In the past two decades, action observation research has been facilitated by technological advancements which allow researchers to easily record and present videos of biological motion. This in turn allowed for the use of video stimuli in controlled neuroimaging and psychophysics investigations of the neural processing of the movements which make up human naturalistic action. Consequently, experimental paradigms involving presentation of pre-recorded videos of various types of bodily motion have become ubiquitous in neuroscience, cognitive science, and psychology. Such investigations have produced great advances in the understanding of biobehavioral phenomena ranging from the cellular to the cognitive level, such as the activity of mirror neurons (Arnstein et al., 2011; Braadbaart et al., 2013; Moriguchi et al., 2009), the Mu and Beta oscillations of the sensorimotor cortex (Avanzini et al., 2012; Brunsdon et al., 2019; Muthukumaraswamy and Johnson, 2004; Quandt and Marshall, 2014), motor affordances (Bach et al., 2011; Tipper et al., 2006), action identification and discrimination (Orban et al., 2019; Urgen and Orban, 2021; Vannuscorps and Caramazza, 2016), gesture recognition (Beattie and Shovelton, 2002), observation learning (Buccino et al., 2004; Malfait et al., 2010), theory of mind (Caillaud et al., 2020; Saxe et al., 2004; Sylwester et al., 2012), and empathy (Rosenthal-von der Pütten et al., 2014; Tholen et al., 2020). To address these phenomena, typically, videos of an agent performing a manual movement are presented to observers either in blocked or in pseudo-random counterbalanced designs while electrophysiological or hemodynamic measures of neural activity are recorded. This approach allows for a resolute assessment of the neurocognitive processing of observed actions, carried out by bottom-up perceptual as well as top-down cognitive neuronal mechanisms. Thus, the approach allows for the investigation of the underpinnings of perception, comprehension, and acquisition of complex action and intention in both neurotypical (Biagi et al., 2016) and neurodivergent (Scott et al., 2020; Spengler et al., 2010) populations.

To effectively capture the neural correlates of the detection and comprehension of the nuanced movements which comprise complex naturalistic action, however, the presented video stimuli must be constructed in accordance with the highest psychometric standards. Duration, frame rate, pixel resolution, color palette, luminosity, filming angle, depicted body segments, gender and handedness of the actor, familiarity with the action and the actor, and recognizability of the action have all been controlled for, at least to a certain extent, in many individual action observation studies. Yet, across studies, there is substantial heterogeneity in the presented video stimuli, with some authors presenting videos depicting only the hand, others presenting videos depicting the entire upper body of the actor; some authors presenting videos from a first-person perspective, others from a third-person perspective; and some authors presenting videos in black and white, others in color. With the advancement of video recording equipment, the overall quality of the video stimuli increases and novel tools have been developed for monitoring changes across video recordings, including nuanced changes in posture (Zouba et al., 2008) or motion kinematics (Trettenbrein and Zaccarella, 2021) of the actor across videos. Still, standardized sets of high-quality stimuli, made specifically for action observation research in neuroimaging settings, which could be used across experiments and laboratories, are scarce, and most investigators in the field construct their own stimuli or rely on non-dedicated freely-available videos from the internet. The reliance on such stimuli which likely differ in either technical properties (e.g., pixel resolution and frame rate) or higher-order characteristics of motion (e.g., goal-directedness and intention relation) prevents the comparison of results across experiments, complicates the conduction of robust and reliable meta-analyses, and hence negatively impacts the reproducibility of action observation research. More recently, open-access multipurpose databases of video stimuli have been developed (Cipriano et al., 2023; Di Crosta et al., 2020; Umla-Runge et al., 2012; Urgen et al., 2022), however, videos therein feature only kinematically correct natural actions, which does not allow for investigations of observation and comprehension of bodily motion with a varying degree of kinematic and goal accuracy. In addition, in spite of the high quality of the videos in these databases, the motion depicted therein was not characterized in terms of when within the video the action begins and when a human observer would actually recognize the action as intended by the experimenter. The absence of a marker indicating when the desired evoked electrophysiological or hemodynamic neural response is likely to occur complicates the use of these databases in neuroimaging settings.

An initial challenge of the serial presentation of a large number of videos in neuroimaging settings is that differences in the timing of movement onset can exist across videos, even among the videos of a carefully constructed stimulus set where all videos are controlled for duration and frame rate. Although some authors have identified and reported the timing of motion onset of their stimuli (e.g., Platonov and Orban, 2016), this practice is often overlooked or underreported in the literature. This is especially an issue for electrophysiological neuroimaging experiments since these typically require averaging of neural responses evoked by a large number of videos (Huettel, 2012), and both time domain and time-frequency domain analyses have to be precisely timed to video onset. In that setting, seemingly small differences in the actual visual detection of movement across videos are likely to contaminate the contrasts between the experimental conditions. For instance, if an onset trigger is sent at the very first frame of each video stimulus' presentation, a discrepancy in movement onset across videos of as little as 3 frames at 60 Hz translates to a jitter of 50 ms. Such jitter can substantially impact the shape and timing of the observed evoked neural responses, resulting in noisier and flattened out averaged neural responses and hence, difficulties in detecting differences between experimental conditions. In hemodynamic neuroimaging, the negative impact of discrepancies between video onset and actual movement onset also exists, although it may be smaller due to the slow nature of the blood-oxygen-level-dependent (BOLD) signal.

In an analogous fashion, the issue of timing and triggering also concerns the timing of the actual conscious recognition of the higher-order properties of movement, such as the degree of integrity of movement kinematics, goal, or intention across different videos, which are the central manipulation of numerous studies addressing the neural and cognitive representations of normal and abnormal movement (e.g., Stapel et al., 2010; Cheng et al., 2017). Such studies often imply presentation of videos that either depict natural or unnatural actions with varying degrees of goal integrity. The videos of natural actions typically comprise kinematically correct movements, as would normally be performed and observed in everyday settings, whereas the videos of unnatural actions comprise kinematically incorrect odd movements, which are unlikely to be performed or observed in everyday settings. However, the motor redundancy and abundance of any complex human action (Gera et al., 2010; Steen et al., 2011) suggest that it is highly likely that the kinematic nature and goal integrity of the actions in different videos are recognized at very different time points. Once again, averaging neural responses with jittered onsets will impact the shape and timing of the observed evoked neural responses and hence, severely contaminate any contrast between natural and unnatural, and between goal-intact and goal-violating action categories.

Therefore, determining when within a video stimulus movement onset and kinematic and goal category are actually perceived is crucial for effective experimental design. Although precise motion characterization of videos can be obtained with complex artificial intelligence computer vision algorithms (Vrigkas et al., 2015), such algorithms are not always freely available and require advanced expertise in deep learning. With respect to motion onset detection, more user-friendly motion detection systems based on the frame-differencing method have been widely applied on video recordings within the field of counseling psychology (Paxton and Dale, 2013; Ramseyer and Tschacher, 2011), however, no such methodology has yet been applied for characterizing videos used as stimuli in action observation research. We propose that such a system can successfully be applied for identifying the first frame of motion onset within video stimuli, so that all video stimuli can subsequently be edited to start exactly at the identified frames. With respect to the recognition of higher-level properties of categories of videos with different kinematics and goal-integrity, no accessible and user-friendly systematic stimulus evaluation procedure exists. We propose that a straightforward psychophysics approach based on the classic up-down staircase methods (Levitt, 1971; Kaernbach, 1991; Wetherill and Levitt, 1965) can be applied for identifying the first frame at which a video can be categorized (i.e., recognition of the movement as kinematically correct or incorrect and goal-intact or goal-violating). Namely, if a video is played up until a given number of frames, then paused, and an observer is asked whether the motion depicted in the video was kinematically correct or not, and goal-intact or not, their response can be used for adjusting the number of frames played on the subsequent presentation of the same video to another observer. With enough between-subject iterations, an asymmetric staircase procedure governed by two simple rules: (1) If the category of motion has been correctly detected within the played segment, the number of played frames decreases on the subsequent presentation and (2) If the category of motion has not been detected within the played segment, the number of played frames increases on the subsequent presentation; should converge on the frame where the category of the video becomes discernible for naïve human observers. The identified frames can be used for implementing precise triggering for the identification of brain or behavioral responses to action kinematics and goal integrity.

Aspiring to improve the feasibility of action observation research, we introduce a large psychometrically evaluated open-access database of videos tailored to the demands of neuroimaging experiments. More precisely, we present 135 videos depicting a goal-directed action that is either kinematically correct or not, and goal-intact or not, filmed from a third-person perspective from 3 angles (portrait, left profile, and top). These videos should be useful for future investigations of the neural mechanisms of action and goal perception and comprehension. Moreover, we provide an easy to implement, open-source frame-differencing motion detection system for identifying the frame of motion onset within each video and a straightforward procedure for editing the videos based on their motion onset. We also provide a straightforward open-source up-down staircase procedure for identifying the frame on which kinematics and goal-integrity are recognized within each video. Both procedures are purposefully designed in a way that would allow any reader to easily implement them on our, or another set of similar video stimuli that depict motion of a single agent.

The present article introduces a novel database of 135 video stimuli for action observation research in neuroimaging settings. It includes high definition videos of a male actor performing object-oriented actions from three categories: kinematically natural and goal-intact Normal actions, kinematically unnatural and goal-intact How actions, and kinematically natural and goal-violating What actions. This publicly available dataset will facilitate the investigation of both bottom-up and top-down neural processing of the characteristics of observed human actions.

Psychometric evaluation of the videos indicated high video classification accuracy and inter-rater agreement, conceptually translating to good validity and substantial reliability of the video categories. Although classification accuracy was very high for videos from all categories, we identified that classification accuracy was consistently higher for Normal compared to What videos in the portrait, profile, and top filming angles. While statistically significant, this difference consisted of, on average, one additional video in 15 in the What category not being correctly classified, compared to the near perfect classification of Normal videos. Therefore, we refrain from interpreting it as a mental phenomenon of importance, or as a serious methodological challenge inherent in the use of our database. Instead, in line with the goal of the publication of our database, these findings could be the object of future psychophysics and neuroimaging investigations aimed at elucidating the recognition of different types of actions.

Subsequent to the psychometric evaluation, every video's exact frame of motion onset was identified with a custom-made motion detection system. Some undesirable differences in motion onset of the original video database as well as slight inaccuracies in the frame-tracing procedure were discovered, therefore we re-edited all videos to assure that perceptible motion begins at exactly the second frame in each video. This assures that the Normal, How, and What videos in each filming angle are equivalent in terms of motion onset, which facilitates their presentation in neuroimaging experiments and circumvents the issue of jittered triggering altogether.

Moreover, the frame on which each video becomes discernibly recognizable as Normal, How, or What was also identified with a novel up-down staircase procedure. We identified significant differences in timing of category recognition between the Normal, How, and What videos with the Normal actions recognized 20–25 frames (or 0.33 s−0.42 s) earlier than the How and What actions. Across videos, there were differences of up to ~220 frames (or 3.7 s) in the timing of action category recognition. These results further highlight the importance of incorporating video motion characterization procedures into future experimental designs. Identifying the respective onset frames allows for counteracting the described differences by incorporating appropriately timed triggers. For instance, in addition to sending a trigger at the moment of presentation of each video on the screen, the experimenters will be able to send a second trigger corresponding to the exact moment of recognition of each video as identified by the staircase procedures. As a cautionary note, given the discovered difference in recognition timing between Normal and What videos, this second trigger will tend to come on average 0.38 s later for the What videos. This implies that experimenters should carefully consider the length of their epoch to assure that their offset is within the presented video's duration for each condition. This way contrasts between Normal and What videos will be performed under identical action observation conditions without potential contamination from the video ending and the interstimulus interval. Although epoch length selection depends on the specific research question, commonly used time periods of 1 to 2.5 s can successfully be epoched following a category recognition trigger from each video from our database. This approach would then allow for more precise grand averaging of epochs, precisely timed to the actual onset of the characteristic of interest. It is expected that such an approach will improve the sensitivity of event-related neuroimaging experiments. For this reason, the video database, the psychometric and motion evaluations, and the scripts required for performing these evaluations are made freely available for future scientific use. Moreover, our two motion evaluation procedures are well suited for characterizations of other already existing (Cipriano et al., 2023; Di Crosta et al., 2020; Umla-Runge et al., 2012; Urgen et al., 2022) or to-be-developed sets of video stimuli. Furthermore, any additional evaluations of the videos such as collection of arousal and valence ratings, object familiarity ratings, imaginability ratings, normative validation in special populations, or collection of eye-tracking data are also greatly encouraged. Motor neuroscience studies of motor resonance or embodiment should particularly focus on characterizing the videos' imaginability, whereas cognitive neuroscience studies should focus on characterizing the videos for arousal and valence. Determining the extent to which participants are familiar with the depicted objects is equally relevant for both motor and cognitive experiments, especially with non-western participants. For a successful experimental design any potential differences along these dimensions should be controlled for. Another fruitful direction for future research is a validation of the results of our motion onset frame-differencing algorithm and our category recognition up-down staircase with more advanced artificial intelligence and marker less motion capture methods to assure their precision of capturing nuanced human movements.

A noteworthy limitation of our database is that our videos depict only a male actor. It is conceivable that nuanced gender effects could be present in investigations of motor resonance and embodiment, therefore, depending on the experimental design, future researchers may need to compare their outcome variable between male and female observers and if necessary supplement the video database with videos depicting a female actor. A further limitation is that in our videos filmed from profile orientation the actor is wearing different attire. This was done to provide better contrast between the actor and the background, however, it could potentially cause differences in the perception of these videos compared to the ones from the other angles. In spite of these limitations, our open-access video database greatly facilitates the process of stimulus preparation for subsequent action observation experiments. By relying on the present set of videos, researchers can design controlled investigations of the perception and comprehension of the kinematics of goal-directed human motion without investing resources into filming videos or depending on haphazard videos gleaned from the internet. Indeed, the reliance on standardized videos across experiments reduces the likelihood of introducing undesired variability into the experimental manipulation due to differences of stimuli. Therefore, reliance on the present set of videos could improve the comparability and replicability of action observation research within motor and cognitive neuroscience.

As it currently stands, the present dataset has a wide applicability for action observation research within various fields of inquiry. The videos were explicitly designed for application in electrophysiological (e.g., Electroencephalography and Magnetoencephalography), hemodynamic (functional Magnetic Resonance Imaging, Functional Near-Infrared Spectroscopy and Positron Emission Tomography), and transcranial stimulation (Transcranial Magnetic Stimulation, transcranial Direct Current Stimulation) studies of brain activity during action observation (e.g., Arnstein et al., 2011; Avanzini et al., 2012; Brunsdon et al., 2019; Dinomais et al., 2013; Nedelko et al., 2010). However, they can also be utilized in investigations of the neural encoding of the kinematics of observed action (Bourguignon et al., 2013; Marty et al., 2018; Savaki et al., 2022), the goals and intentions of observed actions (Hamilton and Grafton, 2006; Nicholson et al., 2017), as well as the actions and goals afforded by objects (Bach et al., 2011; Tipper et al., 2006). Since our videos are highly recognizable, they can also be used in investigations of all above mentioned phenomena during action imitation and action learning interventions for clinical populations with impaired action comprehension such as autism spectrum disorder (Kaokhieo et al., 2023) and schizophrenia (Enticott et al., 2008). Such interventions are expected to strengthen the action perception–execution association, facilitate action embodiment (Tschacher et al., 2017), and lead to improved social cognition. In addition, the videos from the Normal category can also serve as excellent training stimuli for action observation therapy—a promising intervention for motor recuperation in patients with movement disorders such as stroke (Ertelt et al., 2007; Fu et al., 2017) or Parkinson's disease (Pelosin et al., 2010) and an equally promising intervention for maintenance of motor ability and physical performance in healthy elderly (Rizzolatti et al., 2021). The videos can also be used as naturalistic visual stimuli for cognitive neuroscience experiments investigating, among others, attention to and memorization of human action, supplementing the frequently used static pictures and non-naturalistic point-light displays (Gao et al., 2015; Lu et al., 2016; Sifre et al., 2018). Further advances in all above-mentioned fields of research are necessary for gaining a more comprehensive understanding of the neural and cognitive processes which allow for perception and comprehension of observed action and, in turn, execution of appropriate corresponding motor reactions. Such understanding could foster training protocols for timely initial development and, in the case of acquired cognitive and motor impairments, effective recovery of these abilities—both crucial for the display of effective interpersonal behaviors within the cohesive structure of a healthy society.

In closing, in hopes of facilitating action observation research in neuroimaging settings and encouraging open methodology, we provide a large open-access database of psychometrically evaluated videos of an actor performing movements explicitly designed to be kinematically correct or incorrect and goal-intact or goal-violating. We also provide the timing of action category recognition within each video for the sake of maximizing the precision of the assessment of the neural activity evoked by the observation of the different actions. The precise characterization of the videos in terms of psychometric properties and motion onset makes the present database a very suitable stimulus set for experiments aimed at investigating the neural correlates of perception, comprehension and acquisition of observed action.