The type of sensor measurements available for an assessment will determine the steps in the analysis pipeline and the nature of information extracted about a subject's sensorimotor state. It is thus crucial for any assessment procedure to clearly state the measurement variables used to quantify a specific construct. In this context, we define measurement space as the following.

Definition Measurement space is the universal set of all possible sensor measurements available from an upper-limb for quantifying the different constructs in an assessment.

We denote this set—the measurement space—by 𝕄 and assume that the same quantities are measured from both upper-limbs for the given assessment procedure. Inertial sensing is one of the most common modalities used in the current literature, where the arm movements are measured using wrist-worn accelerometers (𝕄 = ℝ³) or IMUs¹ (𝕄 = ℝ⁶). For a more elaborate measurement setup that includes wrist position and orientation, along with k joint angles of the arm, 𝕄 = ℝ³ × SO(3) × [0, 2π)^k; where ℝ³ is the set of all possible wrist positions, SO(3) is the special orthogonal group of all rotation matrices representing 3D orientations of the wrist, and [0, 2π)^k is the set of k joint angles.

Measurements for an assessment are made over a finite observation period referred to as the measurement epoch. Let M_l(t) and M_r(t) represent the values of the measurements from the left and right upper-limb, respectively, made at time instant t, where t ∈ [0, T], and M_l(t), M_r(t) ∈ 𝕄; T is the duration of the measurement epoch. We use M_l and M_r to represent the entire time series or signal, where M_l, M_r ∈ 𝕄([0, T]); 𝕄([0, T]) is the set of all possible measurement signals over a measurement epoch of duration T seconds starting at time t = 0. In addition to specifying 𝕄, it is also essential for the reproducibility of an assessment to clearly specify the exact sensors used for the measurements, their accuracy, noise characteristics, resolution, sampling rate, etc. The values of these parameters have practical implications for data analysis and interpretation. These practical issues will be not be considered in this manuscript, and to simplify the presentation the mathematical formalism used assumes all measurements to be continuous in time and space.

Definition Upper-limb use is a binary construct indicating the presence or absence of a voluntary, meaningful movement, or posture of an upper-limb.

In this definition, the boundary of what constitutes a “meaningful” movement/posture must be defined a priori. Some examples of meaningful use include reaching and grasping, turning a doorknob, stabilizing an object with one limb while manipulating it with the other, holding a glass, writing, typing, upper-limb therapy exercises etc. Under this definition, involuntary and passive upper-limb movements/postures are not considered meaningful, e.g., resting the arm on a table, upper-limb moved by an external force, etc. There are, however, cases where the presence/absence of upper-limb use is ambiguous, e.g., arm swing during walking, passively resting the upper-limb on a book to prevent the pages from turning, etc. Such ambiguities are best resolved in an application-specific manner, where the set of tasks considered as meaningful are clearly stated a priori. For instance, in the current upper-limb use literature arm swing during walking is not considered as meaningful, even though these are unlikely to be purely passive movements (Blouin and Fitzpatrick, 2010).

Upper-limb use can be mathematically represented as a binary signal over time computed from upper-limb measurements M_i ∈ 𝕄([0, T]), where i ∈ {l, r}. Let f_u be a function representing a measure that maps a given measurement signal M_i to a binary signal u_i over the same temporal domain, i.e., f_u:𝕄([0, T]) ↦ 𝔹([0, T]); 𝔹([0, T]) is set of Riemann-integrable binary signals in the time interval [0, T].

where, u_i is the upper-limb use signal of the upper-limb i. The choice of f_u is determined by several factors, such as the measurement space 𝕄, computational complexity of the measure, sensitivity/specificity of the measure, etc. All measures of upper-limb use exploit some common structure in functional/meaningful movements present in the measured data to detect upper-limb use. Some examples of the current measures (f_u) that make use of accelerometers or IMUs are:

Upper-limb use as defined in this section is an idealized construct, and its detection in practice using sensor measurements will be error-prone due to measurement noise, the natural intra- and inter-subject movement variability, and the relative sensitivity of the sensor measurements to movements and postures. The nature of the measurements 𝕄 and the choice of measure f_u will influence how well upper-limb use can be quantified (e.g., sensitivity and specificity) in practice.

The upper-limb use signals from the two limbs can be used for defining unimanual and bimanual upper-limb use at time t as the following:

Unimanual use refers to a situation where only one of the upper-limbs is used, i.e., only u_r(t) or u_l(t) is 1 at time instant t, but not both. On the other hand, bimanual use involves the use of both limbs simultaneously, i.e., both u_r(t) and u_l(t) are 1 at a given time instant t. Bimanual use will include a wide range of coordination patterns between the two limbs (Kantak et al., 2017), including completely independent use of the two limb (e.g., writing with one hand while holding and drinking from a cup with the other hand) to movement of the two limbs with tight spatio-temporal synchronization (e.g., holding and balancing a tray of glasses with both hands).

Definition Instantaneous intensity of use is a construct that reflects how strenuous a movement/posture is at a particular instant of time, when the upper-limb is in use.

Some examples of measures (f_μ) to quantify instantaneous intensity of use include the magnitude of movement velocity, acceleration, interaction force, muscle activity, etc. Let μ_i represent the instantaneous intensity of use signal for the upper-limb i. It assumes non-negative values when the upper-limb is used, and is defined to be zero otherwise.

where, μ_i ∈ ℝ_≥0([0, T]), and the function f_μ:𝕄([0, T]) ↦ ℝ_≥0([0, T]) computes instantaneous intensity of use signal from the upper-limb measurement signal M_i.

The exact choice for f_μ is application-specific and dictated by M. We also note that results obtained from different types of measurements and different measures f_μ might not be comparable, e.g., the magnitude of movement velocity can be independent of the magnitude of movement acceleration. Thus, it is imperative to report the exact f_μ and its units when reporting the instantaneous intensity of use. Activity counting, as defined in Bailey et al. (2014), Bailey et al. (2015), and de Lucena et al. (2017), is an example of an instantaneous intensity of use measure in the current literature.

In general, μ_i(t) will not be uniformly zero in a continuous interval of time t ∈ [t₁, t₂] where there is a functional movement. However, μ_i(t) can be uniformly zero in a continuous interval under two circumstances:

Definition Average upper-limb use or relative duration is a construct that reflects the proportion of time an upper-limb is used in a given time period D.

Average upper-limb use at time t, denoted by Ui(t;D), can be computed as the average value of u_i in the past D seconds.

Ui(t;D) is a smoothed version of u_i. We will drop D in the parenthesis in the rest of the manuscript and use it only if it needs to be explicitly mentioned. From Equation (4), we can immediately identify some essential properties of Ui:

Definition Average intensity of use is a construct that reflects the average intensity of movements during upper-limb use in a given time period D.

Average intensity of use Ii(t) can be computed from the upper-limb use signal u_i and the instantaneous intensity of use signal μ_i(t) as the following,

where, Ii(t)∈ℝ≥0. The same ambiguity as μ_i(t) exists when Ii(t)=0 for some time instant t. Ii(t)=0 could mean that the upper-limb was either not used during the time interval (t − D, t] or it was used for performing upper-limb postures, depending on the measure used to quantify instantaneous intensity of use.

The amount of use of an upper-limb during a measurement epoch depends on both the duration and intensity of movements performed during this period, which are captured by Ui and Ii, respectively.

Definition Average upper-limb activity is a construct that reflects of how long and how intensely an upper-limb is used in a given time period D.

High amounts of average upper-limb activity correspond to long duration, high intensity movements, while low activity corresponds to short duration, low intensity movements. Average upper-limb activity Ai of the upper-limb i can be captured by the product of Ui and Ii, which quantifies the co-variation of these two factors. We thus define Ai as,

where, Ai(t)∈ℝ≥0 assumes non-negative values and is upper-bound by Ii(t). A subject with high values for Ai would be referred to as more active, than one with lower values of Ai. Visualization of how much an upper-limb is used during a measurement epoch, and its quantification through a single number using Ai are discussed in section 4.2.

We are also often interested in knowing the region of space around a person's body where an upper-limb is used to reach and manipulate the environment (Ploderer et al., 2016). The in-clinic assessment of active range of motion of an upper-limb only tells us about the space that can be reached by a subject. It does not necessarily convey information about the space a subject regularly moves to carry out daily activities. We refer to the latter as the functional workspace, defined as the following.

Definition Functional workspace of an upper-limb is a quantitative representation of space traversed by an upper-limb when carrying out functional activities during daily life.

In this definition space could be the endpoint (Euclidean) space of the hand represented in an egocentric frame of reference, or the joint space upper-limb composed of the various joints of the limb.

Let Wi(t) represent the functional workspace of upper-limb i computed at the time instant t from a segment of M_i from the last Dsec, i.e., from t − D sec to t sec. In order to have a general and informative representation of the workspace, we define Wi to be a probability density function over the space of interest, where the density at a given spatial location is proportional to the relative amount of time spent in a small volume of space around the spatial location during the last D sec of performing functional movements or postures.

where, fW(Mi,ui;t,D) is the function that computes a probability density function from the segment of the measurement M_i and upper-limb use u_i signals between times t − D and t. Such a probability density function would allow useful visualization of the functional workspace as a heatmap representing regions of space that a subject is comfortable traversing during their daily activities. Such heatmaps have been found to be useful by clinicians to understand the nature of use of the upper-limb (Ploderer et al., 2016). It should be noted that one might also be interested in the workspace of the limb when performing non-functional movements—non-functional workspace, which can be obtained from fW(Mi,1-ui;t,D).

Additionally, the use of probability density functions for Wi also enables one to compute various summary measures about the nature of the workspace, such as volume of the functional workspace, preferred spatial locations during day-to-day activities, symmetry of the functional workspace between the two limbs, etc., which help to characterize different aspects of the functional workspace.

The constructs discussed so far—u_i, μ_i, Ui, Ii, Ai, and Wi—are task-agnostic constructs that only depend on whether or not a meaningful movement or posture is performed, irrespective of its type (e.g., reaching, manipulation, drawing). To elucidate the nature of upper-limb use, task-specific measures are required, i.e., measures that can classify the types of tasks being performed, how well these tasks are performed, etc. This information could be used to target therapy to accomplish specific rehabilitation goals. To carry out task-specific analysis, one must first define a set of tasks of interest that can be identified from the measurement M_i.

Definition Task is any upper-limb movement or postural pattern of interest.

Let the set 𝕋 ≜ {0, 1, 2, …p} ⊂ ℕ be a set of natural numbers representing the p distinct tasks of interest; the numbers from 1 to p correspond to the p tasks, and 0 represents all tasks other than these p tasks of interest. Let τ_i(t) ∈ 𝕋([0, T]) represent the task performed by the upper-limb i at time t.

The function f_τ is a measure that maps the measurement signal M_i to τ_i, i.e., f_τ:𝕄([0, T]) ↦ 𝕋([0, T]). We assume that, in general, the p tasks of interest are functional in nature, which implies τ_i(t) can take on a non-zero value only if u_i(t) = 1. Similar to upper-limb use, the detection of tasks from the measurement data will also be probabilistic in nature due to the natural intra- and inter-subject movement variability (Bulling et al., 2014). Human activity recognition using various sensing modalities is a current area of research in the machine learning and artificial intelligence community (Bulling et al., 2014; Nweke et al., 2018). Wearable sensors such as IMUs and vision systems are two commonly employed sensing modalities for recognizing different activities or tasks. The choice of the algorithm f_τ will depend on several interrelated factors, all of which have a bearing on its overall performance in detecting tasks:

A range of different algorithms have been explored for human activity recognition, such as Hidden Markov Models, Decision Trees, Supper Vector Machines, k-Nearest Neighbors, etc. (Bulling et al., 2014). More recently there has been an increased use of deep learning networks for activity recognition (Nweke et al., 2018), which have shown very promising results even with a single wrist-worn smartwatch measuring accelerations of the wrist (Laput and Harrison, 2019).

Definition Movement quality (MQ) is construct that reflects the quality of the underlying sensorimotor control.

MQ is a high level construct indicative of the motor ability of a user, and is composed of other constructs such as movement smoothness (Balasubramanian et al., 2015), coordination (Levin, 1996; Cirstea et al., 2003), tremor (Mansur et al., 2007), etc. Movement quality includes both tasks-specific and task-agnostic constructs, which differ primarily in terms of their computational procedure, and their interpretation. Task-agnostic measures of movement quality (e.g., amount of tremor) can be computed from the M_i without worrying about the underlying tasks being performed. For instance, the amount of tremor in a frequency band of interest could be computed over short segments of movement data from the entire measurement epoch. The numbers, thus obtained, indicate the change in the amount of tremor as function of time. Although there can be several reasons (e.g., the task currently performed) that influence the amount of tremor experienced by a subject, these reasons do not influence what the numbers mean at a given time instant. On the other hand, task-specific measures (e.g., smoothness, coordination, etc.) must be computed only from complete data segments corresponding to a particular occurrence of a specific task. The appropriate interpretation of such task-specific movement quality measures requires necessary contextual information, which must include at least the task being performed (Balasubramanian et al., 2015). For example, an equally smooth reaching movement and a drawing movement will result in two different values computed from the same smoothness measure, because the spatio-temporal constraints of the two tasks are different (Balasubramanian et al., 2015). Thus, for such task-specific constructs the numbers alone are not sufficient to appropriately interpret quality of the observed movement. The context in which the different tasks are performed are also crucial for meaningfully interpreting these numbers in this scenario (Ploderer et al., 2016).

Similar to the other constructs discussed above, issues such as the nature of the available sensor data, the use of appropriate measures for movement qualities, applicability of these measures with different types of sensor data etc. need to be considered. For instance, recent work on estimating movement smoothness with IMUs has shown that the SPARC measure cannot be used with acceleration data, even though the SPARC has been shown to be a good measure of movement smoothness when applied on movement velocity (Melendez-Calderon et al., 2021).

Given these difficulties, it is not surprising that there is currently little work on assessing movement quality of upper-limb functioning in daily life using sensors (Bulling et al., 2014). We note that it might also be of interest to evaluate the quality of postures (e.g., the amount of scapular elevation used by subject to hold an object against gravity), in which case movement quality can be generalized to mean movement or posture quality. Furthermore, common compensatory strategies, such as the use of the trunk to compensate for shoulder and elbow deficits (Levin et al., 2002, 2009), would also fall within the purview of movement quality. Such compensatory strategies can be seen as some form of task-specific abnormal coordination between the different joints of the trunk and upper-limb.

The plots of u_i, μ_i, Ui, Ii over the course of the measurement epoch, allows the user to see variations in these constructs over the course of a day or days. Sample plots of u_l, Ul, μ_l, Il for a healthy (left column) and an impaired subject (right column) over a period of 90 min are shown in Figure 3. The left upper-limb use u_l is visualized as an event plot in Figures 3A,B, where the presence of a gray vertical line at time t means u_l(t) = 1, else it is 0. The average upper-limb use or duration Ul is displayed in a red trace in Figures 3A,B. Figures 3C,D display the corresponding μ_l and Il for this period in gray and blue traces, respectively. We note that μ_l(t) = 0 in these plots correspond to either the upper-limb not being used or a functional posture, since both the GM score algorithm and the activity counts are insensitive to postures. Similarly, Il(t)=0 when the upper-limb was not used or used in a posture in the last D seconds, i.e., Ul(t)=0.

Il(t) only provides a summary of the intensity of left upper-limb use in a temporal segment by computing the average intensity. A more detailed depiction of movement intensity can be provided by displaying the relative proportions of time the movement intensity is low, medium, or high, in an observation window; the definitions of the three intensity levels are provided in the figure's caption for this particular case. The plots in Figure 3 can aid clinical evaluation. It indicates that the overall amount and intensity of use for the patient (right column) is lower than that of the healthy subject. The patient also has little or no high intensity movements compared to the healthy participant (Figures 3E,F).

A visual summary of the amount of upper-limb use during a measurement epoch can be provided through a scatter plot of Ui(t) vs. Ii(t), ∀t, such that Ui(t)≠0². This plot will be referred to as the Use vs. Intensity plot, UI plot, which provides a simple visual summary of the overall upper-limb activity. With no loss of generality, we have chosen Ii and Ui to be the x and y axes of the UI plot, respectively, which has the following properties:

Data from both upper-limbs can be visualized in a single plot by plotting them in the first and second quadrants as shown in Figure 4. Here, the right and left upper-limbs are depicted in the first and second quadrants, respectively; note that the data in the second quadrant are plotted by negating the value of Ii. The light red colored lines in these plots correspond to constant average upper-limb activity lines, i.e., Ai=Ui·Ii=c, where c is a constant.

Figures 4b,c display the UI scatter plots for a healthy and stroke participant, respectively, using data collected from a single day (6–8 h) of recording (David et al., 2020). For the healthy subject, most points are of short to medium duration (Ui<0.5) and low intensity (Ii<50) in Figure 4b, with some long duration, high intensity movements performed with both limbs. In comparison, most movements of the stroke participant were of relatively shorter duration (Ui<0.2), with low to medium intensity movements (Ii<100); high intensity movements Ii>100 were rare. This observation is also evidenced by reduced number of constant Ai lines that cut through the scatter plot in Figure 4c compared to that of the healthy subject.

The distribution of points in an UI plot can be thought of as a sample obtained from a bi-variate probability density function of U and I, pI,U(x,y)³. The univariate probability densities of Ui, Ii, and Ai can be obtained from pI,U as the following,

We define a quantitative measure of how much an upper-limb is used, Hq, as the q^th percentile of A, which can be computed from its probability density function pA,

where, the subscript in q in Hq indicates that the measure is computed using the q^th percentile.

Properties of Hq. We demonstrate through a set of simulated scenarios (Figure 5) that the measure Hq agrees with our intuition. Consider the scenarios depicted in Figure 5, which shows five UI plots, in the top row, with different distribution of points. In each of these plots, points are assumed to be uniformly distributed in the gray regions shown; the light red colored curves are the Ai=c lines, where c is a constant. The rows of plots below the UI plots show the univariate probability density functions pI, pU, and pA estimated from the data points sampled from the corresponding distributions pI,U shown in the UI plots in the top row, along with the corresponding q^th percentile values of the sample data (q was set to 90).

The following observations can be made about the five scenarios depicted in Figure 5, which are reflected in the measure Hq:

Visualizing the relative use of the upper-limbs has been explored through 2D scatter plots or heat-maps of different variables related to the use of the upper-limbs (Bailey et al., 2014; David et al., 2020). Relative upper-limb use can be visualized and quantified using measures of average upper-limb use (Ur,Ul), average upper-limb intensity (Ir,Il) or average upper-limb activity (Ar,Al); here, we use average upper-limb intensity for demonstration purposes. We only consider data points where at least one of the two upper-limbs was used, i.e., Ir(t)+Il(t)>0⁴; it is meaningless to talk about relative use when neither upper-limb is used.

In general, relative use of the upper-limbs can be visualized by plotting two functions g(Ir,Il) and h(Ir,Il) of the subject's data along the x and y axis, respectively. These two function g(·) and h(·) will determine the nature of distribution of data points in this “gh” scatter plot and its fundamental properties. A qualitative understanding of these properties can be obtained from the following four family of curves L₁ to L₄ in the gh plot:

where, c ∈ ℝ_≥0. L₁ and L₂ are particularly useful in explaining the shape of the distribution of points in the different visualization plots, where the bounding curves of a scatter plot are generated from different L₁ and L₂ curves. We present the analysis of three visualization methods, the first one based on the work of Bailey et al. (2014), the second from the work of David et al. (2020), and the third one is a rotated version of second plot. Two additional visualization methods based on BMMR and LIRI are presented in Appendix A.

A quantitative measure of relative upper-limb use should allow us to distinguish between different levels of relative use of the upper-limbs through a single number. Such a measure should map: (a) the spectrum of pure unimanual behavior to pure bimanual behavior to a compact interval on the real line, and (b) report low values for unimanual, and high values for bimanual behaviors.

We can conceive such a quantitative measure of relative upper-limb use through an approach similar to that of Hq. Consider the joint probability density pIr,Il(r,l) of Il and Ir⁵. We can compute the marginal densities of Ir and Il, and the probability density of Ir·Il from pIr,Il(r,l) using the approach in Equation (9). We define a measure of relative upper-limb use Rq as the following,

where, Rq:ℝ≥0[0,T]×ℝ≥0[0,T]↦[0,1] maps two time signals Ir and Il to the set [0, 1]. The subscript q in Rq indicates that the measure is computed using the qth percentiles, and q_r, q_l and q_rl are the qth percentiles of the probability density functions of Ir, Il, and Ir·Il, respectively. It should be noted that q_r and q_l will never be simultaneously zero as we only include data points where Il(t)+Ir(t)>0.

The mapping of different movement behaviors to the interval [0, 1] by this measure is shown in Figure 8, where the LIRI plot was chosen for depicting different types of unimanual and bimanual movement behaviors. The distribution of points in these LIRI plots are indicated by the gray regions, where we assume the points are distributed with uniform density; plots with just a black line depict scenarios where the points are distributed uniformly along the line. The red diagonal line in each of these LIRI plots is the x = y line. The value of Rq for each of these plots is shown in the respective plots, and their location in the interval [0, 1] on the real-line is shown in the bottom of the figure (thick black line) with colored vertical lines. The Rq measure has the following properties.

The measure Rq only tells us if one limb is used over the other, and is silent about which of the two limbs is used more. This information can be obtained from the sign of the different between q_r and q_l, which is +1 when the right limb is used more than the left, and −1 when it is vice versa. Rq along with the sign of q_r − q_l will provide information amount of bias in using the upper-limbs, along with the preferred limb.

Measurements and measures form the basis of any assessment procedure. Measurements contain “raw” information about an underlying behavior, and measures map measurements to numbers that quantify and summarize constructs of interest. Thus, the choice of measurements and measures determine the quality of information obtained from an assessment.

In the assessment of upper-limb functioning, several practical issues play a major role in the choice of sensing modality, such as the compactness, power efficiency, aesthetics, ease of donning and doffing of the sensors, privacy, etc. These constraints on a chosen sensing modality can limit the nature and fidelity of information about upper-limb functioning. For example, accelerometers have shown to perform a little better at detecting different tasks than gyroscopes alone (Bulling et al., 2014). Upper-limb use involving fine finger, wrist, and hand movements are unlikely to be captured by a single IMU worn on the forearm (Subash et al., 2020). Detecting tasks involving physical interactions with environment will require some form of vision technology (Tsai et al., 2020), and cannot be obtained purely from body segment kinematics. Similarly, the movement qualities that can be quantified also depend on the sensing modality, e.g., smoothness cannot be computed from pure accelerometer data, except under special circumstances (Melendez-Calderon et al., 2021).

Since these are early days in the field of assessment of upper-limb functioning, it would be unwise to make recommendations for the sensing modalities required for accurate assessment of upper-limb functioning in daily life. However, one can confidently speculate that a compact, body worn sensing system [e.g., wrist band (Bailey et al., 2014; David et al., 2020; Lum et al., 2020), sensorized clothing (Lorussi et al., 2016), etc.] with more than one sensing modality (Maceira-Elvira et al., 2019) [e.g., inertial sensor, pressure sensor, physiological sensing, vision, radar-on-a-chip for hand movement tracking (Malešević et al., 2019), magnetic ring finger tracking (Friedman et al., 2014) etc.] will become the standard for assessing upper-limb functioning.

Upper-limb use u_i and instantaneous intensity of use μ_i, their averages (Ui,Ii,Ai), and the functional workspace Wi together provide a measure of how much an upper-limb is used during a measurement epoch. These constructs are independent of the nature of the task being performed by the subjects, as they only demarcate functional behaviors from non-functional ones. The work presented in this manuscript focused only on task-agnostic analysis, given that these have been of primary interest in the recent literature. This is necessary information which only sheds light on the overall incorporation of the upper-limbs in daily life, without divulging the details of how the upper-limbs are used. Although, these task-agnostic construct provide some information about motor impairments, a more fine-grained task-specific analysis is required for identifying limitations in activity and participation.

Task-specific analysis requires the segmentation of measurements based on features of specific tasks of interest. Such analysis can allow the estimation of various impairment, activity, and participation level parameters to help build a comprehensive profile of the subject's disability. The details of the tasks performed during the measurement epoch provide information about limitations at the activity (e.g., time taken and range of motion while performing a task) and participation levels (e.g., limitations in carrying out household and work-related activities). The fidelity of such an analysis will depend on the nature of the available measurements, and algorithms that can accurately and robustly detect the task of interest. To our knowledge, there is currently no work on task-level analysis for assessing upper-limb movement functioning. This too is likely to change in the coming years with advances in human activity classification using sensors (Chen et al., 2020). Recent work by Schambra et al. on a taxonomy for upper-limb motion provides a nice framework for decomposing functional movements into different “functional primitives” (Schambra et al., 2019). They also demonstrated that most upper-limb functional activities carried out during therapy are captured by this taxonomy. One possible approach to leverage this work for task-specific analysis of upper-limb functioning is to develop algorithms to detect the five different functional primitives (reach, reposition, transport, stabilize, idle) defined in this taxonomy, and use these detected primitives to further identify higher level tasks/activities. This bottom up approach to detecting tasks/activities would also help quantify the “functional” composition of day-to-day movements in terms of functional primitives. Such a decomposition might be relevant for therapy planning, allowing therapist to focus therapy on primitives that might be limiting the patient's daily activity and participation.

Ploderer et al. (2016) investigated the usefulness of different visualization methods for conveying information about upper-limb functioning in daily life. They found that temporal plots of the amount of upper-limb activity (similar to Figure 3) can be useful in understanding the use of the upper-limb from over several hours to weeks. The clinicians also emphasized the importance of a visualization method in providing a quick overview of the upper-limb functioning (Ploderer et al., 2016) over those provided by Use vs. Intensity (UI) and the relative upper-limb use plots in Figures 4, 7. Plots of functional workspace in the form of range of motion plots of joint angles and/or heatmaps of hand position in an egocentric frame were also found to be useful, which were not explored in the current study; data from a single wrist-worn IMU does not allow the extraction of hand position information or the arm joint angles. This again highlights the importance of the sensing modality in determining the information that can be obtained about upper-limb functioning.

The Use vs. Intensity plot, UI plot provides information about how much the upper-limbs are used during the measurement epoch, taking into account both the duration Ui and intensity Ii of movements. The nature of distribution of points in a UI plot depends on: (a) the nature of measurements M; (b) the function f_u, f_μ used to quantify u_i and μ_i; and (c) the window length D (Equations 4 and 5) used to compute Ui and Ii. The extent to which the choice of these parameters affects interpretation were not investigated in this paper and requires further investigation.

Three approaches for visualizing relative use of the upper-limbs were analyzed in this paper. To promote the development and standardization of an appropriate visualization method, we make the following recommendations:

The visualization and quantification of relative use of the upper-limb were demonstrated using (Ir,Il). Although the properties of the visualization and quantification using (Ur,Ul) or (Ar,Al) are likely to be similar, there will be some differences. One must be cautious of these differences to ensure proper interpretation of the data. For example, unlike Ii and Ai the LIRI plot with (Ur,Ul) is restricted to the square 0 ≤ x, y ≤ 1.

Information about how an individual uses their upper limbs in every day activities is arguably a fundamental criterion of interest to a clinician. Our proposed framework aims at removing the ambiguity of what is meant by upper-limb functioning by defining key components that are necessary to depict “how” individuals behave in every day life in an objective manner. This information is conveyed by upper-limb use, intensity, and their averages, which relate to the overall duration and strenuousness of upper-limb use. The combination of these two constructs provide a good measure of how active an upper-limb is during daily life. Visualization of this information over the course of the day or across days was found to be useful by clinicians for monitoring upper-limb use in daily life (Ploderer et al., 2016). Asymmetry of upper-limb use can be evaluated from the level of activity of the two limbs which, when compared to normative data, can provide the measure of compensation employed by the patient by using the less affected limb. The ability to measure this asymmetry can help identify the underlying cause of this asymmetry, such as specific sensorimotor impairments, learned non-use (Taub et al., 2006), etc. The detailed characterization of upper-limb use by decomposing it into different tasks can provide ecologically relevant information about activity and participation. Furthermore, the time of occurrence, duration, and frequency of different tasks identified during daily life, and tracking these parameters over time can help evaluate changes in the ability and confidence in using the upper-limb either due to therapeutic interventions or spontaneous recovery. Finally, the quality of movement performed while carrying out different tasks can provide additional clues about the sensorimotor control ability and its relationship with hand preference and behavior.

The realization of the clinical utility of the different constructs in this framework and their incorporation in routine clinical use is at least a few years away, given the numerous technical and clinical hurdles that need to be overcome. To this end, we note some of the limitation of the current work and make suggestions for future research in the following subsection.

This work is an initial attempt toward a framework for the systematic analysis and interpretation of upper-limb functioning using sensors. We hope that the ideas presented here form a base for future work in this area, and anticipate that these ideas will be further refined and improved in the coming years. To aid this process, we make explicit the limitations of the current work, which are as follows:

To address the aforementioned limitations and to advance the state-of-the-art in sensor-based assessment of upper-limb functioning, we make the following suggestions for future research: