Musician’s dystonia (MD) is a task-specific movement disorder that leads to deterioration or loss of control of highly skilled movements at the musical instrument. This career-threatening disorder has a life prevalence of about 1–2% amongst professional musicians (1) and is thus considerably more frequent as compared to writer’s cramps, another form of task-specific focal dystonia, with a prevalence of about 0.08 (2). In the beginning, typical symptoms are subtle: feelings of tension in the forearm muscles when playing, slowing down of fast repetitive movements, and irregularity of movements required to be regular, e.g., scale playing in classical music. Later, curling of fingers or uncontrolled extension whilst playing, “sticking” fingers at the keys due to overactivity of finger-flexor muscles, and visible cramping of wrist extensors and flexors are more obvious (3), and can be observed objectively and assessed via video-rating of informed and trained raters (4).

Risk factors for Musician’s dystonia include genetic predisposition (5), workload at the instrument (1, 6), and over-practice through highly repetitive movements over a long period (7). The diagnosis of the MD is clinical and not straightforward since it requires examination at the instrument and thus, musical knowledge and knowledge of the playing technique of the examiner. For example, the first symptoms usually include unevenness of regular scale-playing, a symptom which develops insidiously and is perceived by musicians; however, it is frequently not perceived by medical experts who have no musically trained ear. Therefore, there is an urgent need to develop a method for diagnosing Musician’s dystonia that can be easily used even by those medical doctors with no expertise in music and musical performance.

The purpose of this study was to develop a “Red Flag” system of biological markers by objectively assessing playing skill at the piano to facilitate medical personnel’s diagnosis of Musician’s dystonia. This diagnosis system draws on artificial intelligence and trained machine learning algorithms. Meanwhile, many diagnostic methods using artificial intelligence (AI) technology are being developed. The application of AI technology in medical imaging diagnosis, for example, is very promising, and there are areas where it surpasses the performance of specialists (8, 9). However, the diagnosis of movement disorders has not progressed as far as medical imaging diagnosis.

In this study, we present a video-based Musician’s dystonia diagnostic system targeting pianists. Previous studies have shown that pianists with focal dystonia have impaired coordination of finger movements (10). Based on this, we use a kinematics-based approach supplemented by video recordings. In addition, we undertook research into the task specificity of Musician’s dystonia and into possible biomechanical risk factors (11). Aberrant anatomical connections between tendons and muscles for example may increase stiffness and the load on specific, mostly extrinsic and intrinsic finger flexor and extensor muscles, which in turn can trigger focal dystonia (12). Indeed, abnormalities in biomechanical conditions impacting finger joint movements were reported to be associated with focal hand dystonia (10, 12), indicating that not only neurological factors but also anatomical and biomechanical factors could be involved. Therefore, we also examined differences in biomechanical functions that are not directly relevant to the piano performance in pianists suffering from Musician’s dystonia and in healthy pianists.

In this study, we proposed a machine-learning algorithm capable of detecting focal dystonia in pianists with an accuracy of approximately 90% by using anomaly detection and a commercially available video camera. The diagnosis of MD has been readily prone to misdiagnosis due to the task-specific manifestation of symptoms and the need for specialized knowledge on complex movement coordination in musical performance. Despite using only video images during the performance, the method proposed in this study achieved high accuracy in the classification of movements between pianists with and without MD. Furthermore, by assessing the finger biomechanical functions, which are unspecific to musical performance, the accuracy of the classification was improved.

Several methods using scale performance and specific tasks on the piano keyboard have been proposed to objectively assess the nature and degree of MD (13). However, these methods require an electronic piano and a non-commercially available program based on data from MIDI technology, which is difficult to handle and requires time-consuming post-processing. This makes them difficult to use in non-specialized institutions in musician’s medicine.

An innovative aspect of the method in this study is the use of simple video footage, allowing for the development of an automatic diagnostic assistance system using a smartphone camera. Furthermore, this study used an anomaly detection method using an autoencoder based on unsupervised learning rather than supervised learning. Unsupervised learning is appropriate here because the prevalence of musician’s dystonia is estimated to be around 1% (1). Therefore, collecting performance data from healthy pianists is much easier than collecting data from dystonia patients. In general, creating a high-performance machine learning model requires a large amount of high-quality training data. In the case of supervised learning, normal and abnormal labels need to be annotated to the performance data to teach the model how to classify. However, when there is a significant imbalance in the number of labels, the features of the minority labels tend to be ignored, such as in the case of rare disorders, which makes it difficult to improve accuracy even by increasing the number of training data (26). By contrast, in the case of anomaly detection methods using unsupervised learning, learning the features of normal data enables the segregation between the performances of pianists with focal dystonia and healthy pianists. Therefore, further accuracy improvement is possible by increasing the training data of healthy pianists. Moreover, by expanding the dataset of healthy pianists, the algorithm can address the diagnosis of an early stage of MD, in which its symptoms only appear on limited passages, and provide follow-up assessments where the pattern of dystonic movements changes with treatment.

The differences in MD detection accuracy among the tasks may offer insights for improving the algorithm. The kinematic features of each task likely account for these differences. For example, tasks requiring minimal finger movement, such as repetitive chord striking (B-9), may not adequately elicit symptomatic movement, which might complicate MD detection. Additionally, considerable inter-individual differences within the training data from healthy pianists could affect the autoencoder’s ability to calculate abnormalities for certain tasks. To enhance the algorithm’s accuracy, carefully selecting performance tasks for training data for the autoencoder and incorporating the domain knowledge of experienced physicians would be beneficial.

The proposed algorithm facilitates the diagnosis of musicians’ dystonia for neurologists and movement disorders specialists who are not experienced in analyzing evenness or sound quality in pianists. More importantly, we now have an easily applicable objective measure for follow-up studies, such as assessing treatment effects. This has been proven to be time-consuming and laborious (27). Plucked instruments can be the next target for extending this algorithm to the diagnosis of MD on other instruments. Video footage of the right hand can be applicable for assessing hyperflexion and hyperextension in a similar manner as proposed in this study. The algorithm for plucked instruments can be extendable for the left hand of string players and in woodwind instruments according to the same principles. Although a single camera was utilized in this study for simplicity, this setup may limit the accuracy of kinematic measurements in less kinematically constrained tasks such as drumming, violin bowing, and guitar playing. This issue could be addressed by altering the source of the kinematic data. Employing conventional motion capture with a machine-learning algorithm appears promising (28). In addition, a whole-body estimation algorithm using multiple RGB or depth cameras is a relatively simple and promising method for such low-constrained tasks. Recently, methods for detecting facial, finger, and whole-body feature points using machine learning have been developed, enabling almost real-time feature point detection (15, 29, 30). Therefore, this method is potentially applicable to the diagnosis of focal dystonia targeting effectors other than fingers, such as embouchure and feet. In principle, the proposed algorithm could potentially be adapted to diagnosing other movement disorders by creating specific patient datasets.

Furthermore, the proposed algorithm can be applied to the treatment of MD. Since it quantifies the degree of abnormality as the autoencoder’s reconstruction error, the algorithm can be directly used for quantitative evaluation of the treatment effect. In addition, this algorithm can quantitatively evaluate the similarity of kinematics, as the performance kinematics are represented by feature vectors in the middle layer of the autoencoder. Therefore, it can assist in establishing treatment plans by analyzing patient data with similar kinematics to determine the most effective treatments applied in comparable cases. An intriguing observation was abnormal asymmetry of the force exertion of all fingers and finger independence between the left and right hand in dystonia patients. These biomechanical functions are not specific to the instrumental performance that triggers symptoms of MD. However, the former is related to simultaneous control of multiple fingers, which is required for musical performance (e.g., playing a chord), whereas the latter is related to a task requiring inhibitory control between adjacent fingers (31). Abnormal surround inhibition of adjacent fingers at resting state in pianists with MD has been reported in neurophysiological studies using transcranial magnetic stimulation (32). The present behavioral task of evaluating finger independence has a clinical advantage over transcranial magnetic stimulation in that the abnormal inhibitory control of the fingers can be readily evaluated. Significant differences between healthy pianists and patients with focal dystonia were evident in motor functions requiring multi-finger coordination, which is unnecessary in daily manual tasks (33). It is unclear whether the multi-finger coordination function was impaired due to focal dystonia or whether the innately unskilled multi-finger coordination function became a factor in the development of focal dystonia. However, abnormalities of generalized, not task-specific, biomechanical dysfunctions associated with task-specific dystonia have been described by Wilson (11) and Leijnse (34). These results emphasize the urgent need for longitudinal assessment of these functions to prevent the development of Musician’s dystonia.

This study has potential limitations. The MD and healthy control groups were not well-matched in terms of gender and age. The demographics of most MD group participants (elderly males with extensive piano training experience) made recruiting a better-matched healthy control group difficult. However, we believe that the kinematic and video parameters in these extremely overlearned and automated tasks in healthy, professional, fully trained pianists are independent of gender and do not alter until senescence (which we excluded). Indeed, we have demonstrated remarkable stability of expert pianists in kinematic and acoustic parameters in scale-playing skills in earlier longitudinal studies (35). Furthermore, Gründahl et al. showed that the severity of symptoms is not age-dependent using video-rating conducted by expert pianists, indicating that the age of the pianists with MD does not affect their movement patterns (4). Additionally, the algorithm currently only discriminates the presence of symptoms and does not classify their severity. Future work will involve establishing the criteria of severity that combine audio criteria and the abnormality of finger kinematics. Creating a large performance dataset of audio and video recordings from a wide range of pianists, including both healthy individuals and patients across various ages and genders, would be beneficial for developing a machine learning-based diagnostic system for MD and its early detection. Furthermore, the proposed algorithm does not provide the reason of the classification because of the lower explainability of the CNN architecture. Other non-parametric classification techniques, such as anomaly detection algorithms using tree structures like the Isolation Tree, may be potential candidates offering better explainability. However, tree-structured algorithms require feature engineering for classification. The variability in the dystonic patterns of the participants, as presented in Table 1, made designing appropriate feature variables challenging. To address the issue of the lower explainability of CNN-based algorithms, techniques for visualizing the basis of image classification in CNN networks, such as Grad-CAM (36), may enhance the explainability of our anomaly detection algorithm. Although to identify the best classifier is out of scope of the present study, it can be significant to apply such classification techniques in future studies.

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

The studies involving humans were approved by Institutional Review Board at Leibniz University Hannover. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

TO: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Software, Validation, Visualization, Writing – original draft, Writing – review & editing. SF: Conceptualization, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Supervision, Validation, Writing – original draft, Writing – review & editing. AL: Conceptualization, Investigation, Methodology, Validation, Writing – original draft, Writing – review & editing. EA: Conceptualization, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Writing – original draft, Writing – review & editing.