Issues of compensation and accessibility are important to consider with any kind of PRO. At in-person visits, patients are often compensated for their time filling out these questionnaires and completing study requirements, as well as ideally reimbursed for travel. In contrast, there might be assumptions that EMAs require less work or less time due to decreased travel or increased convenience. However, at-home methods might be more intensive, since patients might be expected to rate more frequently or consistently within a given time frame, and they could also be perceived as more invasive from a privacy perspective. At-home PROs also have considerably more accessibility requirements, and there might be considerable upfront costs or training to implement an EMA system in a way that serves as many patients as possible (47). For example, patients need to have access to a computer or smart device, as well as internet or a data plan. If any or all of these are not available, teams must decide whether it is monetarily feasible to provide participants with a smart phone and data plan at no cost to them for the study duration. Additional flexibility in terms of user interface design or content collection may need to be implemented, particularly for patients who may require bigger text, read-aloud functions, or different UI interaction methods (e.g., voice control or buttons instead of sliders to accommodate hand mobility differences) (57). Adjusting frequency or time of PRO deployment or reporting to better fit with participants needs might be necessary to accommodate work and family time (58).

It is well-documented that self-reports of pain are influenced by and change significantly based on participants' gender, age, ethnicity, and culture (44, 59, 60). For example, depending on age, previous experiences with pain, and communication ability, utilizing scales with numbers and words alone may not be the best way of measuring patients' pain. In lieu of this, pain drawings—which show the front and back of the body in its entirety—may be a better option for PROs as they allow people to visually indicate where on their body they are experiencing pain. These include simple pain maps where a person circles areas of the body or shades certain dermatomes or myotomes (61, 62), as well as more layered versions that incorporate colors, numbers, or words to indicate intensity and qualia (1–5). Importantly, body maps can be used longitudinally (63) to show how pain somatotopy fluctuates in time, with efforts to digitalize these into EMAs of their own (64). Similarly, the use of pictograms to illustrate pain experiences and sensations has also emerged—these consist of static images or cartoons that often show a person's facial expressions along with body language and additional context about pain location or source, and they have been helpful in studying pain in children as well as those with literacy, speech, or cognitive differences (65–67).

Across nearly all numeric PROs measuring intensity, there is a discrepancy between the underlying neurophysiological pain response and the outward reporting of pain. While there is ample evidence that human perceptions of pain and pain relief are not linear—e.g., in cases like wind-up, hyperalgesia, offset analgesia, temporal summation, or sensitization (68–72)—the large majority of PROs essentially force people to translate this complex non-linear perception onto a linear scale. From a technical perspective, this limitation might be partly addressed via post-hoc statistics or a-priori models using non-linear functions. However, this may also represent a foundational disconnect within pain research and methods more generally. At the very least, it likely points to a need for a re-designed way to assess even the most fundamental dimensions of pain, like intensity⁵.

PROs have been and continue to be a fundamental research tool in pain methods, as they allow for a relatively easy way to communicate various aspects of the pain experience across different time scales and potentially across different contexts. Due to their prevalence and long history of use in the field, many also have a rich literature base and existing set of standard practices and benchmarks [e.g., minimal detectable change (MDC) scores, international population norms, etc.] that researchers can draw upon for comparison and analysis. However, as others have cautioned (73, 74), they should not be placed on a methodological pedestal as somehow being better or less problematic than other methods overviewed here. Critically, the reliance on numbers, pre-selected word anchors, and forced descriptions of qualia or intensity can change how people report their pain (75) and/or profoundly limit patients' ability to share their unique pain stories and define what pain is like for them (76). To measure pain more authentically, we need to understand pain using the person's own language, relying on patients' voices and narratives to define and extract meaningful information about their experiences.

To capture acoustic properties of language, speech is required. Speech can be elicited actively—via responses to short narrative or monolog free-speech prompts⁶, dialogue and semi-structured open-ended interviews (118–121), descriptions of pictures (122), reading passages (123), or verbal motor tasks like phonoarticulatory diadochokinesis (124)—or passively, by randomly sampling voice snippets from the environment (125, 126). Speech data can be recorded using professional sound equipment or with simple hand-held recorders, as well as via built-in microphones in phones and computers. Alternatively, one can also collect non-spoken language or transcribe collected speech into text—in these instances, participants might be asked to write or type their responses. They might also consent to let investigators use prior or on-going free text in the form of social media posts or direct messages, data which can also be obtained via publicly-available datasets, licenses, or crowd-sourcing (127).

The methods chosen and features calculated need to be designed with the specific application or pain condition in mind, as many factors will influence their utility and feasibility. For example, open-ended monolog prompts might be appropriate for adults with pain but may not be easily understood or fully answered by children; similarly, for some populations, speech may be easier or more natural to elicit than typing a response, which might be better for participants who text often or for participants who might find texting more convenient than talking during the workday. Likewise, the use of off-the-shelf or personal recording devices might make sense for larger clinical trials or at-home assessments to scale data collection and analysis, but for smaller studies interested in vocal characteristics of pain, investing in recording equipment and sound-proofing might result in higher quality acoustic data. Additional study design choices could also dictate how burdensome methods like these could be for patients—outside of elicitation methods and frequency, elements such as randomization without repetition and varied prompt or picture topics might also need to be employed to sustain engagement and reduce boredom [for some examples, see (128)].

There are also many procedural and analytic considerations. Hand transcription of voice responses and qualitative coding of language using methods like grounded theory (129) or interpretative phenomenology analysis (130) might work well for smaller studies, but these approaches are not always feasible for larger studies—they may not scale with data collection or analysis needs, nor are they easily reproducible across studies or applicable for acoustic designs. If participants are collecting language using at-home methods, there will likely be additional sources of noise (e.g., additional voices, background TV, movement) (131), which affects transcription accuracy and acoustic feature calculation. These methods will require participant training to minimize these effects, as well as various pre-processing software to filter out these interferences (132–134). In the absence of paid-for transcription services, researchers can also utilize Automatic-Speech-Recognition (ASR) to transcribe spoken language into text for easier data processing (135). However, ASR's transcription quality is only as good as its underlying acoustic models and associated training data (136). Reducing gender and racial biases in ASR and large language models is an area of on-going investigation⁷ (137, 138); instead of relying strictly on population-based language models for transcription, practitioners can pre-train models on small samples of transcribed individual speech from reading passages [e.g., the Rainbow Passage (139)] to improve accuracy and reduce word error rate (140). In addition to issues of scaling collection and transcription, researchers will also need to think about how to scale analyses of such a rich data source. For this, natural language processing (NLP) (141, 142) is commonly used, deploying sub-methods like named entity recognition or topic modeling to classify differences in pain (143–145).

The language we use to represent pain changes as a function of our neuropsychological development, our experiences with pain, and relatedly with age (146–148)—findings or methods that hold true to one age group may not be applicable to another. Similarly, gender identity, associated norms or social conditioning, and racial or ethnic community practices and values can also influence if and how people talk about pain (149–153). Likewise, culture, religious beliefs, and native language also influence how pain is verbalized, described, and conceptualized (154–162). All these elements can affect patient engagement with the method itself, as well as downstream analyses, interpretations of the data, and validation of any results.

Utilizing patients' language as the “ground truth” is in line with the basic premise that pain is inherently subjective and highlights the necessity of respecting patient's reports. However, the utilization of patient language holds normative and even ableist assumptions about the best forms of communication or the legitimacy of different communication sources. There may be some people who prefer not to communicate through talking or may be limited in their ability to use written or spoken language due to cognitive differences or neurodivergence more broadly. There is some research showing existing PROs and language-based measures fall short for people with Autism or Asperger's who may communicate pain differently⁸ (163). This is particularly troublesome given this community may experience more acute and chronic pain than other members of the neurodivergent community or their neurotypical counterparts. Additionally, many of the language models used today carry assumptions about the acoustic tones and rhythms found in “typical” conversational speech. This could negatively impact pain patients with speech impediments (164) or those whose language patterns do not match that of model creators (typically White, English-speaking men) (165, 166)⁹, and could exclude or delegitimize the use of lyrics, rhymes, or singing, all of which can also be used to communicate pain (167).

Thus, although the recent use of language proxies for pain have been an important development in the field—allowing for more intuitive, social, and patient-centric measurements of broader experiences in ways that are relatively flexible and often scalable—there is still work to be done to make this method inclusive. Moreover, sometimes language as a signal simply is not enough, as many authors have already eloquently described; for example, Virginia Woolf said, “…let a sufferer try to describe a pain in his head to his doctor and language at once runs dry (168)” and Emily Dickinson said, “Pain has an element of blank—it cannot recollect when it began or if there were a time when it was not” (169). In moments when pain is particularly intense or unrelenting, people in pain might not be able to speak. Additionally, non-verbal utterances like cries, gasps, or groans may be produced instead of words, and methods and techniques which can both capture these vocalizations and dissociate them from other noises and language features would be useful and appropriate (170).

Because these forms of expressing pain rely on patient-specific communication and can also be time-intensive to create or perform, it makes it difficult to scale or re-apply data collection methods across different populations, cultures, and even across sample sizes of patients (like in the context of larger clinical trials). Likewise, the subjective and intensely personal nature of these methods creates tensions between the need for unique qualitative data and interpretations on the one hand, and current scientific, technical, and medical requirements of reproducible data and quantifiable results on the other. It's challenging to be able to validate these methods across new patient cohorts or new contexts, and even more challenging to form quantifiable measurements of the data (and do so in ways that don't reproduce the pitfalls of other methods or exploit, reduce, or ignore patient experience). While there are a lot of unknowns in this realm, we note that this line of inquiry remains essentially limited to the social sciences and relatively unexplored in the biomedical pain sciences. It would be worthwhile forming and funding interdisciplinary research collaborations between quantitative and qualitative experts, such as but not limited to pain scientists, clinicians, social scientists, artists, and patients.

The visual methods summarized here all try to center the human-experience of pain as expressed by individuals or even collective communities. On their own or in certain therapeutic environments, they can be empowering and disrupting. But within the context of the current healthcare system complex and under the biomedical gaze, they can still perpetuate harmful stereotypes and assumptions, namely that pain is always shown or eventually can be “known” or made obvious (189). Notably, this assumption is found across all the methods reviewed here and is not unique to visual reports, although they illustrate it well. The reliance on appearance or the presumed outward expression of pain in some contexts can function as a form of forced assimilation, as it does not always acknowledge the hidden aspect of suffering and ignores that many pain patients are forced or prefer to hide their pain at work or at home, or present as “normal” in order to function in society [a medical form of “passing” or “covering” (193, 194)]. Due to the assumption that significant or severe pain will eventually be “seen,” there are often incongruences between verbal reports, elicited behavior, and felt internal reality which might cause biased pain assessments or withheld pain treatments. As a community, pain patients already must deal with numerous stigmas surrounding hidden disabilities, perceived burden and guilt, labor and productivity norms, and assumed malingering (195–200). Care should be taken to reduce, minimize, or prevent (if possible) additional stigmatization through the method used or data collected.

Alternative visual reports have played an important role in the pain field, as they have not only allowed for fuller expressions of the pain experience but also largely challenged the traditional status quo of “acceptable ways” to communicate pain. However, there are numerous instances where pain cannot be verbalized, displayed easily, or even “known” at all due to injury, disease, age, neurological difference, or cognitive impairment (e.g., in advanced stages of dementia, in minimally conscious individuals or comatose patients, or in those with severe communication difficulties from ALS or locked-in syndrome). There exist clinician-reported outcomes of pain (CROs) (201, 202) as well as third-person reported outcomes (203–205), which ask care team members, providers, and loved ones to rate their perceptions of the patient's pain. However, these are fundamentally limited since there might be no way for a patient to concur with the pain assessment, advocate for themselves, or rebut care decisions. Additional measures which don't rely on self-reports, self-displays, or others' reports of pain may also be useful and clinically relevant.

When implementing sensor-based measurements, choosing which devices to use or program is not trivial, as there are hundreds on the market. Moreover, many direct-to-consumer devices claim to accomplish similar things or perform similarly to SOTA devices without openly shared data or validation studies (271–273). Additionally, many devices come with their own out-of-the-box or off-the-shelf output metrics that are automatically computed for the user or consumer; care should be taken as to whether or not to take these metrics at face value (since many calculations are “black box” or proprietary and thus cannot be easily examined for noise, privacy, reliability, or accuracy)¹⁶ (274, 275). Likewise, researcher access to raw, unprocessed data and researcher ability to pre- and post-process these kinds of large and noisy data streams need to be considered.

There are also many layers of complexity to implement multiple device collection congruently and privately, particularly if these methods are deployed outside of a clinical or research setting. The sensing system itself, the network connection, data communication and transference, data storage location and requirements (e.g., de-centralized or centralized), data management and security, applications, data standard compliance, and even data analytics (e.g., on the cloud, federated learning, or locally on approved devices) (207, 276) will all affect starting implementation and maintenance costs. Similarly, the reliance on technology for all measurements has some inherent risks for data collection—data quality, accuracy, detection rate, and obtainment can all be influenced by environmental, connectivity, and human factors such as: low light conditions, occlusion, power outages, wearing and charging compliance, fitting issues, or devices going out of network or range of upload hubs. Securing additional secondary methods for collecting data (e.g., diary methodology or questionnaires) should be considered, as well as backup device storage and queued data transfer abilities.

Adaptability is an important design element to consider when deciding on devices and sensors to use. For example, wrist-worn devices need to have adjustable bands to be able to accommodate varying wrist sizes, and different materials should be considered based on skin sensitivities, skin fragility, or skin allergies; the same is true for patches and wireless EEGs, since adhesive materials can be abrasive or irritating to skin (277) or may not adequately capture signals depending on hair type or head fit (278). Patient living conditions and preferences should also be considered, since placement of sensors and co-use by cohabitants also matter (e.g., bed sensors may not be appropriate for people with partners, children, or pets that co-sleep with them, or for patients who sleep in non-traditional locations or environments) (279–282). Importantly, sensor methods might create new forms of participant burden (283, 284). Setting up devices for proper data collection and the need to charge devices might contribute to confusion or additional work for patients (285, 286)—having systems, methods, or devices which can reduce manual set-up, automate data uploads, or maximize battery life will aid in their long-term feasibility and engagement. As discussed with EMAs, there is also the presumption of access—considerations regarding interface design, digital literacy, and digital inequity must all be taken seriously. Time costs associated with patient or caregiver training, and monetary costs for scaled set-up at home or within a clinical infrastructure also should be accounted for up front. As with other methods, consideration of demographic and comorbid health conditions and abilities is also necessary. Gender, age, body mass index, baseline health, exercise regimens, and other factors can influence many of these physiological signals. For instance, HRV as a signal changes as a function of age, activity level, and sex (287, 288), and sensors or technologies themselves can also work differentially depending on patient variables including skin color (289, 290).

Sensor-based measurements of pain are important resources for studying and assessing patients' experiences. Their contribution to the pain field has largely been their ability to collect multiple kinds of data through often passive or minimally burdensome methods, allowing for a much larger breadth of use and the discovery of correlations with other ecological events, physiological signals, or patient activities. However, these measurements also have the potential to perpetuate harmful suppositions about what pain “is.” For example, many of these measures likely follow the recent trend of calling pain a “fifth vital sign”—while pain perception and report should indeed be considered vital to a person's health and well-being, the comparison to a vital sign immediately conjures up mechanisms akin to strictly biological measures like heart rate and blood pressure (72). In some ways, a hyper or a singular focus on these kinds of metrics can work to erase the comprehensive and multidimensional conceptualization of pain that IASP has put forth. This framing ignores the intensely social aspect of pain communication by solely placing the data of interest as something emitted by the patient, as opposed to also being witnessed by others. Likewise, there is often an unsaid assumptions that sensor-based metrics are more objective or less noisy than self-reported metrics, and some even argue that they work to reduce the kinds of human biases seen when measuring pain (291). However, as mentioned, sensors are subject to numerous points of human and environmental interference that can introduce sources of noise. Moreover, we also know that the decisions of which kinds of sensors to use are not neutral (292) and that the underlying sensor data that accompanying algorithms are trained on are not representative. This is easily seen in automatic facial recognition, where pain inference accuracy remains questionable at best (293–295). Setting aside the technology and methods for a moment, we already know that humans aren't reliably accurate at recognizing the emotional expressions of others¹⁷ (172). This effect is seen not only in day-to-day life, but also concerningly within clinical settings, where the facial and bodily expressions of women and people of color are notoriously under-valued, minimized, or ignored altogether (296–303). These existing human errors and pervasive prejudices means that the data sources and the data labels that go into methods like these are in turn quite biased, as are interpretations of the results, which has implications not only for downstream use but also how to even define accuracy in the first place.

There are additional issues related to privacy, surveillance, and consent that are seen using these methods. While most of the data generated are biometric in a generic sense (generated by biology), some data can also be considered biometric in the legal sense (reasonably identifying of an individual), which raises important ethical and legal concerns surrounding protecting privacy, identity, and sensitive health data. This is particularly relevant for passive data collection methods, where someone might easily forget or not notice that a device is on and upload information they may not have intended or wanted to. In addition to privacy concerns, long-term psychological consequences of “the quantified self,” informed and flexible consent (including the right to be forgotten), transparency of data collection and use, and autonomy in decisions made and interpretations formed from these data are all critical issues to think about (304–311).

Each imaging modality comes with unique advantages and limitations. Temporal resolution is highest in modalities that more directly measure brain electrical activity (like EEG) and are much better at capturing fast neural events, but because these measures are most often taken at the surface of the scalp, detecting signals from pain-relevant deeper brain structures is challenging (326). On the other hand, fMRI is more suited to pain-relevant subcortical sites and has considerably greater spatial resolution, but signal detection is dependent on less-direct measures of neural activity and are orders of magnitude slower. Any interpretation about pain/neuronal relationships using this method must take into account blood oxygenation, hemodynamics, and other sources of physiological activity mixed within the signal (327). Electrocorticography (ECoG) has both high spatial and temporal resolution, as well as direct measures of neural activity, but it requires highly invasive surgery, and thus can only be used under special clinical circumstances (328). Additionally, each modality is susceptible to various sources of noise like breathing, heartbeat, eye and body movements, and electrical activity from the surrounding environment. These confounds may be accounted for using a range of signal processing methods (329–331), although there is no agreed-upon field standard.

The limits of these methods to specify and distinguish physiological and cognitive functions relevant for pain remain questionable, particularly for MRI. The “pain matrix,” for example, a highly studied brain circuit which exhibits coactivation of somatosensory, cingulate, and insular cortices with painful stimuli, has mostly been disregarded as pain-specific and is now more commonly recognized as a network which responds to “salient” stimuli, painful or not (314). Similarly, putatively more pain-specific and granular sets of brain regions (332, 333) have shown little to no improvement in distinguishing pain. A recent study showed that using only small fractions of these maps resulted in almost no loss in researchers' ability to decode perceptual pain states (334). Further, the general notion that high spatial resolution in fMRI is even capable of meaningfully explaining cognitive functions is being tested¹⁸. These results further support the need for cross-disciplinary research, suggesting that pain is too complex to be adequately understood using current neuroimaging methods. A harsher take might be that popular neuroimaging methods altogether cannot meaningfully describe fundamental functional properties of the brain (335), and that funding may be more appropriately allocated to methods with greater promise.

Pain research necessarily involves some degree of vulnerability, and this is only compounded when imaging is involved. In attempts to establish experimental control, clinical pain studies are often less than inclusive, with exclusion criteria that may disqualify people with comorbidities, clinical or drug use histories, or who are subject to various other social circumstances. Imaging creates even tighter restrictions: experiments can be lengthy and require multiple visits at special facilities, which can make it difficult for those who need to accommodate work or dependent care schedules, and those without reliable transportation or adequate funds to travel. The imaging process itself can be physically uncomfortable for people who cannot stay still or in certain positions for long periods of time, or those who suffer from anxiety in closed spaces. For MRI experiments, magnetic material must be removed and/or absent from the body during scanning, which makes it impossible for people with certain implants, work histories or medical/cosmetic procedures, or those who prefer not to remove certain items from their body, to participate. Likewise, to obtain stronger EEG signals, researchers may choose to exclude people with coarse and curly hair in order to get closer contact between the sensors and the scalp (278). Enforcing all these criteria, of course, may leave researchers with findings that cannot be generalized outside of a select, homogeneous, subpopulation with limited pain experiences.

The clunky setup of many neuroimaging devices may also prohibit measuring everyday experiences of clinical pain. For example, research participants may not feel their pain while seated or laying in certain positions. Pain may be sporadic and unpredictable, which makes longitudinal tracking or early-stage detection of disease status difficult. Data collection sessions may also be too short to capture slow pain dynamics which can occur over hours, days, or longer. Attempts to evoke feelings of pain while collecting data may impose an unnatural pain experience, and reporting pain during data collection may also affect pain perception (336). For these and other reasons, current neuroimaging methods are not scalable and further limits those who may wish to participate in research. Mobile neurotechnologies may ease some of these restrictions¹⁹, but signal quality from these devices has yet to be fully validated by the research community.

Neuroimaging experiments can also range widely in financial cost. While EEG experiments are relatively cheap, the going rate for a single hour of MRI scanning may be hundreds to thousands of dollars. This price is compounded by a pervasive fascination with big data. “Scale thinking,” or the desire to process and own an ever-growing volume of work, not only frames how industry tends to think about its problems (337) but also influences the research community, as evidenced in efforts to conduct larger and larger neuroimaging datasets like the Human Brain Project²⁰ and Human Connectome Project²¹. The discoveries and solutions yielded from these efforts, however, may not be proportional to the investment. In a recent simulation, increasing participants by orders of magnitude resulted in only incremental gains in predictive performance²². While not disqualifying, results like this one raise questions about what society should expect from these technologies, whether certain methods may reap larger benefits from scientific funding, and at what point these decisions should be made (and who should be making them).

As mentioned, neuroimaging has been a vital method in the field of pain research, providing us with a non-invasive way to interrogate and better understand underlying neurological patterns in pain perception or chronification. Yet, while the brain is undoubtedly key to understanding the mechanisms of pain, neuroimaging researchers should resist promoting neuroessentialist views that reduce the pain experience to mere brain or spinal cord signals. Doing so risks diminishing the importance of measurement from other methods, may misdirect more practical efforts to diagnose and treat pain (338), and may proliferate a social dependence on scientific authority (339).

The healthcare industry generates data in tremendous volumes, and is projected to be one of the fastest-growing data producers in the world by 2025²³. Various types of patient data attract machine learning researchers to healthcare, but the availability of clinical images in particular is pushing radiology toward the forefront of adopting and integrating machine learning into clinical practice (345–347). One recent study using convolutional neural networks (a popular kind of machine learning used for visual object recognition) demonstrated researchers' ability to identify non-standard patterns of knee pathology, and predicted patients' subjective pain reports 61% better than traditional measures (110). Additionally, using this non-standard approach, researchers were able to account for racial disparities in pain perception at 5x the rate than previously accomplished. This finding is particularly important considering growing concern around imbalanced community representation in medical data: it shows that machine learning can uncover pain-related pathologies to the benefit of marginalized communities, despite their historical exclusion in healthcare.

This is a rather uncommon result in machine learning literature, however, which typically results in an elevated risk for maintaining or amplifying existing societal disparities due to the systemic inequities under which data is created, collected, and used (292, 348). This problem is often framed in terms of bias—applying bias correction through data preprocessing methods²⁴ (349) or adjusting model parameters through procedural fairness²⁵ are then frequently suggested as ways to mitigate these risks. Bias, however, can affect machine learning at many stages of development and implementation, and this can be much harder to address with technical approaches which can't account for existing societal inequities and power imbalances (350, 351). An example of this is shown in a recent study that corrected a healthcare algorithm which disproportionately assigned Black patients lower risk scores, despite having worse health outcomes than White patients with the same score (352). Here, the old algorithm used a financial variable as a proxy for health outcome, which differentially corresponded to communities due to differences in needs, access, and trust in healthcare institutions (353, 354). A corrected machine learning model was created through more rigorous evaluation and the use of a dependent variable which more faithfully represented patients' actual health. The result was a reduced disparity of scores across racialized groups.

Of course, designing a machine learning model like this requires more knowledge than is available in a database. It requires a model evaluation process that extends beyond calculating conventional performance metrics, and an ability to critically assess the presumed impartiality of data. This is particularly pertinent in pain research, as pain is neither an objective measure, nor is it measured or treated equally across different patient and societal communities (300). Accordingly, attempts to objectively encode it for the purpose of scaling research or medical solutions through machine learning continues to pose a risk of entrenching societal harms (337)—more data does not guarantee more meaningful discovery or better predictions. Contextual knowledge of data is also needed, as is transparency about the analytics—without either, scientists can end up making the pain experience worse for patients navigating an already unfair system²⁶.

This point is perhaps most germane to the open-source machine learning and data science communities who are known for a culture of sharing, while at the same time publishing highly controversial work in areas where data is plentiful, but subject matter expertise or critique is lacking. The quick surge of machine learning enthusiasts offering their tech expertise in the beginning of the COVID-19 pandemic is a prime example: in a study evaluating 232 COVID-19 predictive models, it was found that none of them were clinically useful, and almost all of them were “poorly reported,” had a “high risk of bias,” and reported performance that was “probably optimistic” (355). This result may be due to the conventional approach of judging machine learning model quality around benchmark metrics that often don't translate well in real-world practice (356, 357). Machine learning models built on the online availability of face data, for example, have spurred studies making claims about the ability to computationally model one's trustworthiness (358), sexual orientation (359), and criminality²⁷, all of which are based on the debunked pseudoscience of physiognomy (360). Facial emotion recognition, as well, stands against long-held observations that emotion and affect are too complex to be captured by facial expression alone (172). Yet, facial expression data remains a valuable commodity for businesses offering affect recognition solutions²⁸, as well as a popular data source for machine learning studies, including those attempting to measure pain (293, 361, 362).

However, open-source data sharing, and the communities that partake, can and do create positive opportunities for the pain research field (363). As of writing this paper, Kaggle, a popular data science challenge forum, had 964 notebooks under the search for “pain”²⁹; GitHub hosts a list of open-source pain databases³⁰; and OpenPain³¹ hosts the largest set of brain images from academic pain studies, available to anyone. The culture of sharing in machine learning presents relatively low barriers to those interested in studying pain (given sufficient time and internet access), opening the field to fresh enthusiasm and unique perspectives. This benefit can be reciprocal as well—pain research might also inspire new machine learning algorithms for single-shot and transfer learning, for example³².

The machine learning field is quickly changing and becoming a central tool for many scientific disciplines—in fact, multiple studies cited in other sections of this review have used it in some form. In some ways, as a tool and as an academic discipline, its use and study can foster the collaborative research we advocate for throughout this paper. But pain researchers must not take for granted that machine learning is a sufficient or superior way to explain pain mechanisms or predict its occurrence, particularly in the absence of contextual knowledge of the complex nature of pain.