With neurodegenerative disease continuing to grow as a major global public health concern (Nichols et al., 2019), identifying risk factors and early biomarkers of neurodegenerative diseases has become increasingly important (Livingston et al., 2017, 2020). Currently, biomarkers for Alzheimer's disease (AD), the most common cause of dementia, tend to be expensive and invasive for the patient or research participant (Peskind et al., 2005; Wittenberg et al., 2019). Commonly used AD biomarkers include amyloid positron emission tomography scans (Chételat et al., 2020), cerebrospinal fluid amyloid and tau levels (Blennow and Zetterberg, 2018; Blennow et al., 2019), and plasma amyloid and tau concentrations (Hampel et al., 2018). These biomarkers are largely limited to specialist hospitals and are not globally accessible. Investigating speech as an AD biomarker may be an important contribution to decentralizing screening and profiling tools on a global scale.

Speech data are relatively easy to collect and require technological solutions that do not rely on specialist staff and facilities. Furthermore, it is a non-invasive and safe technique that may be more acceptable to patients, participants, and the public compared to currently used AD biomarkers, as a first procedure to identify those most likely to be AD biomarker positive. Previous studies have found important associations between features of speech and AD biomarkers (Boschi et al., 2017). Speech markers of interest include semantic processing errors, failure-to-stop errors, changes in connected speech, and declines in syntactic complexity (Ahmed et al., 2013; Venneri et al., 2018; Gollan et al., 2020; Mueller et al., 2021). Assessments of cognition made using speech and language technology have been found to be at least as equally discriminative between different groups when compared to traditional neuropsychological assessment (Garcia et al., 2020). These studies have collected data through face-to-face interactions between researchers and participants, which may again perpetuate the barriers to traditional biomarker testing access. Harnessing technology to gather these speech data remotely, be that via the telephone or videoconferencing platforms, will be an important step in widening access to research opportunities. This may even include digital assistants, such as Alexa, Siri, and Google Assistant, which collect “ambient intelligence” but would need to be considered within a well-designed ethical framework (Simon et al., 2022). These benefits are particularly relevant with the move to decentralized trials and trial access to traditionally underserved groups.

Evidence from studies collecting data remotely has shown comparative performance to human evaluators when making cognitive screening decisions (König et al., 2018; Konig et al., 2018; Themistocleous et al., 2020). A recent review by our group found that remote administration of cognitive tests (via telephone or video-calling) was typically consistent with in-person administration but variable and limited at the person/test level, with stronger evidence existing for videoconferencing over telephone calls, highlighting a need for further research in this area (Hunter et al., 2021). The COVID-19 pandemic drove the need to further explore the feasibility of conducting cognitive assessments via remote means, again with videoconferencing methods more typically considered for the adaptation of traditional cognitive assessments (Geddes et al., 2020). A number of cognitive assessments have also been adapted or even designed to be administered via telephone call with this form of assessment appearing to be viable, although not meant to replace gold-standard in-person evaluations (Carlew et al., 2020).

When developing new digital biomarkers, it should be of high priority that acceptability to users is also evaluated. Successful engagement with a novel tool can only arise when it has a favorable acceptability profile. A previous study using a different tool found that collecting speech data on the phone was acceptable to a group of older adults (Diaz-Asper et al., 2021). This study used a short cognitive task of language fluency, as well as a more narrative speech task, and is a helpful indicator of the likely acceptability of collecting speech biomarkers on the phone. Automated testing in the form of social robot administration and smartphone applications have previously been reported to be most feasible and acceptable to participants (Takaeda et al., 2019; Taylor et al., 2023), although in-person human cognitive testing remains mainstream in most clinical and research settings. The recent paradigm shift to digital neuropsychology warrants consideration from validity and ethical perspectives. Barriers to the digitalization of cognitive testing typically focus on operational issues, such as hardware and software barriers, as well as on the validity of the same test across different devices (Germine et al., 2019), with less attention paid to the acceptability of the digital version to patients and research participants. The use of artificial intelligence and machine learning in dementia detection is also a topic of ethical interest. As models derived from these techniques need to be trained on large data sets, these are commonly initially derived in data sets from research participants who often do not represent the general population living with dementia and, as such, have limits to their generalizability (Ford et al., 2023). Included acceptability in the earliest stages of digital biomarker design may be one opportunity to more holistically consider all ethical aspects of these new tools.

Disclosure of participant-level data in research studies is an area of increasing interest, and the routine disclosure of data collected on the phone is an important part of feasibility and acceptability testing. Previous studies suggest that there is a public demand for greater information about their risk factors and current disease status regarding AD, particularly surrounding disclosure of APOEε4 genotype and AD biomarkers (Unterman et al., 1993; Caselli et al., 2014; Ott et al., 2016). A number of studies have investigated the psychological impact of disclosure of genotype and biomarkers and typically found no associations with long-term psychological distress or risks of anxiety or depression (Green et al., 2009; Lim et al., 2016; Burns et al., 2017; Taswell et al., 2017; Wake et al., 2017; Grill et al., 2020). However, some studies have identified short-term adverse outcomes alongside positive experiences (Vanderschaeghe et al., 2017; Largent et al., 2020). Thus, there is a need to provide psychological safety considerations to prevent possible harm. Cognitive test results completed in research settings are rarely fed back, particularly not to healthy participants or those with mild cognitive impairment (MCI). Nevertheless, cognitive tasks are commonly utilized, and anecdotally many participants express interest in their personal test performance. Understanding the acceptability of providing these test results and the consequences is an important consideration.

The Speech on the Phone Assessment (SPeAk) study (Gregory et al., 2022) was designed to collect speech data on the phone at two time points from participants at risk for AD, using both semi-structured and structured speech tasks. This study aimed to assess the usability and acceptability of the study protocol in this population. There was also an analysis of the acceptability of receiving cognitive test results in the context of a research setting.

The results from this usability and acceptability analysis suggest that completing this brief battery of cognitive tests via a telephone call is both acceptable and feasible in a midlife-to-older adult population in the United Kingdom living at risk for AD. There were no obvious ceiling or floor effects for immediate recall, digit span, or fluency tasks. Some participants did perform at ceiling for the delayed recall task at both baseline and follow-up visits. There was a normal distribution of scores across the cognitive tasks at both visits, suggesting that these tasks perform as we would expect if administered in a face-to-face setting. Previous reviews have found that telephone-based cognitive testing is appropriate in the assessment of cognitive aging (Castanho et al., 2014). Participants enrolled in the study were free of dementia at enrolment. Comparing the scores to studies using the same subtests with in-person administration, we can see participants on average performed in the cognitively normal range (Harrison et al., 2000; Choi et al., 2014; Sirály et al., 2015). A recent review by our group found consistency between in-person and remote administration of cognitive tests for MCI and AD, although noted individual- and test-level variation (Hunter et al., 2021). As this group largely represented a cognitively healthy group, the findings are discussed in this context, with further work needed to understand feasibility and acceptability in populations with more cognitive impairment.

Three of the tasks (immediate recall, delayed recall, and semantic fluency) demonstrated no significant changes between baseline human testing and follow-up automatic phonebot testing, suggesting these are reliable to use in both administration modes. The RCI indicates individual-level changes in performance, which may reflect personal preferences of testing or cognitive fluctuations at the individual level. There were significant differences in performance on the digit span and phonemic fluency tasks between the baseline and follow-up visits. The digit span score significantly decreased between visits. As no other test scores decreased, it is unlikely that the cohort experienced significant cognitive decline in the interim 3-month period, and this decline in score may reflect some hearing, processing, or attentional difficulties when digits were read by the automated voice. This task may need further development before this can be used reliably in a fully automated manner. The increase in the phonemic fluency task may reflect a learning effect, although, interestingly, this was not seen in the semantic fluency task. This improvement in score is driven by women achieving a higher number of words at follow-up compared to baseline, with men's scores decreasing slightly at follow-up. These results reflect data from previous studies, suggesting a small majority of participants improve on a second attempt at fluency tasks (Harrison et al., 2000).

In general, women performed significantly better than men on a number of tasks at baseline and follow-up. This finding aligns with previous literature, where women appear to have an advantage in verbal memory compared to men despite carrying similar pathology burdens (Sundermann et al., 2017). Interestingly, while there is evidence in the literature for women outperforming men on phonemic fluency tasks (Hirnstein et al., 2022), we saw an effect for semantic but not phonemic fluency. It may be that the category of phonemic fluency was advantageous to both men and women to perform well (Hirnstein et al., 2022), leading to a lack of differentiation by gender, and it appears that women had a stronger learning effect compared to men on the semantic fluency task leading to a difference in scores at follow up. Age was significantly associated with most tasks, with education associated with task performance on a small number of tasks with a lack of consistency seen across visits. The importance of gender, age and education should be considered in further work to generate normative scores for these tasks when delivered via the telephone. Education has been reported as associated with performance on these cognitive tasks in previous studies (Zimmermann et al., 2014). Age and gender associations with cognitive test performance for these tasks are inconsistent in previous studies (Van der Elst et al., 2006; Zimmermann et al., 2014; Woods et al., 2016).

Participants found the experience of completing the tasks on the phone to be generally acceptable. The set-up of the appointment was easy for participants to engage with, and participants were comfortable with completing these tasks in their home environment. Although there was an expressed slight preference to complete tasks face-to-face, there was agreement that participants would be happy to repeat these tasks on the phone in the future with either a human or an automatic phonebot tester. Participants appreciated receiving the results of their cognitive tests, and there was no clear preference for receiving these after both visits or at the end of their study participation. While some information was provided on the results proforma, which was co-designed with participant advisors, on what scores might be expected for each task based on in-person testing normative scores, some participants did express a desire for more detailed information. This feedback can inform future studies, giving evidence to support the safe feedback of cognitive test results, with even more detailed proformas to establish the context of what the scores meant.

To the best of our knowledge, this is one of the first studies to evaluate participants' experiences of receiving cognitive test score results. Previous studies have evaluated the effects of disclosing APOE genotype and amyloid imaging results to cognitively healthy participants and found there was no increase in psychological distress, anxiety, or depression (Green et al., 2009; Grill et al., 2020). As acceptability of results disclosure was a secondary outcome, we did not gather any data on psychological distress or adjustment before and after results disclosure, and further work should explore this topic in more detail, as well as understand how to present this information to patients and participants in an understandable and meaningful way. However, participants self-reported the experience of receiving the results as overwhelmingly neutral or positive, along with no reports of psychological distress from any participants included in the study, suggesting there is likely little harm caused by disclosing test results from standardized tests used in a different format. As the phonemic fluency task was the only increase in mean scores at the follow-up visit, it is unlikely that result disclosure was associated with any change in test performance. There were a small number of problems that arose due to hearing difficulties. This should be noted as a consideration for wider implementation of phone-based speech and cognitive test data collection.

A key strength of this study was the initial co-design with participant panel members, whose input to the study design and materials was to ensure the project was of interest and comprehensible to the target population. The use of electronic data capture allowed participants to take part in a study during a period when in-person study engagement was limited due to the COVID-19 pandemic. The study has some limitations, with participation restricted to those who had previously engaged in cognitive testing. As such, there is a limit to the generalizability of testing-naïve participants. The participant population was also homogeneous, representing a highly educated cohort, which does not reflect the general population from which they were originally drawn. As this study recruited exclusively from those who had participated in the two earlier studies, there were no mitigations that were able to be put in place for this study; however, future studies should endeavor to explore the use of this technology in diverse populations. As previously mentioned, this should also be tested in a more cognitively impaired population to understand if this form of testing remains feasible and acceptable. It is also important to understand the usability of this tool regarding hearing impairment and the use of hearing aids. Further studies using this tool could include the use of brief hearing tests to establish the level of hearing impairment each participant has as well as record any diagnoses of hearing loss or hearing aid usage. This is important not only as a test that relies on hearing skills but also given the recognized importance of hearing loss as a risk factor for future dementia (Livingston et al., 2020). The testing order was fixed, and as such, we were not able to adjust for test order effects when interpreting changes in test scores.

To conclude, completing a short battery of cognitive tests on the telephone is both feasible and acceptable to an older adult population who are cognitively normal or at risk for future dementia. Future studies are needed to replicate this work in a more diverse participant population with a more pronounced cognitive decline. This study provides initial evidence on the value of feeding back cognitive test results to study participants, with further studies needed to explore more in-depth psychological safety of this approach.

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

The studies involving humans were approved by Edinburgh Medical School Research Ethics Committee (EMREC). The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

SG: Conceptualization, Data curation, Formal analysis, Methodology, Project administration, Writing—original draft, Writing—review and editing. JHa: Conceptualization, Methodology, Supervision, Writing—review and editing. JHe: Formal analysis, Methodology, Project administration, Writing—review and editing. MH: Data curation, Formal analysis, Methodology, Project administration, Writing—review and editing. NJ: Data curation, Project administration, Writing—review and editing. AK: Conceptualization, Methodology, Writing—review and editing. NL: Conceptualization, Funding acquisition, Methodology, Writing—review and editing. SL: Conceptualization, Methodology, Writing—review and editing. EM: Project administration, Writing—review and editing. HP: Data curation, Methodology, Project administration, Writing—review and editing. MW: Data curation, Project administration, Writing—review and editing. SR: Writing—review and editing. JT: Data curation, Formal analysis, Writing—review and editing. CR: Conceptualization, Funding acquisition, Methodology, Supervision, Writing—review and editing.