Diagnostic test cross-sectional study to evaluate the performance of the M@T (study test), compared to the neuropsychological evaluation (gold standard test).

We requested the list of regular users (i.e., assistance frequency >3 times/week) of the elderly care home centers. By means of simple random sampling (table of random numbers), the potential participants were selected until completing a quota of half of available population (sample size = 0.5 N), consented to participate, and provided information necessary to assess compliance with eligibility criteria. The evaluation of cognitive impairment was performed in three successive stages: (1) screening (to detect cases with cognitive impairment); (2) nosological diagnosis (to determinate specific disease that is the cause of cognitive impairment); and (3) final classification (to differentiate disease subtypes).

In the screening phase, an integral clinical evaluation was performed, including measurement of anthropometry and blood pressure, application of Pfeffer Functional Activities Questionnaire (PFAQ) and cognitive screening tests (MMSE and CDT). If any cognitive test was positive for impairment, it was repeated by a different evaluator. The confirmed cases were considered as patients with cognitive impairment (PCI). According to educational level, the cutoff score used was 23 for subjects with 4 years of education, 21 for subjects with 1–3 years of education, and 18 for subjects with less than 1 year of education (Custodio and Lira, 2014). The MMSE and CDT was applied to study subjects, and PFAQ was applied to their caregivers/accompanist.

In the second stage, the PCIs were assessed using blood tests (hemogram, glucose, electrolytes, transaminases, rapid plasma reagin (RPR), urea, creatinine, vitamin B12, folic acid, free T3 and T4, and ultra-sensitive TSH), images studies (brain tomography and/or magnetic resonance imaging), and Beck Depression Inventory II (BDI-II) for discarding non-neurodegenerative causes of cognitive impairment. We applied the DSM-IV (American Psychiatric Association, 2000) criteria to diagnosis dementia, and the Clinical Dementia Rating (CDR; Hughes et al., 1982) for staging dementia. The CDR was applied to both study subjects and caregivers/accompanist.

Finally, in the third stage, we performed the neuropsychological evaluation of patients with MCI or dementia to typify its subtype. We applied the criteria of Petersen (Petersen et al., 1999) and NINCDS-ADRDA (McKhann et al., 1984) to classify as aMCI or AD, respectively. The doubtful cases (regarding typification) were resolved by researchers consensus.

The subjects with negative results in all tests for cognitive assessment were considered as cognitively normal (controls). The M@T was applied to study subjects in first stage and the evaluators were blinded to the results of this psychometric. The results of M@T were not used as part of the neuropsychological battery for diagnosis. The team of evaluators of the second and third phases (expert neurologists and neuropsichologists) was different from the team of the first phase (students of medicine and psychology supervised by expert neurologists).

The corresponding descriptive statistics were performed. The analysis was performed comparing the cognitive groups (controls, aMCI and AD) by pairs. For this purpose we applied T tests (for quantitative variables) and Chi Square (for categorical variables). We assessed the internal consistency (Cronbach’s alpha coefficient) and the concurrent validity (Pearson’s ratio coefficient between the M@T and CDR scores).

We performed a logistic regression (logit) for each pair of study groups (early AD/aMCI, aMCI/control, and early AD/control), using a model of two variables: final diagnosis as dependent variable and test as independent variable. We applied postestimation analysis to compute area under receiver operating characteristics (ROC) curve and graph ROC curve, and calculate validity measures (sensitivity, specificity and positive and negative predictive values).

Additionally, we calculated the diagnostic accuracy (percentage of correctly classified individuals) for M@T, MMSE and CDT. The maximum values of this measure were the standard for the cut-off scores selection of sensitivity, specificity and predictive values. Finally, we compared the AUC of this tests using the method of Hanley and McNeil. The tests were performed at 95% confidence using the STATA software (version 12.0).

This study was carried out in accordance with the recommendations of the Council for International Organizations and Medical Sciences (CIOMS). A written informed consent was obtained from all participants or their carers in accordance with the Declaration of Helsinki. The protocol was approved by the Ethics Committee of the Universidad de San Martin de Porres.

The first stage started with 346 participants, but 41 were missed (14 due to withdrawal of informed consent, 21 due to difficulty in attending scheduled appointments and six due to caregiver or evaluator illness). In the second stage, 22 of 305 participants were missed (seven due to difficulty in attending scheduled appointments, four for lack of blood tests results and 11 for lack of brain tomography).

Finally, 283 participants completed the third stage. However, 36 participants were not included in the present analysis because they were classified as non-amnesic MCI (16), vascular dementia (6), Frontotemporal dementia (4), dementia associated with Parkinson’s disease (2) and other unspecified dementias (8).

Statistical analysis of the sociodemographic data, MMSE scores and M@T scores were performed according to the comparison groups. In patients with AD, compared to those with aMCI, age was significantly higher and test scores (MMSE, CDT and M@T) were significantly lower. On the other hand, in the patients with aMCI the age was significantly higher and the M@T and CDT scores were significantly lower, compared to control subjects (Table 1). The M@T and CDT scores showed a differential distribution according to the comparison group, behaving as a trend (Figure 1). The results of the neuropsychological assessment are detailed in Table 2.

Internal consistency (Cronbach’s alpha coefficient: 0.79) and concurrent validity (r = 0.79; p < 0.01) were good. In relation to the M@T cutoff, a score of 26 allows to discriminate between early AD and aMCI (sensitivity = 100.00% and specificity = 97.53%), with an accuracy of 98.41%. Similarly, a score of 35 allows discriminating between aMCI and controls (sensitivity = 99.17% and specificity = 91.11%), with an accuracy of 96.99% (Table 3).

The performance of the M@T to discriminate between early AD and aMCI was 0.9960 (Figure 2) and to discriminate between aMCI and controls was 0.9956 (Figure 3). The discriminatory performance of M@T was significantly higher than the MMSE (p = 0.000) for all combinations of analyzed group pairs. Furthermore, the performance of M@T was significantly higher than CDT to discriminate between patients with aMCI from controls (Table 3). Additionally, we performed an analysis for assessing if the score M@T is statistically associated with clinical diagnosis (early AD or aMCI; Supplementary Table S1).

This study shows a good performance of M@T to discriminate between early AD and aMCI in subjects with less than 4 years of education. These results are similar to those previously obtained with a sample of 6.5 years of average education (AUC: 0.9986; Custodio et al., 2014) and slightly higher than those obtained in a Spanish sample with 8 years of average education (AUC: 0.9300; Rami et al., 2007).

Similarly, we found a good performance to discriminate between patients with aMCI and controls (AUC: 0.9956), which was slightly lower than that reported previously (AUC: 0.9986; Custodio et al., 2014), but also higher than that obtained in a Spanish sample (AUC: 0.932; Rami et al., 2007). Our research has also shown a good correlation coefficient between M@T and MMSE, which suggests convergent validity. This is a finding similar to that previously obtained with the Portuguese version of the M@T (Sousa et al., 2015).

Additionally, we found that the performance of M@T is higher than MMSE and CDT for discriminating both AD vs. aMCI and aMCI vs. controls. This findings can be explained because M@T evaluates episodic and semantic memory, which have their biological substrate in the hippocampus, the medial temporal lobe and temporal neocortex, areas that are early affected in AD (Rami et al., 2007). In contrast, the MMSE evaluates orientation, language, praxia and general aspects of memory and the CDT evaluates planning, visuospatial and constructive functions. Thus, MMSE is not able to discriminate between AD and aMCI (Tombaugh and McIntyre, 1992; Wind et al., 1997; Rami et al., 2009), and CDT is more appropriate to detect advanced stages of AD (Custodio et al., 2016a).

According to recent UNESCO data, 16% of adults have emerged from education systems without basic literacy skills, which is a major problem in the regions of Sub-Saharan Africa and South Asia, where more than 1/3 of adults are illiterate. Around the world, at least 20 countries have adult literacy rates less than 60% and 43 countries have adult literacy rates less than 75% (UNESCO Institute for Statistics, 2016). Thus, this population constitutes an important group and their needs emerge as public health focus. In this context, valid diagnostic tests for its use in people with low educational level are required.

There are evidence about the demographic influences (e.g., age, gender, education, and residence rural/urban) on the performance of several cognitive tests (Freitas et al., 2015; Li et al., 2016; Xie et al., 2016). Particularly, the education is a key factor since dementia is under-recognized among people with low education levels (Xie et al., 2016). Thereby the international norms of MMSE, the most broadly used cognitive screening instrument, consider different optimal cut-off points depending of educational level to improve screening precision for cognitive impairment (Moraes et al., 2010; Kim et al., 2012; Freitas et al., 2015; Li et al., 2016; Xie et al., 2016). Regarding previous results in Peruvian subjects with at least 6 years of education (Custodio et al., 2014), our data showed that performance with M@T is affected by education and cut-off points should be adjusted.

Additionally, previous studies have shown that non-specialist physicians have difficulties in effectively identifying aMCI and early AD. Thus, it is necessary to develop clinically useful, non-invasive and/or cost-effective, screening tools (Connolly et al., 2011), which must be applicable in primary care centers (Laske et al., 2015)^. In Peru, M@T has been shown to be a reliable test with high precision to discriminate between early AD, aMCI and normal cognition in samples of low educational level (Custodio et al., 2014) and, according to the results of this study, in samples with very low educational level. There are evidence suggesting a progression between various clinical states, beginning with MCI and, after a period of up to 5 years, evolving to dementia in its various sequential stages of severity (De Meyer et al., 2010; Derby et al., 2013). Our results show that, in fact, the average age is higher among patients with AD compared to patients with aMCI and, in turn, they are older than the control subjects.

In addition to age, another important sociodemographic variable is the sex. Several population-based studies have shown nearly two-thirds of individuals diagnosed with AD are females (Dal Forno et al., 2005). In this sense, the sociodemographic profile of the patients included in this study is consistent with that previously reported in the world literature.

In our sample, MMSE and CDT showed a suboptimal performance for discriminating between aMCI and healthy controls. This findings contrasts with previous studies, which found an AUC values higher than 0.80 and 0.70 with the use of MMSE and CDT, respectively (Cacho et al., 2010; Kato et al., 2013). However, a brazilian study showed a low performance of these tests (0.63 and 0.59, respectively; Ladeira et al., 2009). Similarly, other study in high educated sample showed same results (0.70 and 0.61, respectively; Rubínová et al., 2014). Thus, the discrepancy in these topic could be explained for the differences in educational level of participants and, potentially, other regional features.

We have not included rural populations or with native language other than Spanish. Consequently, the results of this study may not be applicable to these population subgroups. The comparison groups were statistically different for the age, a potential confounding variable. However, we performed a secondary sub-analysis for checking that the performance of the logistic regression model is not affected by the age.

The psychometric properties of M@T allow its application in subjects with less than 4 years of primary education in urban settings. Cut-off points should be corrected for educational level and, according our data, values of 35 and 26 are useful for distinguishing patients with aMCI and early AD, respectively, in patients with low level of education. However, M@T should not be used in isolation to define dementia, since it measures memory impairment (episodic and semantic) and orientation well, but no other types of cognitive impairment nor functionality. Therefore, the simultaneous use of brief functional tests to compensate for this deficiency is required.

Recent studies in European populations have evaluated the ability of M@T to discriminate between aMCI and subjective memory complaints (SMC), showing an optimal performance in subjects with medium (Rami et al., 2010) and low educational level (Sousa et al., 2015). Our study did not incorporate this study group. However, we consider that future research should do so because SMC has been reported as a predictor of cognitive decline and AD (Mendonça et al., 2016).

Additionally, the future studies should include population with a broad variability of educational level and higher sample size. Thus, multivariate models could be applied to assess the factors that is statistically associated with clinical diagnosis, which includes the years of education.

The M@T constitutes a brief, non-invasive and reliable cognitive test, which could be applicable for non-specialist physicians to support the discrimination between aMCI and early AD in primary care centers.