Alzheimer’s disease (AD) and frontotemporal dementia (FTD) are the two most common causes of early-onset dementia (Ratnavalli et al., 2002; Lattante et al., 2015), both of which carry a substantial socioeconomic burden (Galvin et al., 2017; Livingston et al., 2020). Differentiating between AD and FTD can be challenging due to their overlapping cognitive and behavioral manifestations (Musa et al., 2020). Previous studies have highlighted the importance of using various strategies, including neuropsychological characteristics (Jiskoot et al., 2021; Maito et al., 2023), multimodal neuroimaging (Li et al., 2022; Mao et al., 2023), electroencephalogram (Si et al., 2023), cerebrospinal fluid (Mattsson-Carlgren et al., 2022; Ortner et al., 2022), and blood (Thijssen et al., 2021; Sarto et al., 2023) biomarkers, to distinguish between patients with AD and FTD. The neuropathology of AD and FTD is characterized by the accumulation of misfolded proteins, which begins several years before the onset of clinical symptoms (Benussi et al., 2022; Jack et al., 2024). The long preclinical stage provides opportunities for early disease detection and timely intervention (Jack et al., 2013; Greaves and Rohrer, 2019). However, evidence regarding non-invasive and early-stage biomarkers, even before symptom onset, to differentiate AD from FTD remains limited.

Optical coherence tomography (OCT), as a non-invasive retinal imaging tool, has provided researchers with enhanced access to detailed retinal neuronal structures (Fujimoto et al., 2000). OCT offers several advantages over brain imaging technologies or fluid biomarker testing. It is non-invasive, relatively low-cost, easy to use in primary healthcare institutions, and provides a higher spatial resolution that reveals cellular-level structures (Men et al., 2016). Currently, OCT is increasingly being applied to the study of dementing disorders for early diagnosis and disease management, including AD, FTD, and Parkinson’s disease (Douglas et al., 2021).

AD is pathologically characterized by the abnormal accumulation of amyloid beta (Aβ) and hyperphosphorylated tau proteins in the brain, which leads to subsequent neurodegeneration and progressive cognitive decline (Bloom, 2014). The pathological hallmarks of AD (amyloid beta and phosphorylated tau proteins) (Hart et al., 2016; Hart de Ruyter et al., 2023; Koronyo et al., 2023) and FTLD-tau (Kim et al., 2019; Kim et al., 2021) have been found in different sublayers of the retina in histopathological studies. Consistent with human histological (Hinton et al., 1986; Blanks et al., 1989) and postmortem (La Morgia et al., 2016) studies, previous cross-sectional studies have indicated that patients with AD dementia exhibit reduced thickness of the peripapillary retinal nerve fiber layer (RNFL) (Larrosa et al., 2014; Sánchez et al., 2018) and macular ganglion cell-inner plexiform layer (GC-IPL) (Cheung et al., 2015; Tao et al., 2019) compared to age-matched cognitively normal controls. Moreover, these retinal structural changes are reportedly associated with reduced brain volume measured by magnetic resonance imaging (MRI) (Casaletto et al., 2017; Mutlu et al., 2017; Mutlu et al., 2018b; Tao et al., 2019) Limited prospective studies with follow-up periods ranging from 12 to 27 months have investigated the longitudinal relationship between OCT parameters and the progression of AD (Mutlu et al., 2018a; Santos et al., 2018), but the findings have been inconsistent across studies. Fewer studies with small sample sizes have demonstrated that patients with FTD have thinner outer retinal thickness compared to controls (Kim et al., 2017; Kim et al., 2019) and that outer nuclear layer (ONL) thickness may be helpful in distinguishing FTLD-Tau from AD neuropathologic changes (Kim et al., 2023). Along with the easily accessible and non-invasive advantages of OCT compared to other existing imaging and fluid biomarkers, the findings above support OCT as a promising candidate for the early in vivo differentiation between AD and FTD. However, the limited evidence remains insufficient to identify reliable retinal biomarkers for the risk of AD and FTD due to small sample sizes and short follow-up periods.

Using a large prospective cohort from the UK Biobank, we aimed to identify retinal biomarkers for the risk of various types of dementia outcomes during long-term follow-up and further explore the underlying brain structural mechanisms linking retinal characteristics to dementia.

The current study provides evidence of the longitudinal association between OCT parameters and the risk of incident dementia based on a large cohort of 30,573 UK Biobank participants. We found that the thinner GC-IPL was associated with an increased risk of Alzheimer’s disease, while the thinner RPE in the inner superior subfield was associated with an increased risk of FTD. In addition, reduced thickness of the retinal layers may potentially contribute to AD by decreasing brain volumes associated with the visual pathway. Our findings strongly support that early changes in the retinal layers may serve as presymptomatic biomarkers years before dementia onset.

In this study, the OCT parameters were significantly associated with AD and FTD, with tau pathology potentially playing a contributory role (Samudra et al., 2023). However, no significant association was found between any OCT parameters and ACD or VaD. These findings suggest that retinal structural changes measurable by OCT may be affected by pathophysiological mechanisms related to tauopathies rather than dementia itself.

We found that the thinner GC-IPL was associated with an increased risk of incident AD, which is consistent with the findings of limited prospective studies (Mutlu et al., 2018a; Santos et al., 2018). As demonstrated in cross-sectional studies, the GC-IPL, one of the inner layers of the retina, exhibited pronounced pathology and thinning in AD during the autopsy (Hart et al., 2016; Koronyo et al., 2023). It was also thinner in the patients with AD and mild cognitive impairment (MCI) compared to the controls (Cheung et al., 2015; Tao et al., 2019) and correlated with clinical disease severity (Kim and Kang, 2019; Liu et al., 2019). Some studies have proposed that the impairment of the cell bodies and dendrites of retinal ganglion cells (i.e., GC-IPL) may be connected with several neurotoxic pathways related to AD pathology, such as increased inflammatory pathways, oxidative stress, and vascular alterations (Wang and Mao, 2021; Koronyo et al., 2023), which may explain the thinning of the GC-IPL in the preclinical stage of AD in our study. However, we did not find any significant association between the axon layer of the retinal ganglion cells (macular RNFL) and the incidence of AD. As a previous study (Santos et al., 2018) was based on a relatively short follow-up period, the inconsistency may suggest that damage to the cell bodies of retinal ganglion cells precedes damage to the axons in the disease progression of AD (Cheung et al., 2015; Ito et al., 2020). Our study further supports that alterations in the cell bodies of retinal ganglion cells are a sensitive retinal biomarker for AD.

Our results indicated that the cortical and subcortical regions involved in the visual pathways mainly mediate the association between the thickness of the GC-IPL and the risk of incident AD. Cross-sectional studies have demonstrated that lower GC-IPL thickness is significantly associated with reduced volume in brain regions mainly located in the occipital and temporal lobes among cognitively healthy individuals and patients with AD (Ong et al., 2015; Tao et al., 2019), even in the preclinical stage of AD with normal cognition (Casaletto et al., 2017; Mutlu et al., 2017) and Aβ deposition (Byun et al., 2021). However, no study has suggested possible associations between the GC-IPL and AD via the mechanism of brain structural alterations. The visual pathways emphasized in mediating this relationship in our study were demonstrated to begin in the retina, travel through the optic tract to the LGN in the thalamus, terminate at the primary visual cortex (V1), and then reach the extrastriate areas (V2-V5) via the ventral and dorsal streams (Joukal, 2017). As key components of the visual pathways, the occipital pole encompassing V1-V4 (Jones et al., 2020) and the thalamus including the LGN were found to have a maximal mediation proportion of over 30%. The regions including the middle temporal area, inferior parietal lobule, inferior frontal gyrus, and lateral occipital cortex were shown to have a potential mediating role in our study as well. These regions were demonstrated to be involved in the dorsal stream for the perception of movement and space (Rizzolatti and Matelli, 2003) and the ventral visual pathway supporting object recognition (Grill-Spector et al., 2001), respectively. In addition, consistent with a cross-sectional study showing the relationship between retinal structure changes and hippocampal atrophy in patients with AD and MCI (Zhao et al., 2020), our study further provides evidence that the hippocampus may serve as a link between the thickness of the GC-IPL and AD. It may be because the hippocampus is not only essential for the memory system (Fortin et al., 2002) but also implicated in spatial perception, eye movements, and scene construction as a visual area (Turk-Browne, 2019).

We demonstrated that the lower integrity (lower FA and/or higher MD) of the visual-related white matter tracts including, the CGC, IFOF, and SLF, mediated the association between the thickness of the GC-IPL and AD. Previous studies have found that the thinner GC-IPL is associated with lower FA and higher MD in the optic radiation of both normal elders (Mutlu et al., 2018b) and patients with AD (Hao et al., 2023). Consistently, our study showed the IFOF, which completely covers the superior optic radiation along its entire course (Sarubbo et al., 2015), may mediate the association between the thickness of the GC-IPL and AD. Furthermore, our study also suggested the potential involvement of lower white matter integrity in the cingulum and SLF, which has been observed in cross-sectional studies of patients with AD (Zhao et al., 2021; Xiao et al., 2022) and has been suggested to serve as a significant predictor of worse visuomotor performance in APOE ε4 carriers (Rogojin et al., 2022). Our findings further suggested that white matter degeneration may be part of the brain’s structural mechanism between the thickness of the GC-IPL and AD.

Our study demonstrated that the thinning of the outer retina was associated with an increased risk of FTD, which is consistent with the findings of previous cross-sectional studies (Kim et al., 2017; Kim et al., 2023). The outer retina is composed of the photoreceptor layers through to the choroid, including the ONL, ellipsoid zone (EZ), and RPE. However, previous research has primarily focused on the thinning of the ONL and EZ in patients with symptomatic FTD (Kim et al., 2017; Kim et al., 2021). In our study, the thinning of the RPE in the inner superior subfield was observed at an earlier stage of FTD. We suppose that the thinning of the RPE may lead to secondary metabolic disturbances in photoreceptor cells, resulting in the loss of the ONL, as the ONL contains the cell bodies of photoreceptor cells and the RPE plays a crucial role in supporting the function and survival of these cells (Wang et al., 2023). Future studies could provide more details on the sequence of retinal changes, including the RPE and ONL, in FTD.

The subgroup analyses suggested age and sex differences in the retinal biomarkers and dementia in the current study. A significant association was observed between the GC-IPL and AD in the older group, while an association between the RPE and FTD was found in the younger group. The finding may be due to the age-specific incidence and prevalence of AD, which increases markedly after the age of 65 (Alzheimer's Association Report, 2023), while approximately 60% of people with FTD are between 45 and 64 years old (Knopman and Roberts, 2011). In addition, we reported that the negative association between the thinning GC-IPL and RPE in the inner superior subfield and AD and FTD, respectively, was more significant in the male patients when compared to the female patients, which has not been found in other OCT studies. However, previous studies have highlighted the role of postmenopausal drops in estrogen levels in increasing the incidence of ocular symptoms and ocular diseases (Peck et al., 2017). Moreover, RNFL thickness was found to be lower with increasing age and hormonal decline (Deschênes et al., 2010; Fathy et al., 2022). This suggests that estrogen may be a protective factor for retinal structures and could partly explain the sex differences in this association. Future studies should place greater emphasis on the sex differences in retinal structures and dementia. In addition, a more significant association between the GC-IPL and AD was found in the individuals with CVD in our study. Vascular pathology has been identified as the most common co-pathology in AD (Robinson et al., 2021) and could accelerate AD pathology and cognitive decline (Agrawal and Schneider, 2022).

The sensitivity analyses showed that the follow-up period and APOE ε4 status might have affected the findings between the retinal biomarkers and the risk of incident dementia. The relationship between the RPE and FTD was not significant when excluding the participants with less than 5 years of follow-up, which may be due to the small sample size of the participants with FTD. APOE ε4 status may influence the association between the GC-IPL and AD, as indicated in our study. Similarly, previous studies also suggested that retinal abnormalities were found in cognitively unimpaired APOE ε4 carriers compared to non-carriers (Sheriff et al., 2023). Future studies should focus on the time span of retinal structural alterations, various types of dementia, and other genetic and co-pathological factors in this association.

The advantages of this study include a large sample size, prospective design, and long-term follow-up. However, the current study also has some limitations. First, the dementia outcomes were ascertained based on the ICD codes derived from electronically linked data, but the status of amyloid-tau-neurodegeneration (ATN) (Jack et al., 2018), a biological classification system for AD diagnosis, was not included. This might have resulted in the potential misclassification of the participants. Secondly, the participants in the UK Biobank cohort appeared to be healthier than the general population (Hanlon et al., 2022), which might have led to a relatively small number of participants with dementia during the follow-up (especially as only eight participants developed FTD) and an even smaller sample size with brain MRI. Therefore, the statistical power of the association between the OCT parameters and FTD might have been affected and the mediation analysis of the RPE, brain structures, and FTD could not be conducted. Thirdly, this study focused on macular imaging using SD-OCT, and the peripheral RNFL, which might be associated with AD, as suggested in previous studies (Shi et al., 2014; Mutlu et al., 2018a), was not included. Finally, the representativeness of the study is limited as the participants were predominantly white. The findings of this study may not be applicable to other ethnic groups. Future research should be conducted in more diverse cohorts.

In conclusion, this study provides evidence for the longitudinal association between macular OCT parameters and various types of dementia in the general population. Given that OCT is widely available, relatively affordable compared to other neuroimaging techniques, and produces rapid, high-resolution images, it might be a practical option for widespread screening, especially when compared to traditional cognitive tests, which can be time-consuming and stressful for patients. Collectively, our study supports OCT as a safe, non-invasive screening tool for early-stage AD and FTD.