Undergoing a diagnostic evaluation for suspected cancer can be perceived as stressful and may cause psychological distress, because of the threat of being seriously ill, the uncertainty of unexplained symptoms, or the invasiveness of various cancer investigations (1). Prior studies analysing the link between cancer investigations, and psychological distress, which commonly include anxiety, low mood, stress, worry, panic, and fear, have been inconsistent. Various studies have linked cancer investigations with a high prevalence of distress (2–5). However, two recent systematic reviews reported a low prevalence of psychological distress across different types of cancer, except for colorectal cancer (CRC) screening due to the invasiveness of the procedures (e.g., endoscopies) as part of their diagnostic work-up (6, 7).

In the context of patients who present with non-site specific (NSS) symptoms concerning of cancer, fewer studies have investigated levels of psychological distress. To expedite cancer and serious benign diagnoses and improve patient’s experience in those with concerning NSS symptoms (e.g., weight loss, fatigue, vague abdominal pain) (8), rapid diagnostic clinics (RDCs) have been established. RDCs aim to provide a NSS pathway by prioritising diagnostics, whilst prehabilitating patients by managing comorbidities, polypharmacy, nutrition, and mental health conditions (9). To our knowledge, only one study has investigated psychological distress in those presenting with NSS symptoms in an RDC setting (1). This Danish study reported on quality of life (QoL) scores of patients with NSS symptoms and noted that those suspected of cancer had similar QoL scores (including anxiety and depression) at the outset of their diagnostic evaluation compared to after their evaluation, irrespective of their final diagnosis. In addition, those individuals who ultimately did not receive a cancer diagnosis experienced an improvement in their QoL scores and a reduction in their symptom burden (1). Furthermore, NSS symptoms may also be indicative of an underlying generalized anxiety or major depressive disorder. Diagnostic criteria outlined in the Diagnostic and Statistical Manual of Mental Disorders, fifth edition (DSM-5), include significant weight loss, decrease of appetite, fatigue, and sleep disturbances for major depressive disorder and fatigue, sleep disturbances, and muscle tension for generalized anxiety disorder (10, 11). Therefore, it is crucial to differentiate whether NSS symptoms stem from a pre-existing mental health condition or an underlying organic cause, so that both the physical and psychological needs of these patients can be properly managed.

Given the limited knowledge about psychological distress in the NSS population we used data from the RDC at Guy’s and St Thomas’ Foundation Trust (GSTT), which was established in 2016 and has seen over 5,000 patients to date to (1): assess the prevalence of anxiety and depression; (2) to identify patterns of patient characteristics and symptoms information that can be associated with anxiety and depression at baseline in this specific patient population. Additionally, we aimed to compare the model performance of a traditional statistical method, with machine learning algorithms in predicting outcomes within our population.

A total of 3,161 patients attended GSTT RDC between June 2019 and January 2023. Of these, 1,734 (55%) patients completed an IMPARTS questionnaire and were therefore included in the study. Supplementary Table 1 summarizes the patient characteristics of those who completed the IMPARTS questionnaire compared to those who did not. Both groups exhibited similar characteristics, except for age groups, whereby older patients (≥75 years) were underrepresented amongst questionnaire completers (40.3% completion rate) compared with younger subgroups (57%-63% completion rates. The demographic characteristics of the patients who completed an IMPARTS questionnaire are summarized in Table 1 . The patient cohort comprised 38.5% (n=667) males, with an average age of 59 years (SD 15.5 years). The distribution of age groups was: 1.7% (n=30) <25 years, 25.6% (n=443) 25–50, 57.6% (n=998) 50–75, and 15.2% (n=263) 75+ years. Ethnicity-wise, 53.2% (n=922) identified as White, 19.3% (n=334) as Black, and 11.5% (n=200) represented other ethnicities. The majority (63.9%, n=1108) of patients had two or more comorbidities, but in terms of performance status (PS), 55.4% (n=960) scored 0. Interestingly, 15.1% (n=262) of participants had a previously identified mental health condition.

Table 2 demonstrates the prevalence of anxiety and depression, based on GAD-7 and PHQ-8 scores, within our study population. Among participants, 8.9% had mild anxiety (95%CI 7%-10%), 12.1% had moderate anxiety (95%CI 10%-13%), and 13.8% experienced severe anxiety (95%CI 12%-15%). Additionally, 7.4% reported mild depression (95%CI 6%-8%), 14.2% had moderate depression (95%CI 12%-15%), 11.9% reported moderately severe depression (95%CI 10%-13%), and 9.3% indicated severe depression (95%CI 7%-10%). Lastly, 6.4% (n=111) of patients had both severe anxiety and severe depression.

In our study of 1,734 patients attending GSTT RDC we found a prevalence of 13.7% for severe anxiety, and 9.3% for severe depression. The combination of the following patient characteristics and symptoms was positively associated with both risk of severe anxiety and severe depression in patients presenting with NSS symptoms in the context of cancer: lower income, weight loss, fatigue, progressive pain, symptom duration exceeding one month, and a previous mental health condition.

As of 2023, the UK Office for National Statistics (ONS) reported a self-reported anxiety rate of 26.6% in women and 20% in men (25). Remarkably, these rates align closely with those observed within our NSS population, where the prevalence of moderate to severe anxiety was 25.9%. Moreover, a recent study focusing on the prevalence of depression, as measured by PHQ-8 scores, in the broader UK population revealed a severe depression rate of 3.3% (26). In contrast, our study identified a nearly threefold higher prevalence of severe depression at 9.3% within the NSS population. This difference may be attributed to the PHQ-8 questions, such as those related to feeling tired or having little energy, potentially inflating scores. This wording could encompass both mental health patients and those with somatic complaints, warranting further examination of the tool’s impact on the observed differences in depression prevalence within our NSS population. Furthermore, when looking at other NSS populations, only one previous study has looked at psychological distress and other QoL measures in patients with NSS symptoms prior to diagnosis and 30 days after referral, a point in time when diagnostics should have been completed (1). In this study, patients undergoing diagnostic evaluations for cancer via the Cancer Patient Pathway for Serious Non-Specific Symptoms and Signs of Cancer (NSSC-CPP) in Denmark exhibited a high prevalence of mild to severe anxiety (~35%) and depression (~25%) before diagnosis. Notably, patients without a cancer diagnosis experienced a substantial improvement in QoL measures 30 days after referral. Our study aligns with these findings, showing that 35% of our patients scored mild to severe anxiety, and 43% scored mild to severe depression at baseline. Further studies are needed looking into post-diagnosis measures of psychological well-being and QoL assessments within our study population.

Furthermore, psychological distress commonly concurs with chronic health conditions, which may negatively impact QoL, and health related outcomes associated with anxiety and depression (27–29). Whilst an RDC is in the first instance designed to speed up cancer diagnostics, only 7% of our patients ultimately have a cancer diagnosis (8). As such, RDCs provide an avenue to triage patients with other comorbidities and ensure support for psychological distress such as anxiety and depression. Fatigue and comorbidity were both identified in the models for anxiety and depression, highlighting the importance of further referrals for these patients – irrespective of a cancer diagnosis. Additionally, RDCs follow the faster diagnostic framework established by the National Health Service in England (NHSE). This framework ensures patients will be diagnosed or have cancer ruled out within 28 days of being referred urgently by their GP (30). Consequently, RDCs offer a promising avenue for swiftly improving QoL in patients distressed by concerning symptoms.

NSS symptoms (e.g., fatigue, chronic pain) have been consistently linked with a high incidence of anxiety and depression (31). In fact, all patient characteristics identified in our models have been independently associated with anxiety and depression in various other studies (32–35). Thus, it is important to note that their combination is indicative of severe presentation of anxiety and depression in NSS symptom patients. Conversely, it is worth considering that underlying anxiety and/or depressive disorders might be contributing to NSS symptoms. For instance, patients with anxiety and/or depression often report higher levels of pain intensity compared to the general population, and the remission of an underlying depression has been linked to reduced pain severity (27, 36). Additionally, as previously mentioned, the DSM-V outlines various NSS symptoms (i.e., weight loss and fatigue), within their criteria for major depressive disorder and generalized anxiety disorder (10, 37). Hence, embedding a psychology specialist and conducting a mental health assessment during diagnostic workup at our centre could prove beneficial for patients presenting with NSS symptoms, in addition to having a direct access to mental health support and other interventions. Moreover, recognizing the impact of a previous mental health condition is paramount. Patients with a history of mental health illness may continue to demonstrate signs of distress, exacerbating their NSS symptoms.

Additionally, the majority of patients presenting with severe anxiety and depression received a final diagnosis of a non-serious benign condition or had no definitive diagnosis. Specifically, 38.5% and 36.4% of patients with severe anxiety and depression, respectively, fell into this category. Furthermore, among cancer patients, only a small proportion—6.7% for severe anxiety and 6.8% for severe depression—were diagnosed with cancer. This observation supports the hypothesis that NSS symptoms may often reflect underlying psychological distress ( Supplementary Table 4 ).

Lastly, this study compared the predictive accuracy of various machine learning algorithms with standard logistic regression modelling in forecasting depression and anxiety. The methods effectively predicted both conditions, with SVM achieving the highest accuracy for depression and logistic regression for anxiety. Despite their success, ML methods did not confer additional advantages within our current dataset due to its nature (electronic health records and patient reported data) and size. ML, although capable of handling several variables and complex interactions, requires larger datasets for accurate analysis (24, 38). In addition, the “black box” nature of ML models (i.e., challenging interpretation), raises transparency concerns, even with tools like variable importance within RF. On the other hand, standard statistics offer simpler interpretation of results, assuming a known relationship between input variables and output (39).

In conclusion, our study reveals a high prevalence of severe anxiety and severe depression among patients attending GSTT RDC. Fatigue, weight loss, progressive pain, and symptoms lasting more than a month emerged as combined risk factors for these psychological comorbidities. Rapid diagnostic clinics, like GSTT RDC, offer an opportunity to alleviate distress in patients with concerning symptoms by expediting diagnostic evaluations. Given the high prevalence of anxiety and depression within our NSS population, as indicated by GAD7 and PHQ8, the use Holistic Needs Assessment would be beneficial. This key screening tool, tailored for patients on an urgent cancer pathway, aids in discerning a range of needs and determining whether observed distress warrants intervention. For individuals consistently reporting distress, an upfront mental health assessment led by a mental health specialist may prove beneficial in addressing underlying mental health conditions contributing to their distress. Future research should concentrate on evaluating patient-reported Quality of Life at different time points (pre- and post-diagnosis) to further understand the trajectory of psychological distress in this NSS population. Moreover, integrating a diverse array of screening tools will provide a more comprehensive insight into the psychological well-being of patients and help identify any unmet and supportive care needs among those presenting with NSS symptoms.

The data presented in this study are available on request from the corresponding author. The data are not publicly available due to ethical reasons.

The studies involving humans were approved by GSTT Cancer Cohort (Ethics reference number: 23/NW/0105) and Health Research Authority South Central – Oxford C (Ethics reference number: 23/SC/0109). The studies were conducted in accordance with the local legislation and institutional requirements. Written informed consent for participation was not required from the participants or the participants’ legal guardians/next of kin in accordance with the national legislation and institutional requirements.

MM-I: Writing – review & editing, Writing – original draft, Methodology, Investigation, Formal analysis, Data curation, Conceptualization. BR: Writing – review & editing, Supervision, Methodology. SM: Writing – review & editing, Data curation. LF: Writing – review & editing, Methodology. CLM: Writing – review & editing, Methodology, Data curation. FB: Writing – review & editing, Data curation. JM: Writing – review & editing, Data curation. LS: Writing – review & editing, Conceptualization. CM: Writing – review & editing, Data curation. CC: Writing – review & editing, Data curation. DB: Writing – review & editing, Data curation. GL: Writing – review & editing, Methodology, Data curation. AS: Writing – review & editing, Methodology. JO: Writing – review & editing, Methodology. AH: Writing – review & editing, Methodology, Funding acquisition. CR: Writing – review & editing. GJ: Writing – review & editing. SS: Conceptualization, Writing – review & editing. AS: Writing – review & editing, Methodology. MH: Writing – review & editing, Supervision, Methodology, Conceptualization. SD: Writing – review & editing, Supervision, Methodology, Data curation, Conceptualization.