Psychotic disorders, such as schizophrenia, comprise one of the world’s leading causes of health-related disability (1) and are linked to a high personal burden to the affected individuals, their families, and their carers (2). The first episode of psychosis (FEP) typically occurs during adolescence or early adulthood (3) and is often preceded by a prodromal phase characterised by attenuated psychotic symptoms (4). Attention has therefore been directed towards the early detection of individuals meeting the clinical high-risk for psychosis (CHR-P) criteria (5). CHR-P individuals are typically between 14 and 35 years of age and accumulate various environmental, genetic, and individual risk factors for psychosis (6, 7). The CHR-P status is associated with an increased risk of developing a psychotic disorder (OR = 9.32), which peaks at 2–3 years following the initial clinical assessment and continues to gradually increase over time to a cumulative risk of 0.35 at 10 years of follow-up (8, 9).

Effective detection of young people who might benefit from specialised support is one rate-limiting step for improving outcomes through preventive interventions (10). Currently, the detection of CHR-P individuals is dependent on the available material and human resources of services (11). Psychosis risk assessments are commonly performed by trained personnel in help-seeking individuals who have already been referred or have self-referred to specialised CHR-P clinics (12). Evidence suggests that this is highly inefficient (10). A study of a local NHS Trust found that only 5% of individuals who had FEP had been detected by a local CHR-P service (13). In Australia, the Headspace initiative is a frontline youth mental health service (14, 15). However, despite their “one-stop shop” approach, estimates suggest that only 12% of individuals with FEP are detected during the CHR-P phase (16). As such, the preventive potential of CHR-P services remains largely untapped.

The delayed detection of prodromal symptoms is a major contributing factor to negative long-term outcomes in at-risk individuals (17, 18). Therefore, detecting young people in the community who are experiencing prodromal symptoms and signposting them to specialist treatment may be a step towards improving the outcomes of psychosis. This also aligns with UK clinical guidelines (19, 20), which require the prompt detection and treatment of CHR-P individuals. Promising future avenues for improving the screening and detection of CHR-P individuals in the general population are currently under investigation and include, among others, individualised transdiagnostic (21) or poly-environmental (22, 23) risk calculators, computerised assessments (24), and e-detection strategies aimed at the general population. Among these, e-detection approaches can allow for quick and easy implementation of risk enrichment and detection strategies via the use of web-based versions of self-report scales, such as the Prodromal Questionnaire (PQ-16) (25). This can potentially be used as an initial filter to identify individuals presenting with attenuated psychotic symptomatology who would benefit from a specialised assessment.

Online research has many other advantages, such as having a greater reach, not being limited by geographical location like specialised CHR-P clinics, having increased accessibility, and being able to recruit larger sample sizes in less time with fewer resources for both researchers and participants (26). For mental health research in particular, it can help recruit hard-to-reach and low-prevalence populations and can encourage disclosure of sensitive information, such as mental health symptoms, due to the increased anonymity (26, 27). On the other hand, online health research is faced with important challenges that call for proactive mitigation strategies, notably the risk of fraudulent responses that have been found to affect as low as 3% to as high as 94% of participants in studies (28).

The multi-site ENTER (E-DetectioN Tool for Emerging Mental DisoRders) study aimed at testing and refining an e-detection strategy for the identification of individuals at risk of psychosis in the general population. A longitudinal design was employed to investigate the ability of a self-administered screener (ENTER) to detect true CHR-P, as measured by clinical interview assessments. This method builds upon the Youth Mental Health Risk and Resilience Study (YouR-Study), which provided initial evidence for the feasibility of a web-based screening platform to detect CHR-P individuals in the general population (12). Expanding upon the YouR-Study, ENTER collected information on the following: 1) environmental risk and protective factors; 2) prodromal symptoms of bipolar disorder; and 3) cognitive and speech analysis. All factors have been previously associated with increased risk of psychosis. The aim of the ENTER study was to determine which factors from the screener improve the detection of CHR-P.

This paper outlines the ENTER procedure and recruitment method for automatic digital community screening for psychosis risk, as well as the general sociodemographic characteristics and the PQ-16 scores in the resulting self-selected sample. In addition, this paper describes the cleaning process employed for sample validation to ensure the integrity of the final ENTER study sample.

The aim of the ENTER study was to implement digital screening for emerging psychosis in the community and identify key factors from the screener that improve the detection of true CHR-P as assessed by the CAARMS. The specific aims of this paper were to outline the ENTER procedure, the recruitment methods for automatic digital community screening, and the data cleaning process and to report on the general sociodemographic characteristics and the results of the PQ-16 in the resulting self-selected sample.

The study screened 2,540 participants from sites in London (England), Glasgow (Scotland), and Pavia (Italy). The majority of the sample were of the female sex (77%), self-described white ethnicity (70%), and had some experience in higher education (82%). The mean age was 23 years (the range 12–35 years was set as an inclusion criterion). Just under half (45%) were migrants, either first or second generation. University invitations were the most effective recruitment strategy, accounting for 60% of the participants, which may explain the demographics of the sample.

Almost half of the sample (48%) scored ≥6 on the PQ-16, reaching the threshold set for CAARMS assessment. This was similar to the YouR-study, which found that 52% of its participants aged 16–35 years met the PQ-16 threshold (12). The Tone-P study also employed a web-based screening approach based on the YouR-study to identify CHR-P individuals and found that 43% of participants aged 18–35 years reached the PQ-16 threshold (36).

In addition, the percentage falls within the 35%–65% range reported by other European studies that recruited adolescents and young people from outpatient mental health and child and adolescent services, despite ENTER recruiting from non-clinical sources (37–41). The pretest risk in community samples is typically lower than that in help-seeking samples. The inclusion of many low-risk individuals can create a risk dilution effect, reducing the proportion of true CHR-P cases. To mitigate this and reduce false positives, community samples can be enriched by focusing on participants with a higher pretest risk, for example, those scoring ≥6 on the PQ-16 (42). The rationale for the PQ-16 threshold was based on the finding of Ising et al. (31) that a score of 6 or more on the PQ-16 has a high true-positive rate and specificity when differentiating people at ultra-high risk (UHR) of psychosis from those who are CAARMS-negative. There is some discussion as to whether this threshold is high enough and should take into account the distress scores, especially in non-help-seeking general population samples (37, 39, 41). Subsequent papers on ENTER will publish the CHR-P status of this sample.

This study also highlighted the significant challenge of potentially fraudulent or inauthentic responses in online community samples. Ultimately, in our UK sample, where the participants were paid, over 70% of the responses were deemed invalid and were excluded from the analysis. It is important to note that the main reason for data exclusion was significantly incomplete responses.

Incentive levels may have played a role; for example, ENTER provided a £10 Amazon voucher upon screening completion for the UK sample, while the YouR-study only offered entry into a prize draw. While financial remuneration can help remove barriers to access, it has been found that studies with incentives receive six times more fraudulent behaviour by participants than studies that do not include participant payments (26, 43). The focus of cleaning the data from the Italy sample was on ensuring that the clinical measures were complete and therefore the validation process is not directly comparable (see Supplementary Material). However, the Italian sample, which was not financially incentivised, had no participant responses invalidated due to not passing at least one attention check, providing a pattern of responses or inaccurate contact information.

The promotion of the study on social media may have further contributed to invalid responses, as this can attract participants primarily seeking incentives and facilitate the sharing of strategies to bypass validation measures (27, 44). In comparison, the YouR-study recruited through e-mail invitations sent to universities and colleges in Glasgow and Edinburgh and through flyers and posters, while the Tone-P study recruited through the undergraduate mailing lists of French and UK universities and Amazon’s Mechanical Turk. In ENTER, the top reported recruitment methods in the invalid sample in order (excluding those who did not finish the screener) were social media (33%, namely, via Facebook), friends (31%), and alleged university invitations (20%) (see Supplementary Material).

Given that the aim of the ENTER study was to identify key factors from the screener that improve CHR-P detection, it was paramount to undergo an extensive data cleaning process for future analyses, particularly with machine learning, to avoid the “garbage in, garbage out” problem. Whilst the detection of CHR-P enriched by an online screener is at the research stage, we encourage others conducting online research to proactively anticipate and implement practical considerations to mitigate the risk of fraudulent responses, in particular if monetary incentives are involved. In our experience, no single measure was single-handedly able to successfully identify and screen out all fraudulent respondents. As suggested in previous studies, a combination of automated and manual measures across the whole life span of the project appears to be the most effective strategy (45).

Overall, the results of this study, thus far, showed that online screening for psychosis risk in the general population is a feasible and promising strategy. In addition, real-world clinical settings may naturally mitigate many of the challenges outlined, as there will be a less intensive social media recruitment drive, no financial incentive, and people will be primarily motivated by help-seeking.

In conclusion, nearly half of the young people recruited from the general population who responded to an advert asking “Are you experiencing mental health difficulties?” reached a score of 6 on the PQ-16, which is consistent with the findings of other European studies. The procedures, data cleaning process, and sample characteristics reported here provide important context for future analyses and publications using the ENTER sample. This paper also contributes to addressing the ongoing challenge of fraudulent and inauthentic responses in online research.