Between 2004 and 2010, 9- to 13-year-olds (N = 1,535) attended a summer camp in the northeastern US that was designed for youths experiencing chronic stress. Based on an adult sponsor letter and interview with a school staff member who knew the youth, camp staff categorized sources of a youth's chronic stress as family poverty, serious family problems (e.g., incarcerated parent), social problems (e.g., severe peer rejection), poor academic performance, or emotional problems (e.g., mood disorder but diagnoses were not made during the study), but did so without using standardized tools. Camp staff ratings suggested that the youths had a mean of 2.4 (SD = 1.3) of these stressors. The participants were from urban, suburban, and rural residences (the data for which were not quantified). The youths completed the YRI and other ALEXSA-R subscales while attending camp as part of the program evaluations. Although many youths attended the camp over multiple years, the data analyzed here were only from the youths' first year of camp attendance. More details regarding camp eligibility, data collection, and longitudinal findings are reported in previous studies (35, 39, 53–55). Chronic Stress sample data were analyzed to evaluate the YRI's latent structure and validity.

Sixth-grade students (N = 1,661) completed the YRI to screen for their eligibility to receive preventive intervention from 2017 to 2021. They were from rural, suburban, and urban public schools across a Midwestern state. Parents of students with moderate or high risk were offered a brief preventive intervention and referral to additional behavioral health services for the student at no cost to the family. Sex was the only variable collected other than YRI screening. The YRI was group-administered to students in classrooms using tablets and website-based surveys. Student responses and YRI scores were stored remotely to protect student confidentiality, and parents were notified by mail of their eligibility to receive the preventive intervention. The Sixth Grader Sample data were included only in the analyses of the YRI's latent structure.

From 2014 to 2015, 641 youths, aged 8–13, were recruited from contiguous US regions other than the Northeast to derive standard scores that were normalized to the US population (standardization analyses included data from previously recruited participants from the Northeast). Participants were recruited from areas around Atlanta, GA; Boston, MA; Chicago, IL; Dallas, TX; Durham, NC; Fort Smith, AR; Los Angeles, CA; San Francisco, CA; and Washington, DC. Participants were recruited by private, for-profit survey companies from their pool of parents and youths who were interested in participating in research. Parents responded to advertisements for the study sent out by the survey companies to their pool of pre-registered, eligible participants. The sole exclusion criterion for youths (in addition to age) was having a severe developmental delay. Standardization Sample data were used in the analyses of the YRI's latent structure and concurrent validity.

From 2012 to 2018, two subsamples of patients attending Well-child Check-ups were recruited from the same healthcare system to in part evaluate the acceptability and feasibility of using the YRI for screening within primary care (32, 33). The first subsample of 61 patients, aged 9–12, was recruited at a suburban pediatric private practice (72.2%) or a rural hospital (27.8%) to evaluate the feasibility of the screening protocol. Youths and parents were consented and screened by nurses while they waited for physicians in an examination room. Inclusion criteria required participants to be English-speaking and not have a moderate or severe intellectual disability (per patient records). The parents of youths who scored in the moderate- or high-risk range were provided with contact information for a family therapist who had agreed to work with families to reduce the youths’ risk. This subsample's data were included in the analyses of the YRI's latent structure and concurrent validity.

The second well-child subsample of 564 patients, aged 10–13, was recruited for a clinical trial at a hospital-based pediatric practice that served patients residing in low-resource, urban neighborhoods (33). After the pediatric staff obtained verbal consent for research contact, the study staff approached families in exam rooms to obtain parent and youth assent and conduct YRI screening. If a youth's (or parent's) screening score was in the elevated risk range, the family was invited to participate in a clinical trial in the Family Check-Up prevention program. The other inclusion criterion was that the child had received needs-based Medicaid or the family income was at or below 150% of poverty guidelines. Exclusion criteria included an inability to speak English or the child having a moderate or severe intellectual disability. Data from this sample were included in the latent structure and validity analyses. The CONSORT diagram for the participant recruitment process is presented in Figure 2.

This community sample of 60 youths, aged 9–18, participated in a neurodevelopmental study on the impacts of stress and trait anxiety on the maturation of neural circuitry that underlies stress regulation, emotion, and reward processing. A subsample of 33 participants completed the YRI, which was added to the protocol after the study was underway. Inclusion criteria were a higher-than-average level of stress, increased risk of psychosis, and a familial background of schizophrenia or bipolar disorder. Participant data were analyzed to evaluate the YRI’s validity for capturing the neurocognitive risk of SU/SUD.

Youth reports of the risk factors that were used for the validity analyses were collected in the Chronic Stress, Standardization, and Well-child Check-up samples using other measures from the ALEXSA-R. The ALEXSA-R is an illustration-based computer-assisted self-interview with audio that measures risk factors for, and early manifestations of, youth SU and problem behaviors. Its items are organized into domain scores and individual subscales (based on factor analyses), which are reliable and valid for youth ages 7–19 of different races, genders, and literacy levels (34, 57). Self-management quantifies learnable skills that reduce the probability and severity of harmful consequences from mistakes, emotional dysregulation, or behavioral disinhibition, including SU, using three subscales: Concentration, Planning, and Problem Solving (54). The Sensation Seeking domain is based on Zuckerman's theory and research, using measures that were adapted for late childhood and early adolescence: Gambling, Social Disinhibition, and Thrill Seeking (34, 58). Parent Fortification is composed of three subscales that quantify facets of parent–offspring relationships that are negatively associated with SU/SUD: Parental Monitoring, Parental Attachment, and Parental Nurturance (34). The School Protection domain's four subscales quantify aspects of youths’ academic performance and school-based relationships that are negatively associated with SU/SUD: Academic Competency, School Bonding, School Commitment, and School Atmosphere: Adults (the last measures a youth's perception of how supportive school staff are of students) (34). Some individual ALEXSA-R subscales were also analyzed: Depression, Count of Substances Used, Conduct Disorder Behaviors (a count of behaviors), Parental Problems with the Law (binary), and Parental Problems from SU (binary) (34).

Parent participants in the larger Well-child Check-up subsample completed well-established and widely used instruments. While useful for testing whether parent reports are associated with YRI scores, we expected small correlations at best, based on (1) the well-replicated finding that parent and youth reports on youth characteristics are largely discrepant (59) and (2) parents reported on different characteristics than those that are measured by the YRI items. The Child Behavior Checklist subscales of internalizing symptoms and externalizing behaviors were hypothesized to correlate positively with YRI scores (60). The three parent report measures, which are included in the Family Check-Up assessment battery, were Parental Monitoring, Parent Supports Good Behavior, and Parental Rule Setting (61).

To estimate the degree to which YRI scores reflect cognitive functioning and underlying neural substrates that contribute to SU/SUD liability, a subset of the CogNIT study measures was analyzed. The Wechsler Abbreviated Scale of Intelligence's (WASI) Forward Digit Span measures attention deficits. The subscales of the Delis–Kaplan Executive Function System (D-KEFS) included the Planning-sequencing Tower Test [quantifying planning and problem-solving abilities (62)] and the Color-Word Interference Test [processing speed and executive functioning (63)]. A fractal (N-back) test quantified working memory (64, 65, 94). The CogNIT participants completed the YRI in person, along with the neurocognitive tests, prior to MRI or EEG recordings.

Most of the YRI items are ordinal, four-option Likert items, whereas YRI06–YRI11 are six-option count variables. Several YRI items were right-skewed and some were extremely zero-inflated, including YRI02 (takes out frustrations on others), YRI04 (yells at someone), YRI08 (friends start fights), YRI09 (friends bully others), YRI10 (friends have trespassed), YRI11 (friends have vandalized), YRI12 (access to tobacco), YRI13 (bothers students who are working), YRI21 (minor illegal dare), and YRI23 (would smoke). Two items had bimodal distributions: YRI07 (friends have skipped school) and YRI09 (friends bully others, which was also zero-inflated). When possible, item recoding was avoided to retain maximum item information and retain consistency with prior research and uses, and because the YRI has traditionally been scored using item means (32, 33). Items with extreme L-shape distributions were recoded for consistency with factor model assumptions (56, 67) while balancing the retention of the variance in responses against severely violating normality assumptions as much as possible (i.e., maintaining skewness at less than ±2 and kurtosis between −4 and 10). Specifically, YRI02 was trichotomized while YRI10, YRI12, YRI21, and YRI23 were dichotomized. In addition, a log link function modeled count items (YRI6–YRI11), except YRI10, which had been dichotomized.

Figure 1 presents the two latent structure models that were compared using relative indices of model fit based on Akaike's Information Criterion (AIC), n-adjusted Bayesian Information Criterion (nBIC), log-likelihood (LL), and inspection of factor loadings. Confirmatory factor analyses were conducted using maximum likelihood with robust standard errors (MLR), assuming normal distributions for all factors, which unfortunately does not provide indices of absolute model fit. The alternative weighted least squares with mean and variance adjustments estimator is not appropriate for count data. The best-fitting model factor loadings were estimated for the aggregate sample, then separately by sex (boys vs. girls) and again separately by YRI application purpose (research vs. screening for intervention) to investigate any latent structure differences.

After determining the YRI's best-fitting latent structure, measurement invariance was tested first between sexes and then between YRI application purposes using moderated non-linear factor analyses (68, 69). Measurement invariance testing was conducted by sequentially comparing the fit to the metric invariance, configural invariance, and scalar invariance data. Competing models were compared using the Rutkowski and Svetina (70) criteria for invariance [−0.02 comparative fit index (CFI) change and 0.03 root mean square error of approximation (RMSEA) change for metric; −0.01 CFI and 0.01 RMSEA for scalar].

When non-invariance was found, we identified the item(s) that contributed to the non-invariance. For each model, we constrained the variance of the factor to one and freely estimated all the factor loadings. When testing for partial metric invariance, we constrained parameters for all but one item at a time. After we identified partial metric invariance, we tested scalar invariance only for the items for which metric invariance was satisfied. Testing for measurement non-invariance involves many hypothesis-free statistical tests, so we used a criterion of α = 0.01 to reject the null hypothesis of invariance to balance Type I and Type II error rates.

The YRI total and subscale scores’ concurrent and predictive validity were estimated using bivariate associations, with the specific statistic based on the distribution of the validity criterion (Pearson’s r, Spearman’s rho, or odds ratio). To estimate YRI thresholds’ improvement over chance as a screening tool, the area under the receiver operating characteristic curve was estimated (71). To preliminarily evaluate the impact of revising the traditional YRI scoring algorithms to account for non-invariance, associations were estimated for both the traditional scores and adjusted scores accounting for non-invariance. Validity analyses were conducted separately within study samples because the specific validity criteria varied among samples, to replicate validity tests, and to inspect the robustness of bivariate associations across different populations. Analyses of neurocognitive functioning were considered preliminary due to the relatively small number of participants, emphasizing effect sizes rather than p-values, and were repeated using multiple regression to statistically control for age as a proxy for developmental variability among participants. Single analysis and/or Benjamini and Hochberg’s false discovery rate (72) p-values were presented.

In the aggregated sample, the Liability and Subscales model better fit the data (AIC = 120,353; nBIC = 120,529; LL = −60,112.49; df = 64) than the Overall Liability model (AIC = 123,190; nBIC = 123,551; LL = −61,533.77; df = 61); the two models were not nested, precluding a likelihood ratio χ² comparison. The Liability and Subscales standardized factor loadings are shown in Table 3. Each first-order factor loaded at least 0.7 onto Liability in the aggregated sample. At the item level, the Disinhibition item loadings were greater than 0.4. The Peer Conduct Problems item loadings were larger at 0.6 or greater. The Social Contagion item loadings were at least 0.5 except for YRI20 (loading = 0.343). When estimated separately for boys vs. girls or research vs. screening purposes using unconstrained two-group models, all factor loadings closely resembled results from the aggregated sample, suggesting that the Liability and Subscales model offers a robust latent structure (configural invariance) across subgroups.

Measurement invariance was tested separately for each latent factor because screening traditionally is based on YRI's total score (i.e., Overall Liability), while interpreting each first-order subscale enhances the information garnered from a youth's screening results. In addition, the fit of the second-order factor is identical to the fit of the three-factor model with correlated factors (because the three off-diagonal elements of the factor correlation matrix are reparametrized into the three second-order loadings). As such, the second-order factor also supports the three-first-order-factor model as a better fit to the data than the traditional single first-order factor. The primary benefit of the second-order factor is that it allows estimation of the higher-order factor score, which incorporates differential loadings at the item and first-order factor levels. Moreover, the second-order factor corresponds to the traditional overall YRI score on which screening decisions are based. Since the data do not support item-level unidimensionality, we explored invariance for each of the three first-order factors in addition to the second-order factor.

For Overall Liability, measurement invariance was found between sexes, but partial scalar non-invariance occurred between purposes (Table 4). Compared to applying the YRI for research, when used for screening, the YRI scores were lower by 0.367 on average. Differential item functioning occurred for two items, YRI13 and YRI14, both of which were originally designed to measure distractibility within school settings. When the YRI13 and YRI14 intercepts were free to vary, measurement invariance was met for the remaining YRI items (the change in fit from metric to scalar invariance was −0.017 for CFI and 0.000 for RMSEA).

When evaluating first-order factor scores in isolation, more items contributed to partial non-invariance compared to Overall Liability (Tables 5, 6). For Disinhibition, YRI01 had partial metric non-invariance between sexes and purposes. The three items for Peer Conduct Problems had partial metric and scalar non-invariance across purposes. Social Contagion's YRI12 had metric invariance across sexes and purposes, while YRI20 and YRI23 had metric invariance across sexes. The current results cannot indicate whether scoring refinements to account for non-invariance due to these items might meaningfully and clinically improve YRI scoring because even non-consequential differential item functioning may be statistically significant with a large enough sample size (74, 75).

Although full practical impact analyses were beyond the scope of this study (75, 76), a preliminary evaluation of the impact of differential item functioning was conducted by comparing traditional YRI scores to scores that were adjusted to account for non-invariance, which were both standardized for comparisons in the same metric. Means and ranges were similar for traditional and adjusted scores, respectively (X¯ = 0.00 for both; minimum = −1.41 vs. −1.46; maximum = 3.29 vs. 4.93). The differences between individuals’ traditional and adjusted scores were negligible on average (X¯ = 0.04, SD = 0.28) but were sizable for some youths (ranging from −0.77 to 2.07). Similar results were found for the three first-order factors (Table 7). Overall, the score distributions differed little between traditional and adjusted scoring, whereas the adjusted score distributions had smaller ranges, with the greatest differences in persons who had high liability scores using the traditional YRI algorithm.

The rank order of scores changed little due to adjusting for non-invariance, as the two scoring algorithms correlated highly for Overall Liability (Pearson’s r = 0.96), Disinhibition (r = 0.99), Peer Conduct Problems (r = 0.89), and Social Contagion (r = 0.93). Similarly, large correlations were found within both sexes and purposes (Table 8). Hence, predictive accuracy and validity were reported when only using traditional YRI scores.

YRI total score (when used for screening) associations were large for SU and conduct problems (2+ conduct disorder behaviors), both concurrently and up to 1 year later (Table 9). Overall predictive accuracy was good in all analyses, based on the area under the receiver operating characteristic curves, with stronger accuracy for conduct problems compared to SU. The results observed in the Chronic Stress sample, in which the YRI items were originally identified, were replicated in the general population (Standardization) and clinical (Well-child Check-up) samples. Moreover, similar results were observed between youths who identified as girls vs. boys (Table 9).

Table 10 presents bivariate associations between YRI scores and youth self-reports of well-known risks and resiliencies to SU and conduct problems. YRI scores correlated positively with risk factors (Sensation Seeking, Parental Problems with the Law, Parental Problems from SU, and Depression symptoms), with mostly medium-sized associations. YRI scores correlated negatively with resiliencies (Self-management, Parent Fortification, and School Protection), with small to medium-sized coefficients. In addition to predicting binary coding of any SU and conduct problems (2+ conduct disorder behaviors) (Table 9), the YRI scores correlated with the Counts of the Number of Different Substances Used and Conduct Disorder Behaviors. No consistent differential associations with vulnerabilities or resiliencies were observed across the YRI first-order factor scores.

As expected, compared to youth self-reports, the YRI scores correlated to a smaller degree with parent reports of youths’ externalizing in the Well-child Check-up sample (Table 10). Parent reports of youth externalizing correlated strongest with Disinhibition subscale scores, even compared to Overall Liability. For the parent reports of their own parenting, only Parental Monitoring correlated negatively with YRI Overall and Disinhibition scores (small effect sizes). Unlike the youth reports of Depression, the YRI scores did not correlate with parent reports of youths’ internalizing symptoms.

In the CogNIT sample, the YRI Overall Liability scores were associated with several measures of cognitive functioning with small to moderate effect sizes (albeit needing a larger sample size to attain traditional critical p-values) (Table 11). After controlling for age as a proxy measure of the considerable developmental maturation in both cognitive functioning and liability to SU/SUD that occurs from ages 9 to 17, the Overall Liability scores were still associated with performance on the Planning/Sequencing Tower and the 0-back fractal test with medium-sized associations (correcting for multiple analyses using the false discovery rate). Importantly, controlling for age presents a stringent criterion because age encompasses not only maturation in cognition but also many increasing age-related risk factors that are known to be strongly associated with SU/SUD (35).