Descriptive information for the sample is presented in Table 2. The sample included 275 participants aged 40 years and older with low health literacy and at least one chronic health condition for which they were being treated. The most common chronic conditions they reported were elevated cholesterol levels, high blood pressure, and type II diabetes. The sample included 147 women (53.5%) 37 Whites, and 238 Black, Indigenous, and other People of Color individuals. Most had few socioeconomic resources, with more than half reporting annual incomes of less than $15,000 per year. Participants' digital literacy was assessed with the eHealth Literacy Scale (39). Interpretation of our findings on this measure is complicated by the lack of formal norms for it, but one review reported finding a weighted mean score drawn from 20 publications of 24.3 (95%CI 17.1–31.6) (40), suggesting that our participants were similar to those in other studies.

Individual item statistics are presented in Table 3. Item-level descriptive statistics (Table 2) indicated uniformly high mean scores, ranging from 4.19 to 4.32 on a 5-point scale. All items showed moderate to strong negative skew (−1.23 to −1.62) and positive kurtosis (3.09–4.90), suggesting that participants tended to endorse the upper end of the response scale and that responses were tightly clustered. This pattern reflects generally favorable perceptions of information tailoring and limited variability among respondents. This pattern is consistent with the scale's intent and the intervention's design but may constrain the ability to detect associations with other variables due to reduced variability.

Inter-item correlations are presented in Table 4. Correlations among the eight SIT items were all positive, statistically significant (all ps < 0.01), and of moderate to strong magnitude (Table 3). Correlations ranged from r = 0.51 to 0.83, with a median of approximately.70, indicating substantial coherence among items. The strongest associations were observed between items reflecting the applicability and thought-provoking qualities of the information (“I could use the information” and “Gave me something to think about,” r = 0.83). In contrast, the weakest associations involved relational or engagement-oriented items (e.g., “I could relate to the information” and “I want to try out some of the things,” r = 0.51). This pattern suggests that while the items assess different aspects of perceived tailoring, they share a core consistent with a single underlying construct.

Exploratory analyses with parallel analysis suggested a single-factor solution (Figure 1). Most SIT items loaded well on the first factor. Table 5 lists the factor loadings for the one-factor solution. Overall, the pattern of loadings is consistent with a unidimensional structure.

To further assess the factor structure of the SIT scale, we fit a single-factor confirmatory factor analytic model. It fit the data moderately well (CFI = 0.92, TLI = 0.89, RMSEA = 0.17, SRMR = 0.05) but not at an acceptable level. Considering this evidence of misfit, we explored several alternate factor models; the one that fit the data best was a bifactor model with one general and two minor factors. The first minor factor included the items “The information would work for me” and “I want to try out some of the things talked about,” while the second included “The information was easy to understand” and “I could relate to the information.” Fit indices were substantially improved (CFI = 0.96, TLI = 0.93, RMSEA = 0.14, SRMR = 0.04). The bifactor model fit the data significantly better than the single-factor model [likelihood ratio test χ² (4) = 42.22, p < 0.001].

The bifactor model thus provided the best overall fit among competing structures. Model-level indices indicated that variance in SIT responses was dominated by a strong general factor representing perceived tailoring. The general factor accounted for 88% of the common variance (ECV = 0.88), with a high proportion of uncontaminated correlations (PUC = 0.93). Reliability coefficients were excellent (Ω_T = 0.96; Ω_H = 0.93), indicating that most reliable variance in total SIT scores reflected the general factor. Construct replicability (H = 0.96) and factor determinacy (FD = 0.98) were also high, supporting stable estimation of the general factor. In contrast, the specific factors showed low unique reliability (Ω_HS = 0.18–0.22) and modest replicability (H = 0.28–0.38), suggesting limited distinct variance beyond the general factor. Together, these results indicate that the SIT functions mainly as a unidimensional measure of perceived tailoring, and that a single total score provides a psychometrically useful representation of the construct.

Configural, metric, and scalar models were compared for the bifactor model across administrations (Table 6). Fit indices changed only slightly moving from configural to metric invariance and stayed within the cutoffs. The CFI and RMSEA changed minimally from configural to metric models [Δ CFI = −.01, Δ RMSEA = 0.00; Δχ² (8) = 8.31, p = 0.40], indicating equivalent factor loadings across assessments. These results supported metric invariance, suggesting that the SIT items operated in a consistent way over time with similar factor loadings. However, fit indices did not support scalar invariance. Constraining item intercepts (scalar model) reduced fit substantially (Δ CFI = −.05, Δ RMSEA = + .02), consistent with the significant χ² difference [Δχ² (6) = 131.16, p < 0.001]. Scalar invariance was thus not supported.

Test–retest reliability across administrations was moderate. The intraclass correlation coefficient (ICC) was calculated as a single measure with a two-way random-effects in both absolute-agreement and consistency models. The ICC for absolute agreement [ICC(A,1)] was 0.60 (95% CI 0.51–0.69), and the consistency ICC [ICC(C,1)] showed a similar pattern (ICC = 0.61, 95% CI 0.52–0.70). The standard error of measurement (SEM) was 3.63, giving a smallest detectable change (SDC) of 10.07. Cronbach's alpha for the scale at time 1 was 0.95; at time 2, it was 0.93. McDonald's omega was 0.95. at time 1 and 0.93 at time 2.

A Bland–Altman plot was used to examine agreement between the two SIT administrations Figure 2. The mean difference between scores was close to zero. However, regressing score differences on their means revealed a statistically significant negative slope (β = −0.26, p = .001), suggesting a proportional bias: participants with lower average SIT scores tended to show slightly higher second-administration scores, whereas those with higher average scores showed smaller or slightly negative differences. Despite this trend, most differences fell within the 95% limits of agreement, indicating good overall test–retest consistency. A spaghetti plot of individual SIT scores (Figure 3) shows variability across participants in the two administrations, even though the group means remained stable (red line in Figure 3).

As expected, SIT scores correlated positively with measures assessing related constructs (Figure 4). Perceived tailoring was most strongly associated with participants' ratings of the information's usefulness (User Useful; r = .33, p < .001) and with patient activation (PAM; r = .27, p < .001). Smaller but still significant correlations were observed with health literacy (TOFHLA; r = .25, p < .001; FVA; r = .20, p = .001), internal health locus of control (r = .17, p = .006), and chronic disease self-efficacy (r = .16, p = .010). In contrast, SIT scores were not significantly related to demographic and health variables—including income, age, number of chronic conditions, and systolic blood pressure—and to psychological measures of perceived stress (PSS; r = .08, ns) and depressive symptoms (CESD; r = 0.002, ns). This pattern demonstrates good convergent validity with conceptually related psychosocial constructs and divergent validity with unrelated health and demographic factors.

The latent growth models incorporating patient activation (PAM) and chronic disease self-efficacy (CDSE) showed that SIT scores obtained immediately after the intervention predicted change in outcomes over time. Figure 5 shows the growth curve for patient activation. Patient activation changed significantly over the course of the study (coefficient = 0.17, SE = 0.08, z = 2.08, p = 0.04). The regression of the model's slope on the SIT score was also significant (coefficient = 0.05, SE = 0.02, z = 3.32, p = 0.001) as was the regression of the QOL measure (SF36 General Health scale at three-month follow-up (coefficient = 14.73, SE = 5.51, z = 2.67, p = 0.007). The indirect effect of the SIT on QOL via change in activation was also statistically significant (estimate = 0.71, SE = 0.23, z = 3.16, p = 0.002).

Figure 6 shows the growth curve for chronic disease self-efficacy. It increased significantly over the course of the study (coefficient = 1.26, SE = 0.27, z = 4.64, p < 0.001). The regression of the model's slope on the SIT score was also significant (coefficient = 0.13, SE = 0.06, z = 2.26, p = 0.02), but the regression of the QOL measure (SF36 General Health scale at three-month follow-up; (coefficient = −0.11, SE = 0.08, z = −1.42, p = 0.15) was not. The indirect effect of the SIT on QOL was not statistically significant (estimate = 0.05, SE = 0.04, z = 1.26, p = 0.21).

The purpose of this paper was to provide more detailed information on the SIT's reliability and validity to help support its potential usefulness. We conclude that the SIT does what it was designed to do: it provides a method to measure users' impressions of tailored health information in apps. Factor analyses were consistent with a single common factor, reliability is adequate, and the validity analyses are consistent with our hypotheses. SIT scores obtained immediately after the intervention were related to the pattern of change in the key outcomes of patient activation and chronic disease self-efficacy.

Although the bifactor model yielded the best statistical fit, the pattern of indices clearly supports the practical use of a single SIT composite score. The strong general factor accounted for a large portion of the common variance, while the specific factors contributed little unique and reliable information. This suggests that participants' responses primarily reflect an overarching perception that the information they received was well tailored to their needs, rather than distinct dimensions of tailoring. Accordingly, the SIT scale can be interpreted and scored as a unidimensional measure of perceived tailoring, simplifying its use in both research and applied settings while retaining conceptual coherence with the elaboration likelihood model's emphasis on personal relevance as the key mechanism of tailored communication.

Tailored health communications are more effective because they are perceived as more personally relevant (2). However, for a message to be perceived as relevant, it must first be cognitively accessible which is a concept known as “cognitive fit” (41). Within the Elaboration Likelihood Model (ELM), comprehension is a necessary prerequisite for message elaboration. Therefore, the SIT scale includes items assessing “ease of understanding” not as a measure of the user's skill, but as a measure of the intervention's success in adjusting content to the user's specific cognitive needs. This has been shown to increase message relevance and decrease cognitive load (42). While some items in the SIT explicitly address ease of understanding, our bifactor analysis confirms that these items load onto a robust general factor representing perceived tailoring. This suggests that for users, especially those with low health literacy, the ability to understand the information is inextricably linked to the feeling that the information was “successfully” personalized for them. By reducing the cognitive load required to process the information, the digital tailoring allows the user to focus on the personal utility of the advice.

Overall, the SIT Scale was framed using tailored health communication literature, where personalized message design is viewed as a multidimensional concept that uses person-level assessment to modify content, framing, and feedback so the information fits more appropriately to the person's needs and context (e.g., personalization). In this framework, tailoring appears to work through complementary pathways including strengthening cognitive preconditions for learning (e.g., attention, comprehension, perceived personal relevance) and the ability to impact behavioral determinants (e.g., confidence, skills, intentions) that drive self-management behavior (43, 44). Additionally, consistent with other studies tailoring can be produced through a segmentation, adaptation, and matching process (45). In this scenario, users are segmented based on variables (e.g., age, education, language, cultural belief, disease severity), and, as a result, the delivery of information (e.g., verbal, written, digital) is adapted. Finally, the matching process can be considered the crucial step that encompass how content and delivery are aligned with the user variables. However, data identify key limitations including segmentation rarely using technology related variables, user-interaction data to drive reminders, and a focus on only one type of segmentation variable (46). The SIT scale may not be able to determine which specific segmentation variables are predominantly impacting the user experience. However, the SIT scale may be positioned as a tool that can address whether the personalization is working (and why) and provide an objective score of the tailored content to determine if further improvements are needed.

As a result, the SIT scale was conceptually designed to capture these proximal, receiver-centered effects of tailoring. This includes identifying whether the information is perceived as relevant, useful, and actionable which are theorized mechanisms by which modifiable and tailored content produces changes in self-management outcomes (43, 47). This approach is consistent with data suggesting that tailored (including computer-tailored) delivery of information outperforms non-tailored comparisons. Therefore, the data appear to supports the importance of measuring perceived “fit” and actionability as important features of personalizing and tailoring content to the individual (1, 47). Finally, our finding that SIT scores are associated with related changes in activation and disease self-efficacy appears to align with health literacy frameworks and activation theory that emphasizes confidence, abilities, skills, and knowledge needed to interpret and act on personalized health information (12, 48, 49).

Given the theoretical framework, in order to better frame the SIT scale as a digital measure, it must be noted that eHealth personalization involves adapting not only the content but also the technology driven cues (e.g., interface, delivery, feedback) that impact understanding (46). For example, in the Klooster framework, users can be segmented into demographic/health characteristics and also on digital technology and interaction related variables (e.g., eHealth literacy, usage patterns) which can then be matched to design and delivery adaptations (45, 46). Accordingly, the SIT scale's “ease of understanding” items may be presented as an indicator of successful digital matching (content -interface- delivery cues) instead of a solely text-based construct. As a result, the endpoint is to determine if the adaptations achieved a perceived “fit” for the user in real-world app interactions. Also, consistent with Klooster et al. observations that many personalization efforts underuse technology related variables and user interaction data (e.g., reminders) (45, 46), the SIT scale may be a pragmatic outcome measure. Even though the SIT scale may not isolate which specific digital cues drive user perceptions, it can evaluate whether the implemented digital adaptations are experienced as relevant, useful, and actionable.

Test–retest reliability, measured by ICCs, was moderate. The Bland–Altman plot showed a small but significant average bias but individual variability. In practical terms, the group average was stable, although changes in a single person's score must be interpreted with care. SEM and SDC estimates provide useful guidance on interpreting changes in SIT scores.

The correlations of measured to represent similar and unrelated constructs were consistent with hypotheses. The SIT correlated significantly with participants' ratings of the app's usefulness and with the PAM and CDSE. These findings are consistent with the hypothesized relation as the underlying commonality is that both are about how people see their ability to act on health information. Correlations with unrelated measures were close to zero, supporting divergent validity.

The latent growth curve models showed that people with higher baseline SIT scores made greater changes in activation and disease management self-efficacy over the course of the study. The SIT did not predict initial levels (intercepts); it only predicted the slope of improvement. These findings provide preliminary evidence suggesting that feeling the information was relevant and usable was associated with greater change later.

There are a few caveats. The sample included persons 40 years and older with chronic illness and low health literacy, so results may not apply to other groups. Follow-up was only three months. CFA fit indices were not ideal, and ICCs were moderate, so the scale is better suited for group-level research than individual monitoring. Missing data also may have influenced some results.

We observed a substantial ceiling effect, with many participants responding at high levels. While the bifactor model confirmed that a strong general factor dominates the scale—supporting its use as an essentially unidimensional measure—the results must be interpreted within the context of this observed ceiling effect and restricted variability. These distributional characteristics likely contributed to the high RMSEA values and the absence of scalar invariance, suggesting that while the scale effectively captures group-level perceptions, it is not yet optimized for monitoring subtle individual changes over time. Furthermore, because the SIT scale successfully predicted the slope of change in patient activation and self-efficacy rather than initial status, it is best positioned as a process measure to verify the mechanism by which personalization influences behavioral outcomes.

Future refinements should focus on expanding the response range to mitigate the ceiling effect and validating the scale in diverse, higher-literacy populations to ensure its broader generalizability. The negatively skewed distributions of SIT items indicate that participants overwhelmingly perceived the intervention content as well tailored to their needs. While such skewness and kurtosis are expected in user-centered interventions, they may also signal ceiling effects that constrain variability and attenuate correlations with other constructs. Future refinements to the SIT scale might explore expanding the response range or incorporating items that better differentiate among individuals with uniformly positive perceptions. Nevertheless, the pattern of high endorsement supports the scale's face validity and aligns with the intervention's design to deliver highly personalized information. Although the bifactor analysis indicated strong general-factor dominance, some multidimensionality may still exist, and future research should examine whether distinct facets of perceived tailoring emerge in different populations or intervention contexts.

Based on digital personalization frameworks, the SIT scale may be viewed as a receiver-centered outcome measure of whether digital tailoring and adaptation (e.g., content, interface cues, delivery/feedback features) achieves a meaningful perceived “fit” for the user. The SIT scale provides an interpretable score to identify if the implemented digital adaptations are perceived as relevant, useful, and actionable. As a result, while it lacks the ability to isolate specific digital cues that drive fit, it offers a practical metric to guide iterative optimization of tailored digital health interventions in older adults with low health literacy. The SIT scale is thus primarily intended to serve as a process measure and a mediating variable rather than a primary clinical outcome. While it captures an “outcome” in terms of whether the tailoring process was successful in the user's mind, its greatest value lies in its ability to explain the mechanisms of behavior change within digital health interventions.

Even with its limitations, the SIT scale may provide researchers and program designers with a simple way to determine whether efforts to tailor health information to users are successful. It provides a means of assessing mechanisms of behavior change in digital health interventions. Future studies should examine how the scale performs in other populations, assess longer-term predictive validity, and explore the smaller factors suggested by the CFA results.