This paper further assessed the psychometric properties of the Illusory Health Beliefs Scale. Item refinement/development used the PHBS as a template, with additional items aligning with established aspects of paranormality (e.g., superstition, religiosity, precognition, psi) alongside unsupported convictions related to health, satisfying the definition of paranormal health beliefs as illusory with the potential to influence health practices/behavior (Petrillo and Donizzetti, 2012). Previous research (Denovan et al., 2024) informed the creation of a separate health pseudoscience subscale. In addition to traditional methods, Rasch analysis provided critical psychometric information about item performance (e.g., fit/appropriateness, difficulty), sample targeting, and dimensionality (see Duncan et al., 2003 for a review).

The focus of this research extends previous work. Specifically, Donizzetti and Petrillo (2017) demonstrated that the PHBS possessed satisfactory psychometric properties with an Italian sample. However, Donizzetti and Petrillo (2017) failed to implement tests of item difficulty and items were culturally specific. Moreover, the authors evidenced a relationship with important outcomes (e.g., illness) and emphasized that paranormal/illusory health beliefs could be useful for further appreciating factors that can influence health. Aside from Denovan et al. (2024), academics have not published work on this topic since. This is concerning because the culture-specific orientation of the scale hinders effective application within other contexts/cultures for assessing links between illusory health beliefs and related constructs. Accordingly, validation using empirically supported techniques (e.g., Rasch analysis) alongside construct/measure scrutiny in other contexts (an English-speaking context in this study) is critical for furthering research on the topic. Thus, development of a useful assessment tool for use with English-speaking samples will subsequently advance the research literature on illusory health belief. This includes exploration of latent structure, item/measure performance, and convergent validity using theoretically related constructs/measures (i.e., the PHBS, magical thinking, paranormal, pseudoscientific, and pro-scientific belief, and positive schizotypy).

It is not feasible to specify precise hypotheses concerning relationships with external criteria due to the absence of supporting literature. Using findings from the paranormal belief literature (e.g., Aarnio and Lindeman, 2005; Dagnall et al., 2022a; Irwin, 1990), illusory health beliefs should positively associate with all constructs apart from pro-scientific belief (should be negative). The authors anticipated strongest associations between the PHBS and the Illusory Health Beliefs Scale because the latter is a PHBS variant.

The study sampled 850 UK-based respondents (360 males, 482 females, eight non-binary), mean age 41.29 (SD = 13.84), range 18–80. Demographic information comprised measures of educational level, self-defined ethnicity, religious affiliation, and degree of perceived religiosity, and spirituality (see Table 1). Recruitment occurred through Bilendi, an established supplier of quality data¹. Participants were each allocated credits equalling £1.42 upon survey completion. The researchers instructed Bilendi to supply a representative sample of UK respondents aged 18 years and over from their participant panel. Panels provide data comparable to traditional approaches in quality (Kees et al., 2017). Data screening indicated satisfactory skewness and kurtosis between −2.0 to +2.0 among all study variables.

This study used a range of psychometrically validated self-report measures.

Before taking part, participants read the information sheet detailing the study background alongside study procedures. All participants provided informed consent before completing the study. This included ticking/clicking a box within the online survey confirming that they understood the study’s purpose and agreed to participation. Instruction informed that they could withdraw from the study at any point during completion. Additional instructions emphasized to participants to answer truthfully, take their time, and complete all questions. A forced response option and an inbuilt randomizer minimized incomplete responses and order effects of questionnaires in the online survey. Prior to completing study measures, participants provided demographic information. The Manchester Metropolitan University Ethics Committee (EthOS ID #52313) provided ethical approval.

Validation of the IHBS progressed through iterative stages. Firstly, exploratory factor analysis (EFA) with Principal Axis Factoring and oblique rotation, explored underlying structure using three criteria: Velicer’s minimum average partial (MAP) test, scree plot, and an eigenvalue ≥1. Velicer’s MAP test determines the number of underlying factors by computing partial correlations among residuals until no further common variance remains (O’Connor, 2000). EFA focused initially on IHBS items, and secondly on the ‘Health Pseudoscience’ subscale. This was necessary because although pseudoscience endorsement is related to paranormal belief, the constructs differ. For instance, they are differentially related to ontological confusion (i.e., conflation of mental and physical phenomena) (Lobato et al., 2014). While ontological confusion predicts paranormal belief, it does not predict pseudoscience endorsement. Conceptual differences explain why there is only a medium association (r = 0.36) between paranormal and pseudoscientific belief.

Recognizing differences, the authors presented Health Pseudoscience alongside IHBS items. This enabled parallel assessment of constructs without potential theoretical obtrusion. Moreover, concurrent assessment was advisable from a psychometric perspective since EFA performed on Health Pseudoscience and IHBS items would potentially confound factor identification/coherence. The fact that EFA is exploratory and atheoretical accentuates this concern (Tabachnick and Fidell, 2001). Hence, though related and commonly bracketed as epistemically unwarranted beliefs (Lobato et al., 2014), the constructs required separating for EFA.

Rasch analysis (Winsteps) subsequently examined rating scale efficacy, reliability, dimensionality, item fit and difficulty, and differential item functioning (DIF) (Denovan et al., 2022). Rasch evaluation (Rasch, 1960) was necessary because critics regard classical test theory (CTT) as limited. Explicitly, they disagree with the assumptions that test scores, in the absence of error, are accurate, and measurement error is random. Modern test theory instead contends that error occurs systematically as a function of individual ability and item difficulty. Acknowledging this, Rasch modeling calculates expected item responses. At the polytomous level (Rasch rating scale model) it applies to data with two or more ordinal categories (e.g., Likert type), as in this study. The polytomous model is advantageous because it offers estimates of person locations, item difficulties, and thresholds (fixed across items). This information designates item efficacy (i.e., discriminatory power).

Effectiveness of the rating scale was determined via monotonic increases in response category usage alongside Infit and Outfit (required to be between 0.5 and 2.0; Wright and Linacre, 1994). Item separation/reliability and person separation/reliability indicated reliability. A threshold of 0.7 exists for reliability (Fisher, 1992), and separation indicates the degree of participant or item spread on the ability continuum in addition to the extent of distinct levels of item/person ability (Bond and Fox, 2015). Values >1.5 suggest that items/samples separate into at least two levels (e.g., low, and high ‘complexity/ability’).

Principal Components Analysis of the residuals (PCAR) examined unidimensionality (a key assumption of Rasch analysis) via the following criteria: ≥40% of variance accounted for by the Rasch dimension; ≤15% of variance accounted for by the first contrast in the residuals; and a first contrast eigenvalue <3 (Areepattamannil and Khine, 2018). Additional dimensions exist if these criteria are not satisfied. Infit and Outfit Mean square error (MNSQ) determined whether items ‘misfit’ in relation to the measure. A misfitting item indicates that this is tapping into something distinct from the remainder items on the scale.

Examination of item ‘difficulty’ vs. person ‘ability’ used person-item maps. Efficacious measures should be able to differentiate participants along the ability continuum, with ideal targeting represented by the item mean corresponding with the person mean. DIF assessed equivalency of items across subgroups (gender and religious affiliation). A DIF contrast >0.64 alongside a significant Mantel–Haenszel p-value indicates that subgroups vary in their interpretation of items (Linacre, 2015). Finally, internal reliability testing occurred prior to examining convergent validity associations (using Pearson’s r) with theoretically related measures.

The MAP test recommended extraction of four (Revised MAP Test) and six (Original MAP Test) factors, whereas EFA suggested five factors. Explicitly, the eigenvalue criterion and scree plot (Supplementary material 1). Comparison of competing solutions indicated the five-factor solution yielded the most homogeneous and interpretable factors with minimal cross-loadings (i.e., only item 30, ‘Dreams about the future can suggest ways to avoid illness’, loaded above 0.4 on multiple factors). In comparison, the four-factor and six-factor solutions revealed cross-loadings on three different items. Moreover, the four-factor model excluded a factor with a meaningful eigenvalue, and the six-factor model produced a factor with an unsatisfactory eigenvalue. Accordingly, the researchers selected the five-factor solution, which persisted after removal of item 30. The model accounted for 63.20% of variance and demonstrated satisfactory sampling adequacy, Kaiser–Meyer–Olkin (KMO) of 0.97, and a suitable item correlation matrix (Bartlett’s Test of Sphericity p < 0.001). One item loaded below 0.4 (0.39), retained due to its proximity to the threshold.

Labels derived from conceptual interpretation of factor content. Factor 1, ‘Religious/Spiritual’ (9 items), captured holy/spiritual beliefs about health (eigenvalue = 18.05, 44.19% variance). Factor 2, ‘Superstition’ (12 items), comprised health-related items linked to prediction and ritual (eigenvalue = 2.31, 4.54% variance). The third factor, ‘Precognitive’ (6 items), contained items referencing the ability to influence/affect health via psychic forces (eigenvalue = 2.11, 4.14% variance), and Factor 4, ‘Health Myths’, (5 items), consisted of well-being falsehoods (eigenvalue = 1.49, 2.69% variance). Lastly, Factor 5, ‘Skepticism’, (5 items), included negatively worded items that reflected disbelief in illusory health beliefs (eigenvalue of 1.32, 2.0% variance) (Table 2).

The MAP Test designated a one-factor solution for the pseudoscience subscale. Satisfactory KMO (0.90) and item correlations (Bartlett’s Sphericity p < 0.001) existed. Factor loadings were above 0.4 (i.e., 0.51 and greater), and the solution explained 47% of data variance. The authors subsequently labeled the factor ‘Health Pseudoscience.’

Factors partially aligned with the original PHBS structure, specifically Religious/Spiritual, Superstition, and Precognitive. Additional factors referenced misconceptions/myths about health and doubt regarding the veracity of anomalous practices. This latter dimension of Skepticism was unanticipated. However, this provides a useful counterpoint to antiscientific convictions regarding health. The Health Pseudoscience subscale items also coalesced as one subscale, as expected.

Factors from the IHBS (excluding Skepticism) and Health Pseudoscience were highly positively correlated. Religious/Spiritual associations: Superstition, 0.72; Precognitive, 0.82; Health Myths, 0.61; Health Pseudoscience, 0.50. Superstition associations: Precognitive, 0.78; Health Myths, 0.63; Health Pseudoscience, 0.43. Precognitive associations: Health Myths, 0.65, and Health Pseudoscience, 0.63. Health Myths: Health Pseudoscience, 0.59.

Skepticism demonstrated an inconsistent pattern of weak correlations. Specifically, negative associations with Religious/Spiritual, Superstition, and Precognitive, and positive associations with Health Myths and Health Pseudoscience.

Existence of large correlations among IHBS factors and Health Pseudoscience potentially point to the presence of a global factor underpinning the questionnaire. Prior to computing Rasch analysis separately for IHBS factors and Health Pseudoscience, a PCAR investigated the existence of subdimensions for the total questionnaire. This provided independent confirmation of the dimensionality indicated by EFA (Franchignoni et al., 2013). An eigenvalue of the first contrast >3.0 infers that there is another dimension in the measurement instrument (Linacre, 2012). Findings revealed a first contrast eigenvalue of 8.0, supporting multidimensionality.

Applying Rasch analysis to IHBS factors and Health Pseudoscience revealed that the rating scale functioned appropriately, evidenced by monotonic increases in average measures from response category 1 (Strongly Disagree) to 5 (Strongly Agree), alongside Infit and Outfit MNSQ results between 0.5 and 2.0 (Supplementary material 2).

Measurement reliability was good across IHBS factors and Health Pseudoscience (Religious/Spiritual = 0.97, item separation = 6.12; Superstition = 0.98, item separation = 6.36; Precognitive = 0.98, item separation = 7.36; Health Myths = 0.99, item separation = 8.58; Skepticism = 0.99, item separation = 8.56; Health Pseudoscience = 1.0, item separation = 14.14). Person reliability was also satisfactory for all scales apart from Skepticism (Religious/Spiritual = 0.81, Superstition = 0.74, Precognitive = 0.85, Health Myths = 0.74, Skepticism = 0.58, Health Pseudoscience = 0.84). Moreover, a person separation >1.5 existed for all measures aside from Skepticism (1.18), indicating separation of participants into more than one ability level (Linacre, 2012). The results for Skepticism potentially suggested the need for more items.

The PCAR supported unidimensionality for most scales (Religious/Spiritual %age explained = 60%, eigenvalue = 13.5, %age of variance explained by first contrast = 6.8%, eigenvalue = 1.5; Superstition %age explained = 54%, eigenvalue = 10.6, %age of variance explained by first contrast = 9.4%, eigenvalue = 1.8; Precognitive %age explained = 54.7%, eigenvalue = 14.5, %age of variance explained by first contrast = 7.3%, eigenvalue = 1.9; Health Myths %age explained = 43.9%, eigenvalue = 4.7, %age of variance explained by first contrast = 14%, eigenvalue = 1.5; Health Pseudoscience %age explained = 50.8%, eigenvalue = 10.3, %age of variance explained by first contrast = 10.2%, eigenvalue = 2.1). Skepticism, however, revealed unacceptable explained variance in the first contrast, but acceptable results in relation to the other criteria for unidimensionality (Skepticism %age explained = 40%, eigenvalue = 2.6, %age of variance explained by first contrast = 24.5%, eigenvalue = 1.6).

Items across scales demonstrated satisfactory Infit and Outfit MNSQ (between 0.5 and 2.0) alongside positive and strong Point Measure Correlations (>0.40), inferring a lack of unsuitable or misfitting items (Table 2). Figure 1 shows person ability and item difficulty. For Religious/Spiritual, Superstition, and Precognitive, mean endorsement was lower than average item difficulty, with a noticeable cluster of participants exhibiting low Rasch scaled scores. This indicated that respondents were unlikely to endorse these scales. Health Myths, Skepticism, and Health Pseudoscience evidenced a more even spread of participants relative to items. However, there still existed only a few participants with high scores. Item difficulty calibration indicated no discernible differences (i.e., denoted by a similar spread of items around the mean).

DIF analyses relative to gender (male vs. female) and religion (religious affiliation vs. no religious affiliation) displayed no meaningful concerns. Specifically, although significant Mantel–Haenszel values existed, DIF contrasts <0.64 logits (Table 2) inferred subgroups attributed similar meaning to the items.

Following the guidelines of Taber (2018), alpha and omega reliability for Religious/Spiritual (α = 0.95, ω = 0.95), Superstition (α = 0.94, ω = 0.94), and Precognitive (α = 0.94, ω = 0.94) were excellent. Health Myths (α = 0.78, ω = 0.78) and Health Pseudoscience (α = 0.87, ω = 0.87) demonstrated good reliability. Skepticism evidenced moderate reliability (α = 0.63, ω = 0.62). Convergent validity analysis revealed significant moderate to large associations concerning IHBS factors and Health Pseudoscience with all variables aside from Belief in Science. Indeed, this demonstrated a similar pattern of associations with IHBS factors as Skepticism (Table 3).

As predicted, IHBS paranormal-based factors (Religious/Spiritual, Superstition, Precognitive) correlated the strongest with PHBS factors. Health Pseudoscience correlated the strongest with Pseudo-R, which was appropriate. Some inconsistencies occurred, however, such as Pseudo-R demonstrating the strongest association with Precognitive. This is potentially due to an emphasis on ontological confusion with some of the Pseudo-R items (e.g., ‘The collective memory inherited and shared by the organisms belonging to the same species (‘morphic field’ or also ‘morphic resonance’) explains several biological phenomena’), in addition to an item directly referencing precognition [i.e., ‘It has been scientifically proven that some people have extrasensory abilities (such as telepathy or precognition)’].

Despite establishing validity and internal reliability further investigations should test the stability of the measures in independent samples and across time. Furthermore, the present study was cross-sectional meaning that the investigators collected data at one point in time. While the researchers employed procedural remedies to prevent common method variance (i.e., randomized scale presentation order and instructed participants that scales assessed distinct constructs, see Drinkwater et al., 2020, 2024) it is still necessary to assess scale robustness and verify external reliability (i.e., temporal stability). Consequently, ensuing psychometric evaluation should appraise test–retest reliability. This will ensure that scores are consistent and replicated across time. This is important since reliability across multiple trials, settings, and respondents is often absent or poorly reported (Dagnall et al., 2023). Moreover, now convergent validity has been established relative to mental health/paranormal-based scales, in additon to verifying latent structure (e.g., via confirmatory factor analysis), a further necessary step is to examine IHBS associations with health-based measures.

The present IHBS and Health Pseudoscience subscale combined are lengthy, this potentially restricts their inclusion in large test batteries. Acknowledging this, ensuing studies should create short abridged versions. This process will reduce cognitive load and increase accessibility. However, scale refinement is an iterative process that requires assessment of item performance across a range of samples. This is also necessary to ensure that the evolving brief measures adequately assess construct breadth.