We used data from the 2000 Psychiatric Morbidity Survey (PMS) of Great Britain. The main purpose of the survey was to estimate the prevalence of psychiatric disorders and their correlates using a stratified random multistage design. Participants were 8,580 adults, aged 16–74 years, living in private households in Great Britain. Of these, the 8,338 people (97%) who had complete records of symptoms of interest were the population from which our calibration and verification samples were drawn. Full details of the PMS methods are available in the main survey report (32).

Although Rasch analysis is increasingly used in other medical specialties, it is still largely underutilized in psychiatry (38, 39). As mentioned in the Section “Introduction,” Rasch analysis determines whether mental or psychological scales have the characteristics of physical measures. For this to be the case, five conditions must be met (Figure 1). First, the instrument is designed to measure a single attribute (unidimensionality). Just as a ruler measures length only, depression scales must measure depression alone. Second, the responses follow a Guttman pattern: persons are ranked from lowest to highest levels of the trait, while items are ranked by highest to lowest levels of endorsement. The appearance of a Guttman pattern is like a staircase. The ranking of persons and items in this manner produces units that are interval-level measures called logits (“log odds unit”). Third, endorsing a particular question should be independent of the endorsement of another question except with respect to the attribute being measured (local independence). This requirement guards against spurious correlations – those that are due to external factors, such as wording or position in the scale. Fourth, the items and the overall instrument must not have differential item or test function. This means that at the question and instrument levels, there must be invariance with respect to person characteristics like age and gender. Fifth, the overall scale must be internally consistent (as measured by Cronbach’s alpha) and able to distinguish different strata of respondents along the latent trait (40, 41).

We performed Rasch analysis in two samples of 500 subjects, one serving as calibration and the other as validation sample. The requirements, tests performed, and the criteria in each test are summarized in Table 1. For the complete description of analysis steps, please refer to the Appendix in Data Sheet S1 in Supplementary Material.

Parallel analysis of the 11 candidate items in the calibration sample showed two dimensions (adjusted eigenvalues: 2.82 and 1.02). In the validation sample, a similar result was reached with eigenvalues: 2.84 and 1.04. To determine whether MI and/or irritability were responsible for violating unidimensionality, we omitted them both and performed parallel analysis using only the typical nine depression symptoms. Once again, two dimensions were detected. The eigenvalues were 2.60 and 1.02 for the calibration sample and 2.57 and 1.04 for the validation sample. After misfitting items (i.e., weight/appetite change, agitation/retardation, fatigue, self-blame) were dropped, we tested dimensionality with the retained items (i.e., sad, lack of interest, sleep problems, cognition problems, and suicidal ideas) plus MI and irritability. Only one adjusted eigenvalue was greater than one in both the calibration and validation samples (1.87 and 1.81, respectively). We took these results as evidence that MI and irritability are part of the core indicators of depression.

Of the 11 symptoms we examined, the most common symptom was irritability, while MI was the least common. Please refer to Table 3 for the complete list of item locations in logits. Figure 2 is a visual representation of the item locations and the corresponding fraction of the population that they demarcate.

The symptoms that misfit the Rasch model were weight/appetite change, agitation/retardation, fatigue, and self-blame. With consistent findings in both samples, these symptoms were eliminated. See Table 4 for the initial and final symptom lists and fit statistics.

In the calibration sample, large residual correlations were observed between the item pairs fatigue and cognition, χ² = 14.36, df = 1, p with Holm’s adjustment = 0.004. The validation sample showed a similar large residual correlation for these two items, χ² = 11.26, df = 1, p with Holm’s adjustment = 0.02. After fatigue was eliminated from the symptom list, no large residuals remained.

With the assessment of differential response by gender, no items were flagged for DIF in the calibration sample. In the validation sample, irritability showed both uniform and non-uniform effects. Female respondents more frequently endorsed irritability, and the disparity with male respondents also differed by depression level. The impact of DIF by gender with respect to irritability had a medium sized effect on the test for the validation sample. See Table 5 for details.

With the DIF assessment by age group, cognition showed both uniform DIF. In the calibration sample, respondents above 45 years of age endorsed cognitive problems more frequently. In the validation sample, respondents above age 45 endorsed irritability more frequently. Both cognition and irritability had large test effects in both samples. See Table 6 for details.

The PSI for the initial 11 symptoms was 0.60 for the calibration sample and 0.58 for the validation sample. For the seven retained symptoms (i.e., sad, no interest, sleep, cognition, suicidal ideas, MI, irritability) the PSIs were 0.38 and 0.35 for the calibration and validation samples, respectively. Cronbach alphas for the 11 symptoms (nine original, plus MI and irritability) were 0.71 and 0.70 for the calibration and validation samples, respectively, and 0.58 for both samples for the seven retained symptoms.

In post hoc analysis, we examined the PSI and alpha for the standard nine DSM/ICD symptoms. PSIs were 0.52 and 0.51, and Cronbach alphas were 0.70 and 0.67 for calibration and validation, respectively.

We found that the nine depression symptoms in the DSM/ICD systems are not unidimensional. In practice, this means that the “5 of 9” rule (or “4 of 10” for ICD) is probably not warranted because the symptoms do not all tap the same attribute. A more homogeneous set of indicators is achieved by removing weight and appetite change, agitation and retardation, and feelings of worthlessness and inappropriate guilt and fatigue from the core of the major depression syndrome. The findings that MI and irritability fit the Rasch model indicates that they belong to the core network/cluster of symptoms that includes sadness and anhedonia (49).

While irritability appears to be endorsed differentially by sex and by age group, MI is invariant with respect to both characteristics. That irritability is identified as a DIF item should not automatically exclude it from the list. One possibility is to adjust for the DIF effect of gender and/or age in assessing the severity of depression (50). This would be difficult to implement in a paper and pencil test, but could be solved by computer adaptive testing that takes covariates into account.

Seven items – sad, lack of interest, sleep, cognition, suicidal ideas, MI, and irritability – could serve as the kernel for a depression measure, but on their own do not reliably separate depressed persons from the rest. Likewise, the list of DSM/ICD symptoms fails the typical criterion for internal consistency (alpha < 0.80) and also falls short of distinguishing the depressed from the rest (PSI < 0.70). Clinical judgment and contextual information may need to be taken into account apart from the canonical list of symptoms. For use outside of the clinic, scales such as the PHQ-9, Beck Depression Inventory, HADS, CES-D, and the like are only recently being examined using item response theory (51–53).

Mood instability is common in depression (23, 25) and has been shown to be a precursor of depression (24). Our current results provide evidence that MI is a symptom of depression. According to DSM, relatively short durations of MI phases would not meet episode criteria for major depression (2 weeks) or for hypomania (4 days) (4). If the patient reports rapid mood fluctuations, it is typical to either dismiss MI as clinically unimportant or to concurrently diagnose a personality disorder, particularly borderline personality disorder where both MI and irritability are DSM-V criteria (4). Unfortunately, people who do not fulfill duration criteria may also be considered “well” or at least not depressed and receive no treatment (54), but these people are at higher risk of developing future depression (55). The evidence indicates that intense and frequent mood swings are associated with severe distress (56, 57), and there is merely a quantitative difference between the mood fluctuations of normal individuals and those of patients (58). MI is linked to other indicators of distress and impairment such as health care utilization, medication use, and suicidal thoughts (25, 59) and has recently been proposed to fit the characteristics of the Research Domain Criteria (60).

Irritability is associated with emotional lability in patients with unipolar depression (30) and in university students (30, 61) and is common in depression (29, 30). Mixed depression, which may be defined as “an overlapping of manic and depressive symptoms” includes irritability and emotional lability among its symptoms (62). This presentation is characterized by psychic and motor agitation, accompanied by intense suffering, which put the patient at increased risk of suicide (54). Irritability could be a core symptom of depression (29), an indicator of a more severe and chronic course (30, 63) or a feature of bipolar disorders (64). DSM-V has included irritability as a core symptom of mania, generalized anxiety disorder, and borderline personality disorder. but excluded it as a symptom of major depression. It has previously been rejected as a symptom of major depression as it does not appreciably increase the prevalence above that of sadness and loss of interest (29, 65). Increased prevalence is not necessarily a good basis for defining a syndrome. Conversely, irritability, along with MI, could lead to longstanding interpersonal and adjustment difficulties that could lead to depression (30, 49). In summary, both MI and irritability are observed in a range of psychiatric disorders.

It is uncertain whether agitation and retardation are specific distinguishing features of major depression, melancholia, mixed mood states, atypical depression, bipolar II depression, or anxiety comorbid with mood disorders (66, 67). Agitation is a defining characteristic of a proposed mixed depressive state that has both melancholic and excitatory features, but which does not have the levity in mood of hypomanic patients (62). Our finding is more consistent with a major depression study that reported that agitation could be dropped from the definition of major depression with no loss of validity (68).

The Feighner symptom of “self-reproach or guilt” was expanded in DSM-III to include “feelings of worthlessness …” as part of the DSM trend to broaden the criteria for major depression (69). There is a clear semantic difference between guilt (worry) about past misdeeds and anxiety (worry) about future threats and perhaps feeling helpless, but this distinction might not be meaningful for people with common mental disorders, high comorbidity, or high distress (70–73). Our results replicate findings of a previous Rasch analysis of the PHQ-9 scale that guilt was not coherent with the model of depression (51).

We eliminated fatigue because of higher than expected correlation with cognitive problems. This could be the result of similar wording: both symptoms are presented as diminished ability. An alternative to eliminating this item is rewording either fatigue or cognitive problems. Retaining this item is probably the more prudent course of action. Although one study reported that fatigue is not unidimensional with the other depression symptoms (74), several other studies reported that it satisfies the Rasch model (11, 52, 75).

It should be emphasized that the misfitting items are frequently experienced by patients with major depression. What is in question is whether they are central to the network of symptoms comprising the depressive syndrome and whether they are useful in its assessment. The search for underlying biological or psychological aberrations (76) or treatment (77) for depression is probably hampered by a heterogeneous cluster of symptoms.

Our study has several limitations. First, the data are based on retrospective recall with all of the disadvantages of this method (78). People who are depressed have a general negative recall bias that might affect reporting of symptoms (78). Second, the CIS-R was designed to elicit the ICD-10 criteria for depression, although the symptoms are very similar to those of DSM-V. Third, we did not consider duration criteria for the individual symptoms, and thus cannot be certain that all symptoms occurred at the same time. However, all duration criteria of major depression and similar groupings are arbitrary and, theoretically, symptoms may occur sequentially and still indicate the same syndrome (55, 79). Fourth, we studied MI and irritability, but other common symptoms such as anxiety, rumination and physical symptoms should be studied (73, 80). Fifth, we performed DIF analysis only by age and gender in a British sample, so further analysis is required to determine if MI and irritability are part of the syndrome across cultures. The differential item status of sleep, irritability, and MI should be addressed by future work. Sixth, while MI and irritability were shown to load on a single factor with depression, we did not have an independent external criterion to serve as a reference. Finally, the PMS did not assess bipolar disorder so it is possible that some of the respondents had bipolar, instead of unipolar depression. We do not think this limitation undermines our findings because MI is a feature of a wide range of psychiatric disorders (22, 81) and, second, the prevalence of people with bipolar depression in the sample in comparison to unipolar depression is likely to have been small. It would be beneficial in replication studies to correlate the scores of patients in our proposed 7-item scale to standard psychometric questionnaires, such as the Mood Disorders Questionnaire or the Affective Lability Scale.

A strength of our study is that it was based on empirical data obtained from an epidemiological sample of the population. Accordingly, it was not constrained by a pre-selected sample with major depression as diagnosed by the criteria being studied.