It is estimated that around 65.3 million individuals—the vast majority from non-Western, low-income countries—are currently displaced by violent conflict and humanitarian disasters, including 21.3 million refugees and 40.8 million internally displaced individuals (United Nations High Commissioner for Refugees, 2016). It has been well-established that due to exposure to multiple chronic stressors, this population has high levels of mental illness (Fazel et al., 2005). One of the more widely prevalent and widely studied mental illnesses in this population is depression (Steel et al., 2009), which is a significant contributor to the global burden of disease (Whiteford et al., 2015). Governments, as well as international and local agencies servicing survivors of conflict and disaster, often have limited resources for mental health services (Saxena et al., 2007) and thus, it is important that cross-culturally valid instruments measuring depression and other mental illnesses exist so that survivors who have clinically significant depression and other mental illness can be identified.

With regards to the assessment of depression, the Beck Depression Inventory (Beck and Steer, 1987) is one of most commonly used self-report instruments for assessing the severity of depression worldwide (Richter et al., 1998; Shafer, 2006). The ease with which the BDI can be administered makes it an attractive measure to efficiently and economically determine levels of depressive symptomology in populations of interest. Despite the development of the revised BDI-II 20 years ago (BDI-II; Beck et al., 1996), the original BDI is still widely used in clinical and research settings in both North America (Foa et al., 2005; Gillespie et al., 2009; Veerman et al., 2009; Delisle et al., 2012; Jayawickreme et al., 2014; Udo et al., 2015) and Europe (e.g., Suija et al., 2012; Lahlou-Laforet et al., 2015; Wardenaar et al., 2015). Strikingly, the BDI continues to be widely used to assess for depression in refugees and internally displaced and war-impacted populations (Mghir and Raskin, 1999; Bhui et al., 2003; Ghazinour et al., 2003; Yurtbay et al., 2003; Basishvili et al., 2012; Luitel et al., 2013; Kuittinen et al., 2014).

However, despite the wide use of the BDI and other measures developed for Western clinical populations in non-Western, post-disaster settings, there is a dearth of studies examining the cross-cultural validity of these measures (Van Ommeren, 2003; Bass et al., 2007). In a review of 183 research studies focusing on refugee populations, Hollifield et al. (2002) found that half of the studies did not provide any evidence of validity and reliability for the instruments used. Given that there is now considerable evidence for cross-cultural differences in both the experience and presentation of depression (e.g., Paniagua, 2000; Karasz, 2005; Ryder et al., 2008), it cannot be assumed that just because an instrument is valid and reliable in one context, it will be so in other contexts. Furthermore, one cannot assume that scores on a particular instrument are meaningful in the same way across cultural settings. This point is especially relevant to those providing services to displaced and war-affected populations, who often rely on instrument clinical cutoff scores to identify those who are mentally ill. For example, BDI cutoff scores have been used to determine depression severity in war-affected Lebanese individuals (Farhood and Dimassi, 2012), war-affected Iraqis (Magruder et al., 2015), and Somali refugees (Mölsä et al., 2014). Furthermore, researchers and policy makers use these cutoff scores to compare the relative rates of mental illness across different countries (e.g., Van Hemert et al., 2002).

Before comparisons of scores between different cultural groups can be made, one must first ensure that the instrument in question meets measurement equivalence (also referred to as measurement invariance in the literature); in other words, the instrument must measure the same construct in the same manner across different cultural contexts (Chen, 2008; Millsap, 2011). Measurement equivalence is established by meeting four increasing restrictive criteria: (1) functional equivalence; (2) structural equivalence; (3) metric equivalence; and (4) scalar (or full score) equivalence (Fischer and Fontaine, 2011; Millsap, 2011)¹. Functional equivalence is established when the same construct (e.g., depression) can be observed across different groups. Functional equivalence is typically established through a combination of ethnographic research, case studies, and epidemiological surveys (Lopez and Guarnaccia, 2000). There is now a large body of research indicating that depression is indeed seen globally (Kessler and Bromet, 2013) and there is some evidence (e.g., Jayawickreme N. et al., 2012; Haroz et al., 2016) that measures of depression developed in the Western world at least partially captures the experience of depression in low and middle-income countries in the global south.

Structural equivalence is established when the factor structure of the measure is the same across different groups. With regards to the BDI, Beck and Steer (1993) note that the measure can be divided into two factors, a subscale of cognitive and affective symptoms consisting of the first 13 items, and a subscale of somatic and performance-affecting symptoms consisting of the last eight items (see Table 1). This factor structure has received considerable factor analytic support (Steer et al., 1992; Beck and Steer, 1993). Another factor structure, comprising of three factors, Negative Attitudes toward the Self, Performance Impairment, and Somatic Disturbances, has also been identified in numerous samples (Beck et al., 1988; see Table 1). A meta-analysis of factor-analyses for the BDI (total number of individuals = 13,643, total number of studies = 33) found that this three-factor solution provided the best fit (Shafer, 2006). It should be noted, however, that these factor solutions have been tested predominantly in European-American, middle class samples in the United States (Shafer, 2006).

The heterogeneity of factor models for depression measures, including the BDI, has led some to argue that bifactor factor models better account for the nature of depression (Ward, 2006; Brouwer et al., 2013). Bifactor models have a single general factor that consists of all items in the measure, and subfactors consisting of a subset of items that reflect a narrower construct. As such, these models identify a general factor and any subfactors that account for variance over and above the general factor (Dere et al., 2015). de Miranda Azevedo et al. (2016) tested a bifactor model for the BDI using data from 13,100 patients with myocardial infarction from five countries. The best fit was found for a model that had a general factor that included all 21 items in the measure, and two subfactors, a somatic/affective subfactor and a cognitive/affective subfactor.

The third level of measurement equivalence is metric equivalence. A measure has metric equivalence when individual items load onto factors with the same strength across different populations. This type of variance allows one to make indirect comparisons between these populations, for example, whether predictive relationships found in one population can be found in the other or if changes in symptoms across time are similar across populations. However, in order to directly compare total scores across multiple populations, the fourth and last level of measurement equivalence—scalar equivalence—has to be met. If a measure has scalar equivalence, then the same score on the measure refers to the same position on the underlying latent variable across populations. Scalar equivalence is usually determined by examining whether individual items have the same point of origin, or intercept in the groups being compared. Failing to establish metric and scalar equivalence for a particular measure can result in both the identification of false group differences and the inability to detect true group differences (Chen, 2008).

Two studies have looked at the measurement equivalence of the BDI across different countries. Azocar et al. (2001) examined measurement equivalence in the BDI in medical patients who were either Spanish-speaking Latinos or English-speaking U.S. nationals using differential item functioning (DIF) analysis; they found that Latinos were more likely to endorse crying and punishment and less likely to endorse inability to work compared to English-speaking U.S. nationals with the same total BDI score, indicating a lack of measurement equivalence. Nuevo et al. (2009) examined the measurement equivalence of the BDI across five European countries—UK, Ireland, Spain, Norway, and Finland—and found that while the measure met criteria for structural equivalence, only eight of the 21 items displayed no DIF, thus indicating a lack of measurement equivalence.

No studies have yet examined measurement equivalence of the BDI in non-western, displaced populations, despite its wide use in those contexts. The current study focuses on females from such a population; female refugees and survivors of war have been found to have higher rates of depression (Ai, 2004; Seglem et al., 2011) and more severely impaired global functioning (Song et al., 2015) compared to men. Specifically, we investigate whether the BDI has measurement equivalence in internally displaced female survivors of the Sri Lankan civil war, which lasted from 1983 to 2009 (Jayawickreme et al., 2010). It is estimated that there are currently almost 45,000 internally displaced individuals in Sri Lanka (United Nations High Commissioner for Refugees, 2016). Similar to other displaced populations, Sri Lankan war survivors have endured multiple traumas and have high rates of mental illness, including depression (Jayawickreme N. et al., 2012). A survey of 1,517 households in the Jaffna district in Sri Lanka—an area that was most severely impacted by the civil war—found that 22% of individuals had a diagnosis of depression, with displaced individuals more likely to report symptoms of depression compared to long-term residents (Husain et al., 2011).

Identifying an equivalent Western sample that one can compare a non-Western war-affected population is challenging, given that experiences that non-Western war survivors undergo—multiple traumas, destruction of community, lack of security—are uncommon in the Western world. In order to have a degree of equivalence of life experience between the two groups, we used a sample of U.S. females from a major city—Philadelphia, PA—with a history of trauma. While the trauma suffered in a war context is more chronic compared to those suffered in Western settings, Western urban contexts are nevertheless associated with high levels of trauma exposure (e.g., Gillespie et al., 2009; Khoury et al., 2010; Brown and Mellman, 2014; Whitbeck et al., 2015). Indeed many other risk factors for PTSD besides trauma exposure are concentrated in Western urban contexts, such as risk of future victimization, fewer resources, poor coping skills, and multiple daily stressors (Kelly et al., 2010).

In the current study, we examined the three types of measurement equivalence that can be established through statistical analyses of responses on a measure—structural, metric, and scalar equivalence. We aimed to first identify an optimal factor structure for the BDI in war-affected Sri Lankan females and U.S. females with a history of trauma. To do this, we tested a series of potential baseline factor models. Following this, we examined the measurement equivalence of the BDI in these two culturally distinct groups.

In order to use measures of mental illness that have been developed for Western settings in survivors of war living in non-Western contexts, it is essential that one first establish measurement equivalence for those measures. It is only when measurement equivalence is established that one can make comparisons between, and use similar cut-off scores across, populations. In the current study, we examined the measurement equivalence of the BDI, a self-report measure of depression that is widely used in non-western refugee and displaced populations, among a sample of Sri Lankan women who were displaced due to the recently concluded civil war. To our knowledge, this is the first comprehensive examination of the measurement equivalence of the BDI in a non-Western, war-affected, displaced population.

We failed to establish measurement equivalence for the BDI across females from Sri Lanka and the United States. Neither the two-factor or three-factor models found in previous studies provided an adequate fit, while a bi-factor model found a similar general distress factor in both the Sri Lankan and U.S. sample, but different secondary dimensions or subfactors. However, while the intercepts of individual items differed to some degree, their differences appear to effectively cancel off. This suggests that BDI total scores are, roughly speaking, comparable, although we recommend caution when interpreting them since respondents from the two countries may obtain the same total score through endorsing different items.

We do not recommend making comparisons on subscales (e.g., cognitive-affective vs. performance-somatic), as the subfactors identified by the bifactor model in the two samples were distinct from one another. The subfactor in the Sri Lankan sample was characterized by somatic symptoms (e.g., sleep changes, tiredness, appetite changes) whereas the subfactor in the U.S. sample consisted of both cognitive and somatic symptoms. These results suggest that somatic symptoms are a key element of the depression syndrome in Sri Lankan women, more so than in U.S. women. This distinction in depression symptoms between Sri Lankan and U.S. women mirrors the distinction in symptoms between Chinese and Canadian individuals identified by Ryder et al. (2008), who found that depression in Chinese individuals was characterized more by somatic symptoms compared to Canadian individuals, whose depression was characterized by both somatic and cognitive symptoms. Our finding is also consistent with prior research indicating that somatic symptoms are more prominently seen in refugee populations compared to non-displaced Western populations (Rohlof et al., 2014). This emphasis on somatization in non-Western populations may be due to the fact that in many non-Western cultures, the body and mind are believed to be integrated, leading to greater expression of mental distress through bodily experiences (Kirmayer and Young, 1998). This is in contrast to Western cultures, where beliefs about mind and body reflect the Descartian separation of the two and there is thus more of a distinction between psychological and physical experiences. Furthermore, non-Western cultures tend to emphasize emotional expression to a lesser degree compared to Western cultures, and to have a higher level of stigma attached to mental illness. Both these factors have also been hypothesized to lead to greater somatization of mental distress (Chentsova-Dutton and Tsai, 2002).

Several caveats should be noted. First, our findings could potentially be the result of sampling error. If we had a larger sample, we would have been able to split the sample and conduct comparisons. In particular, we would have been able to look at cross-validation, i.e., split the sample into two and run factor analysis on each of them to see if the loadings are stable and match each other.

Second, the two samples were different in terms of experienced trauma and age. Even though participants in both samples experienced trauma, the Sri Lankan women, unlike the U.S. women, were exposed to chronic trauma. This differential exposure to trauma in the two groups could underlie the findings of this study. Given that there was no variability in trauma exposure in the U.S. female sample—every participant reported experiencing a single trauma—we were unable to control for trauma exposure in our analyses. It is also possible that the observed differences between the U.S. and Sri Lankan samples could be due to the fact that the Sri Lankan sample was significantly older than the U.S. sample. No studies have examined the measurement equivalence of the BDI across age groups, but Missinne et al. (2014) found that another measure of depression, the eight-item version of the Centre for Epidemiological Studies Depression scale (Radloff, 1977), had measurement equivalence across two age groups (50–64 years of age; 65 years and older) in 11 different European countries. This suggests that age does not have an impact on how individuals respond on measures of depression.

Fourth, given that our U.S. sample was majority African-American, it is possible that that sample's responses on BDI reflect a pattern of symptomatology that is specific to that community. There are no studies examining whether African-Americans respond similarly to European-Americans on the BDI; however, the BDI-II (Beck et al., 1996), which has 17 of the 21 items on the original BDI, has been found to have equivalent responses patterns in African-American and European-American samples (Hambrick et al., 2010; Whisman et al., 2012). Given the similarity between the BDI and BDI-II, these findings suggest that the BDI may also have measurement equivalence in these two communities.

Fourth, the current study focused exclusively on females. There is evidence that there are gender differences in the symptom make up of depression in non-Western displaced populations (e.g., Renner and Salem, 2009); thus, the scope of the current findings is limited to females who have been displaced due to conflict.

Fifth, given the fact that all participants in this study experienced at least one traumatic event, it is likely that many participants had comorbid posttraumatic stress disorder (PTSD), whose symptoms overlap with depression (American Psychiatric Association, 2013). Thus, one could hypothesize that differences in symptom endorsement between Sri Lankan and U.S. females may in part be due to PTSD symptomatology. However, in a study of individuals with comorbid depression and PTSD, Post et al. (2011) found that while the two syndromes had overlapping symptoms, they were distinct empirical constructs. This suggests that the singular focus of the current study on a measure of depression is justified.

Despite these limitations, we believe that our study contributes to our understanding of how measures of depression developed in the Western world function in non-Western war-affected populations by comparing two groups that live in distinct cultural contexts: a Western, urban context and a non-Western, war-affected context. There is a paucity of this type of research, i.e., examining the measurement equivalence of commonly used measures of depression in war-affected contexts. Many researchers and practitioners assume that Western-developed measures such as the BDI have measurement equivalence across different contexts and thus use them to identify depression in non-Western war survivors; demonstrating that these measures actually function differently in those non-Western war contexts has real implications for the use of these measures in those situations. We posit that the current study makes a contribution that is of relevance to practitioners and researchers working with non-Western displaced populations and war survivors by providing evidence that the BDI functions differently in those settings and thus should be used with caution. Our findings indicate that while total scores on the BDI can, for the most part, be interpreted similarly across the two populations we sampled, the measure nevertheless failed to meet the strict requirements of measurement equivalence. Thus, researchers and practitioners who work with non-Western displaced populations cannot assume that just because a measure has been widely used or appears to tap into a construct seen on the ground, it can be used to identify those with clinically significant symptoms or to compare multiple groups. Lastly, our findings add to the growing body of literature indicating that bifactor models provide the best fit for the measurement of depression (e.g., Dere et al., 2015).

While there have been a handful of studies examining the measurement equivalence of popular measures of mental illness in culturally diverse (e.g., BDI-II; Dere et al., 2015) and refugee populations (e.g., Harvard Trauma Questionnaire; Rasmussen et al., 2015), more research is clearly needed to establish the psychometric properties of these measures. We do believe that future studies should attempt to identify specific variables that account for any lack of measurement equivalence of the BDI and other measures in non-Western war-affected contexts. Also, more research is needed to confirm the different response patterns to individual items of the BDI in the U.S. and Sri Lankan populations (as well as other populations impacted by war). In the meantime, caution must be used when interpreting and comparing scores from non-Western, war-affected, displaced populations.

This study was carried out in accordance with the recommendations of the Belmont Report, Institutional Review Board of the University of Pennsylvania with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the Institutional Review Board of the University of Pennsylvania.

NJ conceived the research project, assisted with data collection, conducted data analysis, and wrote up the manuscript. JV conceived the research project, conducted data analysis, and wrote up the manuscript. EJ assisted with data collection and reviewed the manuscript. KA assisted in the write-up of the manuscript. EF assisted with data collection.