The Brief Psychiatric Rating Scale (BPRS) was developed to assess the severity of psychopathology symptoms among people diagnosed with psychosis. Its administration does not take a long time, which makes it appropriate for use in clinical settings in situations when efficacy and speed are needed (1).

The original version of this measure consisted of 16 items indicating symptoms scored on a 7-point scale ranging from 1 (not present) to 7 (severe) (1). Clinicians can easily perform such a rating in 2–3 min after the completion of the 20 min structured interview with the patient. Some items are rated based on the clinician's or interviewer's observation of the patient's behavior, whereas the remaining symptoms are assessed using the information obtained during the interview. In subsequent versions of BPRS, Overall and Gorham (2) included two additional items (BPRS-18 version), while Ventura et al. (3) expanded the scale with six new items/symptoms (BPRS-24 or BPRS-E).

Empirical evidence on the factor structure of BPRS-expanded (BPRS-E) showed 4-, 5-, and 6-factor solutions for this measure. Most of the studies revealed that the symptoms in BPRS-E tend to cluster in four dimensions and found that this factor structure is the most stable and adequately representing groups of symptoms in psychosis. In this context, Velligan et al. (4), using the principal axis factoring method (PAF) with Varimax and Promax rotation, identified four factors of BPRS-E among outpatients with major depression, schizophrenia, and bipolar disorder, i.e., anxiety or depression, psychosis, negative symptoms, and activation. The authors concluded that BPRS-E provides a more comprehensive assessment compared to the initial version of BPRS and should be used with people diagnosed with mental health disorders. Similarly, Kopelowicz et al.'s (5) study found that the same 4-factor solution of BPRS-E [positive symptoms, negative symptoms, anxiety/depression, activation/mania extracted with principal component analysis (PCA), and Varimax rotation] was stable across the course of schizophrenia. Furthermore, research conducted on a sample of individuals with psychosis (non-affective) from 11 European countries (6) used a 6-factor structure of BPRS-E with PCA with Varimax rotation applied during their admission, i.e., mania, negative symptoms, positive symptoms, depression, agitation, and anxiety. In another cross-national study involving outpatients with schizophrenia from five Western European countries, Rugerri et al. (7) identified a 4-factor solution of BPRS-E using PCA (mania/disorganization, positive symptoms, negative symptoms, depression/anxiety) with mania and disorganization symptoms loaded as one factor. Previously, Ventura et al. (8) used PCA with Varimax rotation and identified a 5-factor solution among a relatively small sample consisting of patients with recent-onset schizophrenia and schizoaffective and bipolar disorders; however, they concluded that the fifth factor was not interpretable. In their model, mania and disorganization symptoms were clustered together as the same factor. Two additional studies revealed five underlying factors for the BPRS-E symptoms among general psychiatric inpatients (9, 10). In general, when comparing these results, it is evident that identified models with the same number of factors showed slightly different content. However, core items were typically loaded on the same factors across all models.

Despite these insights, it was suggested that a 4-factor solution is better than a 5-factor structure, even though both are very similar (9, 11). Following this, a meta-analysis of the BPRS-18 factor structure (12) showed that the model with 4 factors represented the core BPRS structure, but it also implied a 5-factor solution that included affect (anxiety, guilt, depression, and somatic concern); positive symptoms (unusual thought content, conceptual disorganization, hallucinations, and grandiosity); negative symptoms (blunted affect, emotional withdrawal, and motor retardation); resistance (hostility, uncooperativeness, and suspiciousness); and activation (excitement, tension, mannerisms-and posturing). Based on a more recent meta-analysis of the BPRS-E structure, Dazzi et al. (13) concluded that the 4-factor model of affect (anxiety, guilt, depression, and suicidality), positive symptoms (hallucinations, unusual thought content, suspiciousness, and grandiosity), negative symptoms (blunted affect, emotional withdrawal, and motor retardation), and activation (excitement, motor hyperactivity, elevated mood, and distractibility) was statistically supported and relatively invariant. Furthermore, the authors recommended adding a fifth factor, the factor of Disorganization (conceptual disorganization, disorientation, self-neglect, and mannerism-posturing), to the model.

The BPRS-E factor structure, derived in the aforementioned meta-analysis, along with models extracted in other studies were tested on a large sample of hospitalized patients. These patients were diagnosed with schizophrenia, major depressive disorder, bipolar disorder, manic episodes, followed by addiction-related disorders, adjustment disorders, dementia, and personality-impulsive disorders (14). The authors reported that the meta-analytic 4-factor model with 12 core BPRS items had an excellent fit, but additionally, a model consisting of 5-factors (including Disorganization) with 15 core BPRS-E symptoms also showed an acceptable fit. Studies conducted among outpatients with unipolar depression (15) and patients with manic episodes (16) revealed a 6-factor and 4-factor solutions of BPRS-E, respectively, with factors specific to the disorder or illness.

Differences in the number of factors in BPRS-E might be linked to differences in the sample size across studies, the type of disorder, and participants' status as inpatients and outpatients, and, possibly, different factor extraction and factor rotation methods applied while examining its latent structure. In addition, all findings come from an American and/or Western European context; thus, further exploration of the BPRS-E structure in other and different contexts can reveal new empirical evidence. Even more, recent studies (13, 14) suggest that additional evidence on the Disorganization factor and the 5-factor model is necessary.

To the best of our knowledge, the factor structure of BPRS-E has not been investigated among outpatients with schizophrenia spectrum disorders and bipolar disorder (F20-29 and F-31) who are in a remission phase, particularly in low- and middle-income Southeast European countries. Moreover, previous findings on the factor model of the BPRS-E psychopathology symptoms have not been consistent.

Therefore, this study aimed to explore the factor structure of BPRS-E on a sample of outpatients with psychotic disorders in the following five Southeast European countries: Bosnia and Herzegovina, Montenegro, Kosovo (UN resolution), North Macedonia, and Serbia. More precisely, its first aim was to examine the underlying groups of psychopathology symptoms as measured by BPRS-E using exploratory factor analysis. The second aim of the study was to further investigate the goodness-of-fit of the identified factor model, applying confirmatory factor analysis, as well as to compare it with three competing, previously derived factor models of BPRS-E, i.e., the 4-factor model with 12 core items, the 4-factor model with 15 items, and the 5-factor model with 15 items [see (13, 14)].

Descriptive statistics (mean, standard deviation, and minimum and maximum scores) of all variables, items, and symptoms in BPRS-E are presented in Table 2. Data on median, skewness, and kurtosis are given in Supplementary Table 1.

Six factors with an eigenvalue >1 were extracted, accounting for 47.95% of the variance of the BPRS-E symptomatology. The proportion of explained variance by each factor is given in Table 3.

The first factor, named anxiety/depression (affect), consisted of somatic concern, anxiety, depression, suicidality, and guilt. Flat affect, emotional withdrawal, motor retardation, and mannerism were loaded on the second factor called negative symptoms. The third factor, called positive symptoms, consisted of hallucinations, unusual thought content, bizarre behavior, suspicion, and grandiosity. The fourth factor, called activation, comprised motor hyperactivity, excitement, tension, and elevated mood items/symptoms. Conceptual disorganization, distractibility, and disorientation were loaded on the fifth factor named Disorganization. The sixth factor consisted of three items, namely, elevated mood, disorganization, and grandiosity, that were all cross-loaded on other factors (i.e., on the fourth, fifth, and third factors, respectively). As a result, these three items were retained in those factors. In addition, the scree plot indicated a solution with five factors as well. Hostility, self-neglect, and uncooperativeness did not load to any extracted factor (factor loadings < 0.32). These items were removed from the obtained model and further analysis.

Cronbach's alpha reliability coefficients ranged from 0.65 to 0.77 (see Table 3). As seen in Table 3, an association of negative symptoms with anxiety and depression (affect) group of symptoms, positive symptoms, and activation was statistically significant but weak (r = 0.13, p < 0.01; r = 0.14, p < 0.01, and r = −0.13, p < 0.05, respectively). Positive symptomatology and negative symptomatology factors were significantly and moderately related to the disorganization group of symptoms (r = 0.26, p < 0.001, and r = 0.30, p < 0.001, respectively). This implied that the factors of Disorganization and cognitive impairment symptoms go along with other identified groups of symptoms characteristic of psychotic disorders. The relationship of disorganization to anxiety and depression and activation was found to be weaker (r = 0.20, p < 0.01 and r = 0.15, p < 0.01, respectively).

A follow-up analysis using CFA revealed mixed results regarding model fit, as shown in Table 4. More precisely, CFI was under the recommended value, while RMSEA and SRMR implied an acceptable data fit. In addition, the factor loading of grandiosity symptom on positive symptoms factor was very low and non-significant. Grandiosity, along with the tension symptom, had correlated residuals with other items/symptoms. When these two items were deleted, the CFI value increased (ΔCFI = 0.13), and RMSEA and SRMR indices slightly decreased, demonstrating evidently a better fit of the tested factor model (Table 4). Further, the results showed that the primary factor loadings of elevated mood, mannerism, and bizarre behavior items were 0.24, 0.28, and 0.30, respectively (Table 5, 19-item 5-factor model of this study). Given that these factor loadings were statistically significant (p < 0.01), the items were retained in the factor model.

The analysis of the competing BPRS-E factor models revealed that the best-fitted model in this study was the 4-factor solution with 12 core items. While the model with five factors and 15 items or symptoms showed an acceptable fit, the model with 15 symptoms grouped into four factors (Table 4) did not. However, the difference between the 19-item model obtained in this study and the two competing models was trivial, particularly with the 15-item 5-factor alternative model (Table 4). Factors and factor loadings for all tested models are presented in Table 5.

This cross-national study provided evidence on the factor structure of the BPRS-E among outpatients with psychosis in Southeast European countries, thus contributing to the existing empirical findings that generally come from Western countries. To the best of our knowledge, the previously mentioned study on BPRS-E structure (6) included participants with acute psychopathology symptoms during their hospital admission from two Southeast European countries, both of which are EU member states, along with participants from nine West and East European countries. Additionally, participants from nine West and East European countries with acute psychopathology symptoms were included, and these data were collected during their hospital admission.

Exploratory factor analysis revealed a factor solution with 21 items and similar clustering of core psychopathology symptoms, as measured by BPRS-E, found in previous research [e.g., (9)]. However, there were differences in the distribution of some symptoms across the extracted factors that might be explained by different factor extraction and factor rotation methods used, alongside variations associated with sample size and participants' diagnoses.

It should be particularly emphasized that the CFA findings in this study supported the BPRS-E model, which contains 19 items grouped into five factors, with Disorganization as a fifth factor. Furthermore, its content was similar to that reported in Shafer et al.'s (14) study, clearly referring to cognition impairment (thinking, speech, attention). For instance, this factor consisted of conceptual disorganization, disorientation, and distraction symptoms. The symptoms of conceptual disorganization, disorientation, and distraction were all part of the Disorganization factor, in the study by Horton and Silverstein (29), but along with positive symptoms. However, the core symptoms of conceptual disorganization and disorientation were both found to load on this factor in other studies as well [e.g., (5, 13, 30)].

Consistent with other findings [e.g., (7, 8, 10)], in this study, the symptom of mannerism-posturing loaded on the negative symptoms factor along with blunted affect, emotional withdrawal, and motor retardation. However, three out of four symptoms that were excluded from the BPRS-E model, i.e., uncooperativeness, hostility, and tension, included resistance as a sixth factor in the factor structure obtained among hospitalized patients in the study using EFA (14). These items were also found to load on the resistance factor in other studies [e.g., (31)]. Such findings might imply that these symptoms emerged as a particular group in hospitalized or more severely ill patients but did not contribute to the symptomatology, as measured by BPRS-E, when it comes to mildly ill patients or outpatients in the remission stage of illness.

It should be noted that the model identified in this study showed an acceptable fit, very similar to that of the competing models (12-core item 4-factor model and 15-item 5-factor alternative model). More precisely, the factor structure of the BPRS-E psychopathology symptoms identified in this study showed identical clustering of the core symptoms as in the competing 15-item 5-factor alternative model. However, it is more inclusive since it contains a greater number of items−19 in total. In addition, this model confirmed the stability of affect (anxiety/depression) and negative symptomatology factors. Furthermore, the findings implied that the main symptoms of psychotic disorders with all factors show acceptable internal consistency.

This study has several strengths and limitations. The data were collected by trained researchers whose ratings demonstrated high inter-rater reliability. The research was carried out in five low- and middle-income countries in Southeast Europe with similar cultural, socioeconomic, and healthcare systems. The sample was relatively large, consisting of outpatients with psychotic spectrum disorders in remission and stable stage of the illness. In that context, this study provides an important contribution to the existing findings on the BPRS-E factor structure obtained in Western countries. It should be noted that most of the previous studies used principal component analysis to investigate the factor structure of BPRS-E. In this study, factor analysis was applied as a more suitable method when underlying factors needed to be identified. However, some variations in the BPRS-E factor structure are possible, but considering the aforementioned similarities, it could be assumed that the identified structure is applicable across five countries. Data on illness duration, number of past hospitalizations, and pharmacotherapy were not taken into account, which could be considered as a limitation of the obtained results on psychopathology symptoms measured with BPRS-E. In addition, the research was focused on construct validity; therefore, future studies aiming to investigate the predictive and convergent validity of BPRS-E are needed.

Exploratory factor analysis of BPRS-E yielded a solution with six factors, among which five were clearly defined. Extracted factors consisted of 21 items/symptoms, generally in accordance with the previously produced factor models. A follow-up confirmatory factor analysis revealed a 19-item solution (i.e., a model with two additional items removed) that fitted the data well. The results implied that the two previously identified BPRS-E factor models tested in this study were stable: the 12-core item 4-factor model and the 15-item 5-factor alternative model.

The findings of this study clearly indicate the stability of four BPRS-E factors, i.e., affect, negative symptoms, positive symptoms, and activation. They further pointed out the existence of a fifth factor—the disorganization group of symptoms in a sample of outpatients with psychosis in the Southeast European context. In addition, this model was found to include more items and symptoms compared to competing models. The results revealed acceptable to good reliability of all five factors/groups of symptoms.

It could be concluded that the scale may be used for research purposes considering its factor structure and internal consistency. Furthermore, this measure may be used in a clinical setting for assessing the severity of psychopathology symptoms among outpatients with psychotic disorders in the Balkans, along with other psychopathology assessment methods. The results could be useful to researchers and clinicians, particularly in the Balkans, representing additional empirical evidence on the factor structure of BPRS-E that supported the existence of the fifth factor of Disorganization.

The datasets presented in this article are not readily available for confidentiality reasons. Requests to access the datasets should be directed to NJ.

The study was approved by the Ethical Committees in all participating countries: Bosnia and Herzegovina–approval No. 03-02-4216 and 02.8-408/19 (Klinicki Centar Univerziteta u Sarajevu –Eticki Komitet 03-02-4216, Eticki komitet JU Psihijatrijska bolnica Kantona Sarajevo i JU Zavod za bolesti ovisnosti Kantona Sarajevo, 02.8-408/19), Serbia–approval No. 2650/XII-20 and 01-36/1 (Eticka komisija Medicinskog fakulteta u Beogradu 2650/XII-20 and Eticka komisija Specijalne bolnice Dr. Slavoljub Bakalovic Vrsac, 01-36/1), Kosovo-approval No. 209-85 (Hospital and University Clinical Service of Kosovo–Ethics Committee 2019-85), Republic of North Macedonia–approval No. 03-24219 (Eticka komisija za istrazuvanje na luge, Medicinski Fakultet pri UKIM vo Skopje, 03-24219), and Montenegro-approval No. 03/01–29304/1 and 01-47 (Javna zdravstvena ustanova Klinicki centar Crne Gore–Eticki komitet 03/01–29304/1, ZU Specijalna bolnica za psihijatriju Dobrota Kotor–Eticki komitet, Eticki komitet JZU Dom ydravlja Dr. Nika Labovic Berane 01-47). All participants provided written informed consent prior to the study. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

SB: writing—review and editing, conceptualization, resources (study materials and participants), leadership, and coordination responsibility for research activity and execution. BB: study conceptualization, methodology, validation, statistical analyses, data collection and curation, and writing—original draft. MR: conceptualization, validation, data curation, writing—review and editing, and coordination of research activity planning. SR: coneptualization, data collection and curation, and review and editing. NM: review and editing, resources (study materials and participants), leadership, and coordination responsibility for research activity and execution. AD, AA, and LS: resources (study materials and participants), leadership, and coordination responsibility for research activity and execution. LN, ER, JK, and IR: data collection and data curation. AN: writing—review and editing, supervision, project administration, resources, and investigation. NJ: review and editing, management and coordination responsibility for research activity planning and execution, and funding acquisition. All authors contributed to the article and approved the submitted version.