Social or public stigma is a prevalent and multifaceted social phenomenon usually described as involving negative evaluations and stereotypes associated with a distinguishing characteristic (Goffman, 1963; Link and Phelan, 2001). Such devalued characteristics range from gender, race, or sexuality (e.g., Frost, 2011) to disability, weight, or mental illness (e.g., Stangl et al., 2019), as well as infectious diseases like COVID-19 (e.g., Saeed et al., 2020). In addition, labeling people as different and linking these differences to negative stereotypes creates a clear separation between the superior “us” and the inferior “them” (Jones et al., 1984; Link and Phelan, 2001; Smith, 2007). Relatedly, stigmatization is a complex process that refers to excluding, discriminating, or marginalizing a person or group based on a particular characteristic stigmatized within a society or social context (Major and O'Brien, 2005). Stigmatization can have far-reaching negative consequences for the affected individuals. Due to its potential to increase social and structural inequalities and lead to severe cognitive and affective consequences for individuals, it constitutes a primary global health concern (Hatzenbuehler et al., 2013; Link et al., 2014; Pescosolido and Martin, 2015).

In light of the potential hazard stigmatization poses for the social life of the individuals being stigmatized as well as for their mental and physical health, research in various disciplines such as sociology, psychology, or public health has acknowledged the importance of these issues by thoroughly investigating the occurrence of stigma and how it can be overcome (Bos et al., 2013; Pescosolido and Martin, 2015). One crucial and overarching characteristic of quantitative studies examining public stigma in the general population is that they usually include an assessment of the level of the stigma toward a particular group or the tendency to enact stigma, with (validated) self-report scales (Link et al., 2004). Because of their centrality, standardized scales constitute a central instrument and belong to the standard methodological repertoire in quantitative research investigating social or public stigma. To assess the multifaceted nature of stigma as a social phenomenon, most scales consist of multi-item measures that aim to cover the central dimensions involved in the given stigma at hand (Bresnahan and Zhuang, 2011; Pescosolido and Martin, 2015). While scholars regularly note a lack of consistency (partly due to the specificity of different stigmas) and clarity among measures, stereotyping and discrimination belong to the most commonly measured dimensions and can be seen as overarching and unifying elements in stigma research (Link et al., 2004; Pescosolido and Martin, 2015; Fox et al., 2018).

Beyond the challenge of measuring social stigma, research on stigma is confronted with significant ethical challenges. One potential risk of investigating the phenomenon in the general population is the unintentional reinforcement of stigmatization and its adverse outcomes (Millum et al., 2019). This is particularly important, as the central goal of research in this area is to provide evidence on how to fight and overcome stigma. Considering this challenge and risk, it is surprising that studies have not devoted more attention to the issue of unintended effects of stigma research. In contrast to the discussions in the context of, especially qualitative, research with stigmatized groups (Millum et al., 2019; Gabbidon and Chenneville, 2021), quantitative studies investigating public stigma in the general population—if at all—account for the unintentional reinforcement of stigma through research by providing recommendations on avoiding the negative consequences of stigmatizing language. In their article on measuring mental illness stigma, Link et al. (2004), for example, formulate guiding questions to be asked when selecting measures, among them whether “…the words and phrases used to refer to or describe people with mental illnesses [are] appropriately sensitive and respectful” (p. 518).

However, besides respectfully referring to the people affected by stigma, self-report scales usually include items that directly confront people with the negative stereotypes and evaluations underlying the stigma itself. For example, study participants are asked how much they agree that a person with schizophrenia is “dangerous” (e.g., Corrigan et al., 2015), a person with an eating disorder is using it “to gain attention” (e.g., Roehrig and McLean, 2010), or a person experiencing homelessness is “lazy” (e.g., Knecht and Martinez, 2009). By directly confronting participants with stigmatizing statements, the measurement itself might potentially activate and reinforce existing cognitive stigma-relevant patterns or even create new ones. Detrimental effects from these mechanisms likely manifest on different levels, such as increasing implicit and explicit stigma. Considering the lack of evidence about the occurrence and nature of potential detrimental effects, this study will examine the following research question:

By posing and investigating this research question, we would like to, first and foremost, promote reflection on the possible unintentional negative effects of our research practices in general, but more specifically regarding the use of self-report scales of stigma beliefs. Investigating these risks is not meant to disregard or devalue the invaluable insights previous researchers have generated, nor does it constitute a call for abandoning stigma research altogether. On the contrary, we aim to contribute to responsible research practices in studying stigma. This exploratory study on the possible detrimental effects of stigma scales is one step within the specific context of quantitative research using self-report scales.

Applying priming theory and related empirical findings provides a reasonable basis for investigating this phenomenon in more detail (for an extensive overview, see Carpentier, 2020). Priming theory has its roots in free associations techniques and thought associations and focuses on cognitive structures and how concepts in memory are stored, linked with one another, and accessed (e.g., Tulving, 1972). Further developments in this domain resulted in specific work on political priming (Iyengar et al., 1982), media priming (Berkowitz, 1984), but also—and in the light of this research most relevant—stereotype priming (Devine, 1989). Building on these findings and more recent knowledge about stereotype activation and application (Hilton and von Hippel, 1996; Bargh, 1999; Macrae and Bodenhausen, 2000; Greenwald et al., 2002), it can be assumed that exposure to stigma scales assessing negative stereotypes (i.e., words, statements, or questions) to measure the phenomenon creates or (re-)activates (existing) associations between those characteristics and the respective stigmatized group. This, in turn, might increase implicit and explicit stigma. To assess effects on explicit stigma without directly asking about stigma beliefs, we focus on downward social comparison as an indicator of outgroup devaluation and desired social distance as an indicator of discrimination tendencies. We therefore hypothesize:

In addition, it is vital to consider an individual's knowledge about a topic and its personal relevance to them. More knowledgeable individuals are likely—by way of spreading activation (e.g., Collins and Loftus, 1975)—to have a wider variety of information for both judgment and control of stereotype application compared to less knowledgeable individuals who should be more likely to apply the negative stereotypes primed by scale exposure (Devine, 1989). Research has shown that education interventions can reduce stigma (e.g., Corrigan et al., 2012). On the other hand, individuals who perceive a topic as less relevant personally are less likely to engage in motivated inhibition of stereotype application (Kunda and Sinclair, 1999; Devine and Sharp, 2009; Rees et al., 2019). Consequently, prior knowledge and topic involvement can moderate the effect of stigma scale exposure. Therefore, we assume that:

This study investigated the potentially detrimental effects of employing self-report scales to measure public stigma in research projects using a general population sample. Within the contexts of WLS and PrEP we tested whether the use of scales that explicitly include negative stereotypes can promote stigmatizing tendencies in participants who are asked to respond to such statements. Based on theory and research in priming (Carpentier, 2020) and stereotype activation/application (Macrae and Bodenhausen, 2000), we assumed that stigma scale exposure will have a positive effect on indicators of stigmatization (i.e., implicit stigma, downward social comparison, and desired social distance). Moreover, we expected this effect to be moderated by individuals' knowledge about the topic as well as their issue involvement. The results demonstrate that the underlying mechanisms of potential detrimental effects of stigma scale exposure are more complex and as multifaceted as the concept of stigma itself.

First, regarding general scale exposure effects on stigma (H1), we found increased accessibility of stigma scale items (weak-minded and cowardly) after exposure. Even though there was no influence on participants' tendency to agree with those statements, the increased accessibility could strengthen the connection between these negative stereotypes and the target group if activated repeatedly over time. Of course, the accessibility effect on these two items should not be overrated: This type of priming effect was expected for those stigma adjectives contained in the scale whose exposure was manipulated. At the same time, there was no heightened accessibility effect for negative stereotypes most closely associated with the respective stigma (PrEP: promiscuous, WLS: lazy), but scale exposure did influence agreement with these stereotypes. Curiously, while it increased endorsement of the stigma for PrEP, scale exposure reduced endorsement for WLS patients. In terms of the lack of similar accessibility effects for weak-minded and cowardly, this might result from stronger existing associations between lazy/promiscuous and WLS patients/PrEP users that are less affected by one exposure. An explanation for the opposite effect of scale exposure on agreement with these negative stereotypes is more complex. The stigmatizing effect for PrEP users, which aligns with our hypothesis, might be rooted in the nature of the stigma, i.e., differences between the topics, with PrEP being less familiar and more stigmatized. The de-stigmatizing effect for WLS patients might result from more explicit processing, e.g., a form of impression management or even a protest against perceived stigma. These explanations are only speculative, and future research needs to disentangle such short-term accessibility effects of exposure to such statements, including longer-term accessibility and conditions for translation into an agreement. Besides effects on the directly related stigma items, scale exposure effects did not extend to more general dimensions of the stereotype content model, downward social comparison, or desired social distance.

However, the findings concerning H2, the moderating effects of prior knowledge and personal relevance, show that scale exposure may affect these variables depending on prior knowledge and personal relevance. Independent of the topic, participants to whom PrEP or WLS was less personally relevant judged PrEP users/WLS patients as less competent when previously being confronted with a stigma scale compared to when not. In contrast, individuals who reported greater personal relevance responded faster and judged PrEP users/WLS patients as more competent when exposed to the scale. Thus, the more relevant the topic was to the participants, the more positively they responded to a more generalized dimension of stigma under time pressure after scale exposure. Exposure to the stigma scale resulted in more stigmatized responses by participants with low relevance. While in line with hypothesis two, this negative effect for people who regard the topics as less relevant is quite concerning. Further research is needed to examine the underlying mechanisms.

Like the topic-specific effects for the stigma-specific items, lazy and promiscuous, we found different effects of scale exposure for different levels of prior knowledge about the topic on social comparison as a dependent variable. In contrast to the expected stronger effects for individuals with less knowledge, exposure to a stigma scale about PrEP users increased downward social comparison for increasingly knowledgeable individuals, while the opposite is observed for WLS stigma scale exposure. The findings for WLS correspond to the de-stigmatizing effects of scale exposure on the topic-specific stereotype and point to the potential that a more consolidated attitude makes the negative influence of exposure to stereotyping content less likely but rather leads to the opposite effect. Whether that is a genuinely positive effect or the result of short-term impression management processes or protesting is a question for future research. An even more puzzling question to be addressed by future studies is the detrimental effect of scale exposure on participants with high knowledge of PrEP. A very preliminary, speculative explanation could be that there are self-stigmatizing processes activated by scale exposure, as participants with very high knowledge of PrEP are more likely to be part of the targeted stigma group. Again, more research is needed to disentangle these relationships.

In light of our findings, it is crucial to consider the wider practical implications for anti-stigma communication, particularly regarding the use of negative stereotypes in interventions such as public health campaigns. Some campaigns employ stereotypes as a strategy to convey the message that labeling individuals in certain ways, such as dangerous (in the case of mental disorders) or lazy (pertaining to overweight and obesity), is both morally wrong and socially unacceptable. However, our study reveals that this approach might lead to unintended effects and emphasizes the need for careful consideration and sensitivity when employing stereotypes in anti-stigma interventions.

There are limitations of this study that need to be addressed. This includes the single-exposure nature of the study design. As such, we cannot be certain whether the emerging short-term effects also translate into long-term effects or even accumulate, especially when participants are repeatedly confronted with such statements, i.e., in longitudinal investigations or other settings, such as on social media. Therefore, especially longitudinal designs with measurements weeks or months after the main study are needed to examine whether scale exposure effects are persistent.

Moreover, even though using the panel service to distribute the survey provides more heterogeneity than regular student samples, it must be noted that the sample is still a convenience sample skewed to higher education. Future studies should recruit samples with more varied levels of education.

Our study was conducted within the context of two topics with their respective stigma contexts and the question remains whether the findings can be replicated for other topics. Of note, by choosing two topics, we aimed to increase the generalizability of the underlying effects. However, the topic-specific findings show that more factors need to be considered when investigating the potential detrimental effects of self-report scales measuring stigma beliefs. Thus, it is relevant for future research to systematically consider topic-specific or stigma context-specific characteristics.

Relatedly, our attempt to streamline the specific stigma items to make the self-report scales of stigma beliefs comparable across the two topics and avoid confounding due to differences in scale composition might have resulted in a less specific fit of the applied stigma scales and the low reliability for the stigma items in the implicit measure task. The decision to use single-item dependent variables for the stigma adjectives is not ideal. On a more general note about our implicit measure, two aspects need to be discussed, its validity as a measure of implicit stigma generally and the exploratory nature of our measure. In the methodological literature there is a continued debate about the validity of implicit measures, especially in terms of what they measure (De Houwer et al., 2009; Gawronski and Hahn, 2019). For example, in our case, reaction times for a decision under time pressure of whether or not an adjective applies or not are first and foremost an assessment of how long it takes participants to make that decision. However, by comparing two groups of participants randomly assigned to the experimental conditions, differences between participants exposed to the stigma scale and those not can be interpreted regarding the accessibility of the association between the adjective and the target group. Regarding the agreement decision, this is likely not an implicit measure in the pure sense of automaticity (De Houwer et al., 2009), because participants are aware of the nature of the judgment. However, due to the time pressure under which they are asked to make their decision, their responses are less controlled and more likely to represent their “gut reaction” to the task. At the same time, we need to emphasize that we have created this measure for this study, and it needs to be validated and refined, especially with respect to the reliability issues mentioned above. This underlines the need for future research to develop different sensitive measures to tap into implicit stigma.

Our research aimed to investigate the potential negative effects of standardized stigma scales, including items associated with negative stereotypes. In this context, it is important to consider the potentially harmful impacts of other measures used in this study, such as social comparison, social distance, and the reaction time task. We argue that these three measures do not meet the criteria of stigma messages as defined by Smith (2007) because they lack explicit statements connecting the stigmatized group to negative stereotypes. Moreover, these measures include either exclusively positive items or a combination of positive and negative items. Finally, all three measures are not exclusive to stigma research but can be used to assess perceptions of any group or individual.

However, it is still possible that, within the context of stigma research, a question regarding whether one can imagine being friends with or having a romantic relationship with someone from a stigmatized group may trigger a thought process leading to a desire for greater distance from that person. Nevertheless, it is practically impossible to eliminate all stigmatizing effects of every measure for all participants. The importance of comprehensive debriefing procedures in stigma research cannot be overstated. Nonetheless, it is essential to identify and address unintended effects that are more substantial and consistent if we discover them.

In conclusion, this study is the first exploratory attempt at identifying potential adverse effects of utilizing stigma scales assessing negative stereotypes in quantitative studies of public stigma in the general population. Overall, our results underline the relevance of the issue by showing that being confronted with judgmental or stereotypical items on a stigma scale—even only for a short time—can facilitate scale-related stereotype accessibility, both increasing and decreasing stigmatizing tendencies. Specifically, scale exposure increased endorsement of a negative stereotype for PrEP, increased downward social comparison for individuals highly knowledgeable about PrEP, and reduced perceptions of competence for individuals with low personal relevance for both topics. At the same time, scale exposure reduced endorsement of a negative stereotype of WLS, increased upward social comparison for individuals highly knowledgeable about WLS, and increased perceptions of competence for high personal relevance. In short, stigma scale exposure can have both adverse and positive effects on stigmatizing tendencies. Future research must replicate and extend these results to determine the range and strength of significant adverse effects. A systematic investigation would allow the examination of the mechanisms and conditions under which such detrimental effects occur and how they can be avoided.

At the same time, researchers need to discuss and develop alternative ways of measurement or strategies to avoid or counter any stigmatizing effects of self-report scales assessing stigma beliefs. For example, scales could be constructed with only positive wordings. Alternatively, specific debriefing methods might be developed and tested to counter any negative effects of scale exposure. Our exploratory study is a first step to raise awareness and contribute to ethical research practices that avoid unintended negative consequences of exacerbating the stigma under examination.

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

The studies involving human participants were reviewed and approved by Ethikbeirat Universität Leipzig, 2021.04.01_eb_82. The patients/participants provided their written informed consent to participate in this study.

AO and FS contributed to the conception, design of the study, secured data access, performed the statistical analysis, wrote the first draft of the manuscript, and contributed to the manuscript revision. Both authors read and approved the submitted version.