Front. Commun. Frontiers in Communication Front. Commun. 2297-900X Frontiers Media S.A. 10.3389/fcomm.2024.1378557 Communication Original Research Do COVID-19 related primary emotions affect risk perceptions, efficacy beliefs, and information seeking and behavior? Examining emotions as audience segments Turner Monique M. 1 Lim Jong In 1 * Jang Youjin 2 Heo Ruth Jin-Hee 1 Ye Qijia 3 Kim Miyeon 1 Lapinski Maria Knight 4 Peng Tai-Quan 1 1Department of Communication, Michigan State University, East Lansing, MI, United States 2Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States 3Annenberg School for Communication, University of Pennsylvania, Philadelphia, PA, United States 4Department of Communication and Michigan AgBio Research, Michigan State University, Philadelphia, PA, United States

Edited by: Julie Volkman, Bryant University, United States

Reviewed by: Mustafa Can Gursesli, University of Florence, Italy

Tara McManus, University of Nevada, Las Vegas, United States

 *Correspondence: Jong In Lim, limjong1@msu.edu 16 07 2024 2024 9 1378557 29 01 2024 26 06 2024 Copyright © 2024 Turner, Lim, Jang, Heo, Ye, Kim, Lapinski and Peng. 2024 Turner, Lim, Jang, Heo, Ye, Kim, Lapinski and Peng

This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

 Introduction

Primary emotions among the populace during global health crises, such as the COVID-19 pandemic, provide valuable insights. Guided by appraisal theories of emotions, this study explores emotional segmentation by mapping primary emotions related to COVID-19 and investigating their impact on cognitive and behavioral outcomes, including risk perceptions, efficacy beliefs, behavioral intentions, prevention behaviors, and information seeking/avoidance.

 Methods

Study 1 surveyed young adults (N = 1,368) to investigate their emotions about COVID-19 and examine the effect of these emotions on risk perceptions, efficacy beliefs, and behavioral intentions regarding mask-wearing and physical distancing. Study 2 replicated Study 1 with a quota-based national sample of U.S. adults (N = 8,454) and further tested the effect of primary emotions on preventive behaviors and information seeking/avoidance.

 Results

Results indicated that most people experienced negative emotions, such as sadness, anger, and anxiety. These emotions were associated with varying levels of risk perceptions, efficacy beliefs, behavioral intentions, actual behaviors, and information seeking/avoidance. Notably, across both studies, anxious individuals demonstrated more favorable health-related perceptions and behavioral outcomes compared to those experiencing anger.

 Discussion

These findings offer insights into the emotional experiences of individuals during the COVID-19 pandemic and highlight the significant impact of these emotions on risk perceptions and health-related behaviors. Understanding these emotional responses can inform public health strategies and communication efforts during health crises.

 appraisal tendencies framework (ATF) discrete emotions efficacy beliefs information seeking prevention behaviors section-at-acceptance Health Communication Introduction

The COVID-19 pandemic was an emotional roller coaster for individuals around the globe (Ahmadi and Ramezani, 2020; for a review, Imran et al., 2020; Pedrosa et al., 2020). Research has consistently shown that infectious disease outbreaks cause both psychological consequences (Restubog et al., 2020), including depression and psychological distress (Bai et al., 2004; Jones and Salathé, 2009; Bults et al., 2011; Shultz et al., 2016) worry (Thompson et al., 2017), anxiety about being infected (Horney et al., 2010; Leggat et al., 2010; Rubin et al., 2010; Jehn et al., 2011), and reduced subjective well-being (Lau and Thomas, 2008). COVID-19 specifically has been shown to be related to feelings of anxiety and depression (Al Mutair et al., 2021) as well as anger and fear (Trnka and Lorencova, 2020). The PsyCorona Survey which included 54,845 participants from 112 countries showed that participants reported feelings of fear, depression, anxiety, calm, contentedness, and even joy toward COVID-19 (Han et al., 2021). Individuals also shared various types of emotions they felt during COVID 19 pandemic (e.g., Kim et al., 2022; Metzler et al., 2023). Understanding the emotions surrounding health crises such as COVID-19 is critical as emotion can influence subsequent thoughts and actions (Nabi, 1999; Lerner and Keltner, 2001; Turner, 2007; Turner et al., 2020; Jang et al., 2021). Drawing on the Appraisal Tendency Framework (ATF; Lerner and Keltner, 2001), this study aimed to examine the primary emotions experienced by US adults in relation to COVID-19 and assess their effect on risk perceptions, efficacy beliefs, information seeking, and prevention behaviors. Although several experimental testing the ATF have shown that emotions cause cognitive and behavioral outcomes (Lerner and Keltner, 2001; Turner et al., 2020), little is known about the emotions generated organically amid a global health crisis and their outcomes. The current project seeks to generalize and extend previous ATF findings with larger and more representative samples using survey methodology.

Emotional experience of COVID-19

Notably, there is a long-standing debate on the definition of emotion (Dixon, 2012). Nonetheless, there is a general consensus that emotions are relatively short-lived mental states characterized by several elements: (a) cognitive appraisal; (b) physiological arousal; (c) bodily expression; (d) motivational tendencies driving individuals toward specific cognitive and behavioral responses; and (e) a subjective feeling state (Lazarus et al., 1970; Leventhal, 1979; Plutchik, 1980). While various schools of thought present differing views on the organization and classification of emotions, two primary approaches have guided much of the existing literature in communication (Nabi and Wirth, 2008): the dimensional perspective and the discrete perspective. We adopt the latter.

Emotions are caused by unique patterns of cognitive appraisals in response to events or situations (Lazarus, 1991; Roseman, 2001), with cognition being a necessary but insufficient condition of emotional reaction (Schachter and Singer, 1962). The ATF as well as other appraisal frameworks argues (Lazarus, 1991; Lerner and Keltner, 2001; Roseman, 2001) that stimuli are evaluated in terms of appraisals such as unexpectedness, valence, control, probability, and agency. Moreover, emotions are discrete and unique (Lazarus, 1991). Discrete emotions are targeted at a source or stimulus and are not defined by their valence (i.e., positive/negative). Appraisal-Tendency Framework provides a robust model for understanding the nuanced ways in which emotions influence judgment and decision-making (Lerner and Keltner, 2001). The ATF posits that emotions drive subsequent outcomes that are consistent with the emotions’ appraisal tendencies. For example, anger typically arises from appraisals of certainty and individual control, leading to tendencies for risk-seeking behavior and punitive judgments. In contrast, fear arises from appraisals of uncertainty and situational control, promoting risk-averse behavior and cautious decision-making.

But, it is well accepted that people do not experience one emotion at a time. Discrete emotions can co-occur simultaneously (Zelenski and Larsen, 2000; Harley et al., 2012). For example, research found that guilt manipulation using message stimuli often evoked anger as well as guilt (see Turner and Stephen. Rains., 2021). Moreover, individuals experience seemingly opposite emotions at the same time (Berrios et al., 2015). Certainly, primary (i.e., dominant) emotions are rapid and instinctive reactions to events, such as joy, fear, and sadness. These emotions are universal and typically associated with specific events or circumstances. Moreover, primary emotions are more intense, making them readily recognizable (Ekman, 1992, 1999; Plutchik, 2001). Primary emotions have been conceptualized to be directly felt emotional reactions in the face of a specific situation or an event, while secondary emotions are socially learned emotions about how people feel (Damasio, 1994). Feeling about a situation provides information about which appraisals are made and what behavioral and cognitive responses are more likely to be made (i.e., action tendencies; Frijda, 2007). Thus, we began with investigating the primary emotion people experienced with regard to COVID-19.

RQ1: What emotions will participants self-report as the primary emotion they experienced when thinking of COVID-19?

 Primary emotions and risk perceptions

Emotions are particularly pertinent during infectious disease crises because of their known association with risk perception (Lerner and Keltner, 2001). Risk perceptions indicate the overall level of concern people have about an issue and are comprised of the perceived severity of a threat and the perceived susceptibility to the threat (Witte, 1992). Witte (1992) and Witte et al. (1996) defined perceived severity as the extent to which individuals consider how serious the consequence of a risk would be while perceived susceptibility is conceptualized as the extent to which individuals think how vulnerable they are to the risk. Perceptions about a risk or a threat are strong when both severity and susceptibility are high; it is weak when both components are low (i.e., affect heuristic, Finucane et al., 2000; Slovic et al., 2004; Sjöberg, 2007).

Studies have shown that discrete emotions are related to risk perceptions regardless of whether the emotion was manipulated or measured (e.g., Nan, 2017; Yang and Chu, 2018). Lerner and Keltner (2001, Study 1) showed that individuals’ self-reported anger and fear led to differential outcomes for risk-taking outcomes; Angry participants were more likely to embrace risk relative to fearful participants. They also found (in Study 2) that anger was like happiness; both increased optimism whereas fear decreased optimism. In another experiment on terrorism risk perception, Lerner et al. (2003) showed that people in the anger condition had lower perceived risk of future terrorism, probability for risky events in the coming year, and probability of risk events for the average American in the coming year, relative to those in the fear condition.

In this study we further examine the effect of non-manipulated, organically felt anger and anxiety. Anger can span from slight irritation to intense fury (Spielberger et al., 1983). Anxiety, on the other hand, is described as “an unpleasant emotional state or condition which is characterized by subjective feelings of tension, apprehension, and worry (Spielberger, 1972).” Transition of focus to anxiety is because comparing the impact of fear on risk perceptions to that of anger may be limited due to the uncertain and ambiguous nature of the COVID-19 pandemic (for review, see Koffman et al., 2020). Ambiguity and uncertainty are the cognitions that characterize the feeling of anxiety while fear comes from a sense of imminent danger and threat (Lazarus, 1991). While anxiety is distinctive from fear (Sylvers et al., 2011), people who are anxious about COVID-19, like those who are frightened, may have greater risk perceptions than those who feel anger as their primary emotion. Thus,

H1a and b: Those reporting anxiety as their primary emotion will have greater perceptions of (a) severity and (b) susceptibility to COVID-19 relative to those reporting anger as the primary emotion.

Although the relationship between anger and fear on risk perception has been examined, less is known about other emotions’ relationship with risk perception. Here, we will examine an array of other primary emotions experienced during the COVID-19 pandemic.

RQ2a and b: Will severity (a) and susceptibility (b) vary by other self-reported primary emotions?

 Primary emotions and efficacy beliefs

Emotions also are associated with perceived control. Lerner and Keltner (2001) operationalized the appraisal themes of control as the personal belief that one can control the risks they encounter. Although they did not use the term ‘self-efficacy,’ their definition of perceived control is similar to Bandura (1982) concept of self-efficacy, which is the personal confidence one has that they can enact a behavior; in this case, a risk-management behavior. Their work (Study 4) showed that anger increased perceptions of control and certainty, along with optimistic risk assessments (i.e., lower risk perception) relative to fear. According to the finding, it is possible that those who felt anger when it comes to COVID-19 hold greater efficacy beliefs than those with anxiety. Witte (1992) separated two types of efficacy beliefs; Self-efficacy is the extent to which one believes they perform a suggested action, and response-efficacy refers to the confidence one has in that the suggested action can resolve the problem situation (Witte, 1992). Importantly, efficacy beliefs need to be contextualized with specific behaviors that can potentially fix a problem situation, such as mask wearing and physical distancing in the COVID-19 pandemic.

Although there are limited existing studies that have examined the effect of emotions on one’s efficacy beliefs (i.e., relationships with directions), several studies have explored the relationship between emotions and self-efficacy with cross-sectional data. For instance, happiness was found to be positively associated with self-efficacy (e.g., Hunagund and Hangal, 2014). Shame was negatively associated with self-efficacy, but there was no relationship between guilt and self-efficacy (Baldwin et al., 2006). Moreover, few studies in the public health domain have examined the relationship between discrete emotions and self-efficacy to engage in prevention behaviors but not direct relationship between discrete emotion and efficacy beliefs, which seems to be a critical gap in academic literature.

Thus, we advanced hypotheses and research questions below.

H2a and b: Those who reported anxiety as their primary emotion will have lower (a) self-and (b) response-efficacy beliefs about mask wearing relative to anger.

RQ3a and b: Will (a) self-and (b) response-efficacy beliefs about mask wearing vary by other primary emotions?

H3a and b: Those who reported anxiety as their primary emotion will have lower (a) self- and (b) response-efficacy beliefs about physical distancing relative to anger.

RQ4a and b: Will (a) self- and (b) response-efficacy beliefs about physical distancing vary by other primary emotions?

 Emotion and health preventive behaviors

“Emotions play a central role in the significant events of our lives…. Much of what we do and how we do it is influenced by emotions and the conditions that generate them (Lazarus, 1991, p. 3).” In other words, experienced emotions stimulate a coping response or problem-solving activity (Lazarus, 1991). Such action tendencies are varied by discrete emotions, namely, anger’s tendency is antagonistic (attack) whereas fear can lead to avoidance (if efficacy beliefs are low, see Witte, 1992). Hence, emotions experienced during the COVID-19 pandemic should be related to further self-reported information and prevention behaviors.

Shen and Dillard (2007) pointed out that all action tendencies driven by discrete emotions are some forms of approach/engagement or inhibition/withdrawal. For example, revulsion tends to be an avoidance emotion, while anger tends to be an approach emotion. Fear is predicted to be an avoidance emotion and happiness an approach emotion (see Table 1). In this case, we could argue that fear leads people to want to avoid the virus and its implications. Witte (1992) called this “danger control” and argued that it occurs when fear is accompanied by high efficacy. The present study is not examining fear appeals per se, but we have reason to believe that anxiety could lead to stronger risk perceptions (more on this subsequently) and more prevention behaviors. Anger is an attack emotion and thus is associated with redemption-oriented behavior (Turner, 2007). Angry audiences may focus on “revenge” for those causing their anger and have less focus on protecting themselves (relative to the fear audience).

Core appraisal themes, action tendency and risk perception of emotions.

Discrete emotion Core appraisal theme from Lazarus (1991) Action tendency Risk perception from ATF
Negative Anger Being demeaned Attack on the blamable agent Low risk perception
Sadness Irrevocable lossHelplessness – –
Anxiety Ambiguity or uncertainty Avoidance or escape High risk perception
Disgust Indigestible object/idea Contact avoidance Low risk perception
Shame Failure of ideal self Concealment Low risk perception
Guilt Moral transgressionSelf-blame Expiation or reparation Low risk perception
Positive Happiness Realization of goals Expressing pleasure Low unknown risk perception
Contentment Satisfaction – –
Pride Enhancement of ego Low unknown risk and dread risk perception

H4a and b: Those who report anxiety as their primary emotion will show greater intention (a) to wear masks and (b) to engage in physical distancing than those who self-report anger as their primary emotion.

Also, we advanced research questions below.

RQ5a and b: Will behavioral intention (a) to wear mask and (b) to engage in physical distancing vary by other primary emotions?

 Study 1

Study 1 was conducted with a university sample, and therefore was primarily comprised of young adults. Although we do not predict differences by sample type, we acknowledge the distinctions in risk for COVID-19 by age group (e.g., Gold et al., 2020). Young adults have been attributed to rapid spread of the COVID-19 (Boehmer et al., 2020; Hutchins et al., 2020). This may be because of their optimistic biases about their own vulnerability to the virus, and higher likelihood to engage in risky behaviors like hosting large parties and reduced mask wearing (e.g., Commodari and La Rosa, 2020; Fathian-Dastgerdi et al., 2021; Shukla et al., 2021). To understand what emotions they experience with regard to COVID-19 and its impact on their cognitive and behavioral outcomes received less attention.

Methods Participants and procedures

We ascertained a sample of young adults from a probability-based survey conducted with undergraduate students at a large Midwestern university, in early July 2020 (N = 1,658). A random sample of 8,000 students was obtained from the Registrar’s Office. Male students were oversampled at a rate of 1.5 due to the tendency for male participants to exhibit a lower response rate in online surveys compared to female participants (Porter and Umbach, 2006). The survey instrument measured primary emotions, risk perceptions, behavior intentions, efficacy beliefs, and demographics.

Data collection occurred between July 7 and July 17, 2020. Respondents received one email request to participate in the study ($10 e-card incentive). The study was exempted by the IRB. A total of 1,658 respondents submitted a survey. Participants (1) who completed less than 60% of the survey, (2) did not report the most demographic information and/or (3) aged over 29 were eliminated from the data. Participants who reported pride and guilt as primary emotions were not used for the main analyses due to the small group size (less than 1%). As the statistical software that we used for the main analyses dropped a case with a missing value in the variables of interests (i.e., listwise deletion), the final sample size for the main analysis was 1,368.

Most participants were around 20 years old (M = 20.28, SD = 2.63) ranging 18–29 and slightly over half of the sample were female (55.5%). The majority of the sample identified themselves as a White/Caucasian (78.1%), followed by Asians (14.5%). Slightly over 1% of the sample have tested positive in the COVID-19 (n = 20). More detailed information on demographics is presented in Table 2.

Descriptive statistics of demographics for Study 1.

n %
Age
18–19 688 50.3
20–24 552 40.4
25–29 128 9.4
Biological sex
Female 759 55.5
Male 609 44.5
Race
White 1,068 78.1
Asian 198 14.5
African American 83 6.1
Hispanic 82 6.0
COVID Diagnosis
Yes 20 1.3

Missing values are listwise-deleted.

 Measurement

The reliability of measures was calculated using Cronbach alpha when the items were more than or equal to three. If the 2 items were used to measure a construct, we used Spearman-Brown reliability estimate (Eisinga et al., 2013).

Primary emotions

Primary emotions were measured using a single item, “If you had to use one word to describe what you are mainly feeling with regard to COVID-19 right now, what would it be?” We chose emotions based on their distinctions in appraisal patterns (Table 1). This allows us to test propositions of the ATF and provide data on emotions not studied previously. Participants were asked to choose one of the following eight emotions (e.g., sadness, happiness, anger, anxiety, pride, guilt, surprise, and shame). Six emotions (e.g., sadness, happiness, anger, anxiety, surprise, and shame) were used for the main analysis as less than 1% of participants indicated that they felt proud or guilty as a primary emotion related to COVID-19.

 Risk perceptions

Measures from Witte et al. (1996) were modified and used to measure perceived severity and perceived susceptibility. Perceived severity was assessed by two items on a scale of 0 to 100 (0 = Not at all, 100 = Extremely) (e.g., “How serious of a health threat is COVID-19”; ρ = 0.86). Three items were used to measure perceived susceptibility and were responded on a 0 to 100 scale (e.g., “How worried are you about getting COVID-19 in the future?”; ɑ = 0.89).

 Efficacy beliefs

Efficacy beliefs were measured on a 5-Likert scale (1 = Strongly disagree, 5 = Strongly agree). Each efficacy belief including self-efficacy and response efficacy for mask wearing and physical distancing were measured with two items, respectively, based on the previous study (Witte et al., 1996, p. 92). A total of eight items were used to measure four types of efficacy beliefs (ρ for self-efficacy of mask wearing = 0.82, ρ for response-efficacy of mask wearing = 0.81, ρ for self-efficacy of physical distancing = 0.77, ρ for response-efficacy of physical distancing = 0.81). The example item of self-efficacy includes “I am confident that I can [engage in protective behaviors] when I leave my residence in the town.” The example item of response efficacy includes “I am less likely to spread COVID-19 to others at the university by [engaging in a protective behavior] on and around campus.”

 Behavioral intentions

Behavioral intentions were assessed using modified items from the theory of reasoned action and theory of planned behavior literature (Ajzen and Fishbein, 1970; Ajzen, 1985). Participants were asked to report the percentage of the time they are willing to engage in protective behaviors at the university in the near future on a scale of 0 to 100 (0 = 0%, 100 = 100%). Behavioral intention to engage in mask wearing was measured with three items, including “wear a mask when you are going to class (ρ = 0.89).” Behavioral intention to engage in physical distancing was measured with two items, such as “stay at least 6 feet away from students in my classes” (ρ = 0.69).

 Covariates

Studies have indicated that age, sex, diagnosis of COVID-19, and political affiliation were highly correlated with the COVID-19 risk perception (Dryhurst et al., 2020; Kiviniemi et al., 2022; Martelletti et al., 2022). Yet, given that participants are young adults and only 1.3% of the sample had been diagnosed with the COVID-19, only biological sex and political affiliation were controlled. Political affiliation was measured using a 11-point Likert scale (0, Liberal, 10, Conservative, M = 3.70, SD = 2.77).

 Results

The first research question asked about the distribution of primary emotions. Anxiety was reported as a primary emotion by about half of the participants (49.4%), followed by sadness (17.9%) and anger (16.9%) (Table 3). Interestingly, about 5% of participants indicated happiness as their primary emotion.

Descriptive statistics of variables for Study 1.

Variables n (%) Variables M SD
Primary emotion Severity 67.58 25.14
Anxiety 676(49.4) Susceptibility 56.94 27.37
Sadness 245(17.9) SE (Mask) 4.23 0.99
Anger 231(16.9) SE (Distancing) 3.39 1.15
Surprise 89(6.5) RE (Mask) 4.39 0.90
Happiness 67(4.9) RE (Distancing) 4.35 0.91
Shame 60(4.4) BI (Mask) 81.98 23.54
BI (Distancing) 76.69 23.30

SE, self-efficacy; RE, response-efficacy; BI, behavioral intention. Pride and guilt are not reported due to the small percentage of people reporting them as a primary emotion. Responses from participants with top six primary emotions are analyzed.

Hypotheses predicted greater risk perceptions (H1a, H1b), lower self- and response efficacy beliefs (H2a, H2b, H3a, and H3b), and greater behavioral intentions (H4a and H4b) among those who felt anxiety as a primary emotion, compared to anger. A series of ANCOVAs and post hoc (i.e., Bonferroni) analyses were conducted. For all dependent variables, the omnibus F test results were significant at the 𝛼 level of 0.001 (see Table 4).

Effect of discrete emotions on outcomes for Study 1.

Outcomes Discrete emotions: M (SD) F (df) η2
Anxiety Anger Sadness Surprise Happiness Shame
Severity 72.34 (0.87)ab 56.47 (1.48)cd 66.22 (1.43)ade 65.06 (2.42)abde 63.64 (2.78) abde 70.16 (2.91) abde 17.90 (5,1344) 0.05
Susceptibility 65.01 (25.55)a 43.94 (27.09)b 54.58 (25.44)c 45.97 (26.10)abc 43.86 (28.39) abc 54.50 (25.30) abc 23.68 (5,1331) 0.07
SE (Mask) 4.35 (0.87) 3.76 (1.25)a 4.26 (0.93) 4.24 (0.96) 4.27 (1.03) 4.49 (0.95) 11.35 (5,1359) 0.04
SE (Distancing) 3.44 (1.14) 3.07 (1.18)a 3.40 (1.07) 3.60 (1.11) 3.69 (1.18) 3.38 (1.26) 5.37 (5,1359) 0.02
RE (Mask) 4.50 (0.74) 3.99 (1.16)a 4.46 (0.86) 4.34 (0.85) 4.28 (1.07) 4.56 (0.91) 10.10 (5,1360) 0.03
RE (Distancing) 4.49 (0.75) 3.94 (1.20)a 4.33 (0.92) 4.38 (0.83) 4.35 (0.97) 4.51 (0.89) 13.37 (5,1360) 0.04
BI (Mask) 83.66 (22.12) 71.86 (28.57)a 84.39 (20.83) 80.29 (22.69) 85.12 (21.60) 90.06 (22.04) 9.67 (5,1304) 0.03
BI (Distancing) 78.03 (22.66) 69.38 (25.31)a 77.62 (21.71) 75.37 (24.10) 80.43 (23.05) 82.22 (23.39) 4.97 (5,1279) 0.02

SE, self-efficacy; RE, response-efficacy; BI, behavioral intention. Subscripts indicate the results of Bonferroni tests at alpha level of 0.05. All models were significant at alpha level 0.001, which meets Bonferroni-corrected p-value (p = 0.003).

Pairwise comparison results indicate that the anxiety group felt greater severity, MD = 15.88, p < 0.001, 95% CI = [10.80, 20.96], greater susceptibility, MD = 18.39, p < 0.001, 95% CI = [12.67, 24.11], greater self-efficacy beliefs about mask wearing, MD = 0.51, p < 0.001, 95% CI = [0.30, 0.72], greater response-efficacy beliefs about mask wearing, MD = 0.44, p < 0.001, 95% CI = [0.25, 0.63], greater self-efficacy beliefs about physical distancing, MD = 0.33, p < 0.001, 95% CI = [0.07, 0.59], greater response-efficacy beliefs about physical distancing, MD = 0.53, p < 0.001, 95% CI = [0.33, 0.71], greater behavioral intention to wear masks, MD = 10.23, p < 0.001, 95% CI = [5.14, 15.46], and greater behavioral intention to keep physical distancing, MD = 7.41, p < 0.001, 95% CI = [2.06, 12.77], compared to the angry group. Thus, the data were consistent with hypotheses about severity, susceptibility, and behavioral intentions, but were not consistent with hypotheses about efficacy beliefs.

The second set of research questions were about the effect of discrete emotions on risk perceptions (RQ2a and RQ2b), self- and response-efficacy beliefs (RQ3a, RQ3b, RQ4a, and RQ4b), and behavioral intentions (RQ5a and RQ5b). The results showed that there were systematic differences in severity, susceptibility, self- and response-efficacy about mask wearing and physical distancing, and behavioral intention to mask wearing and physical distancing varied by the type of primary emotions (Table 4). Severity was high among those whose primary emotion was anxiety and shame, while low severity was shown in the angry and happy groups. The highest susceptibility was observed among anxious people. Four types of efficacy beliefs were systematically high in the anger group, while the other emotion group did not differ in efficacy beliefs, except for the self-efficacy beliefs of physical distancing among shame group. Despite high self- and response-efficacy beliefs, behavioral intention to wear mask and keep physical distancing was low among those who felt anger as a primary emotion.

 Discussion

Young adults aged 18–29 years are more likely to engage in risky behaviors and practice fewer preventive behaviors than other age groups (Hutchins et al., 2020). Facing the pandemic, accumulated investigations have shown this pattern worldwide (e.g., Boehmer et al., 2020; Commodari and La Rosa, 2020; Bates et al., 2021). To understand the precursors to their cognitions and preventive behavioral intention, we examined the impact of primary emotions on young adults’ risk perceptions about the COVID-19 as well as efficacy beliefs and willingness to practice two preventive behaviors: mask wearing and physical distancing. We wanted to assess the predictive validity of the ATF in a natural setting.

Most respondents reported negative emotions as their primary emotion: anxiety, sadness, and anger, which together accounted for over 83% of the sample. Interestingly, a notable proportion (4.9%) of participants reported happiness, a positive emotion, as their primary emotion. The underlying reason people feel happy during a global pandemic is understudied. However, given that happiness involves sense of goal congruence and positive future expectation (Lazarus, 1991), the finding implies that unforeseen aspects of COVID-19 and changes in lifestyle (i.e., college lockdown) may have contributed to the pursuit of young adults’ personal goals. For example, for some young adults, school lockdown and the shelter-in-place (SIP) order could be perceived as an opportunity for family reconnection and saving living expenses. Moreover, the transition to online curriculum could provide more flexible time management and lessen anxiety from socializing with others.

Consistent with past experimental research (Lerner and Keltner, 2001), anger was distinctive from anxiety in risk perceptions and behavioral intentions. Angry individuals had lower risk perceptions compared to anxious and sad people. The ATF posits that anger is associated with perceived control over the situation, thereby associated with lower risk perception. Considering the relationship between anger and optimistic bias (Hemenover and Zhang, 2004), these findings are reasonable. An interesting discovery was that anxious young adults exhibited stronger efficacy beliefs compared to angry ones, contradicting the initial predictions.

 Study 2 Overall context

Study 1 was not without limitations. First, although the sample was probability-based, it still only represents one specific population: young adults at a university. Second, Study 1 did not include information seeking measures, which is a critical variable predictive of future proactive prevention behaviors (Turner et al., 2006). Third, preventive behavioral intention was exclusively examined. To address these issues, we conducted Study 2 with a nationally representative sample across the U.S. and including each preventive behavior and information seeking and avoidance measures in our questionnaire. We replaced behavioral intention with behavior for H4a, H4b, RQ5a, and RQ5b.

Anxiety and information seeking has been proposed to be positively related in various models (for review, see Kahlor, 2010). It has been hypothesized that anxiety/worry leads people to seek information to reduce (or avoid) their emotional state. Turner et al. (2006) examined this anxiety-reduction hypothesis and found that participants’ high-risk perception with low efficacy increased anxiety, which subsequently increased their information-seeking behavior. In the same vein, Nabi’s (2003) experimental data indicated that both fearful and angry participants desired information, but content preferences differed: the anger group preferred retributive information and the fear group desired protection-related information.

Likewise, individuals who are anxious about COVID-19 may have different levels of COVID-19 related information seeking and information avoidance from those who felt anger. According to Ahn et al. (2021), anxiety was positively related to COVID-19 information seeking but anger was not. In contrast, anger was positively associated with COVID-19 related information avoidance, while anxiety was not. Given that anxiety is characterized as a high uncertainty emotion (Lazarus, 1991), individuals could be engaged in more information seeking and less information avoidance as they felt anxious compared to those who felt anger.

H5a and b: Those who report anxiety as their primary emotion will show (a) greater information seeking and (b) lower information avoidance than those who self-report anger as their primary emotion.

Also, the effect of other emotions can be asked,

RQ7a and b: Will (a) information seeking and (b) information avoidance vary by other primary emotions?

 Methods Participants and procedures

A rolling-cross sectional survey was conducted using a national quota-based sample of adults aged 18 or older residing in the U.S. during the COVID-19 pandemic from July 6, 2020, to October 16, 2020.1 A total of 32,489 responses were collected from the national Qualtrics panel. Participants who did not pass attention check (n = 12,447),2 did not complete responses (n = 11,202), or indicated that they believe that COVID-19 is fake/hoax (n = 2,424) were excluded. As we did in Study 1, primary emotion groups with small group size (happiness, shame, and guilt, less than 2% of the sample) were excluded from the main analyses The final data used for the main analyses had 8,454 responses.

The participants were on average 45.52 years old (SD = 17.73, ranged 18–93 years). About a third of the sample ranged between 18–34, 35–55, and over 55 years in their ages (Table 5). 51.3% of the respondents identified as biological female (n = 4,489). The majority were White/Caucasians (70.4%), followed by Black/African Americans (16.4%) and Asians (6.8%). Almost 70% of the participants studied (or have studied) in college. Democrats accounted for 36.0% of the participants, whereas Republicans made up of 28.8%. A significant proportion of the participants reported others around them tested positive with COVID-19, but only 0.2% indicated they were infected by COVID-19.

Descriptive statistics of demographics for Study 2.

n %
Age
18–34 2,857 32.5
35–55 3,053 34.8
55+ 2,868 32.7
Biological Sex
Female 4,469 50.9
Male 4,250 48.4
Other 59 0.7
Household Income
<30 k 2,531 28.8
30 k ≦ and <60 k 2,436 27.8
60 k ≦ and <90 k 1,605 18.3
90 k ≦ 2,206 25.1
Race
White 6,179 70.4
Black 1,443 16.4
American Indian 168 1.9
Asian 597 6.8
Native Hawaiian 43 0.5
Others 348 4.0
Education
Less than high school 309 3.5
High school graduate 2,345 26.7
Some College 1,734 19.8
Associates degree 949 10.8
Bachelor’s degree 2,040 23.2
Master’s degree 1,084 12.3
Doctoral degree 135 1.5
Professional degree 182 2.1
Political party
Republican 2,530 28.8
Democrat 3,160 36.0
Other 3,088 35.2
Diagnosed with COVID (Self)
Yes 190 2.2
No 8,540 97.3
Other 48 0.5
Diagnosed with COVID (Others)
Yes 3,420 39.0
No 5,325 60.7
Other 33 0.4

Missing values are listwise-deleted.

The participants who agreed and signed a consent form were asked to fill out questions in the following order: demographics, primary emotions about the COVID-19, risk perceptions, self- and response-efficacy about two COVID-19 preventive behaviors, masking wearing and physical distancing, and COVID-19-related information seeking and avoidance. Additionally, attention check questions and covariates items were randomly shown to respondents.

 Measurement

The internal consistency of measures was calculated using Cronbach’s alpha when there were 3 or more items. If the items were fewer than three, the Spearman–Brown reliability estimate between items were calculated instead. Also, when the number of items was more than four, a confirmatory factor analysis was conducted to validate the measure.

Primary emotions

Primary emotions were measured using a single item, “If you had to use one word to describe what you are mainly feeling with regard to COVID-19, what would it be?” Participants were asked to choose one of the following 10 emotions (sadness, happiness, contentment, anger, anxiety, pride, guilt, surprise, disgust, and shame). We added contentment and disgust to expand emotional dimensions (Russell, 1980). Again, the top six emotions (sadness, happiness, anger, anxiety, surprise, and shame) were used for the main analysis as less than 3% of participants indicated that they felt happiness, pride, guilt, and shame as a primary emotion related to COVID-19.

 Risk perceptions

Perceived susceptibility and severity were measured using a 0–100 scale (0 = not at all, 100 = extremely). Items were adopted from Witte et al. (1996). Two items were used to capture severity (ρ = 0.82), including “COVID-19 is a serious health threat.” To measure susceptibility, two items were used (ρ = 0.77, p < 0.001), including, “When I think carefully about my lifestyle, it does seem possible that I could get COVID-19.”

 Efficacy beliefs

Efficacy beliefs were measured on a 0 to 100 scale (0 = Strongly disagree, 100 = Strongly agree). Response efficacy beliefs related to physical distancing were measured with three items (ɑ = 0.89), including “if I engage [or continue to] in physical distancing, I will be less likely to get COVID-19.” Other efficacy beliefs including self-efficacy and response efficacy for mask wearing and self-efficacy related to physical distancing were measured with two items, respectively, based on the previous study (Witte et al., 1996) (ρ for self-efficacy of mask wearing = 0.82; ρ for response-efficacy of mask wearing = 0.84; ρ for self-efficacy of physical distancing = 0.76). The example item of self-efficacy includes “I am confident that I can [engage in protective behaviors].” The example item of response efficacy related to mask wearing includes “I am less likely to spread COVID-19 to others if I wear (or continue to) a mask in public.”

 Information seeking and avoidance

Information seeking and avoidance behaviors were measured on a scale from 0 to 100 (0 = Strongly disagree, 100 = Strongly agree). Items from Yang and Kahlor (2013) were modified. Two items for information seeking (ρ = 0.88, p < 0.001) include “I regularly seek out information about COVID-19.” Information avoidance was measured using two items (ρ = 0.74, p < 0.001) including “I ignore information about COVID-19.”

 Preventive behaviors

Participants reported their behaviors on a scale from 0 to 100 (0 = 0% of the time, 100 = 100% of the time) using measures specifically developed for this study. 3 Mask wearing behaviors were measured with four items (ɑ = 0.76), including “I wear a mask when I am in a public place.” Physical distancing was measured with seven items (ɑ = 0.89), including “I generally stay away from people.”

 Covariates

Age was a covariate along with sex and political affiliation in Study 2 based on their relationships with one’s risk perception, efficacy beliefs, information management, and preventive behaviors (Dryhurst et al., 2020; Kiviniemi et al., 2022; Martelletti et al., 2022).

 Results

The first research question asked about the distribution of primary emotions. Anxiety showed the largest proportion, accounting for more than 40% of the total responses. Sadness, disgust, and anger followed (see Table 6). Emotions that were chosen by less than 2% of participants were excluded from further analyses: happiness (n = 137, 1.6%), shame (n = 136, 1.5%), and guilt (n = 18, 0.2%). The number of responses with these top six primary emotions was 8,454.

Descriptive statistics of variables for Study 2.

Variables n (%) Variables M SD
Primary emotion Severity 78.83 25.50
Anxiety 3,779(43.1) Susceptibility 45.97 26.99
Sadness 1,582(18.0) SE (Mask) 81.53 25.67
Disgust 1,083(12.3) RE (Mask) 77.35 27.94
Anger 1,054(12.0) SE (Distancing) 79.94 23.47
Contentment 640(7.3) RE (Distancing) 80.90 24.56
Surprise 316(3.6) Mask wearing 65.61 27.88
Social distancing 79.09 21.82
Information Seeking 59.08 31.22
Information Avoiding 23.28 25.51

Other emotions, happiness, guilt, and pride are not reported. SE, self-efficacy; RE, response-efficacy. Due to the small percentage (less than 3%) of people reporting them as a primary emotion. Responses from participants with top six primary emotions are analyzed.

Hypotheses predicted that those who reported anxiety as a primary emotion tend to hold greater risk perceptions (H1a and H1b), lower self- and response-efficacy beliefs (H2a, H2b, H3a, and H3b), greater preventive behaviors (H4a and H4b), greater information seeking (a), and lower information avoidance (H5b) than those with anger. A series of ANCOVAs and post hoc (i.e., Bonferroni) analyses were conducted. The omnibus F test results were significant at the 𝛼 level of 0.001 for all dependent variables (see Table 7).

Effect of primary emotions on outcomes in Study 2 (N = 8,454).

Outcomes Discrete emotions: M (SD) F (df) η2
Anxiety Anger Sadness Disgust Contentment Surprise
Severity 85.20 (19.44)a 72.52 (30.05)bc 82.71 (21.10)a 65.62 (32.23)de 67.92 (30.12)cde 71.75 (25.83)bcd 159.26 (5,8445) 0.08
Susceptibility 50.07 (24.85)a 43.63 (28.11)bc 47.20 (27.57)d 39.44 (28.36)be 36.13 (27.46)be 40.90 (28.28)bce 55.41 (5,8445) 0.03
SE (Mask) 86.30 (19.59)a 74.11 (31.86)b 85.83 (20.16)a 68.51(34.47)c 77.94 (29.28)d 79.52 (25.69)d 111.72 (5,8445) 0.06
SE (Distancing) 82.95 (19.50)abc 74.25 (28.63)d 81.97 (20.52)abce 73.06 (29.61)ad 78.69 (26.59)abe 78.78 (22.82)abce 47.23 (5,8445) 0.03
RE (Mask) 83.79 (20.54)a 69.36 (34.48)b 82.48 (22.46)a 60.70 (36.70)c 70.87 (31.70)b 71.55 (28.32)b 159.42 (5,8445) 0.08
RE (Distancing) 86.62 (17.43)a 7,303 (31.23)bc 84.39 (19.88)a 67.73 (32.91)d 75.76 (28.63)bce 76.81 (24.38)be 142.11 (5,8445) 0.07
Mask Wearing 69.73 (23.99)a 59.78 (31.02)b 72.67 (24.64)c 51.32 (32.51)d 58.79 (31.03)b 63.23 (27.29)b 108.83 (5,8445) 0.06
Physical Distancing 83.72 (16.41)a 73.15 (26.45)b 83.03 (17.20)a 68.23 (28.52)c 72.09 (26.97)b 75.33 (22.08)b 130.14 (5,8445) 0.07
Information Seeking 63.57 (29.22)a 53.56 (32.74)b 64.13 (29.45)a 49.33 (33.35)c 48.17 (32.04)c 54.00 (31.82)b 74.19 (5,8445) 0.04
Information Avoiding 19.10 (22.14)a 27.12 (29.10)b 23.24 (24.94)c 29.36 (28.97)b 28.91 (27.50)b 28.47 (27.11)bc 39.07 (5,8445) 0.02

SE, self-efficacy; RE, response-efficacy. Subscripts indicate the results of Bonferroni tests at alpha level of 0.05. All models were significant at alpha level 0.001, which meets Bonferroni-corrected p-value (p = 0.003).

Pairwise comparison results indicate that anxious people had greater susceptibility, MD = 6.63, p < 0.001, 95% CI = [3.90, 9.35], greater severity about the COVID-19, MD = 12.12, p < 0.001, 95% CI = [9.69, 14.55], greater self-efficacy beliefs about mask wearing, MD = 11.71, p < 0.001, 95% CI = [9.20, 14.22], greater response-efficacy beliefs about mask wearing, MD = 13.84, p < 0.001, 95% CI = [11.15, 16.52], greater self-efficacy beliefs about physical distancing, MD = 8.49, p < 0.001, 95% CI = [6.16, 10.83], greater response-efficacy beliefs about physical distancing, MD = 13.09, p < 0.001, 95% CI = [10.72, 15.46], greater mask wearing behaviors, MD = 9.39, p < 0.001, 95% CI = [6.64, 12.13], and greater physical distancing behaviors, MD = 10.08, p < 0.001, 95% CI = [7.98, 12.19] compared to angry people. The anxious people were also found to more engage in information seeking behavior, MD = 10.18, p < 0.001, 95% CI = [7.06, 13.29] and less avoid the COVID-19 information, MD = −7.37, p < 0.001, 95% CI = [−9.87, −4.86] than angry people. The data were consistent with hypotheses about risk perceptions (H1a and H1b), behaviors (H4a and H4b), and information seeking/avoidance (H5a and H5b), whereas inconsistent with hypotheses about efficacy beliefs (H2a, H2b, H3a, and H3b).

The research questions were about the effect of other discrete emotions on risk perceptions, efficacy beliefs, behaviors, and information seeking/avoidance. All models were significant at significance level of 0.001, indicating systematic differences in severity and susceptibility, self- and response-efficacy beliefs about mask wearing and physical distancing, wearing masks and practicing physical distance, and information seeking and avoiding stimulated by primary emotion (Table 7). Specifically, disgusted people showed different levels of outcomes compared to angry people, except for information avoidance. Those who felt disgust primarily against the COVID-19 scored the lowest level of severity, efficacy, and behaviors, whereas those who felt content scored the lowest level of susceptibility. The anxious people indicated higher levels of severity, susceptibility, self-and response-efficacy, physical distancing, information seeking, and lower level of information avoidance compared to other primary emotion groups. Mask wearing was highest among the sad people.

 Discussion

Study 2 replicated Study 1 with a nation-wide representative sample. Actual practice of preventive behaviors as well as information seeking/avoidance were additionally examined as outcomes of the primary emotions about COVID-19. For primary emotion measurement, we used a more detailed category, by adding disgust and contentment. In general, the results showed similar findings with Study 1.

Negative emotions were dominantly experienced as a primary emotion in Study 2. Anxiety and sadness were the most frequently reported primary emotions. In Study 2, a significant proportion of people reported disgust and contentment as their primary emotion. This provides evidence that people develop various appraisals about COVID-19, such as uncertainty, perceived loss, blame, and acceptance. Anxious and sad people showed higher levels of risk perceptions, efficacy beliefs, and preventive behaviors than others. Both greater information seeking, and less information avoidance were reported among the two groups as well. In contrast, risk perceptions among people who reported disgust and contentment as their primary emotion scored the lowest.

A key finding is the differing effects of anxiety and anger on types of efficacy beliefs. Study 2 reaffirms the findings in Study 1, unveiling the greater efficacy beliefs among anxious individuals over angry people. This consistency across the two studies underscores potential influence of anxiety on fostering a sense of personal control and effectiveness in adhering to preventive behaviors against COVID-19. Higher level of efficacy beliefs, risk perceptions, and greater practice of preventive behaviors among anxious people than those who felt anger, also make sense with the framework of fear appeal studies (Witte, 1992). Again, the level of forms of efficacy beliefs varied greatly by primary emotions, with anxious and sad people on the highest and disgusted group on the lowest.

The impact of primary emotions on both health preventive and information-related behaviors is noteworthy. When asked behavioral intentions, young adults experiencing anger (Study 1) showed lower intention to practice mask wearing and social distancing than other groups whose behavioral intentions did not differ from each other. When it comes to actual behaviors and information seeking/avoiding in general populace, those behaviors were significantly varied by what emotions they primarily experienced.

 General discussion

The U.S. adults varied greatly in how they interpreted or appraised the COVID-19 pandemic. Social media and cable news revealed many blame interpretations, whether the pandemic was due to natural causes or intentional nefarious activities as levels of uncertainty varied across populations. Informed by the ATF, we argued that these appraisal patterns cause distinctive primary emotions. Understanding what primary emotion people self-report feeling “when thinking about COVID-19” can be used to develop audience segments with distinct cognitive and behavioral patterns.

Past research has investigated the effect of discrete emotions on various health related perceptions and preventive behavior intentions. The ATF provides a useful rationale to connect discrete emotions and health related cognitions and behavioral intentions. Based on this framework, we examined what discrete emotions individuals reported as their primary emotion regarding COVID-19, and how their risk perceptions and health related behavioral intentions were varied by the primary emotions among young adults and general populace.

The first objective of this study was to assess COVID-19 related to primary emotions as audience segments with distinct outcomes. The results depicted a spectrum of emotions during the pandemic. We identified the emotions people reported as their primary emotional experience “if they had to choose one main emotion.” By using a representative sample at a university and a national sample, we mapped the distribution of various primary emotions in the context of COVID-19. It is unsurprising that negative emotions, such as anxiety, sadness, and anger, were dominantly reported. In contrast, it is somewhat surprising that a small proportion of participants selected positive emotions such as happiness or contentment. Combined with an appraisal approach, this emotional map expands understanding of cognitive and behavioral responses to the current health crises among the populace.

Our findings are consistent with controlled experiments that revealed that anger caused lower risk perceptions relative to fear or anxiety (e.g., Lerner and Keltner, 2001; Turner et al., 2006). In alignment with the outcome reported by Lerner and Keltner (2000), there were no differences in perceived severity or susceptibility between those reporting anger or happiness as primary emotions about COVID-19. This observation substantiates the assertion that emotional valence alone is a limited predictor of outcomes. While both anxiety and anger are characterized by negative valence, their associations with risk perception distinctly diverge, consistent with past findings (Lerner and Keltner, 2001). Anger and happiness are opposite valence but their relationships with risk perception did not differ. That is, using only valence to examine the effects of emotions could be problematic as it may provide limited and potentially misleading information, especially when one is interested in the relationship between emotions and risk perception.

These data showed that the function of sadness was similar to that of anxiety in its relationship with severity but not with susceptibility. This could be explained by the onus of sadness: loss (Lazarus, 1991). Experiencing loss during COVID-19 would be associated with heightened sense of severity, but this would not necessarily related to the sense of vulnerability (e.g., Fragkaki et al., 2021). This resembles an optimistic bias, which anxious people are less likely to fall prey to compared to those in other emotional states (Lerner and Keltner, 2001).

In theories regarding discrete emotions in the field of communication (e.g., Witte, 1992; Turner, 2007), efficacy beliefs have been mainly considered as a moderator changing the effect of discrete emotions on behavioral outcome. In this study, we wanted to assess efficacy beliefs as a function of discrete emotions. Interestingly, anxious young adults (Study 1) and US populations (Study 2) reported higher efficacy beliefs and preventative behaviors. The results were perplexing since anxiety stems from uncertainty (Lazarus, 1991). In the future, scholars need to examine specific micro-targets of emotion and assess if this moderates the effect of emotion on efficacy.

For communication scholars, the most important findings may be those regarding information seeking and avoiding. Turner et al.’s (2006) experiment found that anxious people are more likely to seek out information to alleviate the level of anxiety. Our findings were consistent with this pattern. The anxious audience reported higher information seeking behaviors compared to angry, disgusted, happy, or surprised audiences. They also reported the least information avoidance behaviors compared to all other groups. However, although our data are insufficient to make a causal claim, it is plausible that people who seek more information related to COVID-19 (and avoid less) might feel anxiety as a primary emotion (Charpentier et al., 2022).

We also had other interesting findings about anger. Even though anger is regarded as an approach-emotion (for review, see Carver and Harmon-Jones, 2009), these data showed that those who reported anger or disgust as their primary emotion were more likely to avoid risk information relative to those who reported other emotions. Thus, the manifestation of the approach tendencies is important. It may be that angry individuals focus on revenge-oriented behaviors and less so on information behaviors. A revenge orientation is not necessarily destructive. In fact, anger can lead people to want to change policies, fight for justice, or “right extant wrongs” (e.g., Skurka, 2019). Yet it is out of scope for this study so that it should be examined further in follow-up studies.

Interestingly, people who felt angry reported a relatively lower intention to engage in preventive behaviors, as well as lower actual preventive behaviors, compared to those experiencing other emotions. As we did not measure the target of anger, we could only speculate that angry people were mad because they felt their freedom was forfeited by being asked to engage in preventive behaviors – thus, they did not intend to engage in those behaviors even though they believed that they had ability to do so.

Lower engagement in mask wearing and physical distancing among people who felt disgusted can be understood in the same way. Even though appraisal theme of disgust has been clearly distinctive from anger (see Lazarus, 1991), laypersons use disgust for indicating broader meaning, including intense anger (Nabi, 2002). Thus, it is possible that the respondent selected disgust to emphasize how much they feel angry toward the COVID situation.

Even though the behavioral intentions of the anxiety group were not statistically different from those who felt sadness, happiness, and shame in Study 1, these sad, happy and shameful people had higher mean values in willingness to wear mask and to keep physical distance than those of anxiety group. This implies that appraisal cognitions such as loss, achievement, and self-concept can be associated with a greater intention to engage in COVID-19 preventive behaviors, potentially even more than the uncertainty activated by COVID-19 (Lazarus, 1991).

These investigations also provide input as to the potential audience segments that emerge during an infectious disease pandemic. These data make clear that people appraised COVID-19 differently, with those distinct appraisal patterns manifesting into distinct primary emotions felt by audiences. Unique discrete emotions lead to distinct patterns in critical outcomes like risk perception, efficacy, information seeking and avoiding, and prevention behavior. Given the findings of the current study, we encourage health practitioners to utilize one’s felt emotion as a tool for audience segmentation. Our findings suggested that people experiencing distinct emotions appraise the risk in a different way, thus they need a unique intervention.

Some limitations should be noted. Despite the importance of primary emotional experience of COVID-19 in this study, it is unclear what aspects of COVID-19 led people to feel their primary emotions. For example, individuals may primarily experience anger when primed with the others non-compliance of preventive behaviors. Mandating e policy may invoke the feeling of restriction in freedom as well. People can have sad feelings because of others’ pain or their own loss. Accordingly, future studies should examine the source of the emotional experience.

Even though the findings overlap with past findings from experiments with manipulated emotions, a dominant discrete emotion does not rule out the presence of other emotional experiences. People may feel mixed emotions, such as both anger and anxiety, about COVID-19. Having both positive and negative emotions is also possible (Diener and Emmons, 1984). Future research would benefit by examining the net effect of discrete emotions and the interplay between more than two emotions on risk perceptions, diverse efficacy beliefs, and behaviors.

Although a vaccine for COVID was not available at the time of the data collection, in the future researchers should examine how primary (or mixed) emotions affect vaccine uptake. In such a case, they would need to assess different targets/sources such as emotions about getting vaccinated versus emotions about others not getting vaccinated. Given that COVID vaccine hesitancy is still high (approximately 27% of the general population, Troiano and Nardi, 2021), it is imperative that we understand if distinct emotions covary with hesitancy. Troiano and Nardi (2021) reported that personal beliefs and politics are important predictors of vaccine hesitancy, and they are also related to emotions. Additionally, recent research by Duradoni et al. (2022) revealed that the emergence of long COVID reduced vaccine hesitancy, it did not reduce their fear of the vaccination. This implies that longitudinal emotion work is needed.

 Conclusion

The empirical evidence from this study reinforces the ATF literature, especially demonstrating that anxiety is related to higher risk perceptions and more proactive health behaviors compared to anger, which is associated with lower risk perceptions and a diminished likelihood of engaging in preventive behaviors. These insights underscore the necessity of considering emotional responses in public health messaging and interventions to enhance their effectiveness. The research also provides a nuanced understanding of how discrete emotions can be related to individual responses to public health crises, highlighting the importance of tailoring communication strategies to different emotional states to improve public health outcomes.

 Data availability statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

 Ethics statement

The studies involving humans were approved by Michigan State University Institutional Review Board. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

 Author contributions

MT: Writing – original draft, Writing – review & editing. JL: Data curation, Writing – original draft, Writing – review & editing. YJ: Data curation, Writing – original draft, Writing – review & editing. RH: Writing – original draft, Writing – review & editing. QY: Writing – original draft, Writing – review & editing. MK: Writing – original draft, Writing – review & editing. ML: Data curation, Funding acquisition, Resources, Writing – review & editing. T-QP: Data curation, Funding acquisition, Resources, Writing – review & editing.

 Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This study was supported by the National Science Foundation Rapid Award DRMS (#2029633) and the USDA National Institute of Food and Agriculture (Hatch project ##1018574).

 Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

 Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

1This project was part of a larger funded grant. To ensure we controlled for incidence rates, we collected data in the top 10 US states with regard to COVID prevalence. We also collected data from a random selection of 5 states with moderate prevalence and 5 states with low prevalence.

2To enhance data quality and identify insincere respondents, we included a single item attention check in the middle of survey questionnaire.

3Those items asked what percent of time respondent do a specific behavior. Four items for mask wearing are “I wear a mask when I am out,” I wear a mask when exercising outdoors (walking, riding a bike or running for example),“I wear a mask when I exercise at the gym,” and “I wear a mask when I am in a public place (like a restaurant or park).” Items for physical distancing behavior are “I generally stay away from people,” “I limit my trips out to only essential activities,” “I engage in physical distancing (at least 6 feet from others; as defined in the instructions),” “I avoid handshaking with people outside your home/germ circle,” “I engage in quarantining (staying at home except for essentials),” “I avoid contact with people who are high risk for COVID-19 (elders, those with heart disease, etc.),” and “I reduce my personal travel.”

 References Ahmadi K. Ramezani M. A. (2020). Iranian emotional experience and expression during the COVID-19 crisis. Asia Pac. J. Public Health 32, 285–286. doi: 10.1177/1010539520937097, PMID: 32633555 Ahn J. Kim H. K. Kahlor L. A. Atkinson L. Noh G.-Y. (2021). The impact of emotion and government trust on individuals’ risk information seeking and avoidance during the COVID-19 pandemic: a cross-country comparison. J. Health Commun. 26, 728–741. doi: 10.1080/10810730.2021.1999348, PMID: 34779340 Ajzen I. (1985). “From intentions to actions: A theory of planned behavior” in Action control: From cognition to behavior. eds. Kuhl J. Beckmann J. (Berlin Heidelberg: Springer-Verlag), 11–39. Ajzen I. Fishbein M. (1970). The prediction of behavior from attitudinal and normative variables, J. Exp. Soc. Psychol. 6, 466–487. doi: 10.1016/0022-1031(70)90057-0 Bai Y. M. Lin C.-C. Lin C.-Y. Chen J.-Y. Chue C.-M. Chou P. (2004). Survey of stress reactions among health care workers involved with the SARS outbreak. Psychiatr. Serv. 55, 1055–1057. doi: 10.1176/appi.ps.55.9.1055, PMID: 15345768 Baldwin K. M. Baldwin J. R. Ewald T. (2006). The relationship among shame, guilt, and self-efficacy. Am. J. Psychother. 60, 1–21. doi: 10.1176/appi.psychotherapy.2006.60.1.1, PMID: 16770913 Bandura A. (1982). Self-efficacy mechanism in human agency. Am. Psychol. 37, 122–147. doi: 10.1037/0003-066X.37.2.122 Bates B. R. Tami A. Carvajal A. Grijalva M. J. (2021). Knowledge, attitudes, and practices towards COVID-19 among Venezuelans during the 2020 epidemic: an online cross-sectional survey. PLoS One 16:e0249022. doi: 10.1371/journal.pone.0249022, PMID: 33857159 Berrios R. Totterdell P. Kellett S. (2015). Eliciting mixed emotions: a meta-analysis comparing models, types, and measures. Front. Psychol. 6:428. doi: 10.3389/fpsyg.2015.00428, PMID: 25926805 Boehmer T. K. Devies J. Caruso E. Van Santen K. L. Tang S. Black C. L. . (2020). Changing age distribution of the COVID-19 pandemic — United States, may–august 2020. MMWR Morb. Mortal Wkly. Rep. 69, 1404–1409. doi: 10.15585/mmwr.mm6939e1, PMID: 33001872 Bults M. Beaujean D. J. M. A. de Zwart O. Kok G. van Empelen P. van Steenbergen J. E. . (2011). Perceived risk, anxiety, and behavioural responses of the general public during the early phase of the influenza a (H1N1) pandemic in the Netherlands: results of three consecutive online surveys. BMC Public Health 11:2. doi: 10.1186/1471-2458-11-2, PMID: 21199571 Carver C. S. Harmon-Jones E. (2009). Anger is an approach-related affect: evidence and implications. Psychol. Bull. 135, 183–204. doi: 10.1037/a0013965, PMID: 19254075 Charpentier C. J. Dezza I. C. Vellani V. Globig L. K. Gädeke M. Sharot T. (2022). Anxiety increases information-seeking in response to large changes. Sci. Rep. 12:7385. doi: 10.1038/s41598-022-10813-9, PMID: 35513397 Commodari E. La Rosa V. L. (2020). Adolescents in quarantine during COVID-19 pandemic in Italy: perceived health risk, beliefs, psychological experiences and expectations for the future. Front. Psychol. 11:559951. doi: 10.3389/fpsyg.2020.559951, PMID: 33071884 Damasio A. R. (1994). Descartes' error and the future of human life. Sci. Am. 271:144. doi: 10.1038/scientificamerican1094-144 Diener E. Emmons R. A. (1984). The independence of positive and negative affect. J. Pers. Soc. Psychol. 47, 1105–1117. doi: 10.1037/0022-3514.47.5.1105 Dixon T. (2012). “Emotion”: the history of keyword in crisis. Emot. Rev. 4, 338–344. doi: 10.1177/1754073912445814, PMID: 23459790 Dryhurst S. Schneider C. R. Kerr J. Freeman A. L. J. Recchia G. Van Der Bles A. M. . (2020). Risk perceptions of COVID-19 around the world. J. Risk Res. 23, 994–1006. doi: 10.1080/13669877.2020.1758193 Duradoni M. Gursesli M. C. Materassi L. Serritella E. Guazzini A. (2022). The long-COVID experience change people’s vaccine hesitancy but not their vaccination fear. Int. J. Environ. Res. Public. Health 19:14550. doi: 10.3390/ijerph192114550, PMID: 36361430 Eisinga R. te Grotenhuis M. Pelzer B. (2013). The reliability of a two-item scale: Pearson, Cronbach, or Spearman-Brown? Int. J. Public Health 58, 637–642. doi: 10.1007/s00038-012-0416-3 Ekman P. (1992). An argument for basic emotions. Cognit. Emot. 6, 169–200. doi: 10.1080/02699939208411068 Ekman P. (1999). “Basic emotions” in Handbook of cognition and emotion. eds. Dalgleish T. Power M. J. (England: John Wiley & Sons Ltd.), 45–60. Fathian-Dastgerdi Z. Khoshgoftar M. Tavakoli B. Jaleh M. (2021). Factors associated with preventive behaviors of COVID-19 among adolescents: applying the health belief model. Res. Soc. Adm. Pharm. 17, 1786–1790. doi: 10.1016/j.sapharm.2021.01.014, PMID: 33558153 Finucane M. L. Alhakami A. Slovic P. Johnson S. M. (2000). The affect heuristic in judgments of risks and benefits. J. Behav. Decis. Mak. 13, 1–17. doi: 10.1002/(SICI)1099-0771(200001/03)13:1<1::AID-BDM333>3.0.CO;2-S Fragkaki I. Maciejewski D. F. Weijman E. L. Feltes J. Cima M. (2021). Human responses to COVID-19: the role of optimism bias, perceived severity, and anxiety. Personal. Individ. Differ. 176:110781. doi: 10.1016/j.paid.2021.110781, PMID: 33623178 Frijda N. H. (2007). What might emotions be? Comments on the comments. Soc. Sci. Inf. 46, 433–443. doi: 10.1177/05390184070460030112 Gold J. A. Rossen L. M. Ahmad F. B. Sutton P. Li Z. Salvatore P. P. . (2020). Race, ethnicity, and age trends in persons who died from COVID-19 — United States, May–August 2020. MMWR Morb. Mortal. Wkly. Rep. 69, 1517–1521. doi: 10.15585/mmwr.mm6942e1 Han Q. Zheng B. Agostini M. Bélanger J. J. Gützkow B. Kreienkamp J. . (2021). Associations of risk perception of COVID-19 with emotion and mental health during the pandemic. J. Affect. Disord. 284, 247–255. doi: 10.1016/j.jad.2021.01.049, PMID: 33602537 Harley J. M. Bouchet F. Azevedo R. (2012). Measuring learners’ co-occurring emotional responses during their interaction with a pedagogical agent in MetaTutor. Berlin Heidelberg: Springer, 40–45. Hemenover S. Zhang S. (2004). Anger, personality, and optimistic stress appraisals. Cognit. Emot. 18, 363–382. doi: 10.1080/02699930341000103 Horney J. A. Moore Z. Davis M. MacDonald P. D. M. (2010). Intent to receive pandemic influenza a (H1N1) vaccine, compliance with social distancing and sources of information in NC, 2009. PLoS One 5:e11226. doi: 10.1371/journal.pone.0011226, PMID: 20585462 Hunagund D. L. Hangal S. J. (2014). Self-efficacy and happiness in youth. J. Indian Acad. Appl. Psychol. 40, 70–73. Hutchins H. J. Wolff B. Leeb R. Ko J. Y. Odom E. Willey J. . (2020). COVID-19 mitigation behaviors by age group – United States, April-June 2020. MMWR Morb. Mortal Wkly. Rep. 69, 1584–1590. doi: 10.15585/mmwr.mm6943e4, PMID: 33119562 Imran N. Zeshan M. Pervaiz Z. (2020). Mental health considerations for children & adolescents in COVID-19 pandemic. Pak. J. Med. Sci. 36, S67–S72. doi: 10.12669/pjms.36.covid19-s4.2759 Jang Y. Turner M. M. Heo R. J. Barry R. (2021). A new approach to audience segmentation for vaccination messaging: applying the anger activism model. J. Soc. Mark. 11, 424–452. doi: 10.1108/jsocm-10-2020-0206 Jehn M. Kim Y. Bradley B. Lant T. (2011). Community knowledge, risk perception, and preparedness for the 2009 influenza a/H1N1 pandemic. J. Public Health Manag. Pract. 17, 431–438. doi: 10.1097/PHH.0b013e3182113921, PMID: 21788781 Jones J. H. Salathé M. (2009). Early assessment of anxiety and behavioral response to novel swine-origin influenza A (H1N1). PLoS One 4:e8032. doi: 10.1371/journal.pone.0008032, PMID: 19997505 Kahlor L. A. (2010). PRISM: a planned risk information seeking model. Health Commun. 25, 345–356. doi: 10.1080/10410231003775172, PMID: 20512716 Kim D. Park C. Kim E. Han J. Song H. (2022). Social sharing of emotion during the COVID-19 pandemic. Cyberpsychol. Behav. Soc. Netw. 25, 369–376. doi: 10.1089/cyber.2021.0270 Kiviniemi M. T. Orom H. Hay J. L. Waters E. A. (2022). Prevention is political: political party affiliation predicts perceived risk and prevention behaviors for COVID-19. BMC Public Health 22:298. doi: 10.1186/s12889-022-12649-4, PMID: 35164719 Koffman J. Gross J. Etkind S. N. Selman L. (2020). Uncertainty and COVID-19: how are we to respond? J. R. Soc. Med. 113, 211–216. doi: 10.1177/0141076820930665, PMID: 32521198 Lau W. Thomas T. (2008). Research into the psychological well-being of young refugees. Int. Psychiatry 5, 60–62. doi: 10.1192/S1749367600002071 Lazarus R. S. (1991). Emotion and adaptation. New York, US: Oxford University Press. Lazarus R. S. Averill J. R. Opton E. M.Jr. (1970). “Toward a cognitive theory of emotion” in Feelings and emotions: The Loyola Symposium. ed. Arnold M. B. (New York: Academic Press), 207–232. Leggat P. A. Brown L. H. Aitken P. Speare R. (2010). Level of concern and precaution taking among Australians regarding travel during pandemic (H1N1) 2009: results from the 2009 Queensland social survey. J. Travel Med. 17, 291–295. doi: 10.1111/j.1708-8305.2010.00445.x Lerner J. S. Gonzalez R. M. Small D. A. Fischhoff B. (2003). Effects of fear and anger on perceived risks of terrorism: a National Field Experiment. Psychol. Sci. 14, 144–150. doi: 10.1111/1467-9280.01433, PMID: 12661676 Lerner J. S. Keltner D. (2000). Beyond valence: toward a model of emotion-specific influences on judgement and choice. Cognit. Emot. 14, 473–493. doi: 10.1080/026999300402763 Lerner J. S. Keltner D. (2001). Fear, anger, and risk. J. Pers. Soc. Psychol. 81, 146–159. doi: 10.1037/0022-3514.81.1.146 Leventhal H. (1979). “A perceptual-motor processing model of emotion” in Perception of emotion in self and others. Advances in the Study of Communication and Affect, vol 5. eds. Pliner P. Pliner K. R. B. Spigel I. M. (Boston, MA: Springer), 1–46. Martelletti C. P. Santirocchi A. Spataro P. Rossi-Arnaud C. Löfstedt R. E. Cestari V. (2022). Predictors of COVID-19 risk perception, worry and anxiety in Italy at the end of the 2020 national lockdown. J. Risk Res. 25, 1306–1320. doi: 10.1080/13669877.2022.2038245 Metzler H. Rimé B. Pellert M. Niederkrotenthaler T. Di Natale A. Garcia D. (2023). Collective emotions during the COVID-19 outbreak. Emotion 23, 844–858. doi: 10.1037/emo0001111 Al Mutair A. Al Mutair A. Ambani Z. Shamsan A. Al Mahmoud S. Alhumaid S. (2021). The impact of COVID-19 pandemic on the level of depression among health care workers: cross-sectional study. PeerJ 9:e11469. doi: 10.7717/peerj.11469, PMID: 34046264 Nabi R. L. (1999). A cognitive-functional model for the effects of discrete negative emotions on information processing, attitude change, and recall. Commun. Theory 9, 292–320. doi: 10.1111/j.1468-2885.1999.tb00172.x Nabi R. L. (2002). The theoretical versus the lay meaning of disgust: implications for emotion research. Cognit. Emot. 16, 695–703. doi: 10.1080/02699930143000437 Nabi R. L. (2003). Exploring the framing effects of emotion: do discrete emotions differentially influence information accessibility, information seeking, and policy preference? Commun. Res. 30, 224–247. doi: 10.1177/0093650202250881 Nabi R. L. Wirth W. (2008). Exploring the role of emotion in media effects: An introduction to the special issue. Media Psychol. 11, 1–6. doi: 10.1080/15213260701852940 Nan X. (2017). Influence of incidental discrete emotions on health risk perception and persuasion. Health Commun. 32, 721–729. doi: 10.1080/10410236.2016.1168004, PMID: 27403680 Pedrosa A. L. Bitencourt L. Froes A. C. F. Cazumba M. L. B. Campos R. G. B. de Brito S. B. C. S. . (2020). Emotional, behavioral, and psychological impact of the COVID-19 pandemic. Front. Psychol. 11:566212. doi: 10.3389/fpsyg.2020.566212, PMID: 33117234 Plutchik R. (1980) “A general psychoevolutionary theory of emotion” in Theory of emotion. eds. Plutchik R. Kellerman H. (London: Academic Press), 3–33. Plutchik R. (2001). The nature of emotions: human emotions have deep evolutionary roots, a fact that may explain their complexity and provide tools for clinical practice. Am. Sci. 89, 344–350. doi: 10.1511/2001.28.739 Porter S. Umbach P. (2006). Student survey response rates across institutions: Why do they vary? Res. High. Educ. 47, 229–247. doi: 10.1007/s11162-005-8887-1 Restubog S. L. D. Ocampo A. C. G. Wang L. (2020). Taking control amidst the chaos: emotion regulation during the COVID-19 pandemic. J. Vocat. Behav. 119:103440. doi: 10.1016/j.jvb.2020.103440, PMID: 32390659 Roseman I. J. (2001). “A model of appraisal in the emotion system” in Appraisal processes in emotion: theory, methods, research. eds. Schorr A. Scherer K. R. Johnstone T. (New York NY, US: Oxford University Press), 68–91. Rubin G. Potts H. Michie S. (2010). The impact of communications about swine flu (influenza A H1N1v) on public responses to the outbreak: results from 36 national telephone surveys in the UK. Health Technol. Assess. 14, 183–266. doi: 10.3310/hta14340-03, PMID: 20630124 Russell J. A. (1980). A circumplex model of affect. J. Pers. Soc. Psychol. 39, 1161–1178. doi: 10.1037/h0077714 Schachter S. Singer J. (1962). Cognitive, social, and physiological determinants of emotional state. Psychol. Rev. 69, 379–399. doi: 10.1037/h0046234 Shen L. Dillard J. P. (2007). The influence of behavioral inhibition/approach systems and message framing on the processing of persuasive health messages. Commun. Res. 34, 433–467. doi: 10.1177/0093650207302787 Shukla S. Mishra S. K. Rai H. (2021). Optimistic bias, risky behavior, and social norms among Indian college students during COVID-19. Personal. Individ. Differ. 183:111076. doi: 10.1016/j.paid.2021.111076, PMID: 36569788 Shultz J. M. Cooper J. L. Baingana F. Oquendo M. A. Espinel Z. Althouse B. M. . (2016). The role of fear-related behaviors in the 2013–2016 West Africa Ebola virus disease outbreak. Curr. Psychiatry Rep. 18:104. doi: 10.1007/s11920-016-0741-y, PMID: 27739026 Sjöberg L. (2007). Emotions and risk perception. Risk Manage. 9, 223–237. doi: 10.1057/palgrave.rm.8250038 Skurka C. J . (2019). Harnessing anger to promote health and environmental activism: exploring the effectiveness of persuasive Counter industry/anger appeals. Ph.D., Cornell University (13887101). Slovic P. Finucane M. L. Peters E. MacGregor D. G. (2004). Risk as analysis and risk as feelings: some thoughts about affect, reason, risk, and rationality. Risk Anal. 24, 311–322. doi: 10.1111/j.0272-4332.2004.00433.x, PMID: 15078302 Spielberger C. D. (1972). Anxiety: current trends in theory and research. Oxford, England: Academic Press. Spielberger C. D. Jacobs G. Russell S. Crane R. S. (1983). Assessment of anger: the state-trait anger scale. Adv. Person. Assess. 2, 161–189. Sylvers P. Lilienfeld S. O. LaPrairie J. L. (2011). Differences between trait fear and trait anxiety: implications for psychopathology. Clin. Psychol. Rev. 31, 122–137. doi: 10.1016/j.cpr.2010.08.004, PMID: 20817337 Thompson R. R. Garfin D. R. Alison Holman E. Silver R. C. (2017). Distress, worry, and functioning following a Global Health crisis: a National Study of Americans’ responses to Ebola. Clin. Psychol. Sci. 5, 513–521. doi: 10.1177/2167702617692030 Trnka R. Lorencova R. (2020). Fear, anger, and media-induced trauma during the outbreak of COVID-19 in the Czech Republic. Psychol. Trauma Theory Res. Pract. Policy 12, 546–549. doi: 10.1037/tra0000675, PMID: 32538654 Troiano G. Nardi A. (2021). Vaccine hesitancy in the era of COVID-19. Public Health 194, 245–251. doi: 10.1016/j.puhe.2021.02.025, PMID: 33965796 Turner M. M. (2007). Using emotion in risk communication: the anger activism model. Public Relat. Rev. 33, 114–119. doi: 10.1016/j.pubrev.2006.11.013 Turner M. M. Richards A. S. Bessarabova E. Magid Y. (2020). The effects of anger appeals on systematic processing and intentions: the moderating role of efficacy. Commun. Rep. 33, 14–26. doi: 10.1080/08934215.2019.1682175 Turner M. M. Rimal R. N. Morrison D. Kim H. (2006). The role of anxiety in seeking and retaining risk information: testing the risk perception attitude framework in two studies. Hum. Commun. Res. 32, 130–156. doi: 10.1111/j.1468-2958.2006.00006.x Turner M. M. Rains S. (2021). Guilt appeals in persuasive communication: a meta-analytic review. Commun. Stud. 72, 684–700. doi: 10.1080/10510974.2021.1953094 Witte K. (1992). Putting the fear back into fear appeals: the extended parallel process model. Commun. Monogr. 59, 329–349. doi: 10.1080/03637759209376276 Witte K. Cameron K. A. McKeon J. K. Berkowitz J. M. (1996). Predicting risk behaviors: development and validation of a diagnostic scale. J. Health Commun. 1, 317–342. doi: 10.1080/108107396127988 Yang J. Z. Chu H. (2018). Who is afraid of the Ebola outbreak? The influence of discrete emotions on risk perception. J. Risk Res. 21, 834–853. doi: 10.1080/13669877.2016.1247378 Yang J. Z. Kahlor L. A. (2013). What, me worry? The role of affect in information seeking and avoidance. Sci. Commun. 35, 189–212. doi: 10.1177/1075547012441873 Zelenski J. M. Larsen R. J. (2000). The distribution of basic emotions in everyday life: a state and trait perspective from experience sampling data. J. Res. Pers. 34, 178–197. doi: 10.1006/jrpe.1999.2275