The experiment featured a representative sample of 150 participants recruited through Prolific, an online participant recruiting platform. In total 121 entries were used for the analysis, with failed attention checks and extraordinary rapid completion times excluded from the analysis. Though we used online sampling methods, previous works have shown that results from similar platforms (e.g., MTurk) have wide external validity (Krupnikov and Levine, 2014; Mullinix et al., 2015). The filtered sample featured 61 females, 59 males and 1 other gender, with an average age of 45.81 (σ = 15.89). The experiment was rolled out in July 2020 on a UK-based sample². Participants were paid a fixed fee for participating in the experiment and could earn additional monetary rewards during the experiment.

All news items were initially tweets posted by well-known and trusted media outlets (e.g., Bloomberg, Reuters, the Economist). To remove any plausible residual bias the tweets were translated into several languages and back to English using DeepL. This was done with the aim of preventing any existent wording bias. Several tweets remained unchanged in this process. All selected tweets were in relation to American politics both internal and external and were factual depictions of reality. From the original selected tweets, we then derived a shorter version, this version though shorter and less information-rich remained an accurate depiction of the original tweet and thus true. Both the original and short Tweets represented true tweets in the experiment. Besides the short version, we additionally created misleading versions of the original tweet, one for each political bias (Democrat- and Republican-leaning). These misleading versions, though correct, presented the information in the favor of their political alignment. Lastly, we derived fake versions of the original tweet. As fake news heavily favored a political party and the information they featured was factually incorrect. Table 1 summarizes this transformation and creation process; Table 2 shows the result of this process from an original claim to a fake version.

The experiment was made of 2 main phases. In the first phase, the reaction phase, participants were asked to react to the tweets as they would on Twitter under normal circumstances. To ensure that participants engaged with all of the items, for those that they did not want to respond to, they had to click an onscreen “ignore” option. Additionally, participants could react with a combination of like retweet and comment, as they are able to on Twitter.

In the second phase, the veracity phase, the participants were tasked and incentivized to assess the veracity of each claim. Correctly identifying the veracity would lead to additional monetary rewards. In order to assess the veracity, participants were given the option of classifying each claim as either true, misleading, or fake. They were shown basic definitions of these terms at the start of the veracity phase. Correct identification lead to higher monetary rewards. Participants were informed of their accuracy at the end of the study.

After completing both phases participants were asked demographic information along with questions assessing the effect of the COVID-19 pandemic on their mental state, their risk and ambiguity aversion and their self-reported political leaning³.

The reaction and veracity phase each featured 8 rounds of 4 tweets. The tweets remained the same across both phases. Every round featured an original tweet and a short tweet, both of which were verifiably true. Additional to those true claims, participants were also shown one or two misleading tweets (correct but politically-biased claims). In three out of eight rounds, only one misleading claim was shown, the initial second misleading claim was replaced by a homologue fake version. That is, if a round did not contain a republican-biased misleading tweet, it would have a republican biased fake tweet. All tweets within a round across both phases were shown in random order. The full experiment and list of tweets is found in Appendix 1. It is worth noting that the experiment features an equal number of true and false tweets, following the tradition of lab-based experiments on fake news (Bond and DePaulo, 2006; Pennycook et al., 2017; Luo et al., 2022). Considering partisanship is a crucial determinant in reaction type and reaction rate (Mourão and Robertson, 2019), an equal number of democrat and republican leaning tweets are selected.

Using a method similar to Park et al. (2021), the experiment data frame was structured on a tweet-participant basis instead of a participant basis. That is, every data entry represents a participant's decisions on a given tweet multiplying the total entries by the number of tweets in the experiment. We use a set of parametric statistical tests to verify the outlined hypothesis. To account for the dataset transformation, subsequent regression analyses control for participant and tweet fixed effects. This restructuration allows for a more understandable representation of the variability.

The dataset featured three main variables. (i) The reaction binary which was activated if a participant did not ignore a tweet in the reaction phase. This variable is used as the dependent variables in the subsequent models. (ii) The (factual) veracity as a categorical variable which indicated the veracity of the tweet participants reacted to (either true, misleading, or fake). (iii) Lastly, the perceived veracity of the participants on tweets, each claim being rated as either real, misleading, or fake news. The dataset also featured general demographic information such as age, gender, nationality, etc. as well as political leaning. In total the dataset featured 3,872 decisions (N = 3,872) for 121 participants. Tables 3, 4 provide an overview of the descriptive statistics of the sample.

Correlation analysis reveals a significant positive correlation between perceived veracity and reaction likelihood [correlation: r_(3,871) = 0.067, p < 0.001]. Though the aforementioned analyses hint at the confirmation of hypothesis 1, they fail to account for participants and tweet characteristics. Table 5 presents multiple logit models testing for the hypothesis, unlike the correlation analysis the logit models account for the fixed effects of both tweets and participants. In all models the perceived veracity significantly predicted the reaction likelihood. This supports our first hypothesis and is in line with current academic literature (Metaxas et al., 2014; Pennycook et al., 2020).

The second hypothesis analyzed if fake news was intrinsically more likely to be shared than real news. Figure 2 shows graphically that this is indeed the case with fake news being reacted to more often than other types of news. We denote that the reaction likelihood for fake news is higher despite its lower perceived accuracy. Moreover, a one-way ANOVA confirmed this difference [F_{(1, 3, 871)} = 17.19, p < 0.001]. Modal evidence, without fixed effects, is also in line with Hypothesis 2, as shown in Table 5. However, when including fixed effects of tweets, the statistical significance of the effect is reduced. This partial confirmation is in line with a social media platform's reality where fake news spreads more widely than its true counterparts (Silverman et al., 2016; Vosoughi et al., 2018). Because the experiment displayed multiple claims of varied political biases within a round (and thus provided equal information), we note that this difference in reaction likelihood holds even outside possible “echo chambers” and “filter bubbles”.

Through the confirmation of hypotheses 1 and 2, we note a surprising distinction; whilst participants are most likely to react to tweets that they perceived to be true, fake news remains the most susceptible to gather reactions.

Hypothesis 3 tested for the interaction effect of tweet veracity and perceived veracity on reaction likelihood. Figure 3 shows the variables in this hypothesis and their interaction. Across all biases tweets considered to be “Real News” by the participants were the most likely to be reacted to, for fake tweets this effect was further magnified. Table 5 shows the results of the analysis. The interaction term of tweet type (fake) and perceived veracity (true) is statistically significant across all models, confirming that the effect of perceived veracity is magnified for fake news. This result provides a mechanism for the well-known stylized fact that fake news spreads more widely than real news. Specifically, even when controlling with fixed effects for tweets and individuals, perceived veracity of specifically fake news is associated to higher reaction likelihood.

This paper derives three main findings from its analysis, they are synthesized in Table 6. We first confirm that (i) higher perceived veracity of a claim leads to a higher reaction likelihood to said claims. We define such behavior as activist-behavior, where the belief of claim leads to increased reaction likelihood. (ii) Fake news is more likely to be reacted to than real news. Lastly, we find (iii) a statistically significant interaction effect of claim (factual) veracity on perceived veracity, i.e., the activist behavior is amplified for claims that are factually false.

Understanding the reasons that drive social media users to share (fake) news is central to limiting the spread of fake news. The literature suggests that veracity is central to a user's decision to share news or not. Yet this finding is often studied by asking users about their own behavior, not the perception they have on a particular news item, i.e., the perceived veracity (Metaxas et al., 2014; Pennycook and Rand, 2019). The experimental setting of this paper allows us to study the perceived veracity of users on all claims. We confirm the initial finding of the literature.

Social media data suggests that fake news spreads more widely than real news (Silverman et al., 2016; Vosoughi et al., 2018). However, this difference could also be explained, at least in part, by social media network effects (i.e., the latent algorithms used to place a posts in a user's feed). We confirm that users react more to fake news even outside the typical social media environment. This entails that the popularity of fake news cannot be solely attributed to the network effect present in social media, rather it has an inherent individual component.

Hypothesis 3 (activist behavior is amplified for fake news) provides a behavioral explanation to the sharing of fake news. It partially explains why fake news spreads more widely than real news despite an initial smaller user base (Grinberg et al., 2019; Guess et al., 2019). This oversharing of fake news is commonly attributed to online echo chambers, which are known to be present across social media platforms (Quattrociocchi et al., 2016). However, the true magnitude of their effect remains uncertain (Spohr, 2017; Guess et al., 2018a). We suggest that these network effects coexist with behavioral reasons and that they simultaneously contribute to the wider spread of fake news. This characterization yields support to the headlines blaming zealotry (i.e., a stronger version of activism) for the role of social media in spreading fake news (Aaronovitch, 2017; Lohr, 2018).

Besides the analysis presented in this section Appendix 2 also finds that hypotheses 1 and 3 hold true using the results of Pennycook and Rand (2019)'s experiment. Pennycook and Rand initially derived from their results that fake news belief was caused rather by lack of thinking than by hyper-partisanship.

Notwithstanding the contributions of this paper to the literature, there are some limitations that should be highlighted. First, this is an experiment rather than a natural study. Therefore, it is attached to the early infodemic and pandemic context of the summer 2020 in which the experiment took place. The information overload present at the time could potentially have affected participant opinions on some of the tweets present in our study (Papapicco, 2020).

Second, the experiment features a UK-based sample whilst the topics cover American politics. Though participants may not have been as informed on the topic as an American public would (though we control for familiarity with US politics), this also allows them to have a more detached opinion with less extreme emotions. To the extent that strong emotions are behind individual reacting decisions, our results could be seen as a conservative benchmark for a US sample.

Third, as noted in the experimental design, the experiment displayed tweets by rounds of four centered on a same topic. Hence, the tweets seen by participant in a same round were diverse both in veracity and political biases. This can affect our results in two ways. On the one hand, this might not be reflective of online echo chambers, where participants would supposedly be shown tweets that fit their profile specifications. On the other hand, the perception of the participants could be affected by the display of diversified tweets. This could be seen as a form of inoculation from fake news (van Der Linden et al., 2020), leading to a reduced impact of the false information used in the experiment.

Furthermore, participants remained uninformed of monetary gains and performance through the veracity-phase. Future studies can look at the effect of informing participants after each trial.