This study was approved by the Stanford University IRB, and the requirement for written consent was waived. We randomized subjects using a three-factor (health condition X advertisement exposure X judgment of lifestyle change versus drugs) between-subjects design (Figure 1). Randomization was conducted internally via an online questionnaire design software (2013, Qualtrics, Provo, UT, USA) and was balanced across the 18 experimental conditions.

At the beginning of the questionnaire, subjects either viewed a drug advertisement for the target health condition (Table 1) and completed an attention-check question, or in the no-advertisement conditions, proceeded directly to the effectiveness questions. (Because of the inherent difficulty of identifying a truly “neutral” counterpart to the advertisement stimuli, particularly with regard to norm induction, we did not present any video clip to subjects in no-advertisement conditions.) Advertisements were viewed once. Then, each subject judged the effectiveness of either lifestyle change or prescription drugs for the target health condition. The between-subjects manipulation of intervention type eliminated the possibility that subjects would experience order effects or directly compare the two types of interventions. We tested all questionnaire materials for comprehension on pilot samples and, in the experimental sample, probed for confusion or technical problems.

We sampled 819 subjects via Amazon Mechanical Turk, a crowdsourcing platform allowing workers to complete brief online tasks in exchange for pay. By contractual agreement with Mechanical Turk, workers must be at least 18 years of age. Workers tend to be somewhat younger, more educated, and lower-income than the US general population, but are demographically more representative than typical university-based research samples (Paolacci et al., 2010). Studies performed on Mechanical Turk yield high-quality data, minimize experimental biases, and successfully replicate the results of behavioral studies performed on traditional samples (Paolacci et al., 2010). Given the nearly ubiquitous reach of DTC advertising in the US, its public health implications may extend far beyond patients with the targeted health conditions; we therefore did not restrict the sample to individuals with specific health conditions. The task title and description were vague to minimize sampling bias and demand characteristics. We sampled US workers with excellent performance history (>95% of previous online tasks “approved” as high-quality by requester) and compensated each subject $0.25.

We selected three target health conditions (high cholesterol, diabetes, and depression) that are widespread, familiar to laypeople, and at least somewhat responsive to lifestyle change, specifically of dietary or exercise habits (Gillies et al., 2007; Thomas and Elliott, 2009; Hooper et al., 2012; Rimer et al., 2012). For each of these three health conditions, we obtained approximately 1-min advertisements for two major brands, for a total of six advertisements (Table 1). We obtained them from an online pharmaceutical advertising database¹, manufacturer websites, and YouTube.com. All advertisements had aired since 2008 on United States television networks.

For subjects who viewed an advertisement, we included the multiple-choice attention check question “According to the video, what is the purpose of the drug?”, to which only one of seven possible answers was correct. We eliminated from analysis all subjects responding incorrectly to this question (an a priori decision).

We collected both direct and indirect outcome measures, comprising two measures of perceived effectiveness and one measure of personal intention to use either lifestyle change or drugs (Weinstein, 1998).

To measure perceived general effectiveness, we asked, “In your opinion, how effective is/are [intervention] at helping with [health condition]?”, where words in brackets depended on the subject’s experimental group. Lifestyle change was described as “healthy lifestyle changes (such as improving nutrition, controlling weight, and increasing exercise),” corresponding to recommendations published by the Centers for Disease Control and Prevention [CDC] (Singer, 2009; Centers for Disease Control and Prevention, 2013). Drugs were described as “[health condition] drugs.” Subjects responded on an integer-valued, 100-point visual analog scale (VAS) with four evenly-spaced verbal labels: “Very Ineffective,” “Ineffective,” “Effective,” and “Very Effective” on which higher values indicated higher effectiveness.

For the second outcome measure, disease severity score, subjects read instructions explaining that the lower the score, the better. They were instructed to “Imagine a patient with [health condition] (score 161) who begins using [intervention]. Please estimate the patient’s [health condition] score after 6 months of using [intervention].” Subjects responded on a 180-point VAS.

For the intention outcome, we asked, “Imagine you were recently diagnosed with [health condition]. How likely would you be to use [intervention]?” Subjects responded on a VAS identical to that used for the general effectiveness measure.

We operationalized individual problem status via two questions gaging current adherence to exercise and diet recommendations: “Do you currently engage in at least: 75 min per week of vigorous-intensity physical exercise, OR 150 min of moderate-intensity exercise?” and “On a typical day, do you eat a healthy diet (rich in vegetables, limited in refined sugar, limited in fatty and fried foods, low in highly processed junk food)?” Subjects currently meeting criteria for both exercise and a healthy diet were classified as “low problem status,” while subjects failing to meet either criterion were classified as “high problem status.”

Previous observational research has documented an association between chronic exposure to advertising and prescription drug requests (Mintzes et al., 2003). As a proxy, we measured chronic television exposure with the question: “On a typical day, how many hours of television do you watch?”

We collected data on the following demographic and behavioral characteristics: age, sex, body mass index (BMI) (calculated from height and weight), diet, exercise, education level, number of prescription drugs taken regularly, and duration of television watched on a typical day. These factual questions appeared at the end of the survey to prevent priming and stereotype threat effects on the outcome measures (Steele and Aronson, 1995; Burgess et al., 2010).

We performed all analyses in R (Version 3.0.2, multiple contributors, Vienna, Austria) (Fox and Weisberg, 2010; R Core Development Team, 2014) and defined statistical significance at an alpha level of 0.05. All tests were two-sided.

To investigate baseline perceptions of perceived effectiveness and intention, we first restricted analysis to subjects who did not view an advertisement. We used t-tests to assess differences in each outcome measure between subjects judging lifestyle change and those judging drugs.

To assess the effect of advertisement exposure, we fit a linear regression model with normally distributed error terms to predict each outcome measure (Supplementary Methods, Model 1). Each model included main effects for advertisement exposure (subject viewed, versus did not view, an advertisement), type of health intervention judged (subject judged lifestyle change versus drugs), health condition (subject judged the outcome measure for high cholesterol, diabetes, or depression). We were primarily interested in the interaction of advertisement exposure with type of health intervention judged, as it estimates a possible differential effect of advertisement exposure for lifestyle change versus drugs. For the general effectiveness and personal intention measures, a negative interaction would indicate that advertisement exposure disproportionately improved perceptions of drugs versus lifestyle change. For the disease severity measure, lower scores indicate greater effectiveness, so a positive interaction would indicate the same. We also included interactions accounting for the possibility that some health conditions are perceived as especially conducive to either lifestyle change or drugs.

To assess whether individual problem status moderates a potential boomerang effect of advertisement exposure, we also analyzed a secondary model including a three-way interaction term of problem status.

If perceived ineffectiveness is a significant barrier to adopting a healthy lifestyle, then individuals who do not currently follow lifestyle recommendations for exercise and diet would be expected to perceive lifestyle as less effective than individuals who do follow recommendations. We investigated this possibility by restricting analysis to only subjects who did not view an advertisement. Among this subset, we fit a linear regression model (Supplementary Methods, Model 2) predicting each outcome measure with type of health intervention judged and current self-reported lifestyle (the subject meets both exercise and dietary requirements or does not). The interaction between the two variables is the term of primary interest, as it represents a differential effect of intervention type between subjects with high and low problem status.

Data quality was excellent. Reported comprehension of all questionnaire items was very high, with only four subjects reporting any confusion. Of 850 subjects who finished the study and who had not previously completed a pilot trial, we excluded 31 responses (3.6%) due to quality problems (6 reported technical or comprehension problems, 18 failed the attention check question, and 7 gave unreasonable body metrics; Figure 2). Sensitivity analyses that instead included all subjects yielded nearly identical results. Thus, the final analyzed sample size was 819; the breakdown by condition is presented in Figure 1. Negligible differences in sample size across the conditions are due to dropout.

Subjects were predominantly male (62.0%), young (median age 27 years), and relatively educated (e.g., 38.2% held a 4-year college degree). Slightly more than half met national standards for diet (55.8%) and for exercise (55.6%), and 24.1% reported regularly taking a prescription drug. Subjects were on average borderline overweight (median BMI 25.0 kg/m²) and reported watching a median of 2.0 h of television daily (Supplementary Table S1). Although we did not collect data on ethnic characteristics in this study, our recent work using nearly identical recruitment procedures (Mathur and Reichling, 2016) found the ethnic background of Mechanical Turk users to be approximately 80% Caucasian, 11% Hispanic, 6% Southeast Asian, 5% Native American, and <5% each: East Asian, South Asian, Pacific Islander, Black, and Middle Eastern (percentages sum to more than 100% due to multiracial subjects).

We assessed baseline perceptions of lifestyle and pharmaceutical intervention effectiveness among the n = 303 subjects who did not view a drug advertisement (Table 2), finding that subjects considered lifestyle change to be more effective, or as effective, as drugs for all outcome measures. Specifically, perceived general effectiveness was much higher for lifestyle than pharmaceutical interventions (mean VAS units 80.2 and 62.8, p < 0.001). A similar pattern occurred for personal intention (means 71.8 and 62.5, p = 0.004). However, disease severity score did not differ significantly between intervention types (means 106.4 and 104.0, p = 0.45).

We also assessed whether baseline perceptions of lifestyle change and drugs depended on whether subjects themselves followed lifestyle recommendations for exercise and diet. Personal adherence to lifestyle recommendations was unrelated to relative perceived effectiveness of lifestyle change and drug: Among subjects not exposed to an advertisement, there was no significant interaction of meeting lifestyle criteria with intervention type for either the general effectiveness (b coefficient = 5.44, p-value = 0.20, partial R² = 0.004) or the disease severity score measure (b = -6.50, p = 0.32, partial R² = 0.003)².

In primary regression analyses (Table 3; Figure 3), the endpoint of interest was the interaction coefficient of advertisement exposure with intervention type, which measures the relative impact of advertisement exposure on perceptions of lifestyle change versus drugs (the boomerang effect). We found that advertisement exposure increased perceived general effectiveness slightly more for drugs versus lifestyle change. However, the opposite was true for the personal intention measure: advertisement exposure disproportionately increased personal intention to use lifestyle versus drugs. That is, there was a small, but significant, negative interaction for the general effectiveness outcome (b = -6.2, p = 0.01, partial R² = 0.007), no significant interaction for the disease severity score outcome (b = -3.9, p = 0.32, partial R² = 0.0009), and a positive interaction for the personal intention outcome (b = 10.6, p = 0.006, partial R² = 0.008). Omitting the main effect of health condition and interactions of health condition with intervention type, or stratifying on health condition (Supplementary Figure S1), did not appreciably change results. Including three-way interaction terms of health condition with intervention type suggested some significant differences in the interaction of interest across diseases, but these were not consistent across outcome measures.

To estimate the difference in potential “boomerang” effects between subjects with high versus low problem status, we fit three-way interactions of individual problem status (meeting both dietary and exercise requirements) with advertisement exposure and intervention type. We found no evidence for moderation by problem status: the interaction terms were non-significant for all outcomes (general effectiveness: b = -3.2, p = 0.54, partial R² = 0.0004; disease severity: b = -0.4, p = 0.97, partial R² < 0.0001; personal intention: b = -14.1, p = 0.07, partial R² = 0.0006).

Because it appeared counter-intuitive that drug advertisements would not substantially increase perceived effectiveness or personal intention to use the advertised products, we performed post hoc secondary analyses assessing the role of time spent watching television. If subjects were already saturated by everyday exposure to DTC advertisements, then the lack of effect of a single experimental exposure to advertisements might belie a cumulative shift in perceptions as a result of chronic exposure to such advertisements.

We conducted two analyses to investigate this possibility. First, to assess whether subjects with little or no chronic exposure to DTC television advertisements would be sensitive to experimental exposure, we repeated analyses among only those subjects who reported watching no television on a typical day (n = 163). Advertisement exposure had no significant interaction with intervention for the general effectiveness (b = -3.2, p = 0.62, partial R² = 0.008) or disease severity score measures (b = -13.2, p = 0.18, partial R² = 0.007). As in the main analysis, there was a large positive interaction for the personal intention measure (b = 19.7, p = 0.02, partial R² = 0.03).

Additionally, we investigated whether, among subjects who did not view an advertisement (n = 303), increased television-watching was associated with a more favorable perception of drugs versus lifestyle change. We fit a linear regression model (Supplementary Methods, Model 3) to predict each outcome measure with main effects and the interaction of the number of hours of television watched on a typical day with type of intervention judged (lifestyle change versus drugs).

A negative interaction between television-watching time and lifestyle change (versus drugs) would indicate that individuals watching more television had a relatively less favorable view of lifestyle (versus drugs) than individuals watching less television. In fact, there was no such interaction for any of the outcome measures (general effectiveness: b = -0.9, p = 0.34, partial R² = 0.002; disease severity score: b = 0.2, p = 0.89, partial R² < 0.0001; personal intention: b = -1.4, p = 0.33, partial R² = 0.003). Therefore, regularly consuming more television, and by proxy more DTC television advertisements, did not appear to shift perceptions of relative effectiveness or personal intention to use lifestyle change versus drugs.

Because our sample was relatively young (median age 27 years), we performed a sensitivity analysis to assess whether the primary endpoint (the interaction of advertisement exposure with health intervention judged) differed by subject age by introducing a three-way interaction in analysis models. These coefficients were small and not statistically significant, suggesting that primary findings were fairly robust to differences in subject age. Power may have been limited by low variability in subject age.