Insightful and well-written scientific papers are sometimes difficult to understand. Even researchers like ourselves at times encounter papers, perhaps focusing on new techniques or concepts that we are unfamiliar with, that we find hard to understand. This is more likely to occur when reading papers from disciplines outside our field of expertise – at least if these papers were not explicitly written for an interdisciplinary audience. Simply grasping the main ideas and results of this kind of paper might constitute an arduous task and could potentially hinder the interdisciplinary exchange of knowledge. If this is the case for scientists, imagine the difficulties that non-scientists (i.e., laypersons) face when they are interested in (the findings of) scientific publications with high societal or personal relevance.

Understanding science in general and especially scientific evidence published in journal articles is not easy for laypersons for multiple reasons (Bromme and Goldman, 2014). For a start, scientific articles often employ technical terms and refer to statistical concepts without explaining them to their readers (e.g., Rakedzon et al., 2017; Hauck, 2019). This procedure is undoubtedly often well-justified, as the main audience of scientific journals consists of fellow researchers who are frequently working in the same field as the authors and who have considerable theoretical and statistical expertise (this is closely linked to the notion of the “arena of internal scientific communication” by Peters, 2013). To specifically address the needs and interests of their scientific audience (and relevant gatekeepers such as editors), researchers may pay more attention to the scientific relevance of their findings and less attention to their practical relevance to the broader public when discussing their results in their publications (see also Radford, 2011; Salita, 2015). This strong focus on a scientific audience may, however, make it burdensome for non-scientists to access the current scientific literature to fulfill their need for valid and understandable information (see also Nunn and Pinfield, 2014). There are manifold relevant groups of laypersons in this context: Stakeholders (e.g., for decisions on policies), patients (for health-related decisions in medicine), practitioners, but also the interested public in general may benefit from access to research findings (Kaslow, 2015; Rodgers, 2017). For example, the current COVID-19 crisis highlighted that the public yearns – at least to some degree – for reliable scientific information and guidance (e.g., Post et al., 2021).

To close this gap (i.e., to make knowledge contained in scientific articles available to the public), plain language summaries (PLS) – easily comprehensible research summaries that complement scientific abstracts – were introduced for biomedical research (FitzGibbon et al., 2020). The pioneering role this particular area of research has taken is probably due to the high relevance that its findings have for laypeople’s everyday life (see Santesso et al., 2008). We argue that psychological findings are also of considerable interest to laypeople (see also Kaslow, 2015). Yet most guidelines on how to write PLS focus solely on biomedical research, for example, the Cochrane guidelines (Cochrane Methods, 2013; Cochrane Norway, 2016, 2019), and the scarce empirical research to validate PLS guidelines has almost exclusively taken place in the biomedical field (e.g., Santesso et al., 2015; Raynor et al., 2018; Anzinger et al., 2020; Buljan et al., 2020). Writing PLS for psychological research is thus not a straightforward task. Validated discipline-specific guidance – based on robust evidence from experimental research and taking characteristics idiosyncratic to psychology as a discipline situated at the crossroads between the humanities and the sciences into account – is practically non-existent (see also Teo, 2017). For example, how does a researcher communicate psychological concepts like “attention” or “attachment,” terms commonly used in everyday language that actually describe very sophisticated issues, views, or theories as scientific terms? Alternatively, how should meta-analytic findings that deviate considerably from the usual methodological approach in medicine be explained? This may, for instance, be the case when personality variables are correlated with indicators of well-being. The typical strategy of comparing intervention groups with each other and a control group, as used in drug effectiveness studies, does not apply here. Therefore, we argue that there is a considerable need to develop empirically validated guidelines for writing PLS in psychology. To tackle this issue, we present two randomized controlled studies investigating psychology-specific aspects of how to optimally communicate meta-analytical evidence to laypersons. Based on the findings from these studies, we subsequently provide rules on writing PLS.

In a recent systematic review of theory, guidelines and empirical research on PLS, we included a total of 90 records. These include 36 theoretical articles primarily discussing the rationale behind and the aims of PLS, 21 guidance-related articles and 33 articles reporting empirical studies (Stoll et al., 2021). Based on extracted information from these records, we developed a conceptual framework of the aims, characteristics, criteria, and outcomes investigated in PLS research. The aims of PLS (e.g., to impart knowledge about scientific findings to laypersons) determine the characteristics taken into account when composing PLS (e.g., the kind of words that are used). PLS criteria outlined in guidelines are specifications of such characteristics (e.g., the PLS should not be written in jargon). The effectiveness of such criteria, in turn, is evaluated by their impact on predefined outcomes constituting measureable operationalizations of the aims (e.g., knowledge tests). We found that the aims of PLS mentioned, discussed or investigated in the literature could be classified into six categories: accessibility, understanding, knowledge, empowerment, communication of research, and improvement of research. In the current studies, we focus on the first four aims because they target the individual PLS user level, whereas improvement and communication of research aims relate to (the relationship between) society and academia at a higher level. Accessibility, understanding and empowerment directly relate to a subjective user’s perspective and can, therefore, also be described as aims that refer to user experience, as user experience research focuses primarily on the attractiveness, understandability, and usefulness of a product or medium (e.g., Buchanan, 1992; Morville, 2004; Rosenbaum, 2010). Outcomes measuring user experience aims of PLS in experimental studies comprise subjective statements and self-reports on perceived readability of a PLS (accessibility), perceived ease of comprehending the meaning of a PLS on a content level (understanding) and/or usefulness of a PLS for making decisions (empowerment). Knowledge can be measured using (objective) knowledge tests (e.g., Alderdice et al., 2016; Buljan et al., 2018; Kerwer et al., 2021).

In our review, we identified several shortcomings of the empirical research on PLS carried out so far. Of the 33 empirical articles, we found only eight (24%) contrasted different versions of PLS, whereby only six of them (18%) used an experimental research design to do so (Santesso et al., 2015; Alderdice et al., 2016; Kirkpatrick et al., 2017; Buljan et al., 2020; Silvagnoli et al., 2020). Many empirical studies on the effectiveness of PLS, however, either compared PLS to different summary formats (e.g., Buljan et al., 2018) or only evaluated one single PLS (e.g., Santesso et al., 2015), making it hard to infer the suitability of certain characteristics of a PLS from the observed outcomes or to generalize study results. Furthermore, most empirical studies on PLS employed rather small samples that were far from representative for the general public. Participants were often highly educated (e.g., students, Alderdice et al., 2016, Kerwer et al., 2021; or healthcare professionals, Opiyo et al., 2013; Maguire and Clarke, 2014). The empirical evidence that PLS are effective for the public in general – outside narrow settings of professionals or student samples – is therefore still scarce. Research on PLS needs to address this issue by collecting data of larger and more diverse groups. This is especially important since PLS, for example, when accompanying scientific abstracts in journals, sometimes do not even aim to address a specific target group but are rather directed toward anyone who might be interested in psychological research or wants to know something about the topic at hand. To allow PLS to fulfill their purpose of educating and empowering the public, we thus need to investigate their performance in a heterogeneous readership, as well as how reader characteristics affect this performance (specifically, individual differences in interest in psychological research).

Against the background of an ongoing replication crisis in psychology (Open Science Collaboration, 2015; Świątkowski and Dompnier, 2017; Klein et al., 2018), we have argued that PLS may be a powerful tool for rebuilding public trust in psychological science, and first experimental findings (Kerwer et al., 2021) seem to support this notion (see also Carvalho et al., 2019 for non-experimental findings that point in the same direction). Because meta-analyses possess a higher quality of evidence and are instrumental for summarizing findings and guiding practitioners (Borenstein et al., 2009; Bastian et al., 2010), we further argue that the lay-friendly communication of psychological meta-analytical findings is of particular importance – even though legitimate criticisms on the validity of their findings may exist (e.g., Sharpe and Poets, 2020; Burgard et al., 2021; Patall, 2021).

When it comes to communicating meta-analytic psychological evidence to a broad audience via PLS, several aspects quite specific to psychological research or especially relevant in the context of psychological research exist (e.g., the vocabulary or the operationalization of psychological constructs in primary studies). Our reasoning for choosing such aspects here does not arise from the notion that they are especially difficult to explain in the context of psychology or even that they serve a unique function in this field. Rather, according to Shneider (2009), use of specific language (technical terms describing subject matter) as well as methodology (e.g., operationalization) constitute the core characteristics of a scientific discipline, which distinguish it from others. In psychology, for example, the necessity of operationalizing latent psychological constructs is distinct from other disciplines. Other aspects exist that are highly relevant in the context of psychological research – but not discipline-specific by nature – and for which empirical evidence beyond biomedical research is lacking (e.g., research designs and effect size measures in the social sciences). More specifically, in the life sciences, laypeople might be unfamiliar with the jargon typically used in scientific publications due to the fact that researchers use scientific names for species or illnesses uncommon in everyday language. In this case, technical terms need to be “translated” for some readers. For example, a plain language translation of the title “Salvage systemic therapy for advanced gastric and oesophago-gastric junction adenocarcinoma” is “Which treatments work best for advanced stomach cancer that has not responded to standard chemotherapy?” (Tomita et al., 2020). In psychological research, however, this problem is complex in another way, since the vocabulary of psychological science often does not differ from everyday language (e.g., the word “attachment,” see above). The underlying meaning might differ, however, or these words might represent very specific ideas. Without prior knowledge, laypersons, for example, might not understand what developmental psychologists mean when they are referring to “secure attachment.” In guidelines on writing PLS in medicine (Cochrane Methods, 2013; Cochrane Norway, 2016, 2019) and other disciplines (AGU, n.d.; People and Nature, n.d.), typical approaches for handling technical terms are: (1) Replacing the technical terms by non-technical terms within the text of the PLS. Transferred to the context of psychological PLS, this means that instead of using technical terms, these terms are circumscribed by using everyday language (e.g., “feeling safe in relationships” instead of “secure attachment”) and this simplified description is used consistently throughout the PLS text (replacement); (2) using technical terms within the text but providing an additional glossary on technical terms in which these terms are explained (glossary). In our research, we wanted to compare these two approaches to simply ignoring the problem, that is, to using technical terms within the text of the PLS without any explanation (no explanation). In hypotheses 1 and 2, we assumed that both replacing technical terms with non-technical terms and providing a glossary on technical terms would improve the user experience (perceived accessibility, perceived understanding, and perceived empowerment) and knowledge acquisition of laypeople:

Second, we have argued above that meta-analytic research findings constitute an important source of information for laypeople and that this is especially true for psychology in times of the replication crisis. However, communicating meta-analytic evidence in a lay-friendly manner can be challenging. Most laypersons will likely not know that meta-analytical findings draw on multiple studies and therefore have a higher level of evidence quality than individual research findings based on one single study. Thus, to provide laypersons the opportunity to grasp the quality of evidence more accurately when evaluating a meta-analysis, they need to understand that the presented evidence does not stem from a single individual study (see Cochrane Norway, 2019). We argue that laypersons’ further understanding is improved by providing them with a lay-friendly explanation of the methodological approach underlying meta-analytic research. In fact, very current and therefore non-peer reviewed research suggests that this approach (i.e., an explanatory statement) allowed non-scientists to take the quality of evidence of preprints compared to peer-reviewed articles into account when it comes to perceived credibility (Wingen et al., 2021). In hypothesis 3, we assumed that the communication of quality of evidence by an explanation of the methodological approach affects laypersons’ knowledge on the quality of evidence (i.e., factual knowledge on the concept “meta-analysis”) and the understanding of the quality of evidence (taking this knowledge adequately into account in a transfer task).

Third, exactly how psychological studies investigate their research questions is often not self-evident – at least to laypersons – thereby establishing a certain need to “translate psychological science” (see Kaslow, 2015). For example, how is arithmetic competence in very young infants that cannot even speak studied? How are anxiety or resilience measured? One might argue that providing laypersons with (exemplary) information on how the individual studies included in a meta-analysis (i.e., the synthesized studies) operationalized their research questions will substantially improve their understanding of what was actually analyzed in the meta-analysis and their ability to draw conclusions based on its findings. In line with this reasoning, Hoogeveen et al. (2020, p. 269) showed that laypeople were able to rate the likelihood of successful replications when “a short description of the research question, its operationalization, and the key finding” were provided in plain language. However, one might also argue that presenting this additional information may increase task-related cognitive load (e.g., Sweller et al., 2011) and therefore impede laypersons’ user experience and knowledge acquisition regarding the findings of the meta-analysis. To shed some light onto how providing information on the operationalization of the research question in the synthesized studies affects the perception of PLS (i.e., user experience) as well as the knowledge acquired through them, we examined effects of providing laypersons with this information. More specifically, we varied whether or not a subsection on the operationalization of the research question in the individual studies was included in the PLS (information on the operationalization provided vs. not provided).

Hypotheses 1–4 were tested in a first empirical study (Study 1). Hypotheses 5–8 were tested in a second empirical study (Study 2) and partly built upon results of Study 1. For example, participants repeatedly asked for more information about what “effect size” means, how effect sizes are computed and how to interpret their values after being provided with a qualitative statement and an effect size estimate in Study 1. Against this background, the fourth aspect addressed by our research, albeit not completely psychology-specific, becomes especially relevant for psychological meta-analyses. Laypeople are most likely not familiar with the statistical concepts and especially effect size measures employed in scientific publications. In spite of the fact that first studies on communicating statistical evidence to non-scientists via PLS exist – especially for Cochrane PLS (e.g., Glenton et al., 2010; Santesso et al., 2015; Buljan et al., 2020) – broad consensus on this issue is still lacking in relevant guidelines. For example, Cochrane (Cochrane Methods, 2013; Cochrane Norway, 2016, 2019) states that statistical terms need to be explained if included. For this purpose, Cochrane offers specific guidelines on how to report descriptive statistics as well as standardized statements for reporting effect sizes. Other guidelines recommend that researchers remove all statistics (e.g., American Psychological Association, 2018) or do not provide any guidance on this issue (see Stoll et al., 2021 for a comprehensive review). This illustrates the need for further research in this area. We compared four approaches for explaining statistical terms and effect sizes in psychological meta-analyses: (1) the effect size is not reported – instead, a qualitative statement on the interpretation of its size is provided, e.g., “The relationship was medium-sized.” (qualitative statement); (2) the effect size is reported and a qualitative statement on the interpretation of its size is provided, e.g., “The correlation was r=0.40. This is a medium-sized relationship.” (effect size+qualitative statement); (3) no qualitative statement is provided, only the effect size is reported in the text and an explanation of the effect size, its boundaries and cut-off values at assist interpretation are provided below the PLS in a glossary, e.g., “What is r and what value of r implies a large effect?” (effect size+glossary); (4) the effect size, a qualitative statement and a glossary are provided (effect size+glossary+qualitative statement). In hypotheses 5 and 6, based on user feedback of Study 1, we propose that providing qualitative statements in the text of a PLS and a glossary on statistical terms promotes user experience and knowledge acquisition:

The fifth aspect, we investigated is not psychology-specific but applies to PLS in general. Cochrane guidelines (Cochrane Methods, 2013; Cochrane Norway, 2016, 2019) and existing evidence on research summary characteristics clearly point toward subheadings as an important formal element that should be included in research summaries and PLS (e.g., Kerwer et al., 2021 for PLS; but see also Hartley, 2004, 2014; Hartley and Betts, 2007 for structured abstracts in general). Additionally, first evidence suggests that further structuring the text of PLS (e.g., by means of bullet points) improves the user experience for information recipients (e.g., Ellen et al., 2014; Raynor et al., 2018; Anzinger et al., 2020). To test if these findings can be transferred to PLS of psychological meta-analyses (and can also be replicated on a conceptual level in a considerably larger and more representative sample in a randomized controlled experimental design), we compared structured text with formatted text blocks (e.g., by bullet points) to unstructured text without further formatting of text blocks within subsections. More specifically, we made the following assumptions in hypothesis 7:

The rationale for hypothesis 8 was derived from our experience testing hypotheses 1 and 2 in Study 1. We found that our participants faced problems with regard to knowledge acquisition and user experience when confronted with PLS that reported complex meta-analytical designs (i.e., results of a meta-analytic structural equation model). One might argue that meta-analytic structural equation models belong to the most complex meta-analytic techniques in the current literature. Still, we had to concede that the statement explaining the methodological approach of meta-analyses (i.e., the quality of evidence statement; see hypothesis 3 and Supplementary Material 1) solely focused on an overly simple type of “univariate” meta-analytical design (i.e., a design testing whether one effect is different from zero in the literature). This type of statement might not promote laypersons’ understanding of more complex meta-analytic designs. This is especially important because an integral feature of (even simple) meta-analytic designs is that they can model heterogeneity in effect sizes, and, as a consequence, estimate moderator effects (e.g., Sánchez-Meca and Marín-Martínez, 2010). Simple meta-analytical designs including moderator effects are arguably still not very complex compared to structural equation modeling approaches (Steinmetz et al., 2020). They are, however, significantly more complex than the design presented in our statement to explain the meta-analytic approach in hypothesis 3. Drawing on this notion, we chose this “slightly more complex” meta-analytical design (i.e., moderator analyses) as a starting point for investigating how to support PLS readers in their understanding of more complex meta-analytic design types. For this purpose, we extended the quality of evidence statement, we used for examining hypothesis 3 to outline moderator analyses. The following hypothesis was specified for comparing this extended quality of evidence statement compared to the regular, “unextended” quality of evidence statement that was used to test hypothesis 3:²

Hypotheses 1–4 and hypotheses 5–8 were preregistered at PsychArchives. Hypotheses 5–8 were preregistered after results regarding hypotheses 1–4 were available but before data collection for Study 2 began.

In our first study, we investigated the following PLS characteristics: approaches for explaining technical terms, approaches for communicating the quality of evidence, and approaches for communicating how the research question was operationalized in the synthesized studies. Study procedures, as well as the original German items and full texts of the PLS, were preregistered at PsychArchives.³ English translations of items are reported in the manuscript and exemplary English translations of PLS, the quality of evidence statement as well as knowledge items are provided in Supplementary Material 1.

In our second study, we investigated the following PLS characteristics: approaches for explaining statistical terms, approaches for communicating complex meta-analytic designs via an extended quality of evidence statement, and approaches for structuring PLS. Study procedures, as well as the original German items and full texts of the PLS, were preregistered at PsychArchives.⁵ English translations of items are reported in the manuscript and exemplary English translations of PLS, the extended quality of evidence statement as well as knowledge items are provided in Supplementary Material 1.

Data and code for this publication are available at PsychArchives: data (http://dx.doi.org/10.23668/psycharchives.5211), code (http://dx.doi.org/10.23668/psycharchives.5210).

The studies involving human participants were reviewed and approved by the ethics committee of Trier University. The participants provided their written informed consent to participate in this study.

AC, MK, and MS contributed to conception and design of Study 1. All authors contributed to the conception and design of Study 2. MK managed the data, performed the statistical analysis, and wrote the first draft of the manuscript. AC, GB, MJ, and MS wrote sections of the manuscript. All authors contributed to the article and approved the submitted version.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a conflict of interest.

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.