<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Publishing DTD v2.3 20070202//EN" "journalpublishing.dtd">
<article xml:lang="EN" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="brief-report">
<front>
<journal-meta>
<journal-id journal-id-type="publisher-id">Front. Commun.</journal-id>
<journal-title>Frontiers in Communication</journal-title>
<abbrev-journal-title abbrev-type="pubmed">Front. Commun.</abbrev-journal-title>
<issn pub-type="epub">2297-900X</issn>
<publisher>
<publisher-name>Frontiers Media S.A.</publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="doi">10.3389/fcomm.2024.1235055</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Communication</subject>
<subj-group>
<subject>Perspective</subject>
</subj-group>
</subj-group>
</article-categories>
<title-group>
<article-title>Risk communication: lessons from an ethnographic, pragmatic, and Canadian regulatory perspective</article-title>
</title-group>
<contrib-group>
<contrib contrib-type="author" corresp="yes">
<name><surname>Bhuller</surname> <given-names>Yadvinder</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
<xref ref-type="corresp" rid="c001"><sup>&#x0002A;</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/2332821/overview"/>
</contrib>
<contrib contrib-type="author">
<name><surname>Trevithick-Sutton</surname> <given-names>Colleen C.</given-names></name>
<xref ref-type="aff" rid="aff2"><sup>2</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/2517965/overview"/>
</contrib>
</contrib-group>
<aff id="aff1"><sup>1</sup><institution>Interdisciplinary School of Health Sciences, University of Ottawa</institution>, <addr-line>Ottawa, ON</addr-line>, <country>Canada</country></aff>
<aff id="aff2"><sup>2</sup><institution>Consumer and Hazardous Products Safety Directorate, Healthy Environments and Consumer Safety Branch, Health Canada</institution>, <addr-line>Ottawa, ON</addr-line>, <country>Canada</country></aff>
<author-notes>
<fn fn-type="edited-by"><p>Edited by: Tracylee Clarke, California State University, Channel Islands, United States</p></fn>
<fn fn-type="edited-by"><p>Reviewed by: Anna Kapu&#x0015B;ci&#x00144;ska, Kazimierz Wielki University of Bydgoszcz, Poland</p>
<p>Nien-Tsu Chen, California State University, Channel Islands, United States</p></fn>
<corresp id="c001">&#x0002A;Correspondence: Yadvinder Bhuller <email>ybhul063&#x00040;uottawa.ca</email></corresp>
</author-notes>
<pub-date pub-type="epub">
<day>17</day>
<month>01</month>
<year>2024</year>
</pub-date>
<pub-date pub-type="collection">
<year>2024</year>
</pub-date>
<volume>9</volume>
<elocation-id>1235055</elocation-id>
<history>
<date date-type="received">
<day>05</day>
<month>06</month>
<year>2023</year>
</date>
<date date-type="accepted">
<day>04</day>
<month>01</month>
<year>2024</year>
</date>
</history>
<permissions>
<copyright-statement>Copyright &#x000A9; 2024 Bhuller and Trevithick-Sutton.</copyright-statement>
<copyright-year>2024</copyright-year>
<copyright-holder>Bhuller and Trevithick-Sutton</copyright-holder>
<license xlink:href="http://creativecommons.org/licenses/by/4.0/"><p>This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.</p></license>
</permissions>
<abstract>
<p>In a regulatory context, it is important to understand how effective risk communication fits into the overall risk assessment, management, and decision-making process. This includes recognizing the intersections between risk analysis and the 3Ps: policy, politics, and publics, and understanding the barriers to effective communication. Risk communication is especially challenging when it requires the audience to follow and act on authoritative information or advice. Risk communicators must factor attributes such as risk perception, tolerance, and behaviors, and tailor the delivery of messages to diverse audiences. This paper captures the discourse from an intradepartmental workshop on risk communication with participants from Health Canada and the Public Health Agency of Canada. The workshop provided an opportunity to discuss and share references to existing frameworks, pertinent documents, and examples of effective risk communication strategies based on the authors&#x00027; ethnographic and pragmatic experiences. The workshop aimed to strengthen risk communication by better understanding the value in collaborating with interdisciplinary teams, applying a systems thinking lens, and finding opportunities to experiment and evaluate risk communication strategies for regulatory purposes.</p></abstract>
<kwd-group>
<kwd>Health Canada</kwd>
<kwd>regulatory context</kwd>
<kwd>risk communication</kwd>
<kwd>ethnographic</kwd>
<kwd>pragmatic</kwd>
</kwd-group>
<counts>
<fig-count count="1"/>
<table-count count="1"/>
<equation-count count="0"/>
<ref-count count="64"/>
<page-count count="8"/>
<word-count count="5919"/>
</counts>
<custom-meta-wrap>
<custom-meta>
<meta-name>section-at-acceptance</meta-name>
<meta-value>Science and Environmental Communication</meta-value>
</custom-meta>
</custom-meta-wrap>
</article-meta>
</front>
<body>
<sec id="s1">
<title>1 Introduction</title>
<sec>
<title>1.1 Background &#x00026; regulatory context</title>
<p>Health Canada is responsible for maintaining and promoting the health of people living in Canada by regulating and communicating risk information on diverse products such as pharmaceuticals, vaccines, medical devices and natural health and pest control products (Government of Canada, <xref ref-type="bibr" rid="B21">2014a</xref>). The organization&#x00027;s governance structure and legal framework supports the various programs and activities at the national/federal level.</p>
<p>The COVID-19 pandemic provides an excellent example of the government&#x00027;s responsibilities for addressing health risks at the federal, provincial/territorial, and local levels (Government of Canada, <xref ref-type="bibr" rid="B26">2022c</xref>). Under the <italic>Food and Drugs Act</italic>, Health Canada, federally, held responsibility for evaluating, approving, and communicating risk-related information on the messenger RNA vaccines. Health Canada also worked alongside portfolio partners in the Public Health Agency of Canada (PHAC), who were responsible for implementing public health measures while working in collaboration with the provinces and territories (Government of Canada, <xref ref-type="bibr" rid="B24">2022a</xref>,<xref ref-type="bibr" rid="B25">b</xref>).</p>
<p>Health Canada has an overarching, health risk decision-making framework (Health Canada, <xref ref-type="bibr" rid="B29">2000</xref>), a strategic risk communications framework (Health Canada, <xref ref-type="bibr" rid="B30">2006</xref>), and a range of tailored, program area-specific documents on risk decision-making (Health Canada, <xref ref-type="bibr" rid="B29">2000</xref>, <xref ref-type="bibr" rid="B32">2012</xref>; Government of Canada, <xref ref-type="bibr" rid="B19">2013a</xref>,<xref ref-type="bibr" rid="B20">b</xref>; Health Canada&#x00027;s Consumer Product Safety Program, <xref ref-type="bibr" rid="B38">2015</xref>; Bureau of Microbial Hazards, <xref ref-type="bibr" rid="B9">2017</xref>; Pest Management Regulatory Agency, <xref ref-type="bibr" rid="B52">2021</xref>). <italic>A Primer on Scientific Risk Assessment at Health Canada</italic> (Saner, <xref ref-type="bibr" rid="B57">2010</xref>) provides additional perspectives on established risk assessment processes while publications on next generation risk assessments (Bhuller et al., <xref ref-type="bibr" rid="B5">2021</xref>; Stucki et al., <xref ref-type="bibr" rid="B60">2022</xref>) show Health Canada&#x00027;s commitment toward more modern approaches to evaluating risks.</p>
</sec>
<sec>
<title>1.2 Risk communication</title>
<p>Health Canada&#x00027;s health risk decision-making framework defines risk communication as: &#x0201C;Any exchange of information concerning the existence, nature, form, severity or acceptability of health or environmental risks&#x0201D; (Health Canada, <xref ref-type="bibr" rid="B29">2000</xref>). This overarching decision-making framework is also integrated in Health Canada/PHAC&#x00027;s <italic>Strategic Risk Communications Framework and Handbook</italic>, which provides guiding principles, a seven-step process, and a detailed handbook for communicating risks (Health Canada, <xref ref-type="bibr" rid="B30">2006</xref>).</p>
<p>Health Canada communicates risks through product-specific information, such as labels, inserts, and monographs, as well as messages provided using other platforms (e.g., social media). For each communication, the message is tailored for the audience (Council of Canadian Academies, <xref ref-type="bibr" rid="B13">2015</xref>; Lee and Lee, <xref ref-type="bibr" rid="B44">2022</xref>). This presents one of the greatest challenges in communicating risks as audiences can be extremely broad and heterogeneous. Consequently, the communication modalities need to be diverse and flexible so that they can go from two-way to more complex approaches involving &#x0201C;multi-way&#x0201D; communication. This requires accommodating a varying degree of knowledge, perceptions, attitudes, and behavior of all parties involved (Balog-Way et al., <xref ref-type="bibr" rid="B3">2020</xref>). Further, the ethical (respect) and instrumental (reciprocity) imperatives for effective science communication, and the importance of recognizing the audience&#x00027;s reaction, can also improve clarity and delivery of a message (Moore, <xref ref-type="bibr" rid="B47">2022</xref>).</p>
<p>Some of the Canadian regulations include requirements for communicating risks to help people make informed decisions. For example, <italic>Division 5</italic> of the <italic>Food and Drug Regulations</italic> describes the requirements for human clinical trials. <italic>Section C.05.010</italic> of these regulations provides requirements for communicating risks and anticipated benefits arising from participating in a clinical trial. A published guidance document further expands and clarifies the intent of these regulations (Health Canada, <xref ref-type="bibr" rid="B34">2022a</xref>). The challenge is that policies and guidance documents can make the presentation and language more technical. Further, as enforcement activities are typically focused on the regulated industry, certain risk communication products, such as product labels, are also legally enforceable. Therefore, risk communication, in these instances, is intentionally more rigid, when compared to other types of messages.</p></sec>
</sec>
<sec id="s2">
<title>2 Role of risk science</title>
<p>Risk science provides a mechanism to generate knowledge based on the information acquired from the multiple dimensions of risk analysis (i.e., from assessing to managing and communicating risks). This includes identifying gaps in current approaches to communicating risks and how to address them. It can also play an integral role in developing or modernizing risk-based regulations, policies, research, and communication approaches to better reflect the current processes and methodologies for risk analysis (Krewski et al., <xref ref-type="bibr" rid="B43">2014</xref>; Aven, <xref ref-type="bibr" rid="B1">2018</xref>, <xref ref-type="bibr" rid="B2">2020</xref>).</p>
<p>Knowledge mobilization and translation are also an integral component for sharing best practices (Graham et al., <xref ref-type="bibr" rid="B27">2006</xref>; Health Canada, <xref ref-type="bibr" rid="B33">2017</xref>). This includes insights from applying a risk science-based mindset to understanding and improving current approaches to communicating risks. At Health Canada, this extends to staff-driven initiatives aimed at enhancing the capacity, reputation, and overall excellence of organization&#x00027;s workforce. Examples of these endeavors include the <italic>Task Force on Scientific Risk Assessment</italic> (Health Canada, n.d.) and a working group on developing and sharing learning opportunities around science literacy and communication (e.g., the <italic>Workshop on Risk Communication</italic> described in this paper).<xref ref-type="fn" rid="fn0001"><sup>1</sup></xref> Another example are ongoing collaborative initiatives aimed at clarifying the role of risk analysis, risk science knowledge, and understanding the underlying elements of contemporary and future risk decision-making at Health Canada. One of these elements is the principle of communicating in an effective way, throughout the risk decision-making process (see <xref ref-type="fig" rid="F1">Figure 1</xref>; Health Canada staff, personal communication, February 17, 2022).</p>
<fig id="F1" position="float">
<label>Figure 1</label>
<caption><p>Integrated risk decision-making. This visual incorporates <italic>Health Canada&#x00027;s Decision-Making Framework for identifying, assessing, and managing health risks</italic> into a sphere with broader contextual factors. At the center/core of the image is Health Canada&#x00027;s mandate (principle 1) with the layers of attributes relevant to decision-making around it. The importance of communicating in an effective way (principle 3) is relevant from the beginning of the decision-making process (first slice) and then throughout the decision-making process.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fcomm-09-1235055-g0001.tif"/>
</fig>
<p>Effective risk communication &#x0201C;&#x02026; involves determining the types of information that interested and affected parties need and want, and presenting this information to them in a useful and meaningful way&#x0201D; (Health Canada, <xref ref-type="bibr" rid="B29">2000</xref>). Dr. Vincent T. Covello&#x00027;s <italic>Communicating in Risk, Crisis, and High Stress Situations: Evidence-Based Strategies and Practice</italic> provides additional and detailed insights on the principles, theories, tools, and techniques for communicating risks (Covello, <xref ref-type="bibr" rid="B14">2022</xref>). Risk communication is also an integral component of emergency preparedness and response. Therefore, publicly available and reputable manuals and tools [e.g., the <italic>Crisis &#x00026; Emergency Risk Communication</italic> platform (Centers for Disease Control and Prevention, n.d.)] and models designed for spokespersons [e.g., the <italic>IDEA</italic> model (Sellnow and Sellnow, <xref ref-type="bibr" rid="B58">2019</xref>)] further strengthen effective communication while complementing the guidance provided in the <italic>Strategic Risk Communications Framework and Handbook</italic> (Health Canada, <xref ref-type="bibr" rid="B30">2006</xref>).</p>
<p>When determining the most &#x0201C;useful and meaningful way&#x0201D; for communicating risks effectively, risk communicators should also consider risk science-based knowledge on how to strengthen the messaging through visualizing the science. For example, Lee and Lee (<xref ref-type="bibr" rid="B44">2022</xref>) showed how participants in a study recalled more information and had favorable attitudes toward genetically modified food when the science news was presented using infographics. It is also useful to consider less contemporary material, such as the National Research Council&#x00027;s <italic>Improving Risk Communication</italic>, as these documents provide historical contexts and can help inform what attributes for communicating risks have not changed (National Research Council, <xref ref-type="bibr" rid="B48">1989</xref>).</p></sec>
<sec id="s3">
<title>3 Workshop objectives &#x00026; key messages</title>
<p>The virtual workshop (Microsoft Teams) provided an interactive venue for discussing effective risk communication considerations and best practices. The event engaged the scientific, regulatory, and research community across Health Canada and PHAC, and over 200 participants attended the session.</p>
<p>The workshop&#x00027;s format included an armchair discussion (between the two authors) followed by an open session where the principal author addressed the top ten pertinent questions raised by the participants and collected using <ext-link ext-link-type="uri" xlink:href="https://Slido.com">Slido.com</ext-link> (<xref ref-type="table" rid="T1">Table 1</xref>). Based on the participants&#x00027; feedback during the workshop, this format was well received and supported the sharing of strategies for effective risk communication.</p>
<table-wrap position="float" id="T1">
<label>Table 1</label>
<caption><p>Top 10 questions.</p></caption>
<table frame="box" rules="all">
<tbody>
<tr>
<th valign="top" align="left">1. How do you communicate when risk is uncertain? How do you accurately communicate when risk is changing?<break/>2. Although risk can often be calculated objectively, it is often perceived subjectively. How can we overcome these differences in short, one-off communications?<break/>3. How can one be proactive in our messaging to address those sharing counter messages/misinformation to the public while being transparent in communicating risks?<break/>4. How does one deal with a situation where your audience has a deep level of mistrust against the source of information (e.g., government, drugs/chemicals, industry, etc.)?<break/>5. Many have emotional reactions when talking about risk. What is an elevator pitch to leaders to successfully communicate without being seen as &#x0201C;fear mongering&#x0201D;?<break/>6. How does behavioral science work in today&#x00027;s age of social media, vs. in the &#x0201C;old days&#x0201D; where people had far fewer avenues of information?<break/>7. How to communicate the difference between hazard and risk and what is best approach to communicate risk when there are many factors that affect risk?<break/>8. What is the most common and experienced misunderstanding about risk? How can we reduce misunderstandings?<break/>9. There is tension between policy and science at times, conflicting information overload, and polarized views of issues&#x02014;what is prioritized or deemed a risk?<break/>10. What are common issues new science/risk communicators have? Are there any quick recommendations on how to deal with these issues?</th>
</tr>
</tbody>
</table>
</table-wrap>
<p>The objectives of this workshop were to: (i) engage staff from Health Canada and PHAC interested in learning more on risk communication at Health Canada, and (ii) use the <italic>Strategic Risk Communications Framework and Handbook</italic> as a starting point for a broader discourse on communicating risks. The key points discussed during this workshop are summarized below using four discrete themes and includes information addressing the ten questions raised by the participants during the workshop.</p>
<sec>
<title>3.1 Consider a holistic, systems thinking approach</title>
<p>Complex health and environmental issues, such as the opioid crisis (Belzak and Halverson, <xref ref-type="bibr" rid="B4">2018</xref>), require collaborative efforts and governance from all sectors of society and levels of government. Systems thinking, defined as &#x0201C;&#x02026; seeing how things are connected to each other within some notion of a whole entity&#x0201D; (Peters, <xref ref-type="bibr" rid="B53">2014</xref>), provides a holistic strategy for decision-makers to consider multiple perspectives necessary for addressing these types of wicked problems.</p>
<p>Using a systems thinking lens, <xref ref-type="fig" rid="F1">Figure 1</xref> shows how Health Canada&#x00027;s risk management decision-making process is integrated through various steps (visualized as circles) which are further positioned inside a broader system of elements (visualized as slices and bubbles with text). The significance of systems thinking is the ability to recognize the complexity, intersections, and interactions of the overall system and how an impact in one area can affect another or the entire decision-making process. Further, systems thinking and tools to mobilize additional knowledge, such as causal loop diagrams (Haynes et al., <xref ref-type="bibr" rid="B28">2020</xref>), provide insights which could assist in optimizing the strategy for addressing the risk issue. For example, an international decision to ban a particular product could trigger the first step of the risk management process (i.e., identify the issue and its content) at the national level. Once the process starts, a systems thinking approach could help determine the strategy for communicating the risks. For complex issues this includes considering a Pan-Canadian approach and collaboration at all levels of Government (i.e., Federal, Provincial/Territorial, and Local). A systems thinking lens also provides an opportunity to adapt the message based on an understanding of the science behind human behavior and attributes such as risk perception and tolerance (Krewski et al., <xref ref-type="bibr" rid="B42">2006</xref>; Council of Canadian Academies, <xref ref-type="bibr" rid="B13">2015</xref>; Kelly and Barker, <xref ref-type="bibr" rid="B41">2016</xref>).</p>
<p>Health Canada (<xref ref-type="bibr" rid="B29">2000</xref>) has historically relied on this type of broad thinking and collaborative approach and the federal government&#x00027;s decision to re-label certain over-the-counter cough and cold products to no longer permit the use of certain products in children under 6 years of age provides one example. To aid in communicating this decision, Health Canada collaborated with various sectors of society, including the public. Further, each sector also discussed their role in delivering the message once the revised labels were in the marketplace (Health Canada, <xref ref-type="bibr" rid="B31">2008</xref>, <xref ref-type="bibr" rid="B36">2023</xref>; Shefrin and Goldman, <xref ref-type="bibr" rid="B59">2009</xref>). More recent examples include the federal approach toward addressing antimicrobial resistance (Government of Canada, <xref ref-type="bibr" rid="B22">2014b</xref>; Public Health Agency of Canada, <xref ref-type="bibr" rid="B55">2015</xref>, <xref ref-type="bibr" rid="B56">2017</xref>), incorporating new approach methodologies into risk assessment (Bhuller et al., <xref ref-type="bibr" rid="B5">2021</xref>; Bury et al., <xref ref-type="bibr" rid="B10">2021</xref>; Clippinger et al., <xref ref-type="bibr" rid="B12">2022</xref>; Gilmour et al., <xref ref-type="bibr" rid="B17">2022</xref>; Stucki et al., <xref ref-type="bibr" rid="B60">2022</xref>; Van Der Zalm et al., <xref ref-type="bibr" rid="B64">2022</xref>), and the management of the COVID-19 pandemic (Government of Canada, <xref ref-type="bibr" rid="B24">2022a</xref>,<xref ref-type="bibr" rid="B25">b</xref>,<xref ref-type="bibr" rid="B26">c</xref>).</p>
<p>Some of the challenges of taking a systems thinking approach include the time, resources (both human and financial), and governance structure required to establish and maintain the collaborative space (Trochim et al., <xref ref-type="bibr" rid="B63">2006</xref>; Boswell et al., <xref ref-type="bibr" rid="B7">2021</xref>). Another challenge is effectively communicating risks within an organization when collaborating with diverse programs. Consequently, while developing strategies for communicating risks externally, sufficient time is also required to ensure a mutual, internal understanding.</p>
<p>While systems thinking is not required for all health and environmental risk issues, applying a systems thinking lens can help guide complex issues (Peters, <xref ref-type="bibr" rid="B53">2014</xref>; Gates, <xref ref-type="bibr" rid="B16">2016</xref>). Further, for global public health risks, systems thinking can provide mechanisms for incorporating agile approaches to knowledge mobilization by developing concepts through experimentation (Haynes et al., <xref ref-type="bibr" rid="B28">2020</xref>). For example, improving the communication of risk within an internal and interdisciplinary team using several rounds of the (<italic>broken) telephone game</italic>. For those unfamiliar with this game, each person whispers a message to their immediate neighbor; at the end of the line, the last person says the message aloud to the room. In this example, members of an interdisciplinary team relay the risk communication message with a goal for making the outcoming response less technical. This type of exercise helps guide a team&#x00027;s understanding of choosing practical and easily understandable words and language for use in communicating risks.</p>
</sec>
<sec>
<title>3.2 Incorporate BroadCAST-3Cs</title>
<p>Experts in risk communication have published key attributes, principles, and characteristics that can generate and strengthen messages (Health Canada, <xref ref-type="bibr" rid="B30">2006</xref>; Lundgren and McMakin, <xref ref-type="bibr" rid="B45">2013</xref>; Council of Canadian Academies, <xref ref-type="bibr" rid="B13">2015</xref>; Aven, <xref ref-type="bibr" rid="B2">2020</xref>; Covello, <xref ref-type="bibr" rid="B14">2022</xref>; Friedman and Rogers, <xref ref-type="bibr" rid="B15">2023</xref>; Peters, <xref ref-type="bibr" rid="B54">2023</xref>). Another tool is a memory aid known as: <italic>BroadCAST-3Cs</italic> which brings together several of these risk communication attributes.<xref ref-type="fn" rid="fn0002"><sup>2</sup></xref> &#x0201C;Broad&#x0201D; is a reminder that regulatory risk communication, by necessity, is often for a large and heterogeneous population (Goerlandt et al., <xref ref-type="bibr" rid="B18">2020</xref>). &#x0201C;CAST&#x0201D; stands for <bold>core</bold>, the underlying reason for the risk communication, <bold>audience</bold>, <bold>spokesperson</bold>, and the importance for providing the risk communication message in a <bold>timely</bold> manner. The &#x0201C;3Cs&#x0201D; is a mnemonic for conveying information that reflects the risk <bold>context</bold>, must be <bold>clear</bold> (sometimes written as <bold>clarity</bold>), and <bold>concise</bold>. Risk communicators, therefore, should adjust messages to meet the 3Cs of each target audience thereby informing them in a useful and meaningful way.</p>
<p>When identifying the <bold>core</bold> of a health or environmental risk issue, it is important to develop this concept from a principle-based mindset and not a position-based one (e.g., risk sciences vs. policy). Further, if the core principle is promoting health and safety, each member of an interdisciplinary team may approach a risk issue from a unique perspective, but what unites them is the core principle. It is also important to recognize that the technical leads for an issue may not be the <bold>spokesperson</bold> responsible for relaying the information to the public. Similarly, the <bold>audience</bold> can also change from internal staff to external stakeholders, partners, and publics. As a result, the <bold>spokesperson</bold> must adjust the messaging according to the <bold>audience</bold> while delivering the message in a <bold>timely</bold> manner, and models, such as &#x0201C;IDEA,&#x0201D; can also provide useful insights for this process (Sellnow and Sellnow, <xref ref-type="bibr" rid="B58">2019</xref>).</p>
<p>Given the technical nature surrounding risk sciences, assessment, and management, there sometimes is a misconception that the plain language principles <bold>clear</bold> and <bold>concise</bold> are not applicable to risk communication. There are also expectations that any communication with the public must be at a certain grade level, which is not the case. Plain language principles can be applied to all types of communication. Updates to the international standards (ISO Standards, <xref ref-type="bibr" rid="B40">2023</xref>) and national guidance documents (Government of Canada, <xref ref-type="bibr" rid="B23">2020</xref>) aim to provide additional guidance in this regard (plain language expert, personal communication, June 17, 2021).</p>
</sec>
<sec>
<title>3.3 Address uncertainty (data gaps), misinformation, and advances in scientific knowledge</title>
<p>During the COVID-19 pandemic, governments at all levels had to manage uncertainty, misinformation, and advances in science. Health Canada continued to update and communicate regulatory advice using various platforms. Further, providing independent, regulatory, and timely input aligned with recommendations on how to maintain public trust and confidence during a global crisis (Mihelj et al., <xref ref-type="bibr" rid="B46">2022</xref>). Health Canada officials also successfully used this approach to address the changing environment by providing clear communication for the safe use of diverse products, such as ultraviolet radiation-emitting devices (Health Canada, <xref ref-type="bibr" rid="B35">2022b</xref>).</p>
<p>With advancements in technology, access to information and opinions is easier than ever before. Users acquire risk information from traditional sources, such as radio, articles, and television, and more recent forms of social media and artificial intelligence technologies. This imposes an additional burden to release effective regulatory risk communication in a timely manner, especially when dealing with uncertainty and misinformation (Howell and Brossard, <xref ref-type="bibr" rid="B39">2021</xref>).</p>
<p>Regulatory risk communication must also address the concerns of the audience by understanding their position, behavior, and level of receptivity. This is no small feat to deliver on the multitude of platforms currently in use. Consequently, checking the &#x0201C;pulse&#x0201D; on how the audience received the information is also important in confirming that the message was delivered in the right context and by the appropriate spokesperson. Further, risk communicators must also ensure that audiences can access and use credible and trustworthy sources, a responsibility that extends to all public servants as they can inform individuals in their respective sphere of influence.</p>
</sec>
<sec>
<title>3.4 Plan time to learn from setbacks and successes</title>
<p>In 2015, the Council of Canadian Academies published <italic>Health Product Risk Communication: Is the Message Getting Through?</italic> These experts described the regulatory context for health product-related risk communication, documented existing best practices and tools, and recommended methods to evaluate the extent of reach to the target audience and impact of the conveyed message (Council of Canadian Academies, <xref ref-type="bibr" rid="B13">2015</xref>). Additional recommendations included the importance of performance measurements using theory- and paradigm-based evaluations, such as realism (Pawson and Tilley, <xref ref-type="bibr" rid="B51">2004</xref>; Pawson et al., <xref ref-type="bibr" rid="B50">2005</xref>; Blamey and Mackenzie, <xref ref-type="bibr" rid="B6">2016</xref>; Breuer et al., <xref ref-type="bibr" rid="B8">2016</xref>; Pawson, <xref ref-type="bibr" rid="B49">2017</xref>; Treasury Board of Canada Secretariat, <xref ref-type="bibr" rid="B61">2021</xref>, n.d.).</p>
<p>There are several barriers in undertaking evaluations and here we identify three key barriers to robustly evaluating risk communication: resource constraints (time, human, and funding), complex performance measurement indicators, and limited expertise. Federal resources are typically devoted to core activities and projects. Developing an underlying theory for evaluation and measurement can be time consuming and complex. Consequently, it may be difficult to find teams with appropriate expertise. However, these barriers should not prevent authorities from dedicating time to, at least, discuss and reflect on lessons learned. This is especially important for initiatives without a formal evaluation component.</p></sec>
</sec>
<sec id="s4">
<title>4 Discussion</title>
<p>Risk communication is complex because it requires one to consider all the layers and attributes in the decision-making process (see <xref ref-type="fig" rid="F1">Figure 1</xref>) as well as how the audience will receive the message. However, this complexity must not prevent one from taking measures to effectively communicate risk. This means recognizing how to adapt messages to meet the requirements of each target audience. One way to help clarify and further strengthen the message, for real world risk communication, is to incorporate approaches, such as the (<italic>broken) telephone game</italic> and lessons learned activities.</p>
<p>Effective communication occurs throughout the decision-making process and is an essential element for bringing together the risk assessment (science) phase with the management (science-policy) and broader (policy-political-publics) space. As the risk communication message develops within an organization, the individual/team responsible for optimizing the message relies on the input from various staff. Further, regulatory risk communication often depends on established policies, guidance documents, templates, and in-house expertise to help develop and strengthen the message. We also include pertinent references to help encourage everyone to review this information along with other models and frameworks designed to strengthen risk communication.</p>
<p>In the workshop, we started from the existing federal risk communication framework and presented the memory aid, <italic>BroadCAST-3Cs</italic>, which integrate key attributes for effective risk communication. We see the pragmatic application of this mnemonic in assisting risk assessors, managers, communicators, and staff who are part of a risk team or are providing input to such a team by addressing some of the challenges associated with communicating risks (see <xref ref-type="table" rid="T1">Table 1</xref>). For example, taking a &#x0201C;broad&#x0201D; approach and seeking feedback from an interdisciplinary team can help the spokesperson understand how to address uncertainties, be more proactive, account for the subjective perceptions of risks, and tailor the messages from the &#x0201C;science&#x0201D; to the &#x0201C;policy, political, and broader publics&#x0201D; spaces. This insight includes an understanding of how any adaptations to the communication are not based on wordsmithing alone or providing information in plain language. Rather, it is a deeper reflection of the diversity in the underlying values, ideological orientations, and sociocultural beliefs related to the risks of concern. Further, a systems thinking lens provides a mechanism to recognize how a principle-based mindset is important (e.g., at the interface between science-policy); however, this does not preclude the need to understand diverse positions and the underlying reasons for such stances, especially at the policy-political-publics space, as this is critical in tailoring the message according to the audience. A systems thinking lens also ensures that the spokesperson is aware of how risk communication in one space can have an impact on other areas within the system and that the risk assessment, management, and communication steps are positioned within a broader set of elements, which are also important for the overall risk decision-making process (see <xref ref-type="fig" rid="F1">Figure 1</xref>).</p>
<p>In closing, our key messages align well with recent publications on risk communication practices and available models and frameworks for effective risk communication. The participants from the workshop left the session motivated to: (i) Embrace the complexities of effective risk communication; (ii) Incorporate the concepts developed from our lived experience; (iii) Explore other relevant and credible sources of information; and (iv) Apply a systems thinking lens to help develop effective risk communication. We hope this paper encourages you to do the same.</p></sec>
<sec sec-type="data-availability" id="s5">
<title>Data availability statement</title>
<p>The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.</p></sec>
<sec sec-type="author-contributions" id="s6">
<title>Author contributions</title>
<p>YB and CT-S designed the concept of the workshop, contributed important intellectual content, and helped in the writing and revisions of this paper. All authors read and approved the final manuscript.</p>
</sec>
</body>
<back>
<ack><p>The authors would like to acknowledge the contributions of Deanna Chan, Kuini Chuen, and Tara Bower and their insightful thoughts and detailed comments on an earlier version of the paper. The authors would also like to thank Fr&#x000E9;d&#x000E9;ric Bissonnette, Stefania Trombetti, David K. Lee, Jessica Roberts, Brad Fisher, and Dr. Daniel Krewski for reviewing the paper. The authors also thank Dr. Cara Tannenbaum for her ongoing support and guidance.</p>
</ack>
<sec sec-type="COI-statement" id="conf1">
<title>Conflict of interest</title>
<p>While Health Canada&#x00027;s Pest Management Regulatory Agency covered the costs to publish this work, the workshop and subsequent report were conducted in the absence of any commercial or financial relationships which could be construed as a potential conflict of interest. Further, the information provided in this paper reflects the authors&#x00027; views and is not representative of the opinions or policies of Health Canada or the Government of Canada. The remaining author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.</p>
</sec>
<sec sec-type="disclaimer" id="s7">
<title>Publisher&#x00027;s note</title>
<p>All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.</p>
</sec>
<fn-group>
<fn id="fn0001"><p><sup>1</sup>This working group is led by Dr. Colleen C Trevithick-Sutton and the workshop was delivered by both authors.</p></fn>
<fn id="fn0002"><p><sup>2</sup>BroadCAST-3Cs is a memory aid created by the principal author.</p></fn>
</fn-group>
<ref-list>
<title>References</title>
<ref id="B1">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Aven</surname> <given-names>T.</given-names></name></person-group> (<year>2018</year>). <article-title>An emerging new risk analysis science: foundations and implications</article-title>. <source>Risk Anal.</source> <volume>38</volume>, <fpage>876</fpage>&#x02013;<lpage>888</lpage>. <pub-id pub-id-type="doi">10.1111/risa.12899</pub-id><pub-id pub-id-type="pmid">28881409</pub-id></citation></ref>
<ref id="B2">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Aven</surname> <given-names>T.</given-names></name></person-group> (<year>2020</year>). <article-title>Risk science contributions: three illustrating examples</article-title>. <source>Risk Anal.</source> <volume>40</volume>, <fpage>1889</fpage>&#x02013;<lpage>1899</lpage>. <pub-id pub-id-type="doi">10.1111/risa.13549</pub-id><pub-id pub-id-type="pmid">32634258</pub-id></citation></ref>
<ref id="B3">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Balog-Way</surname> <given-names>D.</given-names></name> <name><surname>McComas</surname> <given-names>K.</given-names></name> <name><surname>Besley</surname> <given-names>J.</given-names></name></person-group> (<year>2020</year>). <article-title>The evolving field of risk communication</article-title>. <source>Risk Anal.</source> <volume>40</volume>, <fpage>2240</fpage>&#x02013;<lpage>2262</lpage>. <pub-id pub-id-type="doi">10.1111/risa.13615</pub-id><pub-id pub-id-type="pmid">33084114</pub-id></citation></ref>
<ref id="B4">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Belzak</surname> <given-names>L.</given-names></name> <name><surname>Halverson</surname> <given-names>J.</given-names></name></person-group> (<year>2018</year>). <article-title>The opioid crisis in Canada: a national perspective</article-title>. <source>Health Promot. Chronic Dis. Prev. Can.</source> <volume>38</volume>, <fpage>224</fpage>&#x02013;<lpage>233</lpage>. <pub-id pub-id-type="doi">10.24095/hpcdp.38.6.02</pub-id></citation>
</ref>
<ref id="B5">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Bhuller</surname> <given-names>Y.</given-names></name> <name><surname>Ramsingh</surname> <given-names>D.</given-names></name> <name><surname>Beal</surname> <given-names>M.</given-names></name> <name><surname>Kulkarni</surname> <given-names>S.</given-names></name> <name><surname>Gagne</surname> <given-names>M.</given-names></name> <name><surname>Barton-Maclaren</surname> <given-names>T. S.</given-names></name></person-group> (<year>2021</year>). <article-title>Canadian regulatory perspective on next generation risk assessments for pest control products and industrial chemicals</article-title>. <source>Front. Toxicol.</source> <volume>3</volume>:<fpage>748406</fpage>. <pub-id pub-id-type="doi">10.3389/ftox.2021.748406</pub-id><pub-id pub-id-type="pmid">35295100</pub-id></citation></ref>
<ref id="B6">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Blamey</surname> <given-names>A.</given-names></name> <name><surname>Mackenzie</surname> <given-names>M.</given-names></name></person-group> (<year>2016</year>). <article-title>Theories of change and realistic evaluation</article-title>. <source>Evaluation</source> <volume>13</volume>, <fpage>439</fpage>&#x02013;<lpage>455</lpage>. <pub-id pub-id-type="doi">10.1177/1356389007082129</pub-id></citation>
</ref>
<ref id="B7">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Boswell</surname> <given-names>J.</given-names></name> <name><surname>Baird</surname> <given-names>J.</given-names></name> <name><surname>Taheem</surname> <given-names>R.</given-names></name></person-group> (<year>2021</year>). <article-title>The challenges of putting systems thinking into practice; Comment on &#x0201C;what can policy-makers get out of systems thinking? Policy partners&#x00027; experiences of a systems-focused research collaboration in preventive health&#x0201D;</article-title>. <source>Int. J. Health Policy Manag</source>. <volume>10</volume>, <fpage>290</fpage>&#x02013;<lpage>292</lpage>. <pub-id pub-id-type="doi">10.34172/ijhpm.2020.92</pub-id><pub-id pub-id-type="pmid">32610827</pub-id></citation></ref>
<ref id="B8">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Breuer</surname> <given-names>E.</given-names></name> <name><surname>Lee</surname> <given-names>L.</given-names></name> <name><surname>De Silva</surname> <given-names>M.</given-names></name> <name><surname>Lund</surname> <given-names>C.</given-names></name></person-group> (<year>2016</year>). <article-title>Using theory of change to design and evaluate public health interventions: a systematic review</article-title>. <source>Implement Sci.</source> <volume>11</volume>:<fpage>63</fpage>. <pub-id pub-id-type="doi">10.1186/s13012-016-0422-6</pub-id></citation>
</ref>
<ref id="B9">
<citation citation-type="book"><person-group person-group-type="author"><collab>Bureau of Microbial Hazards F. D.</collab></person-group> (<year>2017</year>). <source>Framework for Initiating and Conducting Risk Analysis Activities on Microbial Hazards in Food</source>. <publisher-loc>Ottawa</publisher-loc>: <publisher-name>Health Canada</publisher-name>.</citation>
</ref>
<ref id="B10">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Bury</surname> <given-names>D.</given-names></name> <name><surname>Alexander-White</surname> <given-names>C.</given-names></name> <name><surname>Clewell</surname> <given-names>H. J.</given-names></name> <name><surname>Cronin</surname> <given-names>M.</given-names></name> <name><surname>Desprez</surname> <given-names>B.</given-names></name> <name><surname>Detroyer</surname> <given-names>A.</given-names></name> <etal/></person-group>. (<year>2021</year>). <article-title>New framework for a non-animal approach adequately assures the safety of cosmetic ingredients - a case study on caffeine</article-title>. <source>Regul. Toxicol. Pharmacol.</source> <volume>123</volume>:<fpage>104931</fpage>. <pub-id pub-id-type="doi">10.1016/j.yrtph.2021.104931</pub-id><pub-id pub-id-type="pmid">33905778</pub-id></citation></ref>
<ref id="B11">
<citation citation-type="web"><person-group person-group-type="author"><collab>Centers for Disease Control Prevention (n.d.)</collab></person-group>. <source>Crisis Emergency Risk Communication (CERC) [Online]</source>. Available online at: <ext-link ext-link-type="uri" xlink:href="https://emergency.cdc.gov/cerc/">https://emergency.cdc.gov/cerc/</ext-link> (accessed January 10 2023).</citation>
</ref>
<ref id="B12">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Clippinger</surname> <given-names>A. J.</given-names></name> <name><surname>Henry</surname> <given-names>T.</given-names></name> <name><surname>Hirn</surname> <given-names>C.</given-names></name> <name><surname>Stedeford</surname> <given-names>T.</given-names></name> <name><surname>Stucki</surname> <given-names>A.</given-names></name> <name><surname>Terry</surname> <given-names>C. E.</given-names></name></person-group> (<year>2022</year>). <article-title>Chemical testing using new approach methodologies (NAMs)</article-title>. <source>Front. Toxicol</source>. <volume>4</volume>:<fpage>1048900</fpage>. <pub-id pub-id-type="doi">10.3389/978-2-83250-859-6</pub-id></citation>
</ref>
<ref id="B13">
<citation citation-type="book"><person-group person-group-type="author"><collab>Council of Canadian Academies</collab></person-group> (<year>2015</year>). <source>Health Product Risk Communication: Is the Message Getting Through? The Expert Panel on the Effectiveness of Health Product Risk Communication</source>. <publisher-loc>Ottawa, Canada</publisher-loc>.</citation>
</ref>
<ref id="B14">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Covello</surname> <given-names>V. T.</given-names></name></person-group> (<year>2022</year>). <source>Communicating in Risk, Crisis, and High Stress Situations: Evidence Based Strategies and Practice</source>. <publisher-loc>Hoboken, NJ</publisher-loc>: <publisher-name>John Wiley and Sons</publisher-name>. <pub-id pub-id-type="doi">10.1002/9781119081753</pub-id></citation>
</ref>
<ref id="B15">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Friedman</surname> <given-names>S. M.</given-names></name> <name><surname>Rogers</surname> <given-names>C. L.</given-names></name></person-group> (<year>2023</year>). <article-title>Scientists and journalists and communicating uncertainty: collaborating with Sharon Dunwoody</article-title>. <source>Sci. Commun.</source> <volume>45</volume>, <fpage>117</fpage>&#x02013;<lpage>126</lpage>. <pub-id pub-id-type="doi">10.1177/10755470221143391</pub-id></citation>
</ref>
<ref id="B16">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gates</surname> <given-names>E. F.</given-names></name></person-group> (<year>2016</year>). <article-title>Making sense of the emerging conversation in evaluation about systems thinking and complexity science</article-title>. <source>Eval. Program Plann.</source> <volume>59</volume>, <fpage>62</fpage>&#x02013;<lpage>73</lpage>. <pub-id pub-id-type="doi">10.1016/j.evalprogplan.2016.08.004</pub-id><pub-id pub-id-type="pmid">27591941</pub-id></citation></ref>
<ref id="B17">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gilmour</surname> <given-names>N.</given-names></name> <name><surname>Reynolds</surname> <given-names>J.</given-names></name> <name><surname>Przybylak</surname> <given-names>K.</given-names></name> <name><surname>Aleksic</surname> <given-names>M.</given-names></name> <name><surname>Aptula</surname> <given-names>N.</given-names></name> <name><surname>Baltazar</surname> <given-names>M. T.</given-names></name> <etal/></person-group>. (<year>2022</year>). <article-title>Next generation risk assessment for skin allergy: decision making using new approach methodologies</article-title>. <source>Regul. Toxicol. Pharmacol.</source> <volume>131</volume>:<fpage>105159</fpage>. <pub-id pub-id-type="doi">10.1016/j.yrtph.2022.105159</pub-id><pub-id pub-id-type="pmid">35311660</pub-id></citation></ref>
<ref id="B18">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Goerlandt</surname> <given-names>F.</given-names></name> <name><surname>Li</surname> <given-names>J.</given-names></name> <name><surname>Reniers</surname> <given-names>G.</given-names></name></person-group> (<year>2020</year>). <article-title>The landscape of risk communication research: a scientometric analysis</article-title>. <source>Int. J. Environ. Res. Public Health</source> <volume>17</volume>:<fpage>3255</fpage>. <pub-id pub-id-type="doi">10.3390/ijerph17093255</pub-id><pub-id pub-id-type="pmid">32392734</pub-id></citation></ref>
<ref id="B19">
<citation citation-type="journal"><person-group person-group-type="author"><collab>Government of Canada</collab></person-group> (<year>2013a</year>). <source>Federal Contaminated Sites Action Plan (FCSAP): Decision-Making Framework.</source> Environment and Climate Change Canada.</citation>
</ref>
<ref id="B20">
<citation citation-type="journal"><person-group person-group-type="author"><collab>Government of Canada</collab></person-group> (<year>2013b</year>). <source>Healthy and Safe Food for Canadians Framework</source>. Canada.</citation>
</ref>
<ref id="B21">
<citation citation-type="web"><person-group person-group-type="author"><collab>Government of Canada</collab></person-group> (<year>2014a</year>). <source>About Health Canada</source>. Available online at: <ext-link ext-link-type="uri" xlink:href="https://www.canada.ca/en/health-canada/corporate/about-health-canada.html">https://www.canada.ca/en/health-canada/corporate/about-health-canada.html</ext-link> (accessed January 9, 2023).</citation>
</ref>
<ref id="B22">
<citation citation-type="web"><person-group person-group-type="author"><collab>Government of Canada</collab></person-group> (<year>2014b</year>). <source>Antimicrobial Resistance and Use in Canada - A Federal Framework for Action. Public Health Agency of Canada</source>. Available online at: <ext-link ext-link-type="uri" xlink:href="https://www.canada.ca/content/dam/canada/health-canada/migration/healthy-canadians/alt/pdf/drugs-products-medicaments-produits/buying-using-achat-utilisation/antibiotic-resistance-antibiotique/antimicrobial-framework-cadre-antimicrobiens-eng.pdf">https://www.canada.ca/content/dam/canada/health-canada/migration/healthy-canadians/alt/pdf/drugs-products-medicaments-produits/buying-using-achat-utilisation/antibiotic-resistance-antibiotique/antimicrobial-framework-cadre-antimicrobiens-eng.pdf</ext-link> (accessed January 10, 2023).</citation>
</ref>
<ref id="B23">
<citation citation-type="web"><person-group person-group-type="author"><collab>Government of Canada</collab></person-group> (<year>2020</year>). <source>Canada.ca Content Style Guide. Tresury Board of Canada Secretariat</source>. Available online at: <ext-link ext-link-type="uri" xlink:href="https://www.canada.ca/en/treasury-board-secretariat/services/government-communications/canada-content-style-guide.html">https://www.canada.ca/en/treasury-board-secretariat/services/government-communications/canada-content-style-guide.html</ext-link> (accessed January 18, 2023).</citation>
</ref>
<ref id="B24">
<citation citation-type="web"><person-group person-group-type="author"><collab>Government of Canada</collab></person-group> (<year>2022a</year>). <source>COVID-19 mRNA vaccines. Health Canada</source>. Available online at: <ext-link ext-link-type="uri" xlink:href="https://www.canada.ca/en/health-canada/services/drugs-health-products/covid19-industry/drugs-vaccines-treatments/vaccines/type-mrna.html">https://www.canada.ca/en/health-canada/services/drugs-health-products/covid19-industry/drugs-vaccines-treatments/vaccines/type-mrna.html</ext-link> (accessed January 10, 2023).</citation>
</ref>
<ref id="B25">
<citation citation-type="web"><person-group person-group-type="author"><collab>Government of Canada</collab></person-group> (<year>2022b</year>). <source>Drug and vaccine authorizations for COVID-19: Overview. Health Canada</source>. Available online at: <ext-link ext-link-type="uri" xlink:href="https://www.canada.ca/en/health-canada/services/drugs-health-products/covid19-industry/drugs-vaccines-treatments/authorization.html">https://www.canada.ca/en/health-canada/services/drugs-health-products/covid19-industry/drugs-vaccines-treatments/authorization.html</ext-link> (accessed January 10, 2023).</citation>
</ref>
<ref id="B26">
<citation citation-type="web"><person-group person-group-type="author"><collab>Government of Canada</collab></person-group> (<year>2022c</year>). <source>Public health ethics framework: A guide for use in response to the COVID-19 pandemic in Canada</source>. Available online at: <ext-link ext-link-type="uri" xlink:href="https://www.canada.ca/en/public-health/services/diseases/2019-novel-coronavirus-infection/canadas-reponse/ethics-framework-guide-use-response-covid-19-pandemic.html">https://www.canada.ca/en/public-health/services/diseases/2019-novel-coronavirus-infection/canadas-reponse/ethics-framework-guide-use-response-covid-19-pandemic.html</ext-link> (accessed January 10, 2023).</citation>
</ref>
<ref id="B27">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Graham</surname> <given-names>I. D.</given-names></name> <name><surname>Logan</surname> <given-names>J.</given-names></name> <name><surname>Harrison</surname> <given-names>M. B.</given-names></name> <name><surname>Straus</surname> <given-names>S. E.</given-names></name> <name><surname>Tetroe</surname> <given-names>J.</given-names></name> <name><surname>Caswell</surname> <given-names>W.</given-names></name> <etal/></person-group>. (<year>2006</year>). <article-title>Lost in knowledge translation: time for a map?</article-title> <source>J. Contin. Educ. Health Prof.</source> <volume>26</volume>, <fpage>13</fpage>&#x02013;<lpage>24</lpage>. <pub-id pub-id-type="doi">10.1002/chp.47</pub-id><pub-id pub-id-type="pmid">16557505</pub-id></citation></ref>
<ref id="B28">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Haynes</surname> <given-names>A.</given-names></name> <name><surname>Rychetnik</surname> <given-names>L.</given-names></name> <name><surname>Finegood</surname> <given-names>D.</given-names></name> <name><surname>Irving</surname> <given-names>M.</given-names></name> <name><surname>Freebairn</surname> <given-names>L.</given-names></name> <name><surname>Hawe</surname> <given-names>P.</given-names></name></person-group> (<year>2020</year>). <article-title>Applying systems thinking to knowledge mobilisation in public health</article-title>. <source>Health Res. Policy Syst.</source> <volume>18</volume>:<fpage>134</fpage>. <pub-id pub-id-type="doi">10.1186/s12961-020-00600-1</pub-id><pub-id pub-id-type="pmid">33203438</pub-id></citation></ref>
<ref id="B29">
<citation citation-type="book"><person-group person-group-type="author"><collab>Health Canada</collab></person-group> (<year>2000</year>). <source>Health Canada Decision-Making Framework for Identifying, Assessing, and Managing Health Risks</source>. <publisher-name>Health Canada</publisher-name>.</citation>
</ref>
<ref id="B30">
<citation citation-type="book"><person-group person-group-type="author"><collab>Health Canada</collab></person-group> (<year>2006</year>). <source>Strategic Risk Communications Framework and Handbook</source>. <publisher-loc>Thorne Butte</publisher-loc>: <publisher-name>Decision Partners Inc</publisher-name>.</citation>
</ref>
<ref id="B31">
<citation citation-type="web"><person-group person-group-type="author"><collab>Health Canada</collab></person-group> (<year>2008</year>). <source>Tear Sheet - Cough and Cold Medicine for Children - MedEffect Canada</source>. Available online at: <ext-link ext-link-type="uri" xlink:href="https://www.canada.ca/en/health-canada/services/drugs-health-products/reports-publications/medeffect-canada/tear-sheet-cough-cold-medicine-children-medeffect-canada.html">https://www.canada.ca/en/health-canada/services/drugs-health-products/reports-publications/medeffect-canada/tear-sheet-cough-cold-medicine-children-medeffect-canada.html</ext-link> (accessed January 16, 2023).</citation>
</ref>
<ref id="B32">
<citation citation-type="web"><person-group person-group-type="author"><collab>Health Canada</collab></person-group> (<year>2012</year>). <source>Health Product Vigilance Framework. Minister of Health</source>. Available online at: <ext-link ext-link-type="uri" xlink:href="https://www.canada.ca/content/dam/hc-sc/migration/hc-sc/dhp-mps/alt_formats/pdf/pubs/medeff/fs-if/2012-hpvf-cvps/dhpvf-ecvps-eng.pdf">https://www.canada.ca/content/dam/hc-sc/migration/hc-sc/dhp-mps/alt_formats/pdf/pubs/medeff/fs-if/2012-hpvf-cvps/dhpvf-ecvps-eng.pdf</ext-link> (accessed January 16, 2023).</citation>
</ref>
<ref id="B33">
<citation citation-type="journal"><person-group person-group-type="author"><collab>Health Canada</collab></person-group> (<year>2017</year>). <source>Knowledge Translation Planner</source>. Health Canada.</citation>
</ref>
<ref id="B34">
<citation citation-type="web"><person-group person-group-type="author"><collab>Health Canada</collab></person-group> (<year>2022a</year>). <source>Guidance Document: Part C, Division 5 of the Food and Drug Regulations &#x0201C;Drugs for Clinical Trials Involving Human Subjects&#x0201D; (GUI-0100)</source>. Available online at: <ext-link ext-link-type="uri" xlink:href="https://www.canada.ca/en/health-canada/services/drugs-health-products/compliance-enforcement/good-clinical-practices/guidance-documents/guidance-drugs-clinical-trials-human-subjects-gui-0100/document.html&#x00023;a510">https://www.canada.ca/en/health-canada/services/drugs-health-products/compliance-enforcement/good-clinical-practices/guidance-documents/guidance-drugs-clinical-trials-human-subjects-gui-0100/document.html&#x00023;a510</ext-link> (accessed January 10, 2023).</citation>
</ref>
<ref id="B35">
<citation citation-type="web"><person-group person-group-type="author"><collab>Health Canada</collab></person-group> (<year>2022b</year>). <source>Regulating ultraviolet radiation-emitting and ozone-generating devices under the Pest Control Products Act: Overview</source>. Available online at: <ext-link ext-link-type="uri" xlink:href="https://www.canada.ca/en/health-canada/services/drugs-health-products/covid19-industry/disinfectanst-sanitizers-cleaners-soaps/ultra-violet-radiation-emitting-ozone-generating-devices.html">https://www.canada.ca/en/health-canada/services/drugs-health-products/covid19-industry/disinfectanst-sanitizers-cleaners-soaps/ultra-violet-radiation-emitting-ozone-generating-devices.html</ext-link> (accessed January 18, 2023).</citation>
</ref>
<ref id="B36">
<citation citation-type="web"><person-group person-group-type="author"><collab>Health Canada</collab></person-group> (<year>2023</year>). <source>Concerns About Children&#x00027;s Medication - Avoiding Cough and Cold Medicines</source>. Available online at: <ext-link ext-link-type="uri" xlink:href="https://www.canada.ca/en/health-canada/services/drugs-medicaldevices/concerns-about-children-s-medication.html">https://www.canada.ca/en/health-canada/services/drugs-medicaldevices/concerns-about-children-s-medication.html</ext-link> (accessed January 16, 2023).</citation>
</ref>
<ref id="B37">
<citation citation-type="web"><person-group person-group-type="author"><collab>Health Canada</collab></person-group> (n.d.). <source>Task Force on Scientific Risk Assessment</source>. Available online at: <ext-link ext-link-type="uri" xlink:href="https://www.canada.ca/en/health-canada/services/science-research/science-advice-decision-making/task-force-scientific-risk-assessment.html">https://www.canada.ca/en/health-canada/services/science-research/science-advice-decision-making/task-force-scientific-risk-assessment.html</ext-link> (accessed March 17 2023).</citation>
</ref>
<ref id="B38">
<citation citation-type="web"><person-group person-group-type="author"><collab>Health Canada&#x00027;s Consumer Product Safety Program</collab></person-group> (<year>2015</year>). <source>Risk Assessment Framework Summary. Health Canada</source>. Available online at: <ext-link ext-link-type="uri" xlink:href="https://www.canada.ca/en/health-canada/services/consumer-product-safety/legislation-guidelines/guidelines-policies/risk-assessment-framework/summary.html">https://www.canada.ca/en/health-canada/services/consumer-product-safety/legislation-guidelines/guidelines-policies/risk-assessment-framework/summary.html</ext-link> (accessed January 16, 2023).</citation>
</ref>
<ref id="B39">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Howell</surname> <given-names>E. L.</given-names></name> <name><surname>Brossard</surname> <given-names>D.</given-names></name></person-group> (<year>2021</year>). <article-title>(Mis)informed about what? What it means to be a science-literate citizen in a digital world</article-title>. <source>Proc. Natl. Acad. Sci. U S A.</source> <volume>118</volume>:<fpage>e1912436117</fpage>. <pub-id pub-id-type="doi">10.1073/pnas.1912436117</pub-id><pub-id pub-id-type="pmid">33876739</pub-id></citation></ref>
<ref id="B40">
<citation citation-type="web"><person-group person-group-type="author"><collab>ISO Standards</collab></person-group> (<year>2023</year>) ISO/FDIS 24495-1 Plain language &#x02014; Part 1: Governing principles and guidelines. Available online at: <ext-link ext-link-type="uri" xlink:href="https://www.iso.org/standard/78907.html">https://www.iso.org/standard/78907.html</ext-link> (accessed January 18 2023).</citation>
</ref>
<ref id="B41">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kelly</surname> <given-names>M. P.</given-names></name> <name><surname>Barker</surname> <given-names>M.</given-names></name></person-group> (<year>2016</year>). <article-title>Why is changing health-related behaviour so difficult?</article-title> <source>Public Health</source> <volume>136</volume>, <fpage>109</fpage>&#x02013;<lpage>116</lpage>. <pub-id pub-id-type="doi">10.1016/j.puhe.2016.03.030</pub-id><pub-id pub-id-type="pmid">27184821</pub-id></citation></ref>
<ref id="B42">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Krewski</surname> <given-names>D.</given-names></name> <name><surname>Lemyre</surname> <given-names>L.</given-names></name> <name><surname>Turner</surname> <given-names>M. C.</given-names></name> <name><surname>Lee</surname> <given-names>J. E. C.</given-names></name> <name><surname>Dallaire</surname> <given-names>C.</given-names></name> <name><surname>Bouchard</surname> <given-names>L.</given-names></name> <etal/></person-group>. (<year>2006</year>). <article-title>Public perception of population health risks in Canada: health hazards and sources of information</article-title>. <source>Hum. Ecol. Risk Assess.</source> <volume>12</volume>, <fpage>626</fpage>&#x02013;<lpage>644</lpage>. <pub-id pub-id-type="doi">10.1080/10807030600561832</pub-id></citation>
</ref>
<ref id="B43">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Krewski</surname> <given-names>D.</given-names></name> <name><surname>Westphal</surname> <given-names>M.</given-names></name> <name><surname>Andersen</surname> <given-names>M. E.</given-names></name> <name><surname>Paoli</surname> <given-names>G. M.</given-names></name> <name><surname>Chiu</surname> <given-names>W. A.</given-names></name> <name><surname>Al-Zoughool</surname> <given-names>M.</given-names></name> <etal/></person-group>. (<year>2014</year>). <article-title>A framework for the next generation of risk science</article-title>. <source>Environ. Health Perspect.</source> <volume>122</volume>, <fpage>796</fpage>&#x02013;<lpage>805</lpage>. <pub-id pub-id-type="doi">10.1289/ehp.1307260</pub-id><pub-id pub-id-type="pmid">24727499</pub-id></citation></ref>
<ref id="B44">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lee</surname> <given-names>N.</given-names></name> <name><surname>Lee</surname> <given-names>S.</given-names></name></person-group> (<year>2022</year>). <article-title>Visualizing science: the impact of infographics on free recall, elaboration, and attitude change for genetically modified foods news</article-title>. <source>Public Underst. Sci.</source> <volume>31</volume>, <fpage>168</fpage>&#x02013;<lpage>178</lpage>. <pub-id pub-id-type="doi">10.1177/09636625211034651</pub-id><pub-id pub-id-type="pmid">34399643</pub-id></citation></ref>
<ref id="B45">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Lundgren</surname> <given-names>R. E.</given-names></name> <name><surname>McMakin</surname> <given-names>A. H.</given-names></name></person-group> (<year>2013</year>). <source>Principles of Risk Communication: A Handbook for Communicating Environmental, Safety, and Health Risks (5th Edition)</source>. Risk Communication. <publisher-loc>London</publisher-loc>: <publisher-name>John Wiley and Sons, Inc</publisher-name>. <pub-id pub-id-type="doi">10.1002/9781118645734</pub-id></citation>
</ref>
<ref id="B46">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Mihelj</surname> <given-names>S.</given-names></name> <name><surname>Kondor</surname> <given-names>K.</given-names></name> <name><surname>&#x00160;t&#x0011B;tka</surname> <given-names>V.</given-names></name></person-group> (<year>2022</year>). <article-title>Establishing trust in experts during a crisis: expert trustworthiness and media use during the COVID-19 pandemic</article-title>. <source>Sci. Commun.</source> <volume>44</volume>, <fpage>292</fpage>&#x02013;<lpage>319</lpage>. <pub-id pub-id-type="doi">10.1177/10755470221100558</pub-id></citation>
</ref>
<ref id="B47">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Moore</surname> <given-names>R.</given-names></name></person-group> (<year>2022</year>). <source>Science Communications, Outreach, and Public Engagement.</source> Government Science and Innovation in the New Normal Discussion Paper Institute on Governance.</citation>
</ref>
<ref id="B48">
<citation citation-type="book"><person-group person-group-type="author"><collab>National Research Council</collab></person-group> (<year>1989</year>). <source>Improving Risk Communication.</source> <publisher-loc>Washington, DC</publisher-loc>: <publisher-name>The National Academies Press</publisher-name>.</citation>
</ref>
<ref id="B49">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Pawson</surname> <given-names>R.</given-names></name></person-group> (<year>2017</year>). <source>An Introduction to Realist Evaluation</source>. <publisher-loc>London</publisher-loc>: <publisher-name>Sage Research Methods</publisher-name>.</citation>
</ref>
<ref id="B50">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Pawson</surname> <given-names>R.</given-names></name> <name><surname>Greenhalgh</surname> <given-names>T.</given-names></name> <name><surname>Harvey</surname> <given-names>G.</given-names></name> <name><surname>Washe</surname> <given-names>K.</given-names></name></person-group> (<year>2005</year>). <article-title>Realist review &#x02013; a new method of systematic review designed for complex policy interventions</article-title>. <source>J. Health Serv. Res. Policy</source> <volume>10</volume>, <fpage>21</fpage>&#x02013;<lpage>34</lpage>. <pub-id pub-id-type="doi">10.1258/1355819054308530</pub-id><pub-id pub-id-type="pmid">16053581</pub-id></citation></ref>
<ref id="B51">
<citation citation-type="web"><person-group person-group-type="author"><name><surname>Pawson</surname> <given-names>R.</given-names></name> <name><surname>Tilley</surname> <given-names>N.</given-names></name></person-group> (<year>2004</year>). <source>Realist Evaluation. Urban Crisis Knowledge Hub</source>. Available online at: <ext-link ext-link-type="uri" xlink:href="https://www.urban-response.org/system/files/content/resource/files/main/pawson---tilley-%282004%29-realist-evaluation.pdf">https://www.urban-response.org/system/files/content/resource/files/main/pawson&#x02014;tilley-%282004%29-realist-evaluation.pdf</ext-link> (accessed January 8, 2024).</citation>
</ref>
<ref id="B52">
<citation citation-type="book"><person-group person-group-type="author"><collab>Pest Management Regulatory Agency</collab></person-group> (<year>2021</year>). <source>A Framework for Risk Assessment and Risk Management of Pest Control Products</source>. <publisher-loc>Ottawa</publisher-loc>: <publisher-name>Health Canada</publisher-name>.</citation>
</ref>
<ref id="B53">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Peters</surname> <given-names>D. H.</given-names></name></person-group> (<year>2014</year>). <article-title>The application of systems thinking in health: why use systems thinking?</article-title> <source>Health Res. Policy Syst.</source> <volume>15</volume>, <fpage>1</fpage>&#x02013;<lpage>6</lpage>. <pub-id pub-id-type="doi">10.1186/1478-4505-12-51</pub-id></citation>
</ref>
<ref id="B54">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Peters</surname> <given-names>H. P.</given-names></name></person-group> (<year>2023</year>). <article-title>Sharon Dunwoody&#x00027;s legacy: three timely lessons for us</article-title>. <source>Sci. Commun.</source> <volume>45</volume>, <fpage>127</fpage>&#x02013;<lpage>137</lpage>. <pub-id pub-id-type="doi">10.1177/10755470221149438</pub-id></citation>
</ref>
<ref id="B55">
<citation citation-type="book"><person-group person-group-type="author"><collab>Public Health Agency of Canada</collab></person-group> (<year>2015</year>). <source>Federal Action Plan on Antimicrobial Resistance and Use in Canada: Building on the Federal Framework for Action</source>. <publisher-loc>Ottawa</publisher-loc>: <publisher-name>Health Canada</publisher-name>.</citation>
</ref>
<ref id="B56">
<citation citation-type="book"><person-group person-group-type="author"><collab>Public Health Agency of Canada</collab></person-group> (<year>2017</year>). <source>Tackling Antimicrobial Resistance and Antimicrobial Use: A Pan-Canadian Framework for Action.</source> <publisher-loc>Ottawa</publisher-loc>: <publisher-name>Health Canada</publisher-name>.</citation>
</ref>
<ref id="B57">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Saner</surname> <given-names>M.</given-names></name></person-group> (<year>2010</year>). <source>A Primer on Scientific Risk Assessment at Health Canada</source>. <publisher-loc>Ottawa</publisher-loc>: <publisher-name>Health Canada</publisher-name>. <pub-id pub-id-type="doi">10.2139/ssrn.1595464</pub-id></citation>
</ref>
<ref id="B58">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Sellnow</surname> <given-names>D. D.</given-names></name> <name><surname>Sellnow</surname> <given-names>T. L.</given-names></name></person-group> (<year>2019</year>). <article-title>The IDEA model for effective instructional risk and crisis communication by emergency managers and other key spokespersons</article-title>. <source>J. Emerg. Manag.</source> <volume>17</volume>, <fpage>67</fpage>&#x02013;<lpage>78</lpage>. <pub-id pub-id-type="doi">10.5055/jem.2019.0399</pub-id><pub-id pub-id-type="pmid">30933307</pub-id></citation></ref>
<ref id="B59">
<citation citation-type="web"><person-group person-group-type="author"><name><surname>Shefrin</surname> <given-names>A.</given-names></name> <name><surname>Goldman</surname> <given-names>R.</given-names></name></person-group> (<year>2009</year>). <article-title>Use of over-the-counter cough and cold medications in children</article-title>. <source>Can. Fam. Physic.</source> <volume>55</volume>, <fpage>1081</fpage>&#x02013;<lpage>1083</lpage>. Available online at: <ext-link ext-link-type="uri" xlink:href="https://www.cfp.ca/content/55/11/1081.full">https://www.cfp.ca/content/55/11/1081.full</ext-link> (accessed January 16, 2023).</citation>
</ref>
<ref id="B60">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Stucki</surname> <given-names>A. O.</given-names></name> <name><surname>Barton-Maclaren</surname> <given-names>T. S.</given-names></name> <name><surname>Bhuller</surname> <given-names>Y.</given-names></name> <name><surname>Henriquez</surname> <given-names>J. E.</given-names></name> <name><surname>Henry</surname> <given-names>T. R.</given-names></name> <name><surname>Hirn</surname> <given-names>C.</given-names></name> <etal/></person-group>. (<year>2022</year>). <article-title>Use of new approach methodologies (NAMs) to meet regulatory requirements for the assessment of industrial chemicals and pesticides for effects on human health</article-title>. <source>Front. Toxicol.</source> <volume>4</volume>:<fpage>964553</fpage>. <pub-id pub-id-type="doi">10.3389/ftox.2022.964553</pub-id><pub-id pub-id-type="pmid">36119357</pub-id></citation></ref>
<ref id="B61">
<citation citation-type="web"><person-group person-group-type="author"><collab>Treasury Board of Canada Secretariat</collab></person-group> (<year>2021</year>). <source>Theory-Based Approaches to Evaluation: Concepts and Practices. Government of Canada</source>. Available online at: <ext-link ext-link-type="uri" xlink:href="https://www.canada.ca/en/treasury-board-secretariat/services/audit-evaluation/evaluation-government-canada/theory-based-approaches-evaluation-concepts-practices.html">https://www.canada.ca/en/treasury-board-secretariat/services/audit-evaluation/evaluation-government-canada/theory-based-approaches-evaluation-concepts-practices.html</ext-link> (accessed January 10, 2023).</citation>
</ref>
<ref id="B62">
<citation citation-type="web"><person-group person-group-type="author"><collab>Treasury Board of Canada Secretariat</collab></person-group> (n.d.) <source>Supporting Effective Evaluations: A Guide to Developing Performance Measurement Strategies</source>. Available online at: <ext-link ext-link-type="uri" xlink:href="https://www.canada.ca/en/treasury-board-secretariat/services/audit-evaluation/guide-developing-performance-measurement-strategies.html">https://www.canada.ca/en/treasury-board-secretariat/services/audit-evaluation/guide-developing-performance-measurement-strategies.html</ext-link> (accessed January 10 2023).</citation>
</ref>
<ref id="B63">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Trochim</surname> <given-names>W. M.</given-names></name> <name><surname>Cabrera</surname> <given-names>D. A.</given-names></name> <name><surname>Milstein</surname> <given-names>B.</given-names></name> <name><surname>Gallagher</surname> <given-names>R. S.</given-names></name> <name><surname>Leischow</surname> <given-names>S. J.</given-names></name></person-group> (<year>2006</year>). <article-title>Practical challenges of systems thinking and modeling in public health</article-title>. <source>Am. J. Public Health</source> <volume>96</volume>, <fpage>538</fpage>&#x02013;<lpage>546</lpage>. <pub-id pub-id-type="doi">10.2105/AJPH.2005.066001</pub-id><pub-id pub-id-type="pmid">16449581</pub-id></citation></ref>
<ref id="B64">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Van Der Zalm</surname> <given-names>A. J.</given-names></name> <name><surname>Barroso</surname> <given-names>J.</given-names></name> <name><surname>Browne</surname> <given-names>P.</given-names></name> <name><surname>Casey</surname> <given-names>W.</given-names></name> <name><surname>Gordon</surname> <given-names>J.</given-names></name> <name><surname>Henry</surname> <given-names>T. R.</given-names></name> <etal/></person-group>. (<year>2022</year>). <article-title>A framework for establishing scientific confidence in new approach methodologies</article-title>. <source>Arch. Toxicol</source>. <volume>96</volume>, <fpage>2865</fpage>&#x02013;<lpage>2879</lpage>. <pub-id pub-id-type="doi">10.1007/s00204-022-03365-4</pub-id><pub-id pub-id-type="pmid">35987941</pub-id></citation></ref>
</ref-list>
</back>
</article>