The initial downward counterfactual that we explore here is the 365CE Cretan earthquake and tsunami occurring at the present day. Many authors (e.g., Altinok and Ersoy, 2000; Tinti et al., 2005; Synolakis et al., 2011; IOC, 2020) have recognised that such an event would likely lead to a disaster, and have more severe consequences than in 365CE, particularly given high and increasing coastal population densities ( > 1000 people per km² in the Nile Delta; Figure 1B). This recognition has led to significant research into the hazard component of the disaster (e.g., Papazachos, 1996; Shaw et al., 2008; Synolakis et al., 2011; Valle et al., 2014; England et al., 2015; Necmioglu and Özel, 2015; Howell et al., 2017). However, consideration of the socio-political and economic consequences has predominantly been limited to specific geographic regions (such as Crete; Synolakis et al., 2011, or EU member states; International Bank for Reconstruction and Development/The World Bank, 2021), or industries (e.g., oil; Cruz et al., 2011). Synolakis et al. (2011, p. 406) recognised the need to use “the impacts envisioned in buildings and lifelines from a disaster scenario” in preparing for future tsunamis offshore Crete, that is, for a scenario-based counterfactual analysis of the type we conduct here. The idea of exploring a past event updated to a present or future context is one which has been used in other forms of scenario planning, for example in “storyline approaches” (e.g., Shepherd et al., 2018; Shepherd, 2019), and has potential to be used to identify likely impacts of present day mitigating or exacerbating actions (cf. Durance and Godet, 2010; Trenberth et al., 2015).

The 365CE earthquake occurred off the south-east coast of Crete on 21st July (Figure 1A). The earthquake generated a tsunami, which caused destruction and loss of life around the Eastern Mediterranean, with reports of inundation as far away as Sicily and modern Dubrovnik (Figure 1A; Jacques and Bousquet, 1984; Ambraseys, 2009, pp.151–156). The earthquake uplifted the coastline of western Crete by up to ∼9 m (Pirazzoli et al., 1996; Shaw et al., 2008). Shaw et al. (2008) determined the location and style of the earthquake based on paleoshorelines. They concluded that the earthquake was likely to have been a thrust with magnitude up to Mw 8.5 (with the best-fitting location shown in Figure 1). The earthquake mechanism, however, does not constrain the directionality of the rupture.

The Hellenic subduction zone, where the 365CE earthquake occurred is unusual in that only a small fraction ( < 20 % , Jackson and McKenzie, 1988; Vernant et al., 2014) of the plate movement is accommodated by earthquakes on the plate interface. The low seismicity of the plate interface in the Hellenic Subduction Zone means that it is under-recognised as a source of seismic and tsunami hazard both in academic literature (Hayes et al., 2012) and by the general public (Yalçıner et al., 2017; Aktas et al., 2021). Rather than the plate interface, earthquakes, such as the 365 CE earthquake, occur on steeper dipping splay faults (observed in Greece, the Dodecanese islands and Turkey), which can potentially generate larger tsunamis than similar magnitude earthquakes on the plate interface (England et al., 2015). Similar events are expected to occur somewhere along the Hellenic arc approximately every 800 years (Shaw et al., 2008). No large earthquakes of this sort have been reported in the region since the 365CE Mw ∼8.5 earthquake, meaning that sufficient strain for another similar earthquake to occur is already likely to have accumulated.

Participants for the workshop were recruited via the Cambridge Disaster Research Network (CDRN) and Cambridge Infrastructure Resilience Group (CIRG) mailing lists, which are run by the authors. CDRN is a network for researchers and practitioners interested in disasters and natural hazards. The CIRG is a network of researchers from across disciplines whose work touches on themes of societal and infrastructure resilience. There was no significant overlap in the membership of these mailing lists.

This research underwent a full ethical approval procedure through the Department of Geography at the University of Cambridge (#1848). Prior to the workshop, participants were informed that the workshop would be recorded and used for this research. This information was repeated in a briefing document sent to all participants (Supplementary Document S1). This briefing document also contained a content notice and informed participants that they were free to raise concerns with the workshop convenors (the authors) or to leave the workshop at any time. The use of the data, content notice and freedom to leave were re-stated at the start of the workshop.

The workshop had 22 participants (including the six authors). 18 of these participants were affiliated with the University of Cambridge, the remaining four with other universities, though two worked primarily outside academia. The majority (11) of the participants were postdoctoral researchers (including five of the authors), five participants were graduate students (including one of the authors), and four were faculty. Figure 2 shows the subject expertise of the participants by career stage. Six noted a specialism in volcanoes, volcanology or volcanic risk and four mentioned earthquakes or seismic hazards. Other noted interests included: climate policy, scenario development, risk perception, disaster management, hazard assessment and disaster knowledges. We did not ask whether participants identified as being from a physical or social sciences background, but the majority work on some component of the societal aspect of disasters. Although the participants were from a variety of career stages and disciplines, and thus represented interdisciplinary perspectives, the group was not diverse in that it was dominated by affiliates of the University of Cambridge, most of whom had limited, or no, previous research experience of the Eastern Mediterranean. Since our aim was primarily to test the downward counterfactual methodology, and to explore imaginative aspects of this analysis, the interdisciplinarity of the group was more important than particular backgrounds, however, the issue of who should be involved in downward counterfactual analysis is discussed further in Section 4.2.

The workshop was conducted on Zoom (Zoom Video Communications Inc., 2021) due to Covid-19 (WHO, 2020) restrictions on meeting in person. Groups were separated into “breakout rooms,” each of which was separately recorded. The registered participants had been pre-assigned into five groups, each led by a convenor, but these had to be adjusted based on the turn out. It was agreed in advance that no group should contain fewer than four participants (including the convenor) and that groups should maximise diversity of career stage and expertise. The convenor was allowed to contribute to the exercise; hereafter “participant” refers to both external participants and convenors).

We conducted three rounds of counterfactual exploration of the 365CE earthquake (Section 1.1). Participants were asked to state their suggested downward counterfactual or “turn,” if possible, identifying it with one or more of the turn types in Figure 3). They were also asked to say why this turn might worsen the scenario, and would not make the scenario significantly less likely. The meaning of “worse” was taken to be different in each round: Round 1—a higher number of casualties, Round 2—greater financial losses and Round 3—“participant choice” i.e., what each participant considered to be “worse,” emphasising ideas which had not come up in the previous rounds. What constituted a significant reduction in likelihood was open to interpretation by participants (see Section 4.3) and there was no requirement for quantitative justifications.

Other participants could then comment on, clarify or dispute these turns, with the aim of agreeing on whether a turn was acceptable (i.e., that the turn would worsen the scenario based on that round’s definition of worse and not lead to a significant reduction in the likelihood of the scenario). The end point of each scenario (step 3 in Lin et al., 2020) was the worst case which could be achieved without making the scenario “significantly less likely,” or reaching a pre-specified time limit (25 min for the casualties and financial rounds and 20 min for the participant choice round). The exercise was intended to be imaginative, meaning that scenarios were not expected to be fully specified (either quantitatively or qualitatively) at the multiple geographical and temporal scales of interest.

After the third round, participants were invited to share their reflections on the exercise and the downward counterfactual methodology more generally, with an invitation to follow up with the convenors by email if they had further reflections. Convenors ensured that all participants were given the opportunity to comment.

Audio recordings obtained from Zoom’s internal recording software were uploaded to Otter.ai, (2021), which generated automatic transcripts of the workshop. Each of the authors then manually checked the transcripts from their group against the recording.

Manual coding was carried out using NVivo12 Pro (QSR International Pty Ltd., 2018). Coding refers to the tagging or identification of sections of text with labels, which may be pre-specified (“deductive coding”) or created based on the data (“inductive coding”; Creswell and Creswell, 2017; Gibbs, 2018). We used a combined approach.

Inductive coding was used to identify themes in the workshop discussions. Three major structural elements were clear from an initial investigation: first, themes which emerged in the discussions; second, comments or issues relating to the methodology and, third, justifications given to support proposed turns. These were taken as the basic structure of a codebook, and more detailed code proposals were grouped under these headings. Several of the authors then reviewed the list of proposed codes to identify areas of overlap and redundancy and generated a codebook in NVivo12 Pro. This codebook was then used to code the transcripts. A small number of further codes were proposed after this initial coding and were added to the final codebook (Supplementary Document S2). The same person then reviewed their coding of each section. Finally, a second author coded each section independently using the same codebook. The coders also identified the turns (downward counterfactual changes) made in each group in each round, classifying as a turn any proposal which changed the scenario.

Deductive coding was used to identify turn types, based on the classification proposed by Lin et al. (2020) (Figure 3), with an additional turn type (directionality) introduced to capture changes relating to the direction of energy release from a hazard (which we hypothesised might be of particular relevance for the earthquake/tsunami scenario discussed in this paper). These turn types had been given to the participants prior to the workshop (Supplementary Document S1). If a turn had been identified as a particular type during the workshop it was coded as this turn type, with additional turn types being added where these were felt to more accurately reflect the content of the changes. Each turn was classified into at least one, and up to three, of the categories shown in Figure 3. We then categorised the individual turns as being either social or physical. A number of orienting questions were used to reach a designation, such as, “does this turn involve people in any way?”. We also investigated the type of turns used from a physical vs. social perspective, by dividing Lin et al.’s proposed turn types into those relating to intrinsic properties of the hazard (“timing,” “geographical shift,” “directionality,” “energy release,” “energy release rate”) and those relating to socio-political systems or people (“exposure,” “socio-political change,” “human decision making,” “human error”). The remaining categories (“cascading events,” “coinciding events,” and “environment”) can contain changes in either category.

At least three of the authors manually reviewed each coder’s proposal for what constituted a turn and discussed their identification with turn types as proposed by Lin et al. (2020). This discussion was important because some turns were difficult to distinguish from their impacts and sometimes multiple turns were listed in a single statement. The “social” vs. “physical” designations were rechecked and confirmed by a second author, in reverse order. This approach was only possible because of the relatively small size of our dataset (153 separate turns, ∼12.5 h of transcripts). The identified turns and classifications are still subjective but this approach gives us confidence in the agreement between coders.

We also identified repeated or similar turns. In some cases, a single turn contained aspects of multiple repeated turns (Figure 4). For example, one participant proposed in the financial round that if the earthquake and tsunami were to occur “a bit further past Covid … 2023, potentially, when tourism is really making a comeback and … in the summer” this could lead to expensive damage to super-yachts and that the disaster might become “really ingrained within the public imagination” leading to “damage [to] tourism in the region for a long time”. This turn is clearly about time relative to a pandemic (changing the timing to post-Covid), seasonality (changing the timing to summer) and long term impacts on tourism, so was coded as all of these. This turn also highlights the challenge of identifying turns, as distinct from their impacts, and the issue of participants effectively presenting multiple changes in a single “turn”.

We constructed scenario diagrams (Figure 4 and Supplementary Document S3) for each group and each round, showing both the order in which turns were made (text box numbers) and their interlinkages. In particular, we focused on whether turns followed on from, or relied on, previous turns (“continuing turns”), or could be viewed as direct worsenings of the initial scenario (“independent turns”). These diagrams represent the content and structure of the conversation and proposed turns, rather than the authors’ endorsement of the views or perspectives expressed.

Finally, using inductive coding techniques, we classified the different forms of justification that participants used or referred to into: case studies, disciplinary expertise, lived experience or anecdotal evidence, and regional knowledge (whether “expert” or general), with additional codes for items explicitly referred to as ‘outside a participant’s area of expertise’ and for incorrect information.

Figure 5A shows a heatmap of the number of instances of each theme in different groups and rounds. The most common themes were: critical infrastructure, other natural hazards, and political breakdown, which are broken down further in Figures 5B–D. There was more discussion of critical infrastructure and other natural hazards in the casualties and financial rounds, and more discussion of political breakdown in the financial and participant choice rounds.

Within critical infrastructure, the most coding instances related to water infrastructure, although these were almost all in the casualties round. In the financial round, all groups discussed oil and pipeline infrastructure, often in relation to ports and shipping (particularly in the context of the Suez Canal, which we discuss in more detail in Section 4.1). Electrical power and potential electricity failures were a common theme across all rounds and groups.

The discussion of ‘other natural hazards’ was dominated by infectious diseases, reflecting both the common proposal that the earthquake and tsunami would be followed by waterborne disease outbreaks (via the damage to water infrastructure mentioned above), and frequent references to pandemics (discussed further in Sections 3.2 and 4.1). We note that whilst infectious diseases (i.e., pathogens) themselves are natural, the conditions which create epidemics or pandemics are social, demonstrating the subjectivity of coding, and quantifying, discussion. Weather was also discussed in all groups, often in the context of coinciding rainstorms (leading to slope instability and landslides or, again, waterborne disease) or hot weather leading to more tourists on beaches. Aftershocks were discussed in three out of five groups.

“Political breakdown” was used to encompass national and international political tensions and conflicts, with the most discussion focused on relations between Greece and Turkey, including tensions in Cyprus. This theme also included the breakdown of international organisations. The collapse of the EU or the eurozone were particularly frequently mentioned, either as pre-existing conditions likely to make disaster response more challenging, or as a potential result of the costs of reconstruction (see Section 4.1). Such political breakdown exemplifies the theme “extent of impacts” (sub-divided into spatial and temporal) which emerged, particularly in the financial round, to describe references to the impacts of the disaster extending beyond the Eastern Mediterranean (e.g., collapse of the reinsurance industry) or further in time (e.g., long term loss of tourism). This theme is important for considering the utility of downward counterfactual analysis because it demonstrates the difficulty of assigning causality to complex and evolving risks.

In addition to these general themes, we investigated which specific counterfactual turns were most frequently repeated across rounds and groups. Figure 6 shows the 24 turns which were repeated at least three times during the workshop. Most of these repeated turns (n = 15, 65%) occurred in multiple rounds, and all occurred in multiple groups.

The most frequently repeated turn (n = 8) was; “time relative to a pandemic,” used to describe turns which located the occurrence of the earthquake and tsunami temporally either relative to the Covid-19 pandemic, which was ongoing at the time of the exercise, or to a potential future pandemic.

The casualties and financial rounds each had one turn which was made by all groups: seasonality and loss of income from tourism respectively. Both of these turns reflect regional and anecdotal understanding of the context of the Eastern Mediterranean (discussed further in Section 3.4). Several other turns occurred in four out of five groups in particular rounds. In the casualties round these were: time of day and damage to water infrastructure. In most cases, damage to water infrastructure was also linked to waterborne disease outbreaks, establishing causality between the earthquake and diseases (Section 4.4). In the financial round the most repeated turns were: food shortages, damage to oil infrastructure and environmental damage due to an oil spill (the latter two turns were always linked as either two separate turns or within the same turn).

In addition to analysing the content of the turns, we also investigated what types of downward counterfactuals participants used. The primary aim of this analysis was to investigate whether the classifications of counterfactuals proposed by Lin et al. (2020) provide a useful framework, either for conducting downward counterfactual analysis, or for classifying the resulting scenarios.

Figure 7A shows the prevalence of different turn types. “Cascading events” (n = 76, 50% of turns); worsenings of the scenario as a result of events following on from the initial hazard, were the most common turn type, followed by “exposure” (n = 53, 35% of turns); changes in the amount of people or infrastructure likely to be affected by the event.

We used multiple turn types to code the majority of turns (blue bars in Figure 7A), to reflect the complexity of turns, the difficulty of distinguishing between a turn and its impacts, and uncertainty in the appropriate categorisation for a particular turn. The overlaps between different turn types are shown in Figure 7B, with darker colours highlighting the more frequently occurring overlaps. The largest overlap is between “cascading events” and “exposure,” often signifying turns where cascading effects resulted from damage to infrastructure. Similarly, “timing” overlaps significantly with “coinciding events,” since for events to coincide often requires a change in timing, e.g., for the earthquake to occur on an election day. The interlinkage between cascading events and infrastructure exposure is seen in one participant’s suggestion of “extensive damage to Piraeus, which is the port in Athens … Potentially, you either block the port, or you start backing up international maritime traffic”. This turn incorporates exposure (of the port) and cascading events (“backing up international maritime traffic”) into a single turn, but also demonstrates the challenge of distinguishing between a turn and its impacts. Whether the turn here is “damage to Piraeus,” which might be an initial consequence of a repeat of the 365CE event, or the cascading effects of such damage, is subjective and requires understanding the context of the discussion, not only the single paragraph where it is stated. Quantifying a discussion through coding does not capture the full picture.

Our results suggest that turn types related to intrinsic properties of the hazard were less commonly used (n = 45, 29% of turns) than social turn types (n = 112, 73% of turns), and that hazard-based turn types also tended to be coded with a social code. “Timing” and “geographical shift,” for example, co-occur with “exposure” in 50% (n = 13) and 46% (n = 6) of their respective occurrences, reflecting turns such as changes in seasonality making the event worse because of larger numbers of tourists on beaches, or moving the epicentre leading to exposure of a particular port or harbour. These findings reflect our general results that 93 of the turns (61%) were primarily social (e.g., failure to warn populations or lack of awareness), 22 of the turns (14%) were primarily physical (e.g., an increase in the magnitude of the mainshock, or the occurrence of aftershocks) and 38 turns (25%) were a mix of both physical and social components. That the turns, and turn types, were primarily social might be attributable to the make up of the group, the set-up of the exercise (including the choice of seed event) or a number of other factors, but suggests the importance of understanding social and qualitative components of disasters.

Participants used various approaches to justify that their turns were both plausible (or not significantly less likely than the original event) and downward (i.e., would actually worsen the scenario). References to other disasters were the most common form of justification, including references to participants’ understanding about how something had happened or was done in a particular place (43 references) or references to a specific disaster or hazard case study (104 references). Participants also used their own disciplinary expertise (90 references), regional understanding of the Eastern Mediterranean (146 references, which included references to regional information in the briefing document) and lived or anecdotal experience (47 references).

42 unique case studies were referenced, several of them by more than one group. The characteristics of 13 other events were referred to, but the names of these case studies or their locations were not mentioned. Table 1 shows events that were talked about in at least three of the groups, with the full list of cases studies given in Supplementary Document S4.

The case studies were not only previous earthquakes, although these were the most common (14 separate events including two earthquakes/tsunamis), but also volcanic eruptions, hurricanes, infectious diseases and non-natural hazards including conflict, terrorism and economic crashes (with four or fewer separate case studies in each of these categories, Supplementary Table S1). These case studies were geographically diverse, though not evenly distributed, with the majority from Asia and Europe (Supplementary Table S2).

Case studies were used to justify turns being “downward” (i.e., making the scenario worse), as the following example shows:

Perhaps due to the qualitative nature of the workshop, the idea emerged in multiple groups that a counterfactual change was “satisfactorily real” (Lin et al., 2020) if it had occurred in a previous disaster, as seen in the following discussion:

We term such scenarios, composed of the worst parts of past disasters, “counterfactual Chimaeras,” after the legendary Greek monster which is part lion, goat and serpent (Homer, n.d.).

Analogues can be both spatial and temporal. The majority of the events referred to occurred recently (Supplementary Table S3), with 29 of the case studies (70%) having occurred since 2000. Only one event referred to occurred pre-1900, which was the Santorini eruption of around 1600 BCE (referred to above).

Whilst the main aim of this study is to explore how qualitative approaches and storytelling might be used in downward counterfactual analysis, it is also instructive to assess whether the scenarios developed bear any relation to risks which have already been recognised in the Eastern Mediterranean. A detailed comparison is beyond the scope of this paper but the themes and turns discussed are intended to give insight into how useful, and realistic, some of the downward counterfactual turns proposed were.

Both this exercise, and the work of previous authors (e.g., Tinti et al., 2005; Synolakis et al., 2011; England et al., 2015) suggest that any major earthquake and tsunami in the Eastern Mediterranean under current conditions is likely to lead to a disaster. A series of recent earthquakes (Heidarzadeh et al., 2017; Papadopoulos et al., 2020; UNESCO, 2020) have demonstrated issues with both the timeliness of tsunami alerts (IOC, 2020; Papadopoulos et al., 2020) and regional tsunami awareness (Yalçıner et al., 2017). These issues are consistent with the frequently repeated turn “lack of (timely) warning” in our workshop (Figure 6), and with discussion about failure to prepare (e.g. convenor C, participant choice round, Supplementary Document S3).

The temporal and spatial scale of the evolved scenarios might seem extreme (and there are certainly issues with the simplistic causality implied; Section 4.4), but the plausibility of several such turns is supported by previous risk analyses across multiple sectors and events after the workshop. One proposed turn which merits particular attention is damage or disruption to the Suez Canal, which was discussed by three separate groups. Some of the possible outcomes of such a blockage proposed by the groups in our workshop were: major disruption to international trade, food shortages, and oil bidding wars. The morning after our workshop took place, on Tuesday 23 March 2021, the vessel Ever Given became stuck in the Suez Canal, blocking all ships from moving through the canal for 6 days (Reality Check and Visual Journalism, 2021). Although the full scale of the impacts are not yet known, the blockage has indeed had significant impacts on trade (Ramos et al., 2021), and complex legal ramifications (Gaunt and Aston, 2021, 41:30). Such legal disputes, resulting from uncertain liability and whether or not an event constitutes force majeure (or in this case “perils of the sea”) were also proposed as a downward counterfactual in the context of the earthquake and tsunami in our workshop, suggesting that some of the seemingly improbable or high impact turns may be sensible to consider in risk analysis. However, the full scale of cascading events from a blockage of the Suez Canal proposed in our workshop (including food shortages, “pathological economic behaviour” and the collapse of the global reinsurance industry) have not, or, at least, not yet, occurred. These proposals can be considered downward counterfactuals for a blockage of the Suez Canal.

Financial instability at the EU level (Section 3.1) might also seem to be an extreme consequence of an earthquake and tsunami. However, the impacts of austerity measures imposed on Greece in the wake of the financial crisis (Matsaganis, 2011) has diminished the country’s capacity to maintain reserves against the event of a disaster (International Bank for Reconstruction and Development/The World Bank, 2021). A recent analysis of EU member states’ financial exposure to seismic hazard (International Bank for Reconstruction and Development/The World Bank, 2021) suggests that currently available EU disaster funds cover less than 4% of the annual expected government losses due to earthquakes. In the event of a major disaster, therefore, or cascading disasters (as discussed in our scenario building) there might indeed be major EU-scale financial ramifications (International Bank for Reconstruction and Development/The World Bank, 2021). The two examples above demonstrate the potential cascade of consequences from such a hazard event, and that scenarios with unquantifiable impacts may nonetheless represent significant risks.

Many of the themes which arose, including relations between Greece and Turkey and precarious financial relations between Greece and the EU, related to the current geopolitical situation or what might be considered “common knowledge” about the Eastern Mediterranean, in the sense of stories which have recently been in the news, or lived and anecdotal experience (Section 3.4). Two examples, which were discussed in all groups, are worth mentioning: tourism (particularly the potential exposure of tourists unaware of the regional earthquake and tsunami hazard, and the long term financial impacts of reduced tourism after such a disaster), and refugees and migration (including the vulnerability of coastal refugee camps on the Greek islands, as well as longer term displacement; e.g., Moris and Kousoulis, 2017). Both of these turns highlight known vulnerabilities (Arce et al., 2017; Alam et al., 2020; IOC, 2020), and systemic factors influencing the desire or ability of affected populations to mitigate the impacts of a potential disaster. The divide between known risks and appropriate risk reduction measures is evident in the Eastern Mediterranean, where the tsunami warning system (NEAMTWS; IOC, 2020) only began to be established after the Sumatra 2004 earthquake and tsunami brought tsunamis into the public consciousness globally, though academics had highlighted the risk of Mediterranean tsunamis for a least a decade beforehand (e.g., Tinti, 1993).

The turns above, based on contemporary regional knowledge, and repeated turns related to pandemics and countries leaving the EU, lead us to suggest that repeated turns in our workshop were more often related to current events or regional common knowledge than the probability of a particular turn or its likely impacts. Although we did not perform the same exercise prior to the Covid-19 pandemic or the United Kingdom leaving the European Union (on 1st January 2021, and still central to United Kingdom news at the time of the workshop, Adler, 2021), it seems unlikely that a pandemic, or the breakdown of the European Union and the associated loss of communal disaster response mechanisms, would have occupied so much of the discussion either at an earlier time or, in the latter case, outside the United Kingdom. One participant reflected:

The dominance of repeated turns by those which are contemporary, rather than those which are likely, suggests that simple quantification of a discursive risk analysis may not capture the desired information.

We observed a similar tempophilia (the prioritisation of current time in thinking about potential futures, Kelman et al., 2017, also referred to as the “availability heuristic” in psychology; Tversky and Kahneman, 1973) in the case studies invoked as justifications (Section 3.4), which were dominantly from the last 20 years. This focus on the contemporary might mean that insights from older events are missed, and lead to the fallacy of planning for the most recent disaster, rather than anticipating the next (Ewing and Synolakis, 2011; Paté-Cornell, 2012).

The scenarios generated in this workshop, therefore, have components consistent with pre-existing analyses of risk in the Eastern Mediterranean, but have a greater emphasis on unquantifiable risks, and are, therefore, potentially useful in identifying risks which might not currently be considered in individual risk analyses.

The above examples demonstrate that the scenarios constructed in our workshop capture known risks in the Eastern Mediterranean and provide connections between others. We now look at the role of interdisciplinary groups in developing scenarios.

Unique turns, which express ideas only thought of by a single participant, frequently came from participants’ specific experience or disciplinary backgrounds, suggesting that using interdisciplinary groups to conduct downward counterfactual analysis may broaden the range of identified scenarios. Some examples from this exercise include: rumours spreading on social media leading to instability in financial markets (specifically, “social networks [report] that Elon Musk, Mark Zuckerberg, the president of Greece and the president of Egypt are missing [as a result of the earthquake/tsunami]”), the loss of household and government records, and a deliberate misinformation campaign which could “lead to … loss of trust in the government, in the authorities, and therefore [lack of] willingness of the public … to respond to advice and communications in a … cohesive way”. The idea that particularly useful perspectives come from unique turns and varied expertise is consistent with Durance and Godet (2010)’s observation that “insofar as an expert represents a fresh perspective, his or her point of view should be taken into consideration.”

Several participants appealed to the known expertise of others in their group, as shown in the quotes in Section 3.4. Such interactions support the idea that a group discussion provides a greater diversity and depth of knowledge than an individual person conducting this exercise on their own (Lewis and Herndon, 2011), and provides a check on the possibility of proposed counterfactuals. These interactions also suggest that it was useful to have groups small enough to remember the other participants’ backgrounds. Our observations of group dynamics lead us to suggest that future downward counterfactual analyses, whether in a financial risk context or for DRR, should include interdisciplinary perspectives to broaden the range of imagined scenarios.

Several groups highlighted the importance of including local stakeholders, rather than only interdisciplinary academics, in future downward counterfactual analyses. Including stakeholders, such as residents and Civil Protection Agencies, could help both to better understand the complex factors leading to disasters and to engage local communities in understanding and mitigating disaster risk. The idea of including local stakeholders was based on the implicit assumption, common to most groups, that downward counterfactual analysis was intended for understanding future disasters with the aim of building resilient communities, rather than improving financial loss models (Woo, 2016; Woo et al., 2017). This assumption was demonstrated by one participant’s comment that, “if [downward counterfactual analysis] was used as a methodology it would be local experts doing it”. Whilst the value of local knowledge is widely recognised in DRR (e.g., Donovan et al., 2012), in the financial and (re)insurance sectors such an exercise might only be conducted by risk analysts.

Local stakeholders might also bring more detailed knowledge of the seemingly minor factors which can significantly worsen a disaster. A specific example from the exercise was that one participant said, “the way the [Greek] Civil Protection Authority is alerted [to a tsunami warning] is by fax” (a proposal confirmed by Papadopoulos et al., 2020, though it is not clear whether faxes operate in isolation as a warning mechanism or in combination with emails) leading to a discussion of specific failures associated with non-standard or outdated technologies. Conducting the analysis with local stakeholders would acknowledge the humanity and fallibility of traditional “experts” (Donovan, 2019) and could uncover additional context-specific vulnerabilities (e.g., Jigyasu, 2019). At the same time, it is important not to “romanticise” the knowledge of local people (Hilhorst, 2003).

There might also be benefits to including other forms of knowledge in downward counterfactual analysis beyond increasing the range of anticipated risks. One participant suggested that “you could do this as an educational exercise,” making the experience about the participants understanding the interconnected nature of disasters, rather than the actual scenarios (similar to the proposal of Gill and Malamud, 2014; Mignan et al., 2016, that risk mapping could be used to engage educators). Another participant commented:

These comments lead us to suggest that downward counterfactual analysis could be used in a wide range of settings. Scenarios should not be seen as the only potential output of such an exercise, the process and communication involved in conducting downward counterfactual analysis could itself be beneficial.

The workshop generated a wide variety of downward counterfactuals, and, perhaps more importantly, conversation between participants. There were very few instances of participants passing on taking a turn, and, as discussed above, many appealed to the expertise of different group members and commented that they enjoyed the interactions. There were, however, a number of challenges, both in conducting the exercise and in the subsequent analysis. We now discuss some of these challenges and possible resolutions, which might be useful for conducting similar exercises in future.

In terms of the actual workshop, a significant challenge was convening comparably with different group dynamics. Although all convenors used the same, pre-agreed structure, there were slight differences in emphasis. As a result, some groups focused more on the relative likelihood of a turn occurring, whilst others concentrated on justifying that their turns were “downward” (i.e., made the scenario worse). These variations are likely to be an inevitable results of different people leading different groups in a conversational exercise (e.g., O’Brien, 2004). We note that convenors made slightly fewer turns than other participants, and that the proportion of “social” vs. “physical” turns did not correlate with the convenors’ subject areas. As such, we do not think that convenors influenced the direction of the counterfactuals more than other group members. Different group members and power dynamics did, however, lead to very different experiences of the exercise. Having slightly larger groups (e.g., Six participants) might have reduced the influence of individual participants on the group dynamic, although keeping groups small enough for participants to have a good sense of each others’ expertise is also important (Section 4.2). These variations also mean that a direct quantitative comparison, for example of the number of turns of each type, between groups would not be appropriate, or adequately capture the different group dynamics.

There were also points to consider for future applications in terms of the set up of the workshop, namely: how to specify the starting scenario, defining and measuring “downward” or “worse” and how to decide if a turn is “satisfactorily real” (Lin et al., 2020). All of these are (relatively) simple for a quantitative analysis focussing on a particular parameter, conducted by a single analyst or team of analysts. However, the idea that only quantifiably worse losses are relevant or significant has been posed as a critique of disaster databases and risk analysis in general (Kar-Purkayastha et al., 2011; Nissan et al., 2019), and exploring the introduction of non-quantifiable or intangible losses into downward counterfactual analysis was the main aim of our study. The exact approaches which would be appropriate will depend on the intended outcome of the exercise.

Instead of modelling the initial scenario to establish a baseline for conducting downward counterfactual analysis, as proposed by Lin et al. (2020), we used qualitative reports from the 365CE earthquake and tsunami (Shaw et al., 2008; Ambraseys, 2009) combined with current maps of exposure and population density (Figure 1B) to give participants a sense of the initial scenario. We told participants that we were “interested in understanding the potential impacts of a similar earthquake and tsunami occurring in the present day or near future” (Supplementary Document S1) i.e., in a generally contemporary setting. This lack of specificity, which was intended to allow participants to determine the direction of the exercise, led to some differences of interpretation. Different groups took different views on whether the scenario involved an earthquake on the day of our workshop, on the same day as in 365CE but in 2021, or a more generally contemporary event. There was also uncertainty about whether outcomes would be expected based on the seed event or would constitute further downward counterfactuals. As an example, one participant proposed, “loss of the infrastructure, infrastructure of the city. Buildings, roads, any public land being destroyed by the, by the tsunami and the earthquake” as a turn. Major infrastructural damage occurred event in the 365CE event (Shaw et al., 2008), so is not obviously a downward turn from the seed event. Several participants suggested that an in-person workshop with maps, note cards and other visualisation aids could help to clarify the starting scenario, and subsequent turns.

Such aids might also help to address the challenge of determining whether turns were “downward,” who for and on what scale. As one participant noted, “in some of the cases when we were trying to build on the previous person’s sort of contribution wasn’t always 100% clear how it did make it worse.” This issue likely reflects the broad scope of the exercise (Section 4.3) which was deliberately open to allow participants to use their wide range of expertise (and some participants felt was successful in doing so, as one noted, “by not binding it … I actually think we uncovered a far broader range of potential implications”). Even for quantifiable losses, such as casualties or financial losses, real-time modelling is still likely to be too time consuming for a discursive workshop. In any case, many of the downward counterfactuals, particularly in the participants’ choice round, were obviously incomparable in terms of their impacts, such as changes to Israel’s blockade of Gaza and the extinction of Mediterranean tuna. The ability to account for and consider such unquantifiable impacts which are not obviously comparable is a key feature of including stories in downward counterfactual analysis. Including such unquantifiable losses is also an expansion of storyline approaches in climate change planning, which focus on qualitative scenarios as opposed to probabilistic hazard mapping (Shepherd et al., 2018), but still consider quantifiable outcomes. It would, however, be useful to consider systematic ways of assessing what a “worse” scenario looks like in a qualitative context. One participant proposed that a constructive approach might be to conduct the analysis on the basis of one loss metric and subsequently analyse the impact of the resultant scenario on other metrics e.g., using more casualties as a definition of “worse” and then investigating the financial impacts of the resulting scenario.

Just as it was hard to determine what constituted a “downward” turn, there was a similar difficulty in defining which counterfactuals were sufficiently “plausible” or likely. The approach arrived at organically in our workshop was the construction of “counterfactual Chimaeras,” scenarios combining aspects of previous disasters. As one participant noted, “I kept drawing on historical examples just to make sure that it was … within the realms of what happens”. Such counterfactual Chimaeras may provide an interesting approach to setting bounds on the risks of interest, with the caveat that further consideration needs to be given to biases towards contemporary analogues. This tempophilia could be reduced by encouraging participants to actively draw upon less well-known examples in the justification of their choices.

What constitutes an acceptably “plausible” turn or scenario depends on the setting in which such an analysis is being conducted. A key advantage of conducting a downward counterfactual analysis in the manner we have described is freedom to be creative. As one participant put it, “it is more a kind of opportunity for idea generation, if you like, rather than necessarily a precise calculation of a possible scenario”. Thus, potentially unlikely turns might be interesting to explore in the context of bringing together stakeholders (Section 4.2), or for expanding the space of imagined risks, but be less useful for a more specific risk analysis, where the type of risk to be explored is already known, and, perhaps, quantifiable, as in the examples given by Lin et al. (2020). The level of choice given to participants in similar exercises in the future should depend on the intended outcome of the exercise, but it is important to note that an exercise using, for example, flash cards (cf. Shepherd et al., 2018) has already pre-determined the space of events which are considered possible, irrespective of whether the analysis is qualitative or quantitative.

The conversational nature of the exercise meant that many participants provided counterfactuals in an unstructured way e.g., stating multiple possible counterfactuals at once or not distinguishing between counterfactuals and their consequences, which made it difficult to clearly identify turns in our subsequent coding. Whether this is an issue for future workshops will depend on the intended outcome–a stricter turn system or notes detailing the scenario, could be used if scenario development is the main outcome, but this might be unnecessary if the main aim is to build communication between stakeholders (Section 4.2).

Much of our analysis focused on the turn types proposed by Lin et al. (2020). Whilst we appreciate that these turn types are not intended to be exhaustive, it was interesting to investigate their impacts on the exercise. It was helpful to have these turn types as a guide, and certainly as examples of the type of counterfactuals which might be proposed. However, our general reflection as coders was that the turn types proposed by Lin et al. (2020) do not have sufficient provision for systems or situations in place before a disaster, or reflect the nuances of such systems (categorising all such turns as “socio-political change”). Many instances of multiple turn types arose from the socially constructed nature of disasters (Cannon, 2008; Oliver-Smith, 2010) meaning that turns altering the hazard are frequently associated with a social change (such as exposure), and also from the discursive nature of the exercise. The issue of how to incorporate the social construction of disasters might, however, relate more to downward counterfactual analysis as a methodology (which we discuss further in Section 4.4) than the turn types themselves, since in their original definitions Lin et al. (2020) specify that socio-political change and environment are meant to apply to the pre-existing conditions under which a hazard occurs.

We now discuss some of the underlying assumptions and issues with downward counterfactual analysis as an approach, based on this workshop and our observations.

The prevalence of cascading events in our scenarios suggests that participants saw the hazard event as the starting point of the counterfactual, both as the incident connected to historical reality and as a temporal beginning to the disaster (Hewitt, 2019, pp.5). Starting with a “seed event” (whether natural or not) presupposes a single starting point, and a disaster which unfolds from that point, whether or not this is the intention of the analysis. Thus, despite Woo (e.g., 2019) theoretically recognising that there are no natural disasters (O’Keefe et al., 1976), the framing of downward counterfactual analysis inherently privileges post-hazard evolution, rather than the pre-existing social factors and systems which enable the hazard to evolve into a disaster (Blaikie et al., 1994; Cannon, 2008; Oliver-Smith, 2010).

The use of cascading events to construct a linear narrative of how events might unfold or progressively worsen also stems from a logic of assumed causality, where the actual connection between events is not clear. Indeed, as Kelman (2018) argues, the concept of cascading disasters often imposes false “start” points that neglect the long term process of vulnerability, suggesting that the various loops and overlaps between events over time can be precisely indicated, contrary to many studies of disasters (Hewitt, 1997; Lewis, 1999). Mignan et al. (2016) give the examples of linking earthquakes to disease outbreaks as an incorrect application of cascading models in that although earthquakes are frequently associated with outbreaks of disease there are necessary causal links (such as damage to, and subsequent contamination of, water systems) which are missed through this direct connection. In our exercise, links between the earthquake and tsunami and disease outbreaks were made explicit in all cases, citing inundation of sewage systems as a result of the tsunami, exposure of sewage plants close to sea level and poor water infrastructure in refugee camps. However, in other cases the causality was less clear, for example two groups had the collapse of the eurozone in their financial round, and a further two groups proposed the collapse of the reinsurance industry, potentially leading to “more systemic issues for … the global financial system”. In all these cases, financial collapse was arrived at based on the large scale of the impact and damage, and concerns about Greek national debt (perhaps based on the Greek financial crisis which threatened the eurozone from 2009, Matsaganis, 2011). These concerns are shared by EU analysts (International Bank for Reconstruction and Development/The World Bank, 2021, see discussion in Section 4.1) but the chain of causality would likely be more complex than those proposed. This limited exploration of causality is important to acknowledge in conducting downward counterfactual analyses, since the role of scenario development is less to predict or precisely envisage futures than to explore the impacts of current actions on future risk (e.g., Durance and Godet, 2010; Shepherd, 2019).

That these cascading scenarios managed to reach global scales in relatively few turns in part reflect a tendency to overlook mitigation or resilience in constructing downward counterfactuals. Possible actions alleviating the inferred causal connections were discussed in several groups and used to challenge particular turns (detailed in the notes in the round summaries in Supplementary Document S3). Participants also discussed generating potential mitigating measures as a key component of encouraging governments and other stakeholders to take an interest in, and act on, such scenarios. Indeed, in disaster risk reduction the motivation for building scenarios is not to identify what will happen but rather to identify key weaknesses or points where advance action might reduce risk. Qualitative downward counterfactual analysis could provide a method to empower stakeholders to understand their current vulnerabilities and the origins of these vulnerabilities. However, anticipating future risks will not in itself prevent them from occurring (in contrast to Taleb’s statement that “11 September 2001: had the risk been conceivable on September 10, it would not have happened”; Taleb, 2007, p. xix).

These issues of causality and connectedness point to one of the key motivations for including stories in downward counterfactual analysis. Many of the turns made have multiple implications, and it is often in the proposed impacts, rather than the turns themselves that we observe the unexpected (at least to us). Whilst it is clear how moving the earthquake to a bank holiday weekend could be included in a casualty model on the basis of the average numbers of tourists in Crete at a particular time of year, the logic for this counterfactual in one of our discussions was that the decision makers in the civil protection agency might also be on holiday and difficult to contact, with associated (and much less easily quantifiable) impacts on both tsunami warning and the emergency response. Similarly, though the number of casualties associated with schools collapsing could be estimated, the long term psycho-social impacts of the deaths of large numbers of young people cannot (Woo, 2019).

The complexity and intangibility of both the risks and the causal links between aspects of disasters highlights the need to move beyond using only quantifiable knowledge in downward counterfactual analysis. Our results demonstrate that some aspects of a discursive exercise, such as the number of each type of turn, can be quantified, but with many caveats. The analysis, even of this relatively simple aspect of discussion, is subjective (although turns and most turn types were agreed upon, each return to the data has yielded new complexities and uncertainties). Further, the numbers do not necessarily convey the information that we might have sought (as with the number of repeats of each turn being more related to current events and regional knowledge than the actual likelihood of the turn). Finally, such quantification fails to capture the nuances and impacts of turns. We conclude that quantifying stories is unlikely to be the best approach to including them in downward counterfactual analysis, but that familiarity with multiple narratives and forms of knowledge will lead to a wider range of scenarios. The importance of recognising and valuing diverse forms of knowledge is consistent with work in DRR emphasising the importance of multiple epistemologies for community engagement (Donovan et al., 2012).

We therefore propose three possible extensions to the current study, which are beyond the scope of this work. The first is to investigate the likelihoods and quantitative impacts of those turns which can be quantified, e.g., loss of life due to disease outbreaks or exposure in refugee camps. The second is to identify mitigation measures for those counterfactuals which are not purely related to the hazard. The third, is the broader “gamification” of the downward counterfactual approach. Given the inherent creativity of interdisciplinary groups and our discussion in Section 4.2 of the importance of engaging stakeholders and using other kinds of knowledge, an interesting extension would be to build a game, accessible to a broad range of participants, which could be used to explore downward counterfactuals. A major challenge in such “gamification” would be how to simultaneously limit scenarios to those which are plausible whilst harnessing the creativity of a wider community.