What might have been: counterfactual thinking in risk analysis
RF Yolanda Lin was recently invited to contribute to the Earthquake Engineering Research Institute (EERI) Younger Members Committee (YMC) blog. This is a repost of that piece, and you can find the original blog post here. Thank you to Ezra Jampole and the YMC for the invitation and editing assistance.
After an earthquake, earthquake engineers and scientists work tirelessly to understand exactly the mechanisms of what happened, what was damaged, who was affected, how we can best move forward. In the process, we typically strive to identify what valuable lessons-learned can be harnessed from the event, so that a future, similar event may be safer and less impactful for the next community.
Are we restricted to only learning from our past disasters? This has been the central question driving my work this past year as a Research Fellow at Nanyang Technological University’s Disaster Analytics for Society Lab (DASL), led by Dr. David Lallemant. Driving our work is the fact that every event that has transpired is driven at least in some part by randomness, and how those hazard events transpire through damage, impact, and recovery are subject to these random realizations. For every event that has happened, there exists a whole family of possible and similar-probability events that didn’t happen, only by chance. And yet, there are lessons to be learned in these as well. Even events that already seem like a disaster could actually be considered a near-miss, when compared to the full range of outcomes that could have transpired. This is counterfactual thinking: reimagining the past somehow different than it actually was. We specifically are interested in downward counterfactual thinking, where the outcomes are worse than in the actual past event. According to counterfactual scholars in psychology research, we are already experts at counterfactual thinking, but we’re not wired to think in a downward way — we usually want to reimagine things for the better, and need some extra scaffolding to think in the negative direction.
Take, for example, the 1971 San Fernando earthquake. At M6.7, it was a surprisingly large event at that point in history, and the consequences of this event included the death of over 60 people, and damages valued at near half a billion dollars. Though already a costly disaster in many ways, this event could have been even more devastating. Exactly a year before in 1970, the Van Norman dam was nearly full at 6.5 billion gallons of water, but in 1971 it was just half full at 3.6 billion gallons of water. According to a study from the University of California, Los Angeles, the 1971 earthquake damaged the top 30 feet of the dam, and if this earthquake had occurred a year earlier, the dam may have failed and consequently claimed between 71,600 and 123,400 lives (Ayyaswamy 1974, Reich 1996). Instead, California has had a relatively safe record for direct deaths due to earthquakes in the last 60 years of about 200 lives lost (Woo 2017).
This August, we hosted the Counterfactual Black Swan Workshop at NTU in Singapore, with twenty-eight participants from six countries across three continents, to build momentum around this concept and establish a new community of practice centered on counterfactual thinking in risk analysis, bridging multiple disciplines and industries, such as earthquake engineering, insurance, human geography, and risk communication, spanning multiple geographic contexts (read more about the workshop here).
In one session of the workshop, we tested an activity of “Counterfactual Round Tables,” which is designed to guide participants through the process of building a counterfactual event, based on a real past event. The challenge is to find ways to make something from the past even worse — possibly even a so-called Black Swan, a low-probability, high-impact, and surprising event. Starting with one participant, the “event expert” describes a past event, using supporting materials (maps, figures, etc) as needed. Other participants are invited to ask clarifying questions, and finally all participants engage in counterfactual thinking to devise ways in which this event could have been worse.
We provided prompt cards (Figure 2) around the table to help guide downward counterfactual thinking. Participants then discussed their various counterfactual ideas and grouped their thoughts, written on post-it notes, onto a poster board in order to build a narrative around one or more counterfactual versions of their event. This process was then repeated for all participants at the table, with one past event per participant. The activity had a facilitator at each table, and there were three participants per table. The resulting counterfactual black swans were then shared during an informal coffee break poster session following the activity, and afterwards we also invited general impressions and feedback to share with the rest of the group.
Figure 2. Example categories for counterfactual changes. These were printed and used as prompts to stimulate downward counterfactual changes to past events.
Many participants at our workshop voiced surprise at the difficulty in pushing back against the automatic “upward” counterfactual thinking, wanting instead to engage in finding ways that the situation could have been better. They needed to consciously remind themselves (and be reminded by the moderator and fellow participants) to think downwards instead, which points to the need for a facilitated group exercise around this process.
Ultimately, this exercise can jumpstart our imagination to understand any lucky near-misses that can be addressed in the future, provide the basis for a guided search through a high-dimensional model space, explore multi-hazard scenarios that are currently difficult to model computationally, and provide a structured framework for gathering expert-opinions for a scenario in planning and preparedness purposes.
Next time you are gathered with two or three of your favorite earthquake engineering colleagues and experts, I invite you to grab some post-it notes and try this activity with a past event of your choice. Full directions for the activity are available here, and printable prompt cards are available upon request. Let me know what you come up with!
This work was supported by the National Research Foundation Singapore and the Earth Observatory of Singapore.
Yolanda C. Lin is a Research Fellow in the Disaster Analytics for Society Lab at Nanyang Technological University in Singapore. Yolanda holds a Ph.D. in Civil Engineering from Cornell University, a M.S. from University of Colorado Boulder, and an A.B./B.E. from Dartmouth College. You can reach her by email at ylin@ntu.edu.sg or on twitter @DisasterYolanda.