by Sam L. Savage
Justin Schell is part of a team bringing the discipline of probability management to Highmark Health, an $18 billion, Pittsburgh-based integrated healthcare firm. So far he is getting a good reception from a number of quarters in his organization. However, as he has introduced simulation trials to management, he has encountered confusion with the concept of scenarios. Justin invited me and Dr. Sarah Lukens, a Data Scientist at GE Digital, to discuss this with him and record our conversation in a video, which he has just posted here.
From my perspective, if simulating a business plan before investing in it is like shaking a ladder before climbing on it, then scenarios correspond to where you will unexpectedly find your ladder when the climbing begins--will it be beside your house, over broken beer bottles, next to a shark tank, etc.?
Although scenario analysis was made famous by Royal Dutch Shell’s fortuitous preparedness for the collapse of the former Soviet Union, Reidar Bratvold, a Professor of Investment & Decision Analysis at the University of Stavanger in Norway, points out a potential big problem with the approach. By focusing on a few, causal stories, it diverts attention from “a broader, more systematic representation of the decision situation.” In this way, there is the potential that it “grossly overestimates the probability of the scenarios that come to mind and underestimates long-term probabilities of events occurring one way or another.” Bratvold also compares scenario analysis to risk matrices, which many people consider worse than useless by providing a palliative that lulls one into the sense that they have done risk management.
Both scenario analysis and risk matrices are often used in an attempt to do probabilistic analysis without using probability. That is like trying to learn how to swim without getting wet. Understanding probability, or what I call the arithmetic of uncertainty, has to come first. Then when you conjure up a new scenario for consideration, you can address whether it is more or less likely than an asteroid strike. I think of simulation and scenario analysis as dual to each other. You will fall into the traps described by Bratvold if you don’t understand the probabilities involved, and you may end up simulating the wrong things if you haven’t explored the parallel universes in which you might find yourself.
Through simulation analysis, new scenarios may suggest themselves and vice versa.
© Copyright 2020, Sam L. Savage