Publications

• FAIR Meets SIPmath to Analyze Risk vs Return across the Enterprise by John Button, Dr. Sam Savage, and Eng-Wee Yeo. This three part blog series for the FAIR Institute explains how The FAIR™ Ontology and open SIPmath™ standard are complementary methodologies for managing risk and communicating uncertainty respectively. Fair Institute, Part 1, Part 2, Part 3, January 2024.

• Silicon Valley Bank and Stress Tests: What Can Local Governments Learn? by Shayne Kavanagh, Sam Savage and Matthew Raphaelson. This article discusses the collapse of Silicon Valley Bank (SVB) and the role of stress testing in risk analysis. GFOA, March 2023.

• Moving Toward a Holistic, Rigorous, Analytical Readiness Framework by Connor S. McLemore, Shaun Doheney, Philip Fahringer, and Dr. Sam Savage. This article proposes that the military adopt a uniform data framework to better quantify readiness predictions and better communicate risks, leading to better cost and risk tradeoff discussions among decision-makers. Center for International Maritime Security, May 24, 2021.

• Chancification: Wiring Your Organization for Probability by Sam Savage, with sidebars by Shayne Kavanagh and Aaron Brown. The article discusses how just as electrification delivers electricity generated by engineers to the general public, “chancification” can now deliver stochastic information generated by analysts to managers for estimating the chances of meeting their goals. ORMS Today, Vol. 28, No. 3, June 2021.

• Military Readiness Modeling: Changing the Question from “Ready or Not?” to “How Ready for What?” by Connor McLemore, Shaun Doheney, Sam Savage, and Philip Fahringer. This article describes an actionable data framework to improve military readiness reporting systems using stochastic scenario libraries that permit calculation of the probabilities of military readiness for specified missions at uncertain future times. Military Operations Research Journal, Vol. 26, No. 1, March 2021. (Requires fee to download for non-MORS members.)

• The Flaw of Averages and ICU Capacity by Dr. Christian B. Smart. This blog post addresses how the Flaw of Averages is affecting management of ICU capacity during the COVID-19 pandemic. July 2020.

• The Amazing Future of Risk Management – SIPs and SLURPs by Alex Sidorenko. This blog post describes the benefits of using databases of simulation values to generate random samples within models. January 2020, RISK-ACADEMY Blog.

• Probabilistic Design of Sustainable Reinforced Concrete Infrastructure Repairs Using SIPmath by Michael Lepech, Melissa Zirps, and Sam Savage. Paper written for the 2019 Winter Simulation Conference presents a case study showing that SIPmath allows designers to engage in sustainable design using probabilistic methods. December 2019, Proceedings of the 2019 Winter Simulation Conference.

• A Multi-Dimensional, Counter-Based Pseudo Random Number Generator as a Standard for Monte Carlo Simulations by Doug Hubbard. Paper written for the 2019 Winter Simulation Conference proposes using a new pseudo-random number generator (PRNG) as part of the standards needed to facilitate the adoption and quality control of simulations. December 2019, Proceedings of the 2019 Winter Simulation Conference.

• The Metalog Distributions and Extremely Accurate Sums of Lognormals in Closed Form by Tom Keelin, Lonnie Chrisman, and Sam Savage. Paper written for the 2019 Winter Simulation Conference explains the flexibility of the metalog probability distributions for representing data. December 2019, Proceedings of the 2019 Winter Simulation Conference.

• Calculating Carrier Air Wing Readiness: An Additive Approach by Shaun Doheney, Connor McLemore, and Sam Savage. Describes the development of a stochastic library that drives a decision dashboard for a hypothetical aircraft carrier air wing composed of all the aircraft on a single aircraft carrier, which allows a commander to input mission requirements and estimate the chance that the air wing will be ready to successfully carry out that mission on short notice. December 2019, Phalanx - The Magazine of National Security Analysis, Volume 52, No. 4.

• Measuring Military Readiness by Shaun Doheney, Sam Gray, Connor McLemore, and Sam L. Savage. The article explains how open standard representations of uncertainty can help military decision-makers move from ambiguous estimates of “ready or not?” to unambiguously calculating, quantifying and communicating “how ready for what?” December 2019, ORMS Today, Volume 46, Number 6.

• Characterization of Historical Methane Occurrence Frequencies from U.S. Underground Natural Gas Storage Facilities with Implications for Risk Management, Operations, and Regulatory Policy by Richard A. Schultz, Douglas W. Hubbard, David J. Evans, and Sam L. Savage. Describes the use of occurrence frequencies, their associated probabilities and uncertainties, and forecasts of severity magnitudes to develop more effective mitigation/monitoring/reduction programs in a risk management plan. Risk Analysis, November 2019.

• Curing the Flaw of Averages in Climate Change by Dr. Sam Savage investigates how simulation can help predict and prepare for flooding and other damage caused by climate change. Public Sector Digest, September 2019.

• Operational Readiness Rollup by Shaun Doheney, LCDR Connor S. McLemore, and Sam L. Savage. Describes how simulation and modeling can provide useful information in military operations readiness. September 2019, Phalanx - The Magazine of National Security Analysis, Volume 52, No. 3. Recipient of the 2020 John K. Walker, Jr. Award given "to recognize the author(s) of the technical article judged to be the best published in PHALANX … during the previous calendar year." 

• The flaw of averages: Why it's risky to assume a fixed rate of return in your financial projections by Paul Kaplan explains why investors need to take uncertainty into account when planning for a financial goal. August 29, 2018, Morningstar.ca.

• A UAV Case Study with Set-based Design by Colin Small, Dr. Gregory S. Parnell, Dr. Randy Buchanan, Dr. Matthew Cilli, Dr. Edward Pohl, Dr. Simon Goerger, and Zephan Wade. Paper written for the 28th Annual INCOSE (International Council on Systems Engineering) International Symposium describes an Army case study using the SIPmath Modeler Tools to leverage the capabilities of model-based engineering to evaluate trade-offs between value and cost. July 2018.

• How to Profit From Probability Management by Matthew Raphaelson and Sam Savage. Explains how probability management helps solve age-old problems in budgeting and forecasting. May 9, 2018, BAI.org.

• Introducing the Open Group Open FAIR™ Risk Analysis Tool. Description of a collaboration between The Open Group Security Forum, San Jose State University, and ProbabilityManagement.org to develop a risk analysis tool that adheres to The Open Group Open FAIR Standard. March 29, 2018, The Open Group Blog.

• To Survive in an Increasingly Unpredictable World, We Need to Train Our Brains to Embrace Uncertainty by Emre Soyer, cites The Flaw of Averages and ProbabilityManagement.org. January 2017, Quartz.com.

• Probability Management and the Flaw of Averages by P.S. Kruger and V.S.S. Yadavalli. This article introduces the topic of probability management and uses the newsvendor problem to illustrate the Flaw of Averages. South African Journal of Industrial Engineering, Vol. 27, No. 4, pp 1-17. December 2016.

• Rolling Up Operational Risk at PG&E by Jordan Alen, Christine Cowsert Chapman, Melissa Kirmse, Farshad Miraftab, and Sam Savage, details how probability management provides an overall risk snapshot that allows diverse stakeholders to assess tradeoffs between safety, reliability and cost. December 2016, INFORMS Magazine.

• The Metalog Distributions by Thomas W. Keelin. Introduces the metalog distributions and describes their applications. Decision Analysis Vol. 13, No. 4, December 2016.

• Monte Carlo for the Masses by Sam Savage. Showcases the 3.0 SIPmath modeler tools and kicks off the Random Number Initiative, September 2016, Analytics Magazine.

• Poisson Formula Write-up by Dave Empey and Marc Thibault, a technical breakdown of the one-cell Poisson distribution generator built in to the SIPmath Modeler Tools, August 2016.

• Speaking Uncertainty to Power, by Shayne Kavanagh and Sam Savage, details the practice of risk aware forecasting and budgeting for CFO's, including applications by financial officers in the cities of Redmond, Washington and Colorado Springs, Colorado. April 2016.

• Towards a Simulation Network, or, The Medium is the Monte Carlo (With Apologies to Marshall McLuhan), by Sam Savage and Marc Thibault, describes how the SIPmath Standard may foster the creation of networks of simulations that bring stochastic modeling to general management. December 2015.

• Embracing the Uncertainty of the Iran Nuclear Deal, Sam L. Savage, ProbabilityManagement.org, September 2015 - Dr. Savage addresses the need to avoid average-based thinking in the Iran Nuclear Deal, and recounts a disaster in an earlier Iran saga. The failed Operation Eagle Claw mission to free the US hostages is still presented as a poster child for the Flaw of Averages by the Naval Postgraduate School. A PowerPoint and Excel model accompany the article.

• MIT Sloan Management Review, December 2014 - Psychologists Robin Hogarth and Emre Soyer present the benefits of interactive simulation and the communication of uncertainty, two of the cornerstones of the discipline of probability management. In “Using Simulated Experience to Make Sense of Big Data,” Hogarth and Soyer cite ProbabilityManagement.org as “a non-profit organization that aims at improving communication of uncertainty,” and applies “simulation-based communication to improve actual managerial decisions and public policies.”

• Teaching Modern Portfolio Theory to 10-Year-Olds, by Sam Savage, a personal account of the surprising results of teaching portfolio theory to middle schoolers, OR/MS Today, October 2014.

• Probability Management 2.0, by Sam Savage and Melissa Kirmse, an overview of the 2nd generation of probability management, ORMS Today, October 2014.

• Probability Management in Financial Planning, Sam L.Savage and Shayne Kavanagh. Discusses examples of the Flaw of Averages in public finance, and how they may be mitigated through the discipline of Probability Management. Excel sample files may be downloaded from our Models page. Government Finance Review Magazine, February 2014.

• The Sequestetron, Sam Savage and Shayne Kavanagh. Describes the application of Probability Management to municipal finance in the face of uncertain tax revenues. November/December 2013, Analytics Magazine.

• Distribution Processing and the Arithmetic of Uncertainty, Sam Savage. An introduction to SIPmath in Excel. November/December 2012, Analytics Magazine. NOTE: The videos linked in this article can be found here.

• Monetising NOLs: Finding the ‘Flaw of Averages’ and Producing Useful Forecasts, by Michael H. Salama. Addresses the issue of managing net operating losses under uncertainty. Tax Management Weekly State Tax Report. (2012).

• Cost vs. Risk in Defense Portfolios, by Philip Fahringer and Sam Savage. Investigates the inherent tradeoffs between cost and military airlift preparedness using libraries of threat scenarios. Phalanx, March 2012.

• The Flaw of Averages in U.S.Corporate Income Taxes: An Evaluation From the Taxpayer’s Perspective, by Michael H. Salama, Virginia Tax Review (2011)

• The Flaw of Averages In Project Management by Philip Fahringer, John Hinton, Marc Thibault and Sam Savage. Explores the use of Stochastic Libraries in simulating project completion. PMI Virtual Library, 2011.

• Monte Carlo, A Lightbulb for Illuminating Uncertainty, by Paul D. Kaplan and Sam Savage. Discusses the history and future of financial simulation including Distribution Strings. Investments & Wealth Monitor, 2011.

• Toward a Consolidated Risk Statement, by Sam Savage and Aaron Brown, appears in the December issue of Risk Professional magazine (no longer available online). It describes the potential of Distribution Strings to enable the aggregation of risk reports across an organization. (2009)

• Accounting For Uncertainty, by Sam Savage and Marc Van Allan. Describes current issues with and best practices for addressing uncertainty in accounting statements. The Journal of Portfolio Management, Fall 2002.

• The Flaw of Averages by Sam Savage. Harvard Business Review, November 2002.

• Blitzograms - Interactive Histograms, in INFORMS Transactions on Education, by Sam Savage (Vol. 1, No. 2, 1 Jan 2001).

• Holistic vs. Hole-Istic E&P Strategies, by Dr. Sam Savage and Ben Ball. Seminal article in the Journal of Petroleum Technology that laid the groundwork for the Shell Application in 2005 and the underlying ideas of the discipline of probability management. Journal of Petroleum Technology, 51 (9): 74–84. Paper Number: SPE-57701-JPT. September 1999.

• Notes on Holistic vs. Hole-Istic E&P Strategies, by Dr. Sam Savage and Ben Ball. Self published online to provide mathematical detail for the JPT article above. Also included is EPPO.xlxs an early portfolio prototype in Excel.

• The Bad Seeds - A Parallel Random Number Generation Problem Weeded Out Long Ago, Crops Up Again, by Sam Savage, Linus Schrage, Peter Lewis, and David Empey (1994). Discusses problems with random number generators generating non-random results and how to avoid these problems.