How Probability Management Helps Solve Age-Old Problems in Budgeting and Forecasting
by Sam Savage
Sandbagging is the practice of padding one’s budget to avoid running out of money in the face of an uncertain forecast. Suppose, for example, that ten managers each have independent uncertain annual expenditures that average $10M. Let’s assume they all cover their butts by forecasting the 90th percentile, which turns out to be $11M (the Sandbags). Now they each have only a 10% chance of blowing their budget.
Next, the CFO rolls these forecasts up to get $110M (the Sum of the Sandbags). And suppose the enterprise can also tolerate a 10% chance of exceeding the overall budget. The problem is that due to the diversification effect, there is only about one chance in 1,000 that the CFO will blow through all $110M. Why? Suppose one manager, Paul, ends up exceeding his budget at the end of the year, while another, Peter, has extra cash. Then the CFO can borrow from Peter to pay Paul, and all is well. So mathematically, given the options to balance across the portfolio at the end of the year, the 90th percentile at the line item level turns into something like the 99.99th percentile at the enterprise level.
To achieve the desired 90% confidence, the CFO might need only $105M, which we refer to as the Sandbag of the Sum. So, in this case, $5M is just lying around gathering dust instead of being available as investment capital. If you don’t think that’s a big deal, go out and try raising $5M sometime. And this problem only compounds as you roll up layers upon layers of fat through a multi-tiered organization. In the above, and most examples, the Sum of the Sandbags is greater than the Sandbag of the Sum (the number you should budget at the portfolio level given your organization’s risk tolerance). But the inequality can sometimes go the other way with asymmetric distributions, and you can’t do this stuff in your head.
When I wrote the first edition of The Flaw of Averages, there was no practical way to solve this problem on a universal scale. Today, however, thanks to the Open SIPmath™ Standard, anyone with a spreadsheet can easily perform the necessary calculations with uncertain budgets. What remains is the re-alignment of the numerous stakeholders involved. Impossible, you say? Someone who has done this the hard way without SIPmath is Matthew Raphaelson, who first introduced me to the Sandbag Problem years ago. Matthew is a former senior banking executive with 25 years of experience, which includes being CFO of a large business unit. He is also chair of Banking Applications at ProbabilityManagement.org.
He stresses that some managers may use probability as an excuse for lack of accountability. “At the end of the day,” says Matthew, “managers – not machines – need to own their forecasts and be accountable for their results.” He warns that “a company that relies solely on centralized models will be met with smirks and shrugs when it attempts to distinguish between forecast errors and performance misses.”
Matthew, who has been on the front lines of numerous budget wars, describes five stages of managerial development for tackling the sandbag issue.
Education
Make managers aware of the problem, and how today there is a practical solution.Communication
Understanding percentiles, and communicating uncertain estimates as auditable data.Models and Data
Convert existing data infrastructures to handle SIP libraries instead of numbers. This is no big deal and can be done with current software.Incentives and Cultural Change
The “nobody gets in trouble for beating a forecast” mentality is the root cause of the sandbagging problem. Gamification can both provide new incentives and train managers to become better forecasters in the face of uncertainty.Analysis and Action
Once uncertainty becomes auditable, it may be systematically reduced in a continual improvement process.
Matthew and I have written on this subject for the Banking Administration Institute (BAI).
And there are two separate documented SIPmath models available below that perform thousands of simulation trials per keystroke to connect the seat of your intellect to the seat of your pants.
SandbagCalc
Demonstrates basic sandbag math
Model from BAI article
Banking example with revenues and expenses
I will end with a war story from Matthew, which foretells the nature of the battle ahead.
“In the 1990s, I asked managers to give me a ‘50th percentile’ forecast to avoid the sandbag problem. Apparently, this guidance wasn't as clear as it needed to be. One manager's monthly expense results kept coming in lower than forecast, to the point where it was clear there had to be some bias. I re-affirmed with the manager that he provided 50th percentile forecasts. ‘Oh, absolutely,’ he said. Probing a bit, I asked if this meant there was a 50% chance that actual expenses would come in lower than forecast each month. ‘Yes, that's what it means,’ he said. And so, is there also a 50% chance that actual expenses would come in higher than forecast each month? ‘Oh no, there is almost no chance of exceeding our forecast....’”