I talked with the brilliant Dr Christian B Smart all about project risk and his new book about project risk management. You can direct download the episode here.
Christian is the Chief scientist for Galorath, and provides consultancy for NASA and the US Department of Defence.
Please subscribe on Apple podcast or Spotify. You can also find me on Twitter @costofpodcast and on LinkedIn.
Why do defence and aerospace projects subject to cost overrun?
There are 2 things. Firstly, the absolute high level of cost – you’re producing something for the Government – e.g. in defence of the country, which is hard to budget. These systems are high end and you have to pay for the performance. It’s a balance of cost, schedule and performance. The last 10% of performance accounts for two thirds of the total cost.
Contractors build these products, not the Government – there is a level of oversight required which is an additional cost on reporting which commercial systems wouldn’t necessarily have.
The other thing is cost overrun – due to complex project development. So many things can go wrong. A lot of risk – cost and schedule overruns – are a result of a lack of project planning at the upfront stage, a lack of realism at the front end, and the planning fallacy driving this optimism. Some risks you can’t avoid, external forces can affect the progress of the project, delays schedules influences costs.
If we don’t consider risk does that mean we are bad at estimating?
The estimator needs to include uncertainty. You can be precise. You have to budget to a single number, not to a range. The mean might be the best single number to give as an estimate, but it depends on the level of uncertainty. Analysable risk gives us some control. We can change scope, mitigate risk, etc.
The estimating method needs to appropriately estimate the mean. Depending on the estimating approach used, we might get a most likely, optimistic or pessimistic estimate. This would need us to understand where the mean is. The sum of the most likely inputs might not be the aggregated most likely estimate.
We talk about confidence level in the estimate – simply summing the most likely inputs and aggregating would give you a 5% confidence level. The distribution would give you 40% confidence. This could lead to significant underestimation of the cost.
Estimators need to understand uncertainty in methods and the risk ranges. Need to be careful not to underestimate which we are more likely to do.
Can we say we’re bad at estimating if we get a number badly wrong?
You are providing an estimate relative to the inputs. Your estimate will change as your assumptions become more realistic as the project goes on. The perception of risk increases as the project goes on. The problem fits a log normal distribution, so if we recognise we are at the early stage and have optimism we can correlate to a log normal distribution to calibrate the project with previous cost and schedule overruns.
“As professionals we don’t track our performance, we don’t look at how we have performed previously, consistently. As a profession we need to track our performance better.”
Measuring performance takes patience and persistence, projects take years to deliver. We have to look back to see our performance. But it is needed. We need to measure our performance over time.
What is the core message in your book?
Risk is important, and we tend to underestimate risk, it’s always riskier than you think, even when you take that into account. A little bit of attention will go a long way. If a company puts in effective risk management, it gives you a competitive advantage.
With big data and AI, what competences do estimators need to have going forward?
The advent of big data is here. We still need to keep risk in consideration. Advanced techniques will be incorporated e.g. regression trees. Beyond our comfort zone of excel, we should look more towards R and Python. Natural Language Processing that gives textual insights e.g. from scope of work. Big data gives us more data, lower level, and faster.
If you are an estimator and you don’t look at these techniques, you will be left behind. For your own development you need these skills, or else you may get replaced at some point.
About Christian
Christian is the Chief Scientist for Galorath Federal, providing consultancy to NASA and Department of Defence.
Christian’s new book Solving for Project Risk Management: Understanding the Critical Role of Uncertainty in Project Management is out on November 3rd 2020, and is available for pre-order now.
Follow Christian on Twitter @ChristianBSmar1
Christian’s website for cool blog posts
The Multidimensional Economic Evaluators Podcast on soundcloud and the website Meevaluators.com
The Cost of Everything Podcast 
One reply on “What is Project Risk, Anyway?”
[…] What is Project Risk, Anyway? […]
LikeLiked by 1 person