by Jon Chua
Technology Strategy projects are some of the most exciting and impactful engagements for Kenway Consulting. During these projects, we have the opportunity to survey an organization and understand their business objectives, align those objectives with technology solutions, and create a vision and/or roadmap as to how the organization will move forward.
The first three steps of this process are fast-paced and collaborative. Kenway combines stakeholder interviews, cross-functional workshops, and brainstorming sessions to quickly gain an understanding of the organization’s current state, their full IT portfolio, and their desired future state. From there, we dig into other current state details, as necessary, through documentation and application reviews to deliver a list of recommended projects that drive the last phase, building the strategic roadmap.
At that point, discussion must shift from conceptual, aspirational topics to more grounded questions around estimated costs, returns, timelines, etc. While these conversations are imperative to defining a technology roadmap, it can be difficult to balance competing priorities or beliefs about initiatives in order to build consensus around which projects to undertake and when.
For example, an evaluation of a project may hinge on the forecasted return on investment (ROI), which could result in the following discussion:
While this is a drastically oversimplified example, you can see how this conversation is going down the path of becoming unconstructive by centering around whose forecast is better. It could potentially be “resolved” by one individual appeasing the other, by both parties being forced into a choice due to timeline constraints, or by decisions being made based on organizational clout or “gut feel” as opposed to evidence-based arguments.
How do you avoid these conversations? In this article, we will discuss some of the shortcomings of forecasts in an effort to decrease the dependence on them for decision making. From there, we will walk through how to use assumptions to drive more constructive conversations.
The Shortcomings of Forecasts
With baseball season upon us, let’s compare Javy Baez’s 2018 season stats to ESPN Fantasy Baseball’s projections as an example of some of the shortcomings of forecasts. We’ll discuss these forecast shortcomings in the context of the “Four Basic Laws of Forecasting” as outlined in The Operations Quadrangle: Business Process Fundamentals.
1st Law of Forecasting – Forecasts are always wrong!
Let’s see how this law applied to ESPN’s forecasts for each of Javy Baez’s 159 games in 2018 (we excluded the one-game playoff):
That’s right — as you can see, ESPN did not forecast a single game correctly. While saying that forecasts are always wrong may be an overstatement, the logic holds that expecting a forecast to be completely accurate (especially when you get to more detailed estimates) will almost certainly cause disappointment.
2nd Law of Forecasting – Forecasts always change!
Although ESPN’s initial forecasts were wrong, they followed this second law and updated their forecasts as the season went on and Baez was performing well:
Which worked until Baez struggled in August and September:
The key takeaway here is that, even if you think you made the best possible forecast at the beginning of a project, you must update your forecasts throughout the duration of the project based on information learned and new factors being introduced.
3rd Law of Forecasting – The further into the future, the less reliable the forecast will be!
This piece is fairly intuitive. Forecasting Baez’s performance today is easier than forecasting it 10 games from now when we do not know the pitcher he will face. Both of those are easier than forecasting his performance 100 games from now when we don’t know whether injuries, team performance, or opponent strategy might change things.
The same goes for technology projects. It’s exceptionally difficult to accurately forecast the value of a project that will end in 18 months, let alone two or three years from now.
4th Law of Forecasting – Aggregate forecasts are more accurate.
While still not 100% correct, we can see that ESPN’s forecasts for Javy Baez were more accurate when we measured them by month or even for the full season:
For technology projects, this means that focusing on forecasts for individual projects is much less reliable than those of large programs.
Focusing on Assumptions
As seen above, focusing on the final number in a forecast can lead to unconstructive conversations and could be seen as a futile effort. It will likely be inaccurate, especially with partial information, long time horizons, and/or low levels of detail. However, this is not to say that number should be disregarded. We need estimates on costs, returns, and ROI to drive decision making.
So, how should we use forecasts? We should present them as the evidence for a decision but focus our discussion on the assumptions that built them!
As an example, let’s look at a mock projection for the Cubs’ Season Opener:
The forecast of 5.50 gives us a starting point for our discussion. If the assumptions around Baez’s estimated performance hold, we would want to play him over an alternative player who is forecasted to score 1.00 point. We can now ground our discussion on the assumptions to develop a range with which we can make a decision.
For example, if one manager believes that the assumption that Baez will have 4 Total Bases is too high and that 2 is a more realistic number, we can set a lower forecast of 3.50 points. If another manager believes that Baez will actually score 2 runs, not 1, we can set an upper forecast of 6.50 points. Time permitting, we can discuss the key discrepancies in our estimates. Otherwise, we can also make a decision based on the forecasted range of 3.50 to 6.50 points — both are higher than the alternative of 1.00 point, so our decision does not change.
Let’s now revisit the initial forecast conversation example:
We can already see that this conversation is going to be more constructive than if the team members were debating final numbers. Keep that in mind the next time your colleague disagrees with your forecast or estimates. Remember, it’s probably not you, it’s your assumptions — try working through those to help the decision-making process!
These are the types of constructive conversations we try to facilitate during our Technology Strategy engagements. When we combine this type of constructive, actionable dialog with our analysis and your organization’s strategy, we can create truly impactful road maps that balance projects’ impacts, timing, and costs to maximize the total return.
Interested in learning more about how Kenway balances forecasts, assumptions, variables, and constraints in our Technology Strategy engagements? Send us a note at email@example.com.
 “Javier Baez.” FantasyPros.com. Accessed February 15, 2019. https://www.fantasypros.com/mlb/projections/javier-baez.php?scoring=E.
 Adelman, Dan, Sergio Chayet, and Don Eisenstein. The Operations Quadrangle: Business Process Fundamentals. Chicago, IL: The University of Chicago Booth School of Business. Revised 2018 by Sergio Chayet
Are you a Business or IT leader accountable for driving change in your organization? Are you a person passionate about helping companies solve business problems by bridging gaps between business and technology? Or just want to say hi?