Decision Trees, Predictive Models, and Risk Management: When You Miss a $1B+ Impact
Unless you have been living off the grid for the past few weeks, you have likely heard of the unfortunate situation on a United flight from Chicago to Louisville where a paying customer was forcibly “re-accommodated” to another flight to make room for United staff who needed to be transported to Louisville. While some of the details around the events that transpired may be in dispute, the outcome was indisputably gruesome. Thus, at one point the following day, United’s market cap had dropped over one billion dollars. Beyond the cost of the pending lawsuit by the customer who was forcibly removed from the plane, United is facing the cost of the public relations nightmare, and the cost of lost customers who may never fly with them again. I’m not here to debate United’s policies or procedures. Rather, I am interested in understanding the $1B+ breakdown that occurred at the intersection of United’s risk management, decision trees, and predictive analytical models.
First: Why do airlines overbook flights?
It is a common industry practice for airlines to overbook some of their flights because events often arise that would result in a fare paying customer failing to board. This includes things such as late arrivals, missed connections, customer cancellations, and rescheduling, among others. In some of these cases, the airline may still recover some revenue in the form of cancellation or rescheduling fees. However, in the case of missed connections, they miss out on revenue if the seat goes unfilled. Further, as the laws of supply and demand would dictate, airlines often make their greatest margin on the last few seats sold on each flight. Rather than increasing the cost of tickets across the board, and potentially being less competitive on pricing versus their competition, the airlines have built predictive models to help determine on which flights they should oversell and the quantity to oversell. These predictive models are heavily based on historical data (i.e. there are certain airports where connections are more likely to be missed), seasonality, and weather, among other factors. These models are (or at least should be) continually fine-tuned.
Second: Predictive analytics aren’t designed to predict exactly what will happen, they are designed to demonstrate what is most probable to happen.
Despite the sophisticated predictive models that airlines have created, when flights are oversold, there are times when passengers are denied boarding. In many cases, it is voluntary (a passenger voluntarily surrenders their ticket in exchange for compensation), but in other less fortunate cases, it is involuntary. As the chart below demonstrates, most airlines do it–some just do it better than others.
Third: Where do risk management and decision trees come into play with predictive modeling?
The predictive model needs to be informed by risk management and decision trees that leverage historical data to determine the acceptable number of seats to oversell to maximize profit on a given flight. If the model is too aggressive, significant compensation is required for passengers denied boarding, and the flight is less profitable. If the model isn’t aggressive enough, the opportunity for additional profitability will be missed.
In the case of United, I suspect they grossly underestimated the risk management when assessing the decision tree to create the expected value based on the number of seats to oversell. For purely illustrative purposes, let’s play out a few scenarios of the risk management and decision tree. Note, this doesn’t account for the cash value differences in value between flight vouchers and cash or other costs.
Probability of a passenger on a United flight:
- Sitting in paid seat: 99.93%
- Voluntarily or involuntarily denied boarding: 0.07%
Risk probability when passenger is denied boarding (in hypothetical one seat overbooking):
- Risk: We overbook the flight by one seat; customer accepts first offer:
- Probability: 50%
- Cost: $400 airline voucher
- Probability Adjusted Cost: $0.14
- Risk: We overbook the flight by one seat and no one voluntarily accepts the first offer, but accepts the second offer
- Probability: 45%
- Cost: $800 airline voucher
- Probability Adjusted Cost: $0.25
- Risk: We overbook the flight by one seat and no one voluntarily accepts the $800 airline voucher; customer is randomly selected and involuntarily denied boarding; customer exits plane with little complaint
- Probability: 3%
- Cost: $1,350 cash
- Probability Adjusted Cost: $0.03
- Risk: We overbook the flight by one seat and no one voluntarily accepts the $800 airline voucher; customer is randomly selected and involuntarily denied boarding; customer exits begrudgingly, vowing to never fly United again, and takes outrage to social media to 100+ followers
- Probability: 1.5%
- Cost: $1,350 cash + $5000 (cost of customer acquisition; possible additional customer loss based on social media following)
- Probability Adjusted Cost: $0.88
- Risk: We overbook the flight by one seat and no one voluntarily accepts the $800 airline voucher; customer is randomly selected and involuntarily denied board after already boarding the plane; customer refuses to leave; airport security is called to remove the customer from the plane; customer is assaulted and sustains a concussion, broken nose, scrapes, and loses two teeth; Entire episode is posted to social resulting in $1B loss in market cap the following day; Customer wins seven-figure lawsuit
- Probability: 0.5%
- Cost: $1B+
- Probability Adjusted Cost: $3,750
In this model, United would need to adjust its pricing strategy to account for a probable cost of a denied boarding at $3,751.30 per passenger. If that were true, United would probably sell very few tickets. I’m of the suspicion that United neither considered the probability nor the adjusted cost of the final risk listed above, and only accounted for $1.30 per passenger in probable costs related to a denied boarding.
Had United accounted properly for the risk management in their model, they might have done a few things differently. First, they may have tweaked their predictive model to lower the probability that a customer is denied boarding (see Hawaiian Airlines). Second, they may have changed their process for dealing with overbookings to reduce the likelihood of a customer being involuntarily denied boarding. News sources indicated that United used random selection to determine the customers that were involuntarily removed from the plane. While random may equate to “fair”, random also equates to dumb. Given the information that is available or that could potentially be acquired, it would be borderline stupid for United to randomly select a customer. Even if they narrowed it down to randomly selecting passengers based upon the lowest flight status, it would still be dumb. Information can be utilized to make better decisions than random. Consider the case of Delta (further details here) which allows customers to define in advance what they would be willing to accept to be bumped from a flight. The decision then becomes far less than random, and more about informed economics.
So, let me ask you: Have you properly accounted for all risks in your predictive models?
Kenway is here to help. If you’d like to learn more about how we help companies build comprehensive predictive models, or if you’d just like to find out more about who we are, please reach out at firstname.lastname@example.org.