More Needles, More Haystack: Or The Irony of Optimization
Meinolf Sellmann
Creator of Optimization Solvers, Architect of the ECB Transaction Settlement System, Inventor of Algorithms
In this article, we discuss examples where an increase in performance potential frequently leads to a decline in actual performance. And what we can do about it.
"Trk Trk Trk" (The Ice Road)
Let us look at two real-world examples to get started. First, consider large digital brokers for trucking. After the spectacular failure of Convoy at the end of last year, even the biggest digital brokers that remain find themselves in difficult waters. Uber Freight, e.g., was launched in May 2017 and yet, to date, its only positive EBITDA quarter was in the third quarter of 2022, at a meager $1 million.
The trucking industry is incredibly fragmented and there is tons of potential to increase efficiency: to save miles, to save hours, to save energy, to save costs, and to save emissions.
All that appears to be required is more information.
Who has stood in congestion and not wondered how many trucks on the opposite highway are on the way to pick up some load from someplace behind, while the truck in front is on the way to pick up some load from somewhere ahead? If only we knew who needs to ship what, surely by doing a better matching of carriers to shippers we ought to be able to avoid empty miles. Right?!
Yet, reality appears to elude this assumption. If Uber Freight and Co would create massive value, then surely they could manage to turn a profit. Why aren't they?
South By Southwest
One may rightfully argue that the trucking industry was different, that it was a relationship business and anyway, that digital freight brokers had not been around long enough to really make a dent. So let us look at a very established industry: airlines.
The common assumption is that there is an economy of scale. If our airline owns one aircraft, has four pilots and eight flight attendants, and only operates New York - Washington four times per day, then we run into problems every time a flight attendant is sick, our aircraft is not working or in maintenance, or a pilot is on vacation or even quitting. It is hard to provide resilient services with such a tight operation.
So we would expect larger airlines to have larger profit margins. Right?!
Below is a chart on the operational profit of Southwest Airlines between 2022 and 2023. During this time, the airline added roughly 30% more flights, from about 1,050,000 to 1,400,000 (which we can puzzle together from here). Yet Southwest's operational profit margins have tanked from 11% at the end of 2021 to half a percent in 2023.
Southwest, of course, operates a decentralized network while most competitors still run the traditional, and more resilient, hubs and spokes model. Southwest had therefore big problems to recover after a winter storm hit most of the USA at the end of 2022. That, and the fact that Southwest operates exclusively problem-ridden and flyer confidence-lacking Boeing 737 aircraft, has certainly not helped with profitability.
So let us look at the other end of the spectrum. What about the poster child of airline profitability (and passenger pain), Ryanair? In 2018, Ryanair gave up its single airline strategy, acquired Laudamotion, launched Ryanair Sun etc. Clearly, business was expanding. The result? In 2018, the profit to revenue ratio was over 20%. Until 2023, this ratio went down to 12% while, at the same time, turnover increased from 7.1 Billion to over 10.8 Billion. That is, the revenue went up by 50%, but the total profit stayed the same, even though the load factor also stayed roughly the same (93% in 2023 vs 95% in 2018).
A Bird in the Hand is Worth Two in the Bush
All of the examples above are unique and there are good individual explanations for the results observed in each case. And yet, a pattern is recurring: As the operation grows, the complexity grows, and, despite more potential, the real results get worse. The irony: The increased optimization potential may decrease resiliency against disruptions rather than increase it.
Potential is one thing, and results are another. As optimizers, we know this all too well. When we model decision problems, we aim to keep the number of decisions to be optimized low, even at the cost of optimization potential. After all, what good is a 1% better solution that exists but that I cannot find?
Is There a Digital Economy of Scale?
When Yuri Malitsky and I started our work on instance-specific algorithm tuning, we had the idea to adjust algorithm parameters based on the specific input it had to churn rather than provide default parameters only. The first main question we investigated was this:
Recommended by LinkedIn
Can the increased potential for performance improvements be materialized reliably?
In the case of instance-specific automatic algorithm configuration, we were able to devise conservative strategies that would answer this question in the affirmative while all methods suggested prior were brittle and did not deliver improvements over one-size-fits-all parameters reliably.
For businesses, like digital freight brokers, airlines, but also retailers and manufacturers with a need to manage their inventory efficiently, it is existential that we provide reliable decision support and realize the potential of large organizations. After all, we cannot ask any of these businesses to shrink to a point where the complexity is manageable as scale is, in fact, a necessity for them.
And therefore, we need to provide operational plans that solve highly complex, inter-connected problems, while being resilient against changes and disruptions.
The Practice
Where does that leave us? We have seen that there are operations with a very high potential for improvements that are not realized yet. At the same time, these operations may need to be large and complex to be viable.
Typically, organizations of the scale we are discussing here are already using technology to optimize their operations. And on paper, the "objective" function values that the automatic planners provide look fantastic. In theory, the plans are formidable and should lead to lean, efficient operations.
So where is the gap?
Quite simply, the models that the optimizers get to optimize do not match reality.
Uber Freight needs to give shippers a fixed price before the company knows for certain what carriers would charge or if there are even any available. Southwest needs to commit to a network, fleet and crew assignment before it knows how many people will want to fly, if their aircraft will work, their crew will be healthy, and the weather will cooperate. Retailers need to produce or buy inventory before they know the demand. And everyone needs to set prices without knowing exactly how the demand is affected.
Under such uncertainty in the forecasts, it is unfortunate practice to optimize against point estimates. We may not know the demand when the price is $1.99, but we simply tell our solver it was 98 and use that estimate as if it were certain. This then leads to plans that massively over-optimize that one expected future. The irony: the very fact that large operations use optimization to be able to manage the complexity at all then leads to brittleness and poor operational results.
Optimization literally undoes itself when using point estimates.
The Alternative
The answer to this conundrum is to incorporate the uncertainty in our forecasts in the optimization. This has to be done with care, as we would otherwise massively increase the number of decisions again and, while increasing optimization potential, decrease solution quality in practice.
However, if we devise a stochastic model with care, we now have the technology to solve the resulting optimization problems efficiently. We built a solver named InsideOpt Seeker that is able to solve stochastic optimization problems efficiently. While there is a cost associated with optimization under uncertainty, we have found a way to make it scale sub-linearly with the number of scenarios considered. This allows us to devise plans that perform robustly even in the face of highly undesirable low-probability events.
If your operational plans look better on paper than their performance in real life, we are here to help.
That's an interesting paradox, it seems that having more options can sometimes hinder progress rather than help it. Can you share an example of a situation where this phenomenon was observed, and how it was addressed?
That's an interesting paradox. What do you think is the main reason for this disconnect between potential and results?
Problem Solver
1moElaborating on the bed optimization picture, a solution here in the tropics is the hammock, which admittedly takes some getting use to, but it offers substantially more logistical benefits compared to beds or mattresses, and in the far away places I went I met people who never used a bed on their life! What I'm getting at, optimization models deals with options as given, either as points,, defined dimensions, forecasted estinates, capacities etc and leaving out "creative " solutions spaces that are sometimes filled by niche companies that "see " the gap of inefficiency.
Problem Solver
1moThanks for sharing, the bed optimization picture it's worth a million thought provoking paths