Demand forecasting, for all of its importance in business, has had a mixed run in retail. Even in fairly predictable categories in general merchandise, it’s far too easy for retailers to start the current year’s plan by loading in all the assumptions made from the year before, rather than starting clean with a new demand forecast. In fact, according to RSR Research’s benchmark, even though 68% of better-performing retailers (“Retail Winners”) and 53% of all other retailers believe that starting with a demand forecast as the basis for the next year’s plan is very valuable, only 49% of Winners and 29% of their peers actually do so today.
Part of the reason why is because forecast error in retail is high, as high as 32% according to some estimates. And, the more sporadic or non-repeatable the demand is, the more forecast error occurs – thus, grocery retailers operating a replenishment strategy have a far easier time using a forecast than a fashion retailer introducing a high-fashion item that responds to a new trend.
Additionally, not all products face the same demand profiles. The demand for a sweater that is red might be different than the demand for the exact same sweater in white. Even with automated forecasting tools, most retailers do not have a large enough planning team to devote the level of attention needed to every color/size combination of every item. So, they tend to pay attention to a few important or “key” items, and just apply a general set of assumptions to the rest.
That reported number of 32% average forecast error is exactly that – an average. That means some items could have had a forecast error of 0%, and some could have had an error rate of 100%. It just happens that all of the misses aggregate to an average forecast error of 32%.
AI promises to change the way demand forecasting works in retail in six key ways. Individually, they are promising, and all together have the potential to be transformative. But there are some important caveats to keep in mind along the way.
- Examining a nearly unlimited number of causal factors simultaneously.
The Hype: In most current forecast models, using standard techniques, the approach is primarily focused on identifying patterns of seasonality in past demand data, and using those patterns to predict future demand. Sometimes specific data points can be applied as causal factors – for example, the weather. Knowing that certain kind of product sells better when it’s hot means you want a seasonality boost in demand during June, July, and August – unless you live Australia or Africa, in which case, you need different model that applies the same methodology, just to a more southern hemisphere time of year.
But just looking at summer months when deciding a “hot” seasonality pattern leaves a lot of room on the table. “Hot” along the equator is almost year-round, and may only reflect 3 weeks in August in Norway. Typically, retailers just don’t have the capacity to manage multiple granular weather seasonality models like that, so they manage to improve their forecast to some degree by at least recognizing weather’s impact, even if they leave some value on the table because they can’t match weather assumptions down to granular locations like cities or counties.
AI promises to solve this problem by being able to look at an enormous amount of data all at once, and sort out not only what data is most important, but also what level of granularity is needed. And AI should be able to apply that analysis to the kinds of unstructured data out there, which retailers know to have some value somehow, but haven’t figured out how to get to that value yet – things like social media sentiment, or celebrity trends, or a new viral video’s emergence.
The Reality: It’s true that AI can cut through the noise of an overwhelming amount of data, and can get you to the top five or top ten most important factors that should be applied when calculating the demand of a specific product. However, this assumes that there is a ton of data out there, and that it is all data that is ready to be used.
Plenty of retailers will undoubtedly argue that there is a ton of data out there. But just because a company feels like it’s drowning in its own data, doesn’t mean it has access to the right kind of data. If all you have are continental-level weather forecasts or trends to start from, your ability to get granular with the impact of weather data is limited.
You may want to track local events to determine the impact of demand on individual stores, but how “local” of an event do you mean? Retailers trying to predict the impact of the local high school football game on Friday nights need to understand a lot of dynamics before they can predict the impact with any expectation of accuracy: high school football in Texas is everything, big town or small. In Denver, the impact could be muddied by shared stadiums that serve multiple schools across a wide area.
If you need further proof that this is harder than it looks, look no further than the impact of customer data on assortment. Retailers have been trying to add a customer dimension to assortment plans for over a decade, and have hardly made any progress at all. Why? Customer data is dirty, so it’s hard to use reliably. And even with AI, it’s still not clear which customer attributes are most important – and that could certainly vary by product and by location. It may very well take AI to figure out the right way to apply customer data to assortments, but we won’t get to that point until we can make customer data clean enough to use in the first place.
- Applying this analysis to every granular SKU, across every granular location where it will be sold.
Hype: Retailers usually only plan key items in their assortment, and leave a conventional forecasting tool or set of heuristics to take care of the rest. AI will apply the same level of “consideration” to every SKU in every location, as a human planning a key item.
Reality: When I see results from AI-driven forecasts, I feel like most of those results come from addressing this particular challenge. Whether the data is perfect and detailed or not, even doing something slightly more than just blindly applying the same forecasting algorithm across every SKU/location combination should yield an improved forecast result.
The real problem arises when you start getting into very sparse or highly intermittent demand history. The idea behind AI-driven demand forecasting is that you take a lot of data, throw it at past history to find patterns that humans would miss and that go beyond identifying seasonality in the data, and then use that new, deeper understanding of what drives demand to predict the future. In some ways, we’re back to the most basic data problem: garbage in, garbage out. If you don’t have enough good demand history to start from, it’s exceedingly difficult to predict the future from it.
That’s why new product introductions in fashion are such a challenge. Last year’s plaid boyfriend shirt is this year’s one-shoulder sweater in the assortment. Trying to figure out what the demand pattern should be for an item you’ve never sold before requires a different way of examining the data – where the AI looks for patterns or similarities in product attributes that might dictate how, even though the two items might be sold in different seasons, they follow the same demand curve. AI is making inroads here, but not nearly at the same pace or scale as something more reliable, like replenishment forecasting.
- Applying advanced algorithms, like Neural Nets, to create new methods of forecasting.
The Hype: If you want to sound as futuristic and science-y as possible in the tech world, outside of quantum computing, your best bet is to start talking about neural nets. Basically, neural nets are a subset of deep learning that tries to replicate “natural” models of decision-making. The term comes from trying to replicate the neural networks of the human brain. If you really want to stretch your own brain, you can check out a list of different neural net models, with explanation. There are other natural models that can fall under this category though, like evolutionary algorithms, which use multiple iterations of competing combinations of results to let a sort of “natural selection” method drive to the best result.
The idea behind using these kinds of algorithms for forecasting is that you’ll be able to use models that are not necessarily constrained by time series thinking – what goes into the usual approach for forecasting (“I needed 10 last month, so how many do I need this month?”).
The Reality: Right now, this kind of application of neural networks – for forecasting retail demand – is very much in its infancy. It seems to work very well, in part because there is a strong machine learning component to how neural networks work. The machine predicts an outcome, and then is shown the actual result, and then adjusts some of the computational weights in the neural network based on what it learned from what actually happened.The problem is, while early results have been good, that could just be luck – a wide enough variety of use-cases and data sets have not been tested to say that a neural net approach to forecasting should outright replace traditional models or even AI models that take a different approach. It may turn out the neural nets are good at only certain kinds of forecasting, like when the data is very noisy or non-linear or with unknown connections to other data sets.
- Selecting the right forecasting model to use for each specific circumstance.
The Hype: Similar to the evolutionary forecasting model, AI can be applied in forecasting to run a lot of different forecast models simultaneously, and then select the one that is the best predictor of results. How can you know which one was right before you know that result? That’s where machine learning comes in. The machine picks a model, it views the result, it learns which model would’ve been better, and then puts that knowledge into the next round of forecasting, putting an extra weight on models that did a better job of predicting the outcome in the last iterations.
The Reality: This is another place where the hype isn’t that too far ahead of reality. Companies specializing in demand forecasting have been trying to create “universal forecasting engines” for a long time. The only differences here are the self-learning approach, and the incorporation of AI-driven forecasting models in the consideration set. Where things get interesting is when you start looking at the next place where AI can be applied to demand forecasting, in the attributes that are used to identify when different products should use similar models.
- Identifying when specific causal factors deteriorate in their contribution to the forecast, and replacing them with new, more important causal factors.
The Hype: Part of the value of AI in forecasting is in using it to identify causal factors that humans can’t see or don’t have the time to find. It’s similar to the original idea of time series forecasts, except instead of looking for seasonality patterns specifically, the AI looks for any kind of causal factor that could be expressed as seasonality or even as a product attribute. Understanding this deeper level of product attributes – and then combining them with the same kind of analysis for customer attributes and location or channel attributes – gets you a potentially much more accurate forecast without, you know, developing outright clairvoyance.
The Reality: While AI-driven forecasting engines do this today, the problem is that they do them inside a deep, murky black box. Some of the attributes that the AI is “assigning” to products are not language-based, or easily expressed in language that humans can understand. This is a big miss for forecasting, because while the machine might learn, it means it can’t teach.
Any business, retail or otherwise, should not outsource its understanding of the core drivers of its business to an algorithm that can’t break down and explain how it arrives at its conclusions. I’ve heard from some AI vendors that this issue is merely one of building trust – “if it’s right all the time, then you should just trust it.” Except that it’s only right until it’s not. And if you don’t understand how it arrives at its recommendations, you’re missing a critical piece for how to make sure that it’s still leading your company in the right direction. Past results do not predict future performance.
- Reacting to changes in demand assumptions much more quickly than a human can.
The Hype: When retailers do actually create a demand forecast as the starting point of their planning process, they often then never look at it again. The assumptions they built that forecast on may change, or things may happen in execution that suggest looking at that forecast all over again, but companies rarely do. One thing that AI can do, though, is correct that. Because it’s looking at an enormous amount of data, and doing so constantly, it can easily identify when conditions have moved too far away from the conditions assumed in the original forecast, and make the call that a complete reforecast is needed – whether you’re talking at the SKU level, the location level, or across the whole company.
The Reality: My sense for this is that it’s still an open question as to whether AI is really good at this, or even if it’s needed. It’s not that you need a complete reforecast, it’s just that as you move from pre-season, to in-season, to the end of the season, different factors have greater importance. We manage that today by moving from a long term demand forecast, to a mid-term forecast, to a short-term forecast. Each of those uses different models and looks at different seasonality patterns or causal factors. Certainly, AI can detect when it’s time to move between these faster than a human could, and it’s in that margin where value can be found. It’s just a question of how much value is actually there.
The Bottom Line
It’s easy to get wrapped up in the hype about AI, and to leap forward into a future where AI drives everything. AI certainly has a lot of value to add to prediction algorithms, and it’s with these algorithms that retailers can most definitely drive new value in their business. Retail forecast error is high, and anything that reduces it not only drives value on its own, but increased value in other levers too, like inventory levels, inventory turns, and margin.
It’s helpful to back away from using “AI” to mean some kind of nebulous future and dive into how, specifically, we might get to that future. But it’s even more valuable to understand which pieces are within reach – and which are not.
Date: February 28, 2019