Last year, many of the world's largest and most technologically mature companies overspent millions of dollars on inventory purchases or production. At the root of it, consumer behavior did not match forecasts.

Planners cannot simply blame consumers for not acting as forecasted—especially when many customers are under historic inflationary pressure in food, fuel, and housing.

"Demand planning was shelved over the past three years," said Rick Jordon, senior managing director at FTI Consulting. "Now they are picking it back up because excess inventory is piling up everywhere. But they either forgot how to do it, or they never did it well."

After three years of pandemic disruptions and economic volatility, optimizing demand forecasting and inventory planning across industries seems to have reached unprecedented levels of urgency and difficulty.

To achieve better results, companies are turning to new data sets and leveraging new technological tools such as artificial intelligence for analysis.

From spreadsheets to AI

A McKinsey executive survey in early 2022 found thatspreadsheets remained the primary planning tool for supply chain leaders, with 73% of respondents relying on them.

But 43% of respondents said they planned to use AI and machine learning for some planning activities; another 17% said they planned to use AI and machine learning for most activities at some point in the future.

A survey this year by supply chain software company BlueYonder showed that demand forecasting and inventory optimization are among the top scenarioswhere supply chain executives apply AI and machine learningtechnologies.

"We are clearly moving beyond Excel to a more advanced set of tools," said Ron Scalzo, senior managing director at FTI Consulting.

In Jordon's view, AI technology will in the future help eliminate human bias in forecasting, drive more systematic predictions, and run multiple scenario simulations simultaneously.

Jordon noted that processing large data sets and helping companies understand their own internal data requires AI intervention. "Most organizations have massive amounts of data," Jordon said. "Are they fully leveraging that data with the best technology and ignoring the noise? That will be the real direction of development over the next three to four years."

Finding the right data, analyzing it the right way

At the most basic level, AI takes in data and outputs forecasts. To get the best forecasts, the data fed in may be even more critical than the AI tool itself.

"Existing systems are already tapping into internal company data, including enterprise systems, POS systems, store inventory levels, and so on," said Meher Dinesh Naroju, director of AI services at Centific. "But very few companies step outside that framework to acquire completely unstructured external data sources."

Such data sources include social media, which may provide clues about emerging market trends. Naroju said customers "are actively engaging, leaving comments and clues about their intentions in the market or in specific categories."

Even images from cameras in retail stores may, at an aggregate level, contain data related to customer behavior, including reactions to specific products, brands, and categories. "It's a bit of a stretch, but it could be helpful," Naroju said.

He also noted that other sources, such as weather data, can provide early signals for supply chain timelines and inventory arrivals.

Nevertheless, all these data sets must be interpreted within the context of local and regional culture and other differentiating characteristics.

"Choosing the data you need and very clearly defining the scope of data you are analyzing is crucial," Naroju said.

Once companies have the best, most refined, and most useful data, AI tools become key. This technology can extract insights from massive amounts of unstructured data, something "no human could accomplish in such a short time," Naroju said.

But even with the most creative data sources and the most advanced tools, forecasts are ultimately just forecasts.

"If you could crack the code of perfectly forecasting today's consumer, you would quickly become the world's richest person," said Jake Self, vice president of operations at Smart Warehousing, a third-party warehousing and logistics company.

"Too much noise"

Generations of companies have relied on the past to predict the future. After the pandemic and subsequent inflation disrupted normal patterns, with volatility cycles stacking on top of each other since 2020, this approach may seem outdated.

But for many companies, historical trends remain the most useful forecasting data. "It's not particularly flashy, but we have always held the philosophy that history is the best indicator for predicting the future," Harvey Kanter, CEO of Destination XL Group, told Supply Chain Dive in April.

"I think there has been too much noise over the past three-plus years, and the roller-coaster swings we experienced were almost unpredictable," he added. "I'm not sure the highest level of AI and machine learning could predict consumer behavior."

Kanter further noted that the apparel retailer still references pre-pandemic revenue and historical revenue contributions broken down by day, week, and month over the past 10 years, which he said have remained remarkably consistent.

Other companies simply do not have the budget to pay for or justify advanced technology tools. Smart Warehousing's clients range from small brands started in garages to Fortune 500 companies. Self said that for some small companies seeking replenishment and warehousing support, their forecasting process is essentially equivalent to "hope and dreams."

For these companies, historical data is far better than no data. "We have invested a lot of time in our replenishment programs, truly building an algorithm that learns from historical data and understands the inventory levels we need to achieve," Self said.

Smart Warehousing can do this because it has invested in forecasting and replenishment technology for over twenty years and has a dedicated team of experts focused on this skill. Self mentioned that an algorithm for the Missouri cold storage replenishment business "looks like something out of 'Good Will Hunting'"—the film starring actor Matt Damon as an unrecognized mathematical genius.

But even with these resources, forecasting remains difficult in the current environment. "If you don't plan for unexpected factors, then your planning itself may be flawed," Self said.