Pandemic exposes weak spots in AI systems trained on normal times
Coronavirus disruptions are testing machine-learning tools built on historical data, from trading algorithms to grocery website defenses.
By Hana Yoshida · Markets Reporter
3 min read
The coronavirus pandemic has become a live stress test for corporate artificial intelligence systems, Fortune’s Jeremy Kahn reported. The problem matters because many machine-learning models depend on patterns in past data, and the pandemic has pushed markets, shopping habits and web traffic outside familiar ranges.
In finance, rare events that jolt markets are often described as black swans, Kahn wrote. In AI and data science, similar surprises are often called edge cases, corner cases or out-of-distribution data points, and Kahn reported that many AI systems struggle when the data they see no longer resembles the data used to train them.
Trading is one example. Kahn reported that many AI-based trading systems have been put into use only in the past five years, meaning their training data may not include the 2008 financial crisis and is unlikely to include a shock like the pandemic’s broad hit to demand across industries.
Some investment strategies driven by AI and designed to perform across different market conditions have done worse than expected in recent weeks, Kahn reported, citing the Financial Times.
Grocery surge triggers AI alarms
The strain is also showing up outside finance. Kahn reported that Ocado, the U.K. online grocer, saw website traffic rise to four times the highest level it had recorded in its 20-year history.
On a call with reporters, Ocado spokesman David Shriver said the company’s machine-learning cybersecurity software interpreted the burst of visitors as a possible denial-of-service attack and prepared to block connections, according to Fortune. Human operations managers stepped in before that happened, Shriver said.
Jay Schuren, a data scientist at Boston-based DataRobot, told Fortune that companies need closer oversight of machine-learning models during abnormal periods. He said businesses should monitor model inputs in real time, especially when demand suddenly departs from normal levels.
Schuren said companies should identify which models and variables are most exposed to pandemic-related changes in human behavior. Systems tied to areas such as electricity use or shopping are likely to be affected by COVID-19 disruptions, he told Fortune.
How companies can limit damage
Schuren told Fortune that businesses should weigh the risk of each algorithm according to what it controls. A malfunctioning ad-placement system carries different consequences from a supply system that sends $1 million in products to a store closed under social-distancing measures.
He also urged data scientists to work with business specialists on simulations. Kahn reported that such stress tests could examine how customers might behave in a crisis and how a supply-management model would respond if many shoppers tried to buy months of toilet paper in a single week.
Schuren said data teams can adjust the inputs their systems use so models recover faster from extreme swings. Fortune cited the example of using percentage changes in prices rather than prices themselves.
Companies can also search their historical data for rough comparisons, Schuren said, such as patterns from Hurricane Sandy or the 1973 oil crisis. He cautioned that teams should decide carefully whether pandemic-era behavior belongs in future training data, since including it could either help models handle a similar crisis or lead them to treat temporary behavior as a lasting pattern.
Schuren told Fortune that companies may benefit from maintaining different model families for different conditions. One set could prioritize efficiency in normal periods, while another could be less efficient but more resilient when unusual data begins to arrive.
This story draws on original reporting from Fortune.