AI is failing businesses with more promise than performance

The former chief economist of Microsoft and the founders of the startup Aible explain why AI needs to solve business problems, not math problems.

When it comes to making businesses run better, artificial intelligence has shown more promise than performance.

AI, which refers to algorithms that learn from and find patterns in huge quantities of data, is fundamentally about giving the right actionable information to someone at the moment of making a decision.

Only AI can do that at scale. AI can be used all across the enterprise, in sales, marketing, customer retention, customer support, fraud reduction—anything that requires a prediction based on data and actionable recommendations in order to make an informed business decision.

But a recent International Data Corporation survey of global organizations that are already using AI solutions found only 25% have developed an enterprise-wide AI strategy. Most organizations reported failures among their AI projects, with a quarter of them reporting up to a 50% failure rate.

Why? Too many times, AI fails to deliver the positive impact that businesses really want from the technology, like more revenue, lower cost, fewer customers lost to churn, higher manufacturing quality, and lower waste and fraud. The mathematics behind today’s AI is impressive (just ask any data scientist). But when it comes to making businesses more profitable, somehow the numbers don’t add up.

One key reason that AI underperforms for business is that most AI used by businesses today has been trained to maximize model accuracy—or the percentage of “correct” answers. But in many real-world business cases, the most accurate AI models aren’t the ones that result in the best business results.

If the AI provides only five good sales recommendations, it’s of little value to a sales rep needing to close 100 deals per month. After nearly two decades working in AI, we’ve heard a lot of frustrated business users say that their AI is never wrong—it’s just completely useless for the business improvement they’re trying to achieve. It tells them things they already know.

AI often fails to respect fundamental business principles, chief among them the Efficient Frontier, a term borrowed from economics and finance. The Efficient Frontier refers to trade-offs and the balance of risk and reward. It’s the set of portfolios or assets that offer the highest expected return for a defined level of risk, or the lowest risk for a given level of expected return.

For example, when the price of oil goes up, an energy company may add more resources—employees, drilling sites, advanced equipment—because the reward is high. But the company can’t add unlimited capacity—it has resource constraints that limit its ability to expand, even during a price surge.

Businesses are constantly calculating and recalculating the Efficient Frontier for every aspect of their operation, from manufacturing capacity and sales team size to inventory levels, marketing budget, and geographic location. That means they’re always seeking the optimal risk-reward trade-off for a given asset under current circumstances.

Today, there needs to be an Efficient Frontier for AI models. The models must change as business realities change, with many small models constantly re-computing the Efficient Frontier. Models can be specialized by geography—one model for sales in Germany, another for China, and a third for the U.S.

Models can also be customized according to current business opportunities and constraints. There might be one model for when a company has a lot of sales leads that week, and a different model for when there are relatively few leads.

As new data is generated, some models that performed well may start to decay, while some poor-performing models may show surprising improvement. The Efficient Frontier of models is all about using the right model at the right time.

Unfortunately, most AI today is generated by off-the-shelf machine learning platforms from companies like Amazon, Google, and DataRobot that often fail to take into account the sort of cost-benefit trade-offs and resource constraints that businesses face every day.

Many models assume that all costs and benefits are equal, but that’s almost never true in business. What if the benefit of winning a deal is 100 times the cost of pursuing a deal? In that case, you might be willing to pursue and lose 99 deals for a single win. An AI that finds only 1 win in 100 would be very inaccurate. But it would boost your net revenue. An AI trained for accuracy would never recognize this cost-benefit trade-off.

Similarly, most AI fails to respect resource constraints. If your company’s current sales capacity is restricted to only 10 leads you can effectively pursue, an AI that tells you to go after 100 leads is worthless. In business, operational constraints such as marketing budget and sales capacity matter—a lot.

Models rarely go out of tune for everything all at once—they first go out of tune in specific subgroups. A company may experience a big spike in customer complaints in the Boston area; then it quickly becomes a Massachusetts-wide problem, and before long, it permeates the entire enterprise. Specific subgroups going out of tune can act as canaries in a coal mine, a signal that the entire model may be starting to break.

AI needs to incorporate the fact that businesses are subject to constant change, and are not a vast collection of numbers that generally follow predictable patterns. AI optimized for business impact must adjust, enabling a company to continually run experiments and find new ways to be efficient.

Above all, companies need AI that solves business problems rather than math problems. Accuracy matters less than business impact. It doesn’t pay to have an AI that’s trained to learn more and more about things that aren’t going to make the company money. Too much of today’s AI gets lost in the complex data science metrics—every number, it seems, except profit.

The gap between AI’s promise and payoff has become a canyon of disappointment for business. AI needs to, and can, do a much better job at responding to the problems that businesses face every day. Businesses shouldn’t have to learn how to speak AI. AI needs to speak business.

R. Preston McAfee most recently served as chief economist at Microsoft, and is an advisor to Aible, an AI startup headquartered in Silicon Valley. Arijit Sengupta is the founder and CEO of Aible. Jonathan Wray is the cofounder of Aible.

Source: https://www.fastcompany.com