Predictive analytics has dramatically changed how we make business decisions in today’s technology-driven world. Although automated decision-making processes have their benefits, we cannot deny the pivotal role that human expertise plays in managing risks and uncovering valuable insights that data offers.
We commissioned a research team from the University of Michigan, University of Wisconsin, Emory University, and WHU Vienna to explore when and why human expertise would override artificial intelligence (AI) recommendations based on predictive analytics to improve store-inventory planning at a car parts retailer in the United States.
The resulting report, Predictive Analytics: Should there be a “human in the loop”?, reveals that 1) the value of human involvement in decision-making is dependent on humans having information that predictive models cannot capture, 2) generalist analysts tend to override AI’s decisions with better recommendations than specialist analysts who play a dedicated role in inventory planning, and 3) effective predictive analytics also relies on the organisational design and management controls that connect the company with its external stakeholders.
Conditions that require human decision-making
When there wasn’t enough historical data or the market was particularly unpredictable, it was more effective to have a corporate analyst, acting as a ‘human in the loop’, provide high-quality decisions, or overrides, rather than relying on recommendations from a predictive model.
High-quality overrides required anticipating changes in an inventory item’s likelihood of selling within three months, as estimated by the predictive model. By anticipating a downward change, the analyst avoided excess inventory costs for the upcoming year; predicting an upward change prevented potential losses in sales.
Human analysts are important for managing risk analytics, evaluating scenarios, and making better informed decisions. AI systems alone can't do the job when there isn't enough information. Living, breathing experts use their experience and knowledge to understand things that AI can overlook.
The link between analyst characteristics and decision-making quality
Although it may sound counterintuitive, the report shows that generalist analysts, such as sales managers, regularly did a better job when recommending high-quality inventory overrides than specialists whose main job is to plan inventory.
Further, the report found that several analysts’ characteristics linked to high-quality decision-making — such as tenure, seniority, competence, or perceived cohesiveness — did not affect the quality of override decisions.
Ultimately, the success of overrides with inventory planning hinged on how well the analyst could access local information and unspoken knowledge from the company’s network of external stakeholders, which includes independent store owners.
AI’s predictive analytics relies on data mining and modeling, but human collaboration, such as an analyst working with a store owner, reveals nuance that AI systems can miss and puts AI suggestions into better business applications.
Engagement with stakeholders is needed for high-quality decision-making
Savvy analysts who were more engaged with the independent store owners and had broader job responsibilities were better able to make well-informed choices when it came to overriding these models, according to the report.
Report findings suggest that including stakeholders more often within the company’s store-inventory planning can help analysts access implied knowledge that is not reflected in historical data used by predictive models. However, this collaboration includes the awareness of stakeholder interests and incentives that may be at odds with those of the organisation.
Management accountants’ role in integrating predictive analytics and human-based decision-making
The report provides important insights on the usage of predictive analytics in the business world and how the integration of human expertise and predictive analytics can be key to achieving superior outcomes.
Our findings also suggest that management accountants can play a role in communicating with external stakeholders. And because of their central role in planning, they can identify and correct weaknesses in management control systems to facilitate integration choices.
By combining the power of AI systems with human decision-making and collaboration, companies can harness the full potential of predictive analytics while managing risks effectively and generating insights with digital intelligence for high-quality analyst override decisions.
Gain greater insight into the value of human expertise in predictive analytics by downloading Predictive Analytics: Should there be a “human in the loop”?.
You can also read more about how human intervention can improve predictive analytics, enhance your IT knowledge with the CITP credential, and access additional AICPA & CIMA resources.