An Anatomy of Algorithm Aversion

Abstract

People are said to show “algorithm aversion” when they prefer human forecasters or decision-makers to algorithms, even though algorithms generally outperform people (in forecasting accuracy and/or optimal decision-making in furtherance of a specified goal). Algorithm aversion also has “softer” forms, as when people prefer human forecasters or decision-makers to algorithms in the abstract, without having clear evidence about comparative performance. Algorithm aversion has strong implications for policy and law; it suggests that those who seek to use algorithms, such as officials in federal agencies, might face serious public resistance. Algorithm aversion is a product of diverse mechanisms, including (1) a desire for agency; (2) a negative moral or emotional reaction to judgment by algorithms; (3) a belief that certain human experts have unique knowledge, unlikely to be held or used by algorithms; (4) ignorance about why algorithms perform well; and (5) asymmetrical forgiveness, or a larger negative reaction to algorithmic error than to human error. An understanding of the various mechanisms provides some clues about how to overcome algorithm aversion, and also of its boundary conditions. These clues bear on the numerous decisions in law and policy, including those of federal agencies (such as the Department of Homeland Security and the Internal Revenue Service) and those involved in the criminal justice system (such as those thinking about using algorithms for bail decisions).