Artificial Intelligence and Discrimination in Health Care

Artificial intelligence (AI) holds great promise for improved health-care outcomes. It has been used to analyze tumor images, to help doctors choose among different treatment options, and to combat the COVID-19 pandemic. But AI also poses new hazards. This Article focuses on a particular type of health-care harm that has thus far evaded significant legal scrutiny. The harm is algorithmic discrimination. Algorithmic discrimination in healthcare occurs with surprising frequency. A well-known example is an algorithm used to identify candidates for “high risk care management” programs that routinely failed to refer racial minorities for these beneficial services. Some algorithms deliberately adjust for race in ways that hurt minority patients. For example, such algorithms have regularly underestimated African Americans’ risks of kidney stones, death from heart failure, and other medical problems. The Article argues that algorithmic discrimination in medicine can violate civil rights laws such as Title VI and Section 1557 of the Affordable Care Act when it exacerbates health disparities or perpetuates inequities. It urges that algorithmic fairness constitute a key element in designing, validating, and implementing AI and that both legal and technical tools be deployed to promote fairness. To that end, we call for the reintroduction of the disparate impact theory as a robust litigation tool in the health-care arena and for the passage of an algorithmic accountability act. We also detail technical measures that AI developers and users should implement.