Algorithms dictate our world. They are used to predict crime, assess credit, screen resumes, and set prices. In health care settings, algorithms screen nearly 200 million people across the United States for admission to care management programs.
In simplest terms, an algorithm is a set of rules computers follow to make decisions and predictions. As our technology has become more complex, so have the models. Algorithms were thought to be objective, race-neutral tools. However, in practice, algorithms reproduce the same disparities we’ve created in our world.
A research team led by Ziad Obermeyer gained access to one commonly used health care algorithm to test whether there were racial biases in its predictions. The algorithm gives “illness risk scores” to predict a patient’s need for monitoring in a care management program. As the graph on the left illustrates, researchers found that Black patients were sicker than White patients even as the algorithm predicted they were equally needing of special care management. The graph on the right shows that, if the algorithm had no racial bias and attributed health risk more accurately, more Black patients would be in care management programs. The algorithm, in place in health care systems today, made predictions for this need based on prior health care spending, meaning racial inequities health care spending for white and black patients in the past was simply being carried forward.
According to the researchers, racial disparities in health care spending between Black and white patients exist for two primary reasons: first: differential access to healthcare because of socioeconomic status, race, and ethnicity, and second, discrimination affecting both a doctor’s willingness to suggest treatment to a Black patient and a Black patient’s willingness to trust a practitioner.
Health algorithms measure risk with data established in conditions of inequity, and as a result, they reproduce racial disparities. The researchers found that shifting from predictions based exclusively on future cost or future chance of developing an illness decreased bias by 84%. If we continue to use algorithms for efficiency, we must make sure these algorithms do not reproduce inequity.
Databyte via Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. Dissecting racial bias in an algorithm used to manage the health of populations. Science. 25 October 2019.
