Traditional statistical methods designed to detect discrimination are often ill-suited when applied to algorithmic decisions, since such algorithms typically exclude race or other protected factors from their decision-making inputs. To address this limitation, Dr. Shroff and colleagues introduce "risk-adjusted regression," a three-step method in which one:
estimates decision risk using all available information in a flexible machine learning model;
measures disparities adjusting solely for that estimated risk; and
assesses the sensitivity of estimated disparities to potential mismeasurement of risk.
When applied to 2.2 million police stops in New York City, this approach revealed that conventional approaches can substantially understate the true magnitude of unjustified racial disparities. This paper was published in the Proceedings of the National Academy of Sciences.