Automated Hiring Systems and Racial Discrimination
Commercial automated hiring systems are increasingly used in recruitment and selection, yet audit studies and matched‑profile experiments reveal persistent disparate impacts by race. This paper reports on a series of audit experiments and vendor documentation reviews that assess selection probabilities across multiple commercial systems. Using matched resumes and controlled experiments, we show that certain algorithmic screening and scoring configurations yield lower selection rates for profiles with names statistically associated with Black applicants, even when qualifications are identical. We analyze vendor transparency practices and procurement terms and propose regulatory and procurement reforms—mandatory bias audits, public reporting of disparate impact metrics, and procurement standards prioritizing fairness—to reduce discriminatory outcomes in automated hiring.
Introduction
Automated hiring tools promise efficiency but can reproduce historical biases embedded in training data. This study evaluates disparate impacts across commercial systems and examines procurement and transparency practices that influence fairness.
Methods
We conducted matched‑profile audit experiments across three commercial systems and reviewed vendor documentation and procurement contracts. Statistical tests compared selection probabilities by name and other proxies for race while holding qualifications constant.
Results
Two of three systems showed lower selection probabilities for profiles with names associated with Black applicants. Vendor documentation often lacked sufficient transparency to evaluate fairness claims, and procurement contracts frequently limited independent audits.
Discussion
Regulatory and procurement reforms—mandatory independent audits, public reporting of disparate impact metrics, and procurement clauses requiring fairness guarantees—are necessary to reduce discriminatory outcomes. Employers should prioritize transparency and fairness in vendor selection.
References
- Barocas S, Selbst AD. Big data’s disparate impact. California Law Review. 2016.
- Noble SU. Algorithms of Oppression. NYU Press. 2018.
- Raghavan M, et al. Mitigating bias in algorithmic hiring. ACM FAT* 2020.