Tests Reveal Racial Bias in OpenAI GPT's Resume Screening
-
Bloomberg's experiment revealed that OpenAI GPT3.5 demonstrates noticeable racial bias when sorting resumes using fictional names. The study extracted names with at least 90% association to specific races or ethnicities from voter and census data, randomly assigning them to equally qualified resumes.
During 1,000 sorting attempts, GPT3.5 consistently favored certain ethnic names more frequently, violating benchmarks for evaluating workplace discrimination against protected groups. Among the four tested positions (HR business partner, senior software engineer, retail manager, and financial analyst), names associated with Black Americans were least likely to be ranked as top candidates by GPT for financial analyst and software engineer roles. Experiments also show that GPT exhibits varying gender and racial preferences across different job positions. Although GPT does not consistently favor any specific group, it selects winners and losers in different contexts. Additionally, when testing with the less commonly used GPT-4, significant biases were similarly detected.
In response to Bloomberg's detailed inquiries, OpenAI stated that the 'out-of-the-box' results from GPT models may not reflect how clients actually use them. Enterprises typically implement additional bias mitigation measures when deploying the technology, including fine-tuning software responses and managing system messages.
Despite the widespread attention on generative AI applications in human resources, this experiment highlights the serious risks of automated discrimination that may arise when using such technologies for recruitment and hiring. Addressing biases in AI models remains a major challenge for AI companies and researchers, while automated hiring systems could further complicate corporate diversity efforts.