Anders Hagstrom, DCNF
Amazon shut down an experimental artificial intelligence recruitment engine after it started preferring male candidates and penalizing resumes including certain all-female colleges or words like “women’s,” Reuters reported Tuesday.
Amazon has had a team working on AI recruitment tools since 2014, seeking to make it easier for employers to find their top applicants. Several members of the team who spoke to Reuters on condition on anonymity said their experimental product began showing a bias against female candidates and had to be shut down in 2015. Amazon disbanded the team in 2017.
“Everyone wanted this holy grail,” one of the people told Reuters. “They literally wanted it to be an engine where I’m going to give you 100 resumes, it will spit out the top five, and we’ll hire those.”
The AI was designed to identify patterns among top applicants and resumes, and the team gave the AI the past 10 years of Amazon applications to use as a sample. The team says that because the majority of those applicants were male, the AI developed a bias against women.
Specifically, the program would penalize applications that used the modifier “women’s” to describe groups that an applicant had been a part of. The program also penalized applicants from two all-female colleges, according to Reuters. The AI’s bias wasn’t purely negative; however, it also favored applications featuring words most often used by male candidates such as “executed” and “captured.”
Amazon used the tool to make its own hiring decisions prior to discovering the bias, but because it was experimental, the company didn’t rely upon it exclusively.
Amazon has been on the forefront of automation and AI learning, facing off with competitors like Google and Facebook. Amazon’s largely automated warehouses are part of the reason it can ship products so cheaply and efficiently.
Amazon’s preference for tech over human workers has long dogged the company’s popularity, a problem it sought to fix last week by raising its minimum wage to $15 an hour in reaction to criticism that it was taking advantage of welfare programs in order to pay its employees low wages. Not everyone was happy with the raise, however, as it came at the expense of worker’s benefits and stock options. Amazon insisted all its employees were getting a net benefit from the change.
“We can confirm that all hourly Operations and Customer Service employees will see an increase in their total compensation as a result of this announcement,” Amazon said in a statement. “In addition, because it’s no longer incentive-based, the compensation will be more immediate and predictable.”