Amazon canceled a multi-year project to develop an experimental automated recruiting engine after the e-commerce giant’s machine learning team discovered that the system was exhibiting explicit bias against women, Reuters reports. The engine, which the team began building in 2014, used artificial intelligence to filter résumés and score candidates on a scale from one to five stars. Within a year of starting the project, however, it became clear that the algorithm was discriminating against female candidates when reviewing them for technical roles.
Because the AI was taught to evaluate candidates based on patterns it found in ten years of résumés submitted to Amazon, most of which came from men, the system “taught itself that male candidates were preferable,” according to Reuters:
It penalized resumes that included the word “women’s,” as in “women’s chess club captain.” And it downgraded graduates of two all-women’s colleges, according to people familiar with the matter. They did not specify the names of the schools. Amazon edited the programs to make them neutral to these particular terms. But that was no guarantee that the machines would not devise other ways of sorting candidates that could prove discriminatory, the people said.
The company scuttled the project by the start of 2017 after executives lost faith in it. By that time, however, it may have already helped perpetuate gender bias in Amazon’s own hiring practices. The company told Reuters its recruiters never used the engine to evaluate candidates, but did not dispute claims from people familiar with the project that they had had looked at the recommendations it generated.
9th US Circuit Court of Appeals (InFootage/Shutterstock.com)
In the past week, the US Court of Appeals for the Ninth Circuit issued two decisions on the class-action status of lawsuits against two major tech companies, denying class-action status to Uber employees seeking a ruling on their employment classification but saying it would hear arguments for a class action against Microsoft in a lawsuit alleging gender discrimination. In the Microsoft case, the Seattle Times reported last Monday, the court said it would hear an appeal of a lower court’s decision denying class-action status:
The plaintiffs in the lawsuit are seeking to add to the case more than 8,600 women who have worked in engineering jobs at the company, making it a class-action suit. U.S. District Judge James Robart denied the class-action motion in June, saying there were not strong enough similarities between the women’s claims to prove companywide bias practices.
The case, Moussouris v. Microsoft, is one of a few high-profile gender-discrimination lawsuits against big tech companies weaving its way through the courts. It has been going on for three years and alleges gender discrimination across Microsoft, particularly tied to the way performance reviews and promotions were conducted. Microsoft has denied the claims, saying its processes do not discriminate against women. Lawyers for the plaintiffs appealed Robart’s ruling this summer that blocked class-action status.
The case against Microsoft was first filed by three female employees in 2015, alleging that the company’s pay and promotion processes systematically discriminated against women in technical and engineering roles. Documents unsealed in the lawsuit last November also suggested the tech company failed to respond appropriately to reports of sexual harassment and assault. The appeals court is expected to hear oral arguments in this case next year.
Meanwhile, on Tuesday, a three-judge panel at the Ninth Circuit ruled in favor of Uber in a lawsuit concerning whether drivers using the ride-sharing app should be classified as employees or independent contractors. According to the Verge, the court determined that Uber’s arbitration clause prohibits drivers from engaging in class actions, so it had no choice but to reverse a lower court’s class certification order: