Amazon canceled a multi-year project to develop an experimental automated recruiting engine after the e-commerce giant’s machine learning team discovered that the system was exhibiting explicit bias against women, Reuters reports. The engine, which the team began building in 2014, used artificial intelligence to filter résumés and score candidates on a scale from one to five stars. Within a year of starting the project, however, it became clear that the algorithm was discriminating against female candidates when reviewing them for technical roles.
Because the AI was taught to evaluate candidates based on patterns it found in ten years of résumés submitted to Amazon, most of which came from men, the system “taught itself that male candidates were preferable,” according to Reuters:
It penalized resumes that included the word “women’s,” as in “women’s chess club captain.” And it downgraded graduates of two all-women’s colleges, according to people familiar with the matter. They did not specify the names of the schools. Amazon edited the programs to make them neutral to these particular terms. But that was no guarantee that the machines would not devise other ways of sorting candidates that could prove discriminatory, the people said.
The company scuttled the project by the start of 2017 after executives lost faith in it. By that time, however, it may have already helped perpetuate gender bias in Amazon’s own hiring practices. The company told Reuters its recruiters never used the engine to evaluate candidates, but did not dispute claims from people familiar with the project that they had had looked at the recommendations it generated.
In recent years, many organizations have been looking for ways to make their recruiting processes less dependent on the bias and subjectivity of hiring managers, whether by using technology to hold blind interviews, making hiring decisions with pre-hire skills tests, or handing the process over entirely to an algorithm. Udemy leadership coach Lawrence Miller has a different approach, as he explains to Fast Company’s Stephanie Vozza, which entails having candidates interview each other rather than be interviewed by a manager:
Miller found the best employees for his Maryland-based management-consulting firm when he turned the interview process upside down, bringing in candidates in small groups, and asking them to interview him and his team and then each other. … When they completed their interviews, Miller gave each person a piece of paper that had these four questions:
- Who would you hire and why?
- Who do you think is most technically competent to do this job?
- Who has the best skills?
- Who would you choose to be stranded with in an airport during a snowstorm?
“The last question was a good indicator of likeability,” says Miller. “We found that question to be the most reliable, because in the kind of consulting we did, it was a really good predictor of who would succeed.”
Other experts Vozza spoke to warned, however, that this process can have drawbacks, such as putting introverts at a disadvantage and making it more difficult for candidates to get a genuine view into the organization. Another major issue with this practice is that having candidates interview each other creates an entirely new opportunity for bias.
Ken Wolter / Shutterstock.com
The development of business software and advancement of analytics are playing a big role in shaping the future of the HR function, and diversity and inclusion is no exception to this change. Last year, SAP announced a commitment to building software to enable corporate diversity on its popular SuccessFactors HCM Suite. Updates beginning this month will aim to remove bias from the hiring, performance review, and promotion processes.
One new feature will scan job descriptions for biased terminology geared towards men and recommend words to replace them. By the end of the year, another new feature promises to prevent managers from making biased decisions in staffing, compensation or promotions. Sarah Kessler has the details at Quartz:
Companies will have the option to set rules for their organizations, such as triggering a notification when a woman who has previously been rated highly gets down-rated after they take a leave of absence (which could be indicative of bias against women who take maternity leave) or if someone who has been rated highly consistently for years has been overlooked for promotions.
Recruiting experts have become increasingly vocal in recent years about the ineffectiveness of unstructured job interviews, which researchers have found cause interviewers to form strong but inaccurate impressions about candidates that often have more to do with the interviewer’s preferences and biases. Unstructured interviews, especially those including generic questions like “Where do you see yourself in five years,” encourage candidates to perform rote answers or say whatever they think the interviewer wants to hear.
Behavioral interviews, which examine how an interviewee reacts to challenges in their professional life, have emerged as the antidote to the unstructured interview. These interviews typically include asking candidates to give examples from their own experience of times when they faced stress, difficulty, or conflict at work and how they handled it—successfully or not. Organizational psychologist and Wharton professor Adam Grant, however, believes this type of question is problematic in its own right.
Why? Quite simply, Grant elaborates to Quartz’s Leah Fessler, they’re biased against candidates who have less work experience, or who for whatever reason don’t have good stories to draw on:
“When you ask questions about the past—’tell me about a time when you…’—interviewees with less experience in that situation are at a disadvantage,” Grant tells Quartz. The more jobs you have, the more you navigate professional conflict and success, and the more workplace anecdotes you accumulate. Meanwhile, even competitive younger candidates haven’t had enough professional exposure to narrate an equally nuanced story.
Last week, the New York City Council passed a law barring employers in the city from inquiring about the salary histories of job candidates. The measure is meant to help curb gender- and race-based pay gaps, drawing on the argument that using a candidate’s salary history to help determine their compensation perpetuates these gaps by making it more difficult for those who are underpaid early in their careers to catch up with their peers. Even if a hiring manager does not deliberately offer a candidate lower pay based on their salary history, Kristin Wong argues at Science of Us, finding out what the candidate currently earns can warp the manager’s perceptions of what they are worth:
When potential employers use your salary history as a reference point, it can work against you due to a cognitive bias called “anchoring.” With anchoring, people rely too heavily on one piece of information to come to a conclusion or a decision. For example, one of the first studies on anchoring, published in 1974, asked subjects to estimate how many African countries were members of the United Nations. Before answering, subjects spun a wheel with numbers on it. Behind the scenes, the study’s researchers controlled where the wheel landed so that it would either hit the number ten or the number 65. When the wheel landed on ten, people estimated that 25 percent of African nations were U.N. members. When it landed on 65, the estimate increased to 45 percent.
Without realizing it, the subjects latched onto the arbitrary number before making their guesses. This is the anchoring bias in action, and when a potential employer asks you about salary history, that number similarly serves as an anchor in the negotiation process. If you earned next to nothing in a past job, a low anchor could limit your chance to earn more in the new job. Plus, raises, bonuses, and pay increases are often based on a percentage of an employee’s salary, too.
Critics of these bans have countered that they could backfire on women by forcing employers to guess at candidates’ past compensation—which they might assume to be lower for female employees. As Bloomberg View points out in an editorial making the case against salary history bans, there is little evidence that they will have the intended effect of closing gender pay gaps. Instead, the editors call them “policymaking by anecdote, driven by politicians eager to say they have taken action”:
At Science of Us, Drake Baer highlights some interesting new research into how signals of social class disclosed on resumes may have an impact on hiring. In the study, professors Lauren Rivera and András Tilcsik sent out job applications to 316 top law firms, each of which received an application randomized by gender and social-class background. Social class backgrounds were indicated with several signals, including last name (“Cabot” vs. “Clark”), extracurricular activities, and hobbies and interests. The responses they received were surprising:
Those 316 applications led to 22 interview invitations, good for a 6.96 percent callback rate. (This, the authors note, is consistent with previous studies on white-collar jobs and expectations for applicants who were at the top of their class but didn’t go to super-élite schools.) What was bananas, however, is how that rate skewed by gender: the lower-class male got just one callback, the lower-class female five, the higher-class woman three, and the higher-class man thirteen. That means the blue-blooded James had a 16.25 callback rate, while his nearly identical siblings had a paltry 3.83 callback percentage.
“Coming from a higher-class background only helps men,” Rivera tells Science of Us. “Given my prior research, we thought that social class background would lift all those people regardless of gender, and that was not the case.”
Marianne Calnan at the CIPD highlights the Social Mobility Employer Index, “a joint initiative from the Social Mobility Foundation and the Social Mobility Commission [that] aims to showcase organisations improving social mobility in the UK by recruiting the best talent for job vacancies, regardless of their social background”:
The index is primarily aimed at companies in ‘elite’ sectors that have a poor track record of encouraging social mobility, including law, accountancy, media, banking and science. Research has consistently shown that people with more affluent backgrounds – including those who attended private schools and elite universities – take a disproportionate number of the ‘best’ jobs available. Around half of diplomats, 47 per cent of newspaper columnists and 38 per cent of members of the House of Lords attended either Oxford or Cambridge University, compared with less than 1 per cent of the British population as a whole, according to government figures.
Many organisations have turned their attention to social mobility in recent months: the BBC has started to collect a broader range of information about job applicants’ backgrounds, while the civil service is also deploying HR metrics to open up access to a broader range of candidates. Companies that wish to be listed on the new index will need to answer questions about their recruitment, selection and career progression practices. They will be ranked by a panel of experts, and receive recommendations of areas for improvement. Those that fail to make the grade will not be ‘named and shamed’, said the Social Mobility Commission.