When it comes to making judgments based on large data sets, machines are often superior to humans, but many business leaders remain skeptical of the guidance produced by their organizations’ data analytics programs, particularly when it comes to talent analytics. That skepticism derives largely from doubts about the quality of the data the organization is collecting, but there is also a natural tendency among people who make strategic decisions for a living to reject the notion that an algorithm could do parts of their job as well as or better than they can.
While this may be true of executives and high-level professionals, some recent research suggests that most people are actually comfortable with the decisions algorithms make and even more trusting of them than of judgments made by humans. A new study from the Harvard Business School, led by post-doctoral fellow Jennifer M. Logg, finds that “lay people adhere more to advice when they think it comes from an algorithm than from a person”:
People showed this sort of algorithm appreciation when making numeric estimates about a visual stimulus (Experiment 1A) and forecasts about the popularity of songs and romantic matches (Experiments 1B and 1C). Yet, researchers predicted the opposite result (Experiment 1D). Algorithm appreciation persisted when advice appeared jointly or separately (Experiment 2). However, algorithm appreciation waned when people chose between an algorithm’s estimate and their own (versus an external advisor’s—Experiment 3) and they had expertise in forecasting (Experiment 4). Paradoxically, experienced professionals, who make forecasts on a regular basis, relied less on algorithmic advice than lay people did, which hurt their accuracy.
Our colleagues here at Gartner have also investigated consumers’ attitudes toward AI and found that these attitudes are more welcoming than conventional wisdom might lead you to believe. The 2018 Gartner Consumer AI Perceptions Study found that overall, consumers are not skeptical of the potential usefulness of AI, though they do have some concerns about its impact on their skills, social relationships, and privacy. The study was conducted online during January and February 2018 among 4,019 respondents in the US and UK. Respondents ranged in age from 18 through 74 years old, with quotas and weighting applied for age, gender, region, and income.
SHRM’s Roy Maurer recently highlighted a survey from KPMG showing that corporate leaders around the world remain distrustful toward their organizations’ data and analytics when it comes to using these tools to make business decisions:
In the survey of 2,190 senior executives from Australia, Brazil, China, France, Germany, India, South Africa, the U.K. and the U.S., just 35 percent said they have a high level of trust in their organization’s use of data and analytics. Another 40 percent said they had reservations about relying on the data and analytics they produce, and 25 percent admitted they have either limited trust or active distrust in their data and analytics. Nearly all respondents (92 percent) worry about the impact flawed data could have on their company’s business and reputation.
“Executives and managers are being asked to make major decisions based on the output of an algorithm that they didn’t create and don’t always fully understand,” said Thomas Erwin, global head of KPMG International’s Lighthouse, the firm’s center of excellence for data, analytics and intelligent automation. “As a decision-maker, you really need to have confidence that the insights you are getting are reliable and accurate, but many of these executives can’t even be sure if their models are of sufficient quality to be trusted. It’s an uncomfortable situation for any decision-maker to be in.”
One barrier to the credibility of analytics for business leaders is the prevalence of incomplete data; another is that the metrics against which organizations are measuring are often ill-defined. HR metrics like source of hire and quality of hire are particularly hard to measure accurately, Kevin Wheeler, founder and president of the Future of Talent Institute, tells Maurer, and there is significant disagreement on how best to define them.
Data scientists are among the most in-demand professionals in the US right now, as more and more industries look to harness the power of data to drive productivity and innovation to new heights. Demand is high and supply is short, so these experts command remarkably high salaries. However, recent research has suggested that many companies’ data isn’t sufficiently high-quality to produce the kinds of insights managers are expecting. This is particularly true in the emerging field of talent analytics, where companies are making major investments but most aren’t seeing them pay off.
In addition to the data quality challenge, Data Quality Solutions President Thomas C. Redman suggested in a recent Harvard Business Review article that senior managers at many organizations are mismanaging their data scientists: placing them in the wrong part of the organization, not focusing the data science program on business outcomes, and not facilitating a transition to a more data-driven culture. He offers some suggestions for how companies can get more out of their data scientists:
First, think through how you want data scientists to contribute, and put them in spots where they can do so. The worst mistake a company can make is to hire a cadre of smart data scientists, provide them with access to the data, and turn them loose, expecting them to come up with something brilliant. Lacking focus and support, most fail. Instead, clearly define the opportunities you want to address using data science, and put your data scientists in places in the organization where they can best pursue those opportunities. …
Diversity, new interviewing tools, data, and artificial intelligence are the four trends set to have the biggest impact on recruiting in the coming year, according to LinkedIn’s latest Global Recruiting Trends report. Based on a survey of over 9,000 talent leaders and hiring managers worldwide, along with a series of expert interviews, the report underscores the growing role of technology in shaping how companies meet their hiring goals, of which diversity is increasingly paramount. Nonetheless, while many HR leaders see these trends as important, the number of organizations fully acting on them lags far behind.
Diversity was the top trend by far, with 78 percent of respondents saying it was very or extremely important, though only 53 percent said their organizations had mostly or completely adopted diversity-oriented recruiting. In recent years, diversity has evolved from a compliance issue to a major driver of culture and performance, as more and more organizations recognize its bottom-line value. This shift was reflected in the LinkedIn report, with 62 percent of the companies surveyed saying they believed boosting diversity would have a positive impact on financial performance and 78 percent saying they were pursuing it to improve their culture. Additionally, 49 percent are looking to ensure that their workforce better reflects the diversity of their customer base.
Diversity was the only top trend identified in LinkedIn’s survey that wasn’t directly related to technology, but technology is definitely influencing how organizations are pursuing it. In the past year, we have seen the emergence of new software and tools to support diversity and inclusion. The aim of these tools is to remove the human error of unconscious bias from the recruiting process, but it’s important to be aware that automated processes can also develop built-in biases and end up replicating the very problem they are meant to solve. This is an issue we’ve been following in our research at CEB, now Gartner; CEB Diversity and Inclusion Leadership Council members can read more of our insights on algorithmic bias here.
The development of new interview tools and techniques was identified as the second most important trend, with 56 percent saying it was important. The LinkedIn survey found that the most common areas where traditional interviews fail are assessing candidates’ soft skills (63 percent), understanding candidates’ weaknesses (57 percent), the biases of interviewers (42 percent), and the process taking too long (36 percent). The report highlights five new interviewing techniques, all enabled by technology, that aim to address these problems:
Employee monitoring technologies represent the cutting edge of workplace gadgets, and these technologies are already becoming increasingly common, from sociometric badges to tracking devices at desks to sentiment analysis and even experiments with microchipping employees. Olivia Solon at the Guardian recently explored the next generation of this tech:
How can an employer make sure its remote workers aren’t slacking off? In the case of talent management company Crossover, the answer is to take photos of them every 10 minutes through their webcam. The pictures are taken by Crossover’s productivity tool, WorkSmart, and combine with screenshots of their workstations along with other data —including app use and keystrokes—to come up with a “focus score” and an “intensity score” that can be used to assess the value of freelancers.
Today’s workplace surveillance software is a digital panopticon that began with email and phone monitoring but now includes keeping track of web-browsing patterns, text messages, screenshots, keystrokes, social media posts, private messaging apps like WhatsApp and even face-to-face interactions with co-workers. …
According to our research at CEB, now Gartner, even though 85 percent of CEOs believe it enhances business performance, only one third of employees are satisfied with diversity and inclusion at their organization, while nearly 60 percent of heads of HR believe their D&I strategy is ineffective. Many organizations are focused on making their cultures more inclusive and ensuring compliance with evolving legislation, but aren’t always seeing the results they had hoped for.
At our recent summit for HR executives in Johannesburg, more than 100 HR executives from 45 organizations had the opportunity to share ideas and hear from a panel of their peers how progressive organizations in South Africa are addressing the challenge of enhancing and evolving their D&I strategies.
1) Bring the Outside In
When defining what successful D&I looks like, our participants highlighted ideas and innovations, deliberate dialogue and co-creation, and thinking about diversity in all aspects: clients, products, and employees alike. The more integrated these are, the greater the impact. Many companies find that hiring employees from more diverse backgrounds gives them a way to engage new markets through new products, ideas or services. By bringing new perspectives into the organization, companies were better able to address the needs of both employees and customers.
2) Tackle Systems and Processes
Organizations that have made progress on D&I stressed the value of accelerated development programs that have yielded results in nurturing internal talent, including C-suite executives developed from within the organization; as well as the need to make hard decisions such as suspending the promotion process because the pool of candidates was not diverse enough.
Even though 91 percent of S&P global companies offer D&I training with 46 percent of organizations conducting their D&I training to mitigate unconscious bias, but as one participant shared, “It’s hard to catch bias in the moment.” One way to mitigate bias is by creating accountability for decision makers. For example, rather than expecting a hiring manager to make unbiased decisions independently, organizations are using a diverse panel when interviewing candidates. (To learn more, CEB Recruiting Leadership Council members can read our research on Driving Diversity Through Talent Acquisition.)
In this half-hour talk posted last week at re:Work, Google’s VP of People Operations Prasad Setty discusses his experience leading the development of the search giant’s talent analytics program, and about the key difference he discovered between having data make decisions for people, and using data to improve the way people make decisions:
When Prasad Setty joined Google ten years ago to build its People Analytics team, he envisioned a workplace where all people-related decisions would be made by data and analytics. If algorithms were spitting out search terms, why couldn’t we use them to make decisions for and about our people?
Setty soon discovered that this was the wrong approach. Despite the ability of analytics to objectively predict outcomes with high accuracy, people were reluctant to rely solely on formulas when it came to making important decisions — especially decisions that involved people, such as a promotion. And so, Setty shifted his vision for the People Analytics team. Rather than using data and analytics to make all decisions at Google, the team’s mission would be to educate Googlers on how they were making decisions and to help them make better decisions over time.
What really stands out about Google’s approach here is that they chose not to use a quantitative focus, even though they had the analytic sophistication necessary to do so. At one point, Setty mentions how HR was able to create a logistic predictive model that was able to accurately predict promotion decisions with an error rate of only 10 percent based on a few easily measurable attributes. Despite this, the engineers involved in the hiring process made it very clear that they did not want to outsource such an important task away to an algorithm.
This is an important lesson in how organizations can effectively use data in managing talent issues, particularly culture change.