SHRM’s Roy Maurer recently highlighted a survey from KPMG showing that corporate leaders around the world remain distrustful toward their organizations’ data and analytics when it comes to using these tools to make business decisions:
In the survey of 2,190 senior executives from Australia, Brazil, China, France, Germany, India, South Africa, the U.K. and the U.S., just 35 percent said they have a high level of trust in their organization’s use of data and analytics. Another 40 percent said they had reservations about relying on the data and analytics they produce, and 25 percent admitted they have either limited trust or active distrust in their data and analytics. Nearly all respondents (92 percent) worry about the impact flawed data could have on their company’s business and reputation.
“Executives and managers are being asked to make major decisions based on the output of an algorithm that they didn’t create and don’t always fully understand,” said Thomas Erwin, global head of KPMG International’s Lighthouse, the firm’s center of excellence for data, analytics and intelligent automation. “As a decision-maker, you really need to have confidence that the insights you are getting are reliable and accurate, but many of these executives can’t even be sure if their models are of sufficient quality to be trusted. It’s an uncomfortable situation for any decision-maker to be in.”
One barrier to the credibility of analytics for business leaders is the prevalence of incomplete data; another is that the metrics against which organizations are measuring are often ill-defined. HR metrics like source of hire and quality of hire are particularly hard to measure accurately, Kevin Wheeler, founder and president of the Future of Talent Institute, tells Maurer, and there is significant disagreement on how best to define them.
Data scientists are among the most in-demand professionals in the US right now, as more and more industries look to harness the power of data to drive productivity and innovation to new heights. Demand is high and supply is short, so these experts command remarkably high salaries. However, recent research has suggested that many companies’ data isn’t sufficiently high-quality to produce the kinds of insights managers are expecting. This is particularly true in the emerging field of talent analytics, where companies are making major investments but most aren’t seeing them pay off.
In addition to the data quality challenge, Data Quality Solutions President Thomas C. Redman suggested in a recent Harvard Business Review article that senior managers at many organizations are mismanaging their data scientists: placing them in the wrong part of the organization, not focusing the data science program on business outcomes, and not facilitating a transition to a more data-driven culture. He offers some suggestions for how companies can get more out of their data scientists:
First, think through how you want data scientists to contribute, and put them in spots where they can do so. The worst mistake a company can make is to hire a cadre of smart data scientists, provide them with access to the data, and turn them loose, expecting them to come up with something brilliant. Lacking focus and support, most fail. Instead, clearly define the opportunities you want to address using data science, and put your data scientists in places in the organization where they can best pursue those opportunities. …
Diversity, new interviewing tools, data, and artificial intelligence are the four trends set to have the biggest impact on recruiting in the coming year, according to LinkedIn’s latest Global Recruiting Trends report. Based on a survey of over 9,000 talent leaders and hiring managers worldwide, along with a series of expert interviews, the report underscores the growing role of technology in shaping how companies meet their hiring goals, of which diversity is increasingly paramount. Nonetheless, while many HR leaders see these trends as important, the number of organizations fully acting on them lags far behind.
Diversity was the top trend by far, with 78 percent of respondents saying it was very or extremely important, though only 53 percent said their organizations had mostly or completely adopted diversity-oriented recruiting. In recent years, diversity has evolved from a compliance issue to a major driver of culture and performance, as more and more organizations recognize its bottom-line value. This shift was reflected in the LinkedIn report, with 62 percent of the companies surveyed saying they believed boosting diversity would have a positive impact on financial performance and 78 percent saying they were pursuing it to improve their culture. Additionally, 49 percent are looking to ensure that their workforce better reflects the diversity of their customer base.
Diversity was the only top trend identified in LinkedIn’s survey that wasn’t directly related to technology, but technology is definitely influencing how organizations are pursuing it. In the past year, we have seen the emergence of new software and tools to support diversity and inclusion. The aim of these tools is to remove the human error of unconscious bias from the recruiting process, but it’s important to be aware that automated processes can also develop built-in biases and end up replicating the very problem they are meant to solve. This is an issue we’ve been following in our research at CEB, now Gartner; CEB Diversity and Inclusion Leadership Council members can read more of our insights on algorithmic bias here.
The development of new interview tools and techniques was identified as the second most important trend, with 56 percent saying it was important. The LinkedIn survey found that the most common areas where traditional interviews fail are assessing candidates’ soft skills (63 percent), understanding candidates’ weaknesses (57 percent), the biases of interviewers (42 percent), and the process taking too long (36 percent). The report highlights five new interviewing techniques, all enabled by technology, that aim to address these problems:
Employee monitoring technologies represent the cutting edge of workplace gadgets, and these technologies are already becoming increasingly common, from sociometric badges to tracking devices at desks to sentiment analysis and even experiments with microchipping employees. Olivia Solon at the Guardian recently explored the next generation of this tech:
How can an employer make sure its remote workers aren’t slacking off? In the case of talent management company Crossover, the answer is to take photos of them every 10 minutes through their webcam. The pictures are taken by Crossover’s productivity tool, WorkSmart, and combine with screenshots of their workstations along with other data —including app use and keystrokes—to come up with a “focus score” and an “intensity score” that can be used to assess the value of freelancers.
Today’s workplace surveillance software is a digital panopticon that began with email and phone monitoring but now includes keeping track of web-browsing patterns, text messages, screenshots, keystrokes, social media posts, private messaging apps like WhatsApp and even face-to-face interactions with co-workers. …
According to our research at CEB, now Gartner, even though 85 percent of CEOs believe it enhances business performance, only one third of employees are satisfied with diversity and inclusion at their organization, while nearly 60 percent of heads of HR believe their D&I strategy is ineffective. Many organizations are focused on making their cultures more inclusive and ensuring compliance with evolving legislation, but aren’t always seeing the results they had hoped for.
At our recent summit for HR executives in Johannesburg, more than 100 HR executives from 45 organizations had the opportunity to share ideas and hear from a panel of their peers how progressive organizations in South Africa are addressing the challenge of enhancing and evolving their D&I strategies.
1) Bring the Outside In
When defining what successful D&I looks like, our participants highlighted ideas and innovations, deliberate dialogue and co-creation, and thinking about diversity in all aspects: clients, products, and employees alike. The more integrated these are, the greater the impact. Many companies find that hiring employees from more diverse backgrounds gives them a way to engage new markets through new products, ideas or services. By bringing new perspectives into the organization, companies were better able to address the needs of both employees and customers.
2) Tackle Systems and Processes
Organizations that have made progress on D&I stressed the value of accelerated development programs that have yielded results in nurturing internal talent, including C-suite executives developed from within the organization; as well as the need to make hard decisions such as suspending the promotion process because the pool of candidates was not diverse enough.
Even though 91 percent of S&P global companies offer D&I training with 46 percent of organizations conducting their D&I training to mitigate unconscious bias, but as one participant shared, “It’s hard to catch bias in the moment.” One way to mitigate bias is by creating accountability for decision makers. For example, rather than expecting a hiring manager to make unbiased decisions independently, organizations are using a diverse panel when interviewing candidates. (To learn more, CEB Recruiting Leadership Council members can read our research on Driving Diversity Through Talent Acquisition.)
In this half-hour talk posted last week at re:Work, Google’s VP of People Operations Prasad Setty discusses his experience leading the development of the search giant’s talent analytics program, and about the key difference he discovered between having data make decisions for people, and using data to improve the way people make decisions:
When Prasad Setty joined Google ten years ago to build its People Analytics team, he envisioned a workplace where all people-related decisions would be made by data and analytics. If algorithms were spitting out search terms, why couldn’t we use them to make decisions for and about our people?
Setty soon discovered that this was the wrong approach. Despite the ability of analytics to objectively predict outcomes with high accuracy, people were reluctant to rely solely on formulas when it came to making important decisions — especially decisions that involved people, such as a promotion. And so, Setty shifted his vision for the People Analytics team. Rather than using data and analytics to make all decisions at Google, the team’s mission would be to educate Googlers on how they were making decisions and to help them make better decisions over time.
What really stands out about Google’s approach here is that they chose not to use a quantitative focus, even though they had the analytic sophistication necessary to do so. At one point, Setty mentions how HR was able to create a logistic predictive model that was able to accurately predict promotion decisions with an error rate of only 10 percent based on a few easily measurable attributes. Despite this, the engineers involved in the hiring process made it very clear that they did not want to outsource such an important task away to an algorithm.
This is an important lesson in how organizations can effectively use data in managing talent issues, particularly culture change.
At the Harvard Business Review, Tadhg Nagle, Thomas C. Redman, and David Sammon present the findings of a study they conducted to assess the quality of data available to managers at 75 companies in Ireland. Using Redman’s Friday Afternoon Measurement method, they asked managers to collect critical data on the last 100 units of work conducted by their departments and mark them up, highlighting obvious errors and counting the number of error-free records to produce a data quality score. “Our analyses confirm,” they write, “that data is in far worse shape than most managers realize”:
- On average, 47% of newly-created data records have at least one critical (e.g., work-impacting) error. A full quarter of the scores in our sample are below 30% and half are below 57%. In today’s business world, work and data are inextricably tied to one another. No manager can claim that his area is functioning properly in the face of data quality issues. It is hard to see how businesses can survive, never mind thrive, under such conditions.
- Only 3% of the DQ scores in our study can be rated “acceptable” using the loosest-possible standard. We often ask managers (both in these classes and in consulting engagements) how good their data needs to be. While a fine-grained answer depends on their uses of the data, how much an error costs them, and other company- and department-specific considerations, none has ever thought a score less than the “high nineties” acceptable. Less than 3% in our sample meet this standard. For the vast majority, the problem is severe.
- The variation in DQ scores is enormous. Individual tallies range from 0% to 99%. Our deeper analyses (to see if, for instance, specific industries are better or worse) have yielded no meaningful insights. Thus, no sector, government agency, or department is immune to the ravages of extremely poor data quality.
The data quality challenge should sound familiar to HR leaders attempting to implement talent analytics strategies.