When it comes to making judgments based on large data sets, machines are often superior to humans, but many business leaders remain skeptical of the guidance produced by their organizations’ data analytics programs, particularly when it comes to talent analytics. That skepticism derives largely from doubts about the quality of the data the organization is collecting, but there is also a natural tendency among people who make strategic decisions for a living to reject the notion that an algorithm could do parts of their job as well as or better than they can.
While this may be true of executives and high-level professionals, some recent research suggests that most people are actually comfortable with the decisions algorithms make and even more trusting of them than of judgments made by humans. A new study from the Harvard Business School, led by post-doctoral fellow Jennifer M. Logg, finds that “lay people adhere more to advice when they think it comes from an algorithm than from a person”:
People showed this sort of algorithm appreciation when making numeric estimates about a visual stimulus (Experiment 1A) and forecasts about the popularity of songs and romantic matches (Experiments 1B and 1C). Yet, researchers predicted the opposite result (Experiment 1D). Algorithm appreciation persisted when advice appeared jointly or separately (Experiment 2). However, algorithm appreciation waned when people chose between an algorithm’s estimate and their own (versus an external advisor’s—Experiment 3) and they had expertise in forecasting (Experiment 4). Paradoxically, experienced professionals, who make forecasts on a regular basis, relied less on algorithmic advice than lay people did, which hurt their accuracy.
Our colleagues here at Gartner have also investigated consumers’ attitudes toward AI and found that these attitudes are more welcoming than conventional wisdom might lead you to believe. The 2018 Gartner Consumer AI Perceptions Study found that overall, consumers are not skeptical of the potential usefulness of AI, though they do have some concerns about its impact on their skills, social relationships, and privacy. The study was conducted online during January and February 2018 among 4,019 respondents in the US and UK. Respondents ranged in age from 18 through 74 years old, with quotas and weighting applied for age, gender, region, and income.
The EU’s General Data Protection Regulation, which went into effect on May 25, imposes new data privacy obligations on all organizations that process the data of EU citizens, whether or not they are based in Europe themselves. The maximum penalties for noncompliance are hefty, so it is essential for businesses to ensure that their practices are GDPR-compliant if they haven’t already.
According to a survey on the eve of the regulation coming into effect, however, most organizations have not yet finished making the required changes, while many do not expect to be fully compliant by the end of this year. Much work still remains to be done to bring organizations into initial compliance with the regulation, and still more work to re-develop data collection, storage, and analytics programs in a compliant manner.
With every organization doing a huge amount of work for the first time and trying to get right with the GDPR as quickly as possible, this makes for a fertile environment for bad information to circulate and for opportunists to take advantage of organizations’ unfamiliarity with the new regulatory terrain. Organizational leaders need to be vigilant about which “experts” to trust for guidance on GDPR compliance, take advantage of the information provided directly by the European Commission, and bear in mind that different functions, particularly HR, face unique compliance challenges.
Step 1: Beware of Charlatans
The proliferation of bad advice and information is a simple matter of supply and demand. Demand for advice is high, both because of the global impact of the GDPR and because so many organizations were not proactive in planning for compliance are now scrambling to catch up. The supply of that advice is scarce and of uneven quality, with no historical track record of performance. Over the past few months, many companies have been assembling data protection functions and hiring data protection officers (DPOs), causing a run on the thin supply of qualified talent for these roles.
In today’s digital organizations, HR departments are increasingly using algorithms to aid in their decision-making, by predicting who is a retention risk, who is ready for a promotion, and whom to hire. For the employees and candidates subjected to these decisions, these are important, even life-changing, events, and so we would would expect the people making them to be closely supervised and held to a set of known performance criteria. Does anyone supervise the algorithms in the same way?
Algorithms don’t monitor themselves. Replacing a portion of your recruiting team with AI doesn’t obviate the need to manage the performance of that AI in the same way you would have managed the performance of the recruiter. To ensure that the decisions of an AI-enhanced HR function are fair, accurate, and right for the business, organizations must establish performance criteria for algorithms and a process to review them periodically.
A recent special report in The Economist illustrates the significant extent to which AI is already changing the way HR works. The report covers eight major companies that are now using algorithms in human resource management, which they either developed internally or bought from a growing field of vendors for use cases including recruiting, internal mobility, retention risk, and pay equity. These practices are increasingly mainstream; 2018 may mark the year of transition between “early adopters” and “early majority” in the life cycle of this technology.
At this point in time, it is essential that leaders ask themselves whether their organizations have management practices in place to supervise the decisions of these algorithms. The Economist concludes their piece with a reminder about transparency, supervision, and bias, noting that companies “will need to ensure that algorithms are being constantly monitored,” particularly when it comes to the prevention of bias.
The EU’s upcoming General Data Protection Regulation (GDPR), which is scheduled to come into force on May 25, expands the reach of existing privacy regulations, applying not just to European organizations but to all companies processing the personal data of EU residents, no matter where the company is located. It also requires organizations to request users’ consent for data collection and grants EU citizens a number of new rights, including the right to access data collected about them and the “right to be forgotten,” or to have that data erased. Organizations caught violating the regulation risk fines of as much as 4 percent of their annual global turnover or 20 million euros.
The GDPR has sent many companies scrambling to establish new data privacy functions and hire data protection officers to manage what they expect to be a hefty compliance challenge. For any organization that does business in Europe, GDPR compliance will involve ensuring that employee data is managed correctly, meaning the HR function has a large part to play. Talent Economy’s Sarah Fister Gale gives a primer on what the impending regulation means for HR:
The main job for HR on these projects is to make sure EU employees and recruits are given notice describing what personal data the company is collecting, how it is being used and how it will be shared and kept. [Neal Dittersdorf, general counsel and privacy officer for iCIMS,] noted that many companies already provide data notifications to these workers, however HR needs to be certain the language and timing of these notifications is updated to reflect GDPR requirements. …
SHRM’s Roy Maurer recently highlighted a survey from KPMG showing that corporate leaders around the world remain distrustful toward their organizations’ data and analytics when it comes to using these tools to make business decisions:
In the survey of 2,190 senior executives from Australia, Brazil, China, France, Germany, India, South Africa, the U.K. and the U.S., just 35 percent said they have a high level of trust in their organization’s use of data and analytics. Another 40 percent said they had reservations about relying on the data and analytics they produce, and 25 percent admitted they have either limited trust or active distrust in their data and analytics. Nearly all respondents (92 percent) worry about the impact flawed data could have on their company’s business and reputation.
“Executives and managers are being asked to make major decisions based on the output of an algorithm that they didn’t create and don’t always fully understand,” said Thomas Erwin, global head of KPMG International’s Lighthouse, the firm’s center of excellence for data, analytics and intelligent automation. “As a decision-maker, you really need to have confidence that the insights you are getting are reliable and accurate, but many of these executives can’t even be sure if their models are of sufficient quality to be trusted. It’s an uncomfortable situation for any decision-maker to be in.”
One barrier to the credibility of analytics for business leaders is the prevalence of incomplete data; another is that the metrics against which organizations are measuring are often ill-defined. HR metrics like source of hire and quality of hire are particularly hard to measure accurately, Kevin Wheeler, founder and president of the Future of Talent Institute, tells Maurer, and there is significant disagreement on how best to define them.
The online survey development SurveyMonkey has launched a new tool for employers called SurveyMonkey Engage, which (you guessed it) surveys employees about their engagement, job satisfaction, future plans, and other talent dynamics. Phil Albinus reports at Employee Benefit News that the platform collects anonymous survey responses, analyzes them, and displays them in a dashboard for HR directors and senior executives to review:
The first Engage poll, called a “core survey,” is 15 questions plus a few questions that require comments, which the company says takes only five minutes to complete. Employees have two weeks to fill out the survey before the polling is closed.
After the first survey, Engage begins a monthly follow-up survey of three to five questions to create what it calls a “check in.” This “conversational cadence” allows SurveyMonkey to poll employees for their opinions on their employer’s workplace, connection with their job, work matters and more. Every six months, employees take another core survey that resembles the first survey to “establish the baseline of employee engagement.” The company just released a new question bank to add additional questions to the check-in surveys if they desire.
SurveyMonkey is only the latest polling company to get in the engagement survey game: Gallup, for example, has been offering employee engagement services for years. With business leaders more focused than ever on capturing the value of talent analytics, these third-party services are likely to proliferate further in the coming years.
Data scientists are among the most in-demand professionals in the US right now, as more and more industries look to harness the power of data to drive productivity and innovation to new heights. Demand is high and supply is short, so these experts command remarkably high salaries. However, recent research has suggested that many companies’ data isn’t sufficiently high-quality to produce the kinds of insights managers are expecting. This is particularly true in the emerging field of talent analytics, where companies are making major investments but most aren’t seeing them pay off.
In addition to the data quality challenge, Data Quality Solutions President Thomas C. Redman suggested in a recent Harvard Business Review article that senior managers at many organizations are mismanaging their data scientists: placing them in the wrong part of the organization, not focusing the data science program on business outcomes, and not facilitating a transition to a more data-driven culture. He offers some suggestions for how companies can get more out of their data scientists:
First, think through how you want data scientists to contribute, and put them in spots where they can do so. The worst mistake a company can make is to hire a cadre of smart data scientists, provide them with access to the data, and turn them loose, expecting them to come up with something brilliant. Lacking focus and support, most fail. Instead, clearly define the opportunities you want to address using data science, and put your data scientists in places in the organization where they can best pursue those opportunities. …