An unengaged workforce is immensely costly. It’s also a stark reality in Canada and the US. According to Gallup, only 31% of employees say they are functioning at their potential in terms of both their value to employers and the sense of fulfilment they derive from work.

Employee engagement is dire, which is why companies are increasing the frequency of their employee survey deployment. Surveys highlight what your workforce cares about, what’s working well, and what needs improvement. They are an effective tool for getting consensus on a choice, measuring interest level or satisfaction, and making employees feel heard.

Unfortunately, traditional employee surveys have significant shortcomings, too. They provide only a partial understanding of what’s happening within a workforce, leaving HR and business teams lacking critical insight. This is why interest in the new survey technology that leverages artificial intelligence (AI) is so high.

3 significant limitations of traditional employee surveys

To better understand this change in the landscape, let’s take a look at the gaps inherent in traditional employee surveys:

Lack of anonymity (real or perceived)
Employees are reluctant to criticize or report discontent because it can have negative repercussions for them if the survey is not anonymous (and let’s face it, often they aren’t). Their less-than-truthful answers skew the survey results—a problem that should not be underestimated! Harvard Business Review cites a costly strike at UPS “just ten months after receiving impressive marks on its regular annual survey on worker morale.” How serious of a problem is this? 42% withhold information when they have nothing to gain or something to lose, according to the Cornell National Social Survey.

Limited purview
The insight we gain is limited to the questions we ask, which are usually questions about strategy, tactics and the progress we’re making on particular initiatives. It’s therefore likely the survey won’t address some employee concerns, or that problems are bubbling up that are completely off management’s radar.

Though an old case, the UPS employee survey is a perfect example of this: “Although the survey had found that overall employee satisfaction was very high, it had failed to uncover bitter complaints about the proliferation of part-time jobs within the company, a central issue during the strike.”

Inability to measure employees’ emotions
Even the questions we ask aren’t giving us the full picture. Question types that work best in traditional surveys such as multiple choice and rating scales do not reveal employees’ feelings about their job and the workplace. Open-ended questions are a better measure, but they are generally interpreted on an individual basis and are subject to reader bias. This is because of “the volume of data generated and the difficulty grouping and analysing them,” says the SHRM. “Organizations regularly conducting surveys will want to be able to do trending analysis, and open-ended questions make that difficult.”

Despite their gaps, these surveys continue to play an important role in understanding employees. This said new technology and techniques are providing us with measures of employee engagement and sentiment that close the gaps in traditional surveys. Let’s look at this technology, and how organizations are layering insights it gives on to their survey data.

The rise of artificial intelligence (AI) in survey platforms
Recently, some survey platforms have begun to employ AI to interpret and analyze large amounts of data in a human-like manner. In some cases, the computer is programmed to learn as it works, such that its accuracy increases over time. Organizations adopting this new generation of survey technology are able to close the big gaps inherent in traditional employee surveys, here’s how:

Interpreting open-ended questions
Survey platforms employing AI are able to comb through, organize and analyze qualitative feedback in aggregate rather than individually, and report holistic unbiased findings in an instant. Known as sentiment analysis, this functionality unearths unique insights about employees’ feelings that we’ve never before been able to capture.

Employee listening
AI is being used to aggregate and interpret data collected through “employee listening”, a type of observational research in which comments employees have posted online, for example on Indeed or Glassdoor, are compiled and analyzed in aggregate. This data is unprompted, (often) anonymous and unsolicited. This data captures people’s unbiased feelings, so it is important to contrast it with sentiment captured through open-ended survey questions. It is also valuable because it isn’t constrained to the organization’s priorities or focus, so it can unveil concerns we are unaware of. Whereas surveys give a point-in-time snapshot, employee listening can be used throughout the year to monitor for issues and proactively respond to changes.

Because traditional surveys provide only partial insight into a workforce, companies are turning to sentiment analysis and employee listening to fill the gaps. It should be treated as an add-on to surveying and other initiatives currently underway, this can be done by employing the services of a separate firm or finding a partner who can assist you with the technology and analytics for all three parts: your survey, qualitative feedback, and employee listening.

Sentiment analysis is causing a huge stir in the HR world because it addresses a long-standing gap. The expectation is that this far more accurate data about employees’ concerns and feelings will give us a lot more clarity on what we need to do to address low levels of engagement.