Can You Really Measure Engagement?

Originally posted by Hugh Tonks, Thymometrics in HRZone on May 16, 2013. If we are going to try to measure something, we need to know what it is we’re measuring. So an essential precursor to any attempt to measure engagement is to pin down a definition of what engagement actually is. Fortunately (and I’m not entirely sure I mean that) there are already many, many different definitions of engagement: some are simple, others highly complex; some appeal to common sense, others to statistics; some are very popular, others obscure. Which to pick? Well, it would be nice if we could plump for the ISO Standard Engagement Model, but sadly it doesn’t exist and is never likely to. So our only choice is to invent our own (appealing, but possibly unwise), or use one from an existing survey provider, and here we have an embarrassment of riches. But select one we must, and this is one situation where, as the saying goes, the best is the enemy of the good. There isn’t a perfect definition of engagement, only definitions which are good or bad fits for your organisation, its people and culture. Unless your survey provider’s methodology omits the obvious things (like asking about salary) or otherwise appears implausible, it’ll probably get you somewhere near the truth, and you should nevertheless get reasonable value from it despite any imperfections. So having picked a likely-looking definition of engagement, and developed or purchased your methodology for the measurement thereof, you can begin. But if you look at the methodology, you’ll discover that you’re not actually measuring something called “engagement” at all; instead you’re measuring proxies for engagement: some combination of factors which are deemed to be good indicators of engagement, and some outcomes which are deemed to be positively affected by a workforce that is engaged. How do we know that these factors are good indicators, and how can we be certain that the outcomes we measured arose because of our engagement levels? It would seem churlish to doubt survey suppliers with many years’ experience; yet due diligence demands a certain level of scepticism, so it’s worth asking, if only to confirm that an explanation does exist. The input to any engagement survey is important (“Garbage In, Garbage Out” applies as much here as anywhere), but even given good quality input, what can we say about the output from this engagement-measuring process, which, after all, is what your action plans will be based on? Often, we see the headline results expressed as an overall “employee engagement” percentage. That could mean one of several things, including:

• The percentage of employees who are engaged (as opposed to not engaged). For a yes/no classification like this, how is the dividing line drawn? How reliable and accurate is the test that classifies employees as being on one side of this line or the other, and what is the typical misclassification rate?
• The average engagement level of the employees as a whole. To calculate this figure, you need the percentage engagement level of each employee, so how is this worked out? How can you be sure that the formula used applies equally well to all employees?

These are some additional interesting questions you might like to ask of your survey supplier. Even given convincing replies, though, one has to question the point of reducing everything to a single number, because it’s not very useful: a once-a-year measurement is going to be as accurate as a stopped clock, and engagement is a complex state with many independent variables. In other words, engagement is a multi-dimensional problem, whereas single numbers are merely one-dimensional entities. In reducing the complexity of the description to a number, we necessarily also lose a lot of context and useful information, such as how engagement scores vary across the workforce. It is no different to trying to draw a full-colour image of a person having seen only their shadow. The solution is not to rely on a single metric, but on as many independent metrics as you need for a full and proper description to be rendered. So even if you find a single number metric useful, it’s never going to be as useful as lots of metrics. Beware, though, of relying on metrics as the sole source of information. Not everything worthwhile is measurable; some things can only be qualitatively assessed (for example, comments). And beware, also, of including metrics simply because they are easy to measure; they may not be relevant. But you will nevertheless get value from most sensible metrics, if not from their actual values, then from comparison of them with the previous set of values; it’s always useful to know about positive and negative trends in your organisation. Of course there is another way of trying to measure engagement, but we don’t use it much – perhaps because it’s too obvious. This methodology comes in two forms, the simple, in which you ask your employees (anonymously, if you like) whether or not they are engaged with their work, and the slightly more complex, in which you ask them to what extent they are engaged with their work. If you don’t think employees are capable of understanding the concept of engagement, you could always ask “your job: how much do you really care?”, or some other equivalent question. But to measure it accurately? The point here is that engagement is not a yes/no thing. It’s not even a sliding scale; it’s lots of sliding scales, each representing a different aspect of the job or a part of the work, and these scales’ values change continually in response to events, interactions, time, successes, failures, and the general ups and downs and ins and outs of working life and of each employee’s personal circumstances. This constantly changing information must surely give you the best possible chance of working out the high and lows, the positive and negative trends, the danger areas and how all this varies across your organisation. And then the really hard work – making the necessary changes – can begin. For information about Thymometrics’ world leading real-time employee engagement surveys, email info@thymometrics.com, call +1 646 760 9323 (US) or +44 (0) 1223 750 251 (Europe) or visit thymometrics.com. Image courtesy of Jeroen van Oostrom / FreeDigitalPhotos.net