Engagement Soup

August 21, 2009 26 Comments

soup

Is optimism a competitive advantage? The link between a company’s employee engagement is real: the more engaged the workers, the higher the sales and profits…. Companies don’t directly measure the optimism of their employees. Instead they rely on engagement scores, typically gathered by outside consultants who exhaustively survey the staff. As in: Does your boss support you in getting your job done? Do you have a best friend at work? (Business Week Online, August 13, 2009).

I am dumbfounded.  How can BW claim that optimism is a competitive advantage when they admit the companies they reference are NOT directly measuring optimism?  No worries, these trusted and well compensated consultants are measuring engagement, and that is the same as optimism, right? Wrong!

And I have more bad news – they are NOT measuring engagement either. These are the twelve magic engagement questions referred to in the BW article.

engagement

Engagement is an effect, such that someone might be able to say “I am engaged” and we should be able to observe and evaluate the degree to which this attitude is manifest.  Are ANY of these twelve questions able to capture the degree to which someone is engaged?  NO!

Should we believe that BECAUSE you are engaged, you are given the materials and equipment you need to do the work right?  Or is it that because you are given the materials and equipment to do the work right you are engaged?  Does your engagement at work cause people to want to be friends with you, or is it that because you have supportive relationships at work that you might be engaged?  Get the picture?

These questions are NOT effect indicators of engagement.  They are instead causal indicators – of something.  The persuasive consulting company believes these things cause engagement, but they don’t measure psychological engagement so it remains an empirical question! Worse, they say they are measuring engagement when they clearly are not, and companies that should know better don’t.

Here are a few examples of effect indicators of engagement: I am immersed in my work; when I get up in the morning, I feel like going to work; I feel very happy when I am working intensely.  To the extent you answer these in the affirmative we know that you are engaged.  We don’t know what caused you to be engaged, but we have good reason to believe you are because we can measure the effects.

Want to know if a tree is healthy?  Look at the leaves and the fruit.  These are indicators of the health of the tree.  Just because we give our tree lots of fertilizer, water, and sunlight does NOT mean it is healthy.  These are all very smart things to do, but they do not guarantee health, and the only way to evaluate health is to examine the tree and not its environment.

But what about the business results, those higher sales and profits?  If you ever do find that data published in a peer-reviewed format (consultant promotional material does NOT count), remember this simple fact: correlation is not the same as causality.  Establishing the case that an individual level psychological construct (that’s what engagement is) can cause a company level aggregate measure is VERY, VERY, VERY difficult.

I remain extremely skeptical.

About the Author:

Comments (26)

Trackback URL | Comments RSS Feed

  1. Joseph Logan says:

    Very good explanation. This sort of simplistic attempt at explanation–without a grounding in logic, apparently–makes it very difficult to intervene effectively with organizations. It plays especially to senior executives who are looking for a magic bullet and the consultants who would prey on their insecurities.

    [Reply]

    Bret L. Simmons Reply:

    Joseph, I appreciate your comment. There is no magic bullet. When dealing with data, you simply must allow your interpretation of the data to be constrained by your measurement methods. Engagement in my opinion is interpretation gone wild. Thanks!! Bret

    [Reply]

  2. Wally Bock says:

    Well you nailed that one, Bret. Engagement is the magic stone of the day. If you rub it, all will be well. The problem that there is no agreed-upon definition or evidence that engagement causes anything, let alone improved profit, seems irrelevant. Thanks for a lucid description of why this particular treatment is rubbish.

    [Reply]

    Bret L. Simmons Reply:

    Wally, my friend, I have never seen anything like this. I am continually amazed how much smart companies do dumb things. You hit the nail on the head that no standard definition or measurement is a BIG problem. It is such a mess. Thanks for sharing your thoughts! As always, I value your opinion very much. Bret

    [Reply]

  3. Bret,

    Thanks for the well-constructed description of a poorly-constructed but money-generating programme du jour.

    Your money line: “. . .correlation is not the same as causality.”

    Perhaps a refresher in cause and effect would go a lot further to enhance performance over the long run.

    Well done. . .

    [Reply]

    Bret L. Simmons Reply:

    Steve: I do so appreciate your input on this. I just don’t get it. You meet with these high level folks much more than I do. How is it that clearly intelligent thinkers have abandoned some of the most basic learning? Thanks for taking the time to comment. Your expert opinion will always be welcome here. Bret

    [Reply]

  4. The questions BW published come from Marcus Buckingham’s book, “First Break All the Rules.” I don’t think they credited him their article. Makes me want to go back to Buckingham’s book to see what he said the 12 questions measured. Nice work, Bret!

    [Reply]

    Bret L. Simmons Reply:

    Hi Lori, thanks for your comment. You are exactly right, it is that book that started all this madness. When you go back to the book, see if you can even find where he give a clear and concise DEFINITION of engagement. And notice at the end when they try to snow us with data that they NEVER show us the data. A dude named Harter and his colleagues later published some data in 2003 in a book chapter (NOT a peer reviewed source), but we have the same two big problems: 1) not measuring engagement (measuring its causes), and 2) correlations and not causality. Thanks for sharing your thoughts!! Bret

    [Reply]

  5. I still work there so I can't leave my name says:

    My experience? We were told taking the online survey, using these exact questions was mandatory. I took it. Some of the answers weren’t very pretty.

    Several months later, the only question that was addressed was the first one: “I know what is expected of me at work.” We had to state, verbally and in detail, our answer to the question in order to prove that we really did know. It was an all-staff meeting that devoted over an hour to that one question.

    When I asked when we were going to go over the staff’s answers to the other questions, the response I got was, “Those questions are copyrighted.”

    True story.

    How engaged is that?

    [Reply]

    Bret L. Simmons Reply:

    Earl, wow your example really adds value to this conversation. I don’t even get into how the administration and follow up on these surveys is another HUGE problem, but you help us see that. When your supervisor is rated on how well you answer those questions, there will be STRONG pressures for you to learn how to answer correctly. Anyone that thinks otherwise is drinking too much of the kool aid. Thanks!!! Bret

    [Reply]

  6. Eric says:

    Great post. I have client organizations who have spent millions on these 12 magic questions. The process turned into a massive charade. Managers and employees focused on getting through the process as quickly as possible, showing good numbers, and minimizing the damage the survey caused.

    Here’s an ancient story indicating that this issue of confusing correlation and causality is long standing:

    The Mullah Nasrudin was beating a drum as hard as he could. The neighbors were fed up at the racket and asked him what he was doing.

    He replied, “Keeping wild tigers at bay.”

    A neighbor shouted, “But Mullah, there are no wild tigers within a thousand miles of here.”

    The Mullah smiled and replied, “It works, doesn’t it!”

    [Reply]

    Bret L. Simmons Reply:

    Eric, I appreciate your comment. Everyone that I have ever known that worked in an organization that used this systems also described it as a great charade. Is it worth all those millions? What a waste, and it destroys employee trust and credibility. Great story about cause and effect! Thanks!! Bret

    [Reply]

  7. Art Petty says:

    Bret, I truly enjoyed this post and the many thoughtful and pointed comments. You’ve offered a nice public service and fair warning.

    I read the article last week and remember thinking that I still like my old fashioned way of checking engagement in my businesses and on my teams. It’s called watching and listening for things like:

    -Do people seem to be having fun as they knock down problems and move on to new challenges, without any prodding.

    -Is there a steadily increasing flow of new ideas to improve and innovate, and do people act on those ideas?

    -Are teams self-policing on performance and quality.

    -Are people comfortable sharing with their peers and leaders (especially with me) that the emperor has no clothes, without fear of retribution.

    -Do people and teams fight and resolve professionally and do they find time to play and have some fun in the process?

    -Do people have a great handle on their priorities connect to the firm’s key objectives?

    -Is there a strong sense of pride in accomplishments and a healthy “let’s go at it again” on failures?

    I could keep rolling here but I’ll look for these and other signs of engagement long before I’ll fall victim to the survey that you so effectively challenged. Thanks for your consistently great and thought-provoking content! -Art

    [Reply]

    Bret L. Simmons Reply:

    Art, no surprise that you and I are on EXACTLY the same page. To me this is management 101, the very basics. You know these things by being present, watching, listening, caring, being patient and being trusted. Any survey should only serve to quantify what you already know.

    Sage advice, Art. I appreciate your time and effort. Your voice will always be welcome here. Thanks! Bret

    [Reply]

  8. Bret:
    Thank for exposing the view that measuring optimism is how you gain competitve advantage: BW author seemed to miss premise behind orginal gallup poll and Marcus Buckinghams’ book.

    It’s my view that organizations are challenged with attracting, retaining and motivating current talent so to that end it’s a managers’ roll to provide leadership in their organizations.

    Also, it was Marcus’ idea for reflection that “The best managers are those that build a work environment where the employees answer positively to these 12 Questions” in the poll. It is interesting that organizations are still implementing these surveys.

    Marcus Buckingham said it well “The great manager mantra is don’t try to put in what was left out; instead draw out what was left in. You must hire for talent, and hone that talent into outstanding performance.”

    ~Naomi

    [Reply]

    Bret L. Simmons Reply:

    Naomi, I appreciate your comments and perspective. No denying that the best managers are those that build an environment for employees to succeed. Maybe the survey could be a learning tool for managers that did not know these things before. I suspect for most they were simply another performance metric, a new hoop to jump through. To his great credit, Buckingham has sold a boat load of books. I must admit to you that I have never been a fan of his core message of hiring for talent. That has always bothered me because it reeks of the fundamental attribution error. I’m a systems thinker, and I concur with Pfeffer and Sutton that the law of crappy systems trumps the law of crappy people. Organizations don’t have talent problems, they have systems problems and as long as they hang their hat on the talent stuff they will never get to work fixing those crappy systems.

    Thanks so much for your input!!! Bret

    [Reply]

  9. Leighton says:

    I was linked here from Bret’s comment made on another article that also confused ‘happiness’ with engagement. Worrying when the author claims to be an expert in happiness, and yet doesn’t concern herself with ensuring that the source of her information actually measures that. It’s amazing how pervasive Gallup’s Q12 is, and the fact that most people don’t realise that it doesn’t even measure what it claims to measure – engagement. The survey is simply an ‘in’ to organizations who then pay hideous sums of money for Gallup’s ongoing consulting services, based around their limited measure.

    And you can’t tell me that making sure people have the tools and resources they need to do their job drives engagement, happiness, or optimism in any significant way…

    [Reply]

    Bret L. Simmons Reply:

    Leighton, I appreciate you finding my site and taking the time to comment. I agree with you 100%. These consulting companies are very big and very influential, and they are good at selling a very persuasive story. but as you point out, the story has serious flaws. Thanks!! Bret

    [Reply]

  10. Long Huynh says:

    I enjoy this post so much that I even write a post to recommend it. I love your analogy of measuring the health of a tree: “Just because we give our tree lots of fertilizer, water, and sunlight does NOT mean it is healthy.”

    Unlike many who should no better, you have seen through the fog of correlative analysis with your critical thinking. Without saying it out loud, you have succeeded in pointing out (at least to me) that this practice of substituting a correlation for a causal effect is at best questionable and at worse, a pile of horse s..t … er, fertilizer.
    The rest of my post is here: http://tao-of-it.com/2009/09/16/the-fallacy-of-measuring-employee-engagement/.

    [Reply]

    Bret L. Simmons Reply:

    Long, I read your post and love it!! Glad to have you among the voices that are trying to bring common sense to all this engagement stuff. Keep up the good work and thanks for visiting and commenting! Bret

    [Reply]

  11. Bret – Love the blog and this thought-provoking post. While I agree your general point, that when it comes to employee engagement cause and effect can be difficult and sometimes impossible to parse out, I think you overstate the case against engagement research, including the usefulness of some of the statements on the Q12 survey.

    Using your own example, just because a tree’s leaves and fruit look good today does not mean that the tree is healthy – it could be diseased now and dead in a few months too. Does that mean don’t measure a tree’s health by looking at its leaves and fruit? Of course not. But I don’t agree that just because water, sunlight and fertilizer don’t necessarily mean health that it’s not a good idea to measure those qualities as well. I think you should examine both the tree and its environment.

    I think Earl’s reply is very instructive. Survey research is simply a tool. It should serve as a platform from which to begin or continue a discussion about the type of work environment a company and its employees want to create together. It gives you a good idea of places to “shine a flashlight” that may be productive areas to work on, or areas of strength to amplify. Just like any other tool, it can be used productively or (like in the case of Earl above) destructively.

    I agree that consultants should be careful about claims regarding business performance (especially financial performance) as a result of engagement interventions. Organizations are organic and constantly changing. You are exactly right that figuring out what is a cause and and what is an effect (and even what cause causes what effect) is probably impossible in most cases. But I can say that I have personally seen many examples where a company has experienced significant improvement in business results after working steadily on employee engagement. I’m sure you have too. That’s not necessarily cause and effect, although in many cases I believe it is.

    I’ve never thought that improved financial performance should be the main thing motivating an engagement project. Whether there is a measurable financial impact or not, creating a great place to work is the right thing to do. But there are clearly measurable impacts of a disengaged workforce (absenteeism, presenteeism, shrink, quality problems, etc.) that do have a financial impact on a company. To whatever extent these things improve – whether they can be specifically tied to a specific survey statement or consulting intervention or not – they help a company succeed.

    Thanks again for a great post and discussion. Phil.

    [Reply]

  12. Phil, I really appreciate you taking the time and effort to leave this excellent comment. I don’t disupte that the Q12 asks important questions. My point is it does not do what it is advertised to do – measure engagement. I’m also a big believer in the link between attitudes and performance. But the leverage lies in satisfaction and commitment, not engagement. As you correctly point out, a better work environment will make employees more satisfied and committed. Engagement is a smoke screen. We don’t even need that term – satisfaction and commitment work just fine and they are MUCH more clearly defined and measured. Thanks for sharing!! Bret

    [Reply]

  13. George says:

    As a supervisor my bonus was linked to the score of this stupid Q12 survey. I’ve never seen such hogwash of a survey.

    Maybe if it was taken without bias that’d be one thing, but we were so under the thumb to ace this thing that we were taking folks out to steakhouses the night before, half jokingly saying “5 is the right answer” because in the end these results would come back and club you over the head. We compared plant to plant and it was very punitive.

    Gallup was making a TON of money off this from my company. With no results on our end.

    [Reply]

    Bret L. Simmons Reply:

    Glad you shared this, George! I hope people will find your comment down here. It is exactly one of the points I want to make about this. Thanks! bret

    [Reply]

  14. High level change management, my consulting area, has far too much focus on “engagement”, “studies” and the time wasted consulting that ties to both.

    If there is cash and fear (especially at the executive level) there will be consultants to fill the vacuum. To get as much of the $ as possible requires lots of assessment and lots of gathering of information. There is certainly engagement in that model…

    Your post rings true.

    I find as an external with leverage (the high level part) and the tag “change management consultant” I can tell you if there is engagement from the first conversation on. And, yes, it may be as simple as people with smiles on their faces getting things done. Each time I do it saves the client thousands of dollars that can be spent connecting vision/strategy to people/work (and vice versa). That is the engagement everyone is looking for.

    Good stuff Brett- thanks for the post.
    GG

    [Reply]

    Bret L. Simmons Reply:

    Welcome, Garrett. Your observation rings true. If you walk in and there are no smiles on faces, why waist time measuring engagement? It’s a charade. Thanks! Bret

    [Reply]

Leave a Reply