Nowadays, short surveys are very popular, often consisting of just a handful of questions or a dozen at the most. In these cases, especially when all questions are related to a single topic, it is typically not necessary (nor recommended) to categorize them. In a sense, there already exists a default categorization, comprised of just one category that includes all the questions. This is often invisible to survey designers until they examine the results, at which point they can look at either the responses to each question separately or rolled up into an overall score. The longer the survey and the more topics it covers, the less this overall score will make sense, hence the need for survey categories.
In a categorized survey, the questions are grouped by topic or theme. The advantage of this is in reporting – when one can examine, not only the results of each individual question, but also the average score for the group of questions in each survey category (e.g., leadership). This makes the results far easier to interpret. In fact, we often recommend looking at the category-level results first and then diving into those categories that are problematic (e.g. below norm or historical levels). Categories also make it far easier to examine sub-group results. Imagine having to compare the scores of 12 different divisions across 25 questions. That’s a difficult task made far easier if you can replace the 25 questions with 5 or 6 categories representing the underlying dimensions in the survey.
Clients often ask how many questions they should include in each category – how many is too many or too few? Although three to five is a good rule of thumb, the key is really to group the questions according to their thematic similarity. If you have more than five questions on the same topic, you may have included more questions than necessary. The ultimate test is to consider if a single average score across all questions in a category would still make sense. If not, you likely need more categories – or fewer questions. Of course, there are some judgment calls to make. For example, should questions about pay and benefits go into a single category called "Rewards"? They can, and often do, but be careful because scores on those two questions can be very different and make the category average less meaningful. Note that a category with only a single question very rarely – if ever – makes sense. This is because using a single question to represent a broader concept, e.g., leadership, is generally bad practice.
A final consideration is whether or not questions should be grouped by category for respondents during survey administration. This is an easy one – the answer is no. Although it may seem like a good idea, grouping questions by category generally makes respondents focus less on each individual question, and respond more similarly to all questions in a category. This leads to a lack of differentiation in the data and far less actionable results.
Log in to Employee Engagement Software today to start creating categories for your next survey or contact us for a demo.
Author
Engage
Adam is the dynamic force behind Engage, WTW’s game-changing employee engagement platform. His goal is to create the world's greatest software for understanding and improving the whole employee experience, at work and in life, driving action, change and impact on organizations’ EX, company culture, and business performance. In his own life outside of work, Adam enjoys off-roading in his Jeep and spending time with his family. Follow Adam on Twitter and LinkedIn.
Related Insights
