Survey comments are a gold mine. They provide detail, depth, and verbatim insight from the source.

But when there are so many the process is daunting – how do you make sense of all the voices?

Thankfully, there are things you can do to make the comments as useful as they should be. Here’s our take on how to make sense of your survey comments.

^{Photo by Red Mirror on Unsplash}

Can’t ChatGP summarise my comments?

Yes. But also, No. It’s not that simple (this is our next blog using ChatGPT to read survey comments).

Are you really listening though?

You have to wonder why ask people to share their opinions, only to feed the words into a bot to regurgitate summaries that are an approximation about what was said. It doesn’t suggest an organisation that’s listening. Even when there are 5,000 voices, at least take a sample and hear what’s being fed back.

ChatGPT to one side, let’s look at what you can do to turn your survey comments into truly useful feedback and insight.

Part 1: Preparation

A good survey takes planning. Ahh, the ‘P’ word again. We wrote about it here – 3 steps to better insight – the same principles apply to gleaning useful comments from a survey.

1. What purpose do the comments serve?

Before you ask an open-ended question, ask yourself:

• Why do we need to ask an open-ended question here?
• How am I going to use the answers?
• What is it exactly we want to know?

Verbatim responses take effort to answer (as well as to read/review). Ask for written answers sparingly.

Think about how the survey data and comments will be reported, and to whom. Consider the decisions that it’ll inform and use this starting point to guide the comments you invite.

2. Don’t ask “Any other comments?”

Unless your survey will have fewer than 100 responses, don’t ask “Anything else?” With no unifying direction or coherent theme, random comments will be wide-ranging, vary in quality and importance, and take many hours to interpret.

3. Instead, ask targeted questions

In our twenty years of experience, the single most important piece of advice on comments is this:

The more specific you are, the more relevant the feedback will be, and the more easily you’ll be able to work with the comments.

Of course, humans are unpredictable. You’ll always get that one answer that comes from left field, but help yourselves by gearing your questions towards workable answers. For instance,

“Are there any changes you’d like to see when we return to the office?”

invites focused, action-orientated responses.

Part 2: Analysing comments

You have survey data, great! Where do you start?

1. Look at the data first

Try not to fixate on the comments as the responses roll in. Keep your attention on key metrics and gauge whether scores are good, fair or poor as a starting point.

Think of it like an appraisal with your boss: there’s usually a lot of great feedback and maybe one or two comments that suggest could do better. It’s easy to get distracted by the negatives, but also certain comments stand out. Don’t forget to look at the big picture to put these comments into context.

When the survey closes, analyse the data before the comments. Look for trends, sentiment, and consistencies or differences across the organisation.

2. Read the comments after

Now, when you turn your attention to the comments, you’ll be able to understand them in context. Comments add richness to the results and help paint a picture. When reporting survey results we regularly draw out select comments that illustrate a key finding.

A quick tip I swear by:
Sort the comments by length (i.e. number of characters). Common, short answers will cluster together and make the initial getting started much easier while giving you a quick feel for themes. As you get to the longer comments, they build further on what you’ve read so far.

It’s possible to filter comments by theme, department – the options are many. It all depends on your intentions: what do you want to find out and what will the information help you decide? Clarity about your goals will help you determine how to approach the comments.

3. Close the loop

The final step harks back to part one – what are you hoping to achieve with the survey? The key is to collect genuine insight that helps to understand and act on the survey data received.

Customer surveys can alert you (or your service team) when you have a seriously dissatisfied customer. Their comments might allow you to quickly turn things around and save a customer relationship that’s become precarious.

Employee surveys are usually anonymous, and closing the loop is probably limited to thanking everyone for sharing their views. Sharing a few of the (anonymised) comments and the next steps is a great way to demonstrate how helpful the feedback is proving.

Good comments start with asking the right questions

Preparation is key. So when your survey has just closed and you’re staring at a few hundred comments, you can uncover those invaluable insights.

Need help? Talk to Surveylab! We can help you make sense of the comments.