Good Survey Question Design Helps You Get Better Customer Feedback

Good survey question design is the key to getting the most out of business intelligence tools (BI) like Power BI and others that it easier to understand what customers like and don’t like. With these technologies, it is incredible how quickly you can analyze large amounts of data.  Analyses that may have taken days in the past are now available in minutes.  It is also interesting to see some valuable insights these tools provide.

With almost any new and highly effective tool, you must do basics well to make the analytics more effective and powerful.  I want to discuss these basics in this blog.  One of those fundamentals is starting with an effective survey.  Here are some things to keep in mind:

Design your surveys with your outcomes in mind

When working with a new client, one of the first questions I ask is, “What do you most want to know from your customers?”  In short, what outcomes do you have in mind?  If you do not have a short list (3-5) of clear outcomes, you run the risk of creating a survey (1) that is too long and/or (2) unfocused.  The result is often low survey response rates, as long surveys frustrate respondents.  Also, you may generate data but very little useful information.

To illustrate the above point, too often, committees design surveys, with every area involved in the design process wanting to get feedback pertinent to their location.  What can happen is that every area gets some feedback, but not enough to satisfy its needs.  For example, the engineering department may need detailed feedback on a new product feature.  However, the more detailed questions are left out to keep it short.  This approach illustrates what happens without clearly defining outcomes.  We purchased a new RV built on a Sprinter chassis several years ago.  I received a PDF with forty questions from Mercedes a few months into ownership.  The format was one problem as it was not easy to use.  The number of questions caused me to delete the entire thing.  Parenthetically, I liked that Sprinter RV!

1. Have clear, precise questions

Having clear, precise questions sounds simple, but it is quite challenging.  Why?  The short answer is words!  Words can be interpreted differently by different people.  One’s education, memories, associations, preferences, background, etc.,  all influence how a person interprets a question.   Let’s take a simple one-word example: puny.  In some areas, puny means someone slight or weak.  In other places, especially the southern US, it means sickly.  Being as precise and straightforward as possible when selecting words in question design is critical.

• • Avoid double-barreled questions:  This is something I see pretty often.  One example is the question that asks the following:  How satisfied were you with the professionalism and friendliness of the service technician?  From the customer’s perspective, the technician may have been very professional but not friendly.  When the customer responds, is it the friendliness or professional dimension?
  • Keep individual questions short:  The longer the question, the more excellent the opportunity for confusion.  This advice is especially true for phone surveys.  If the question is so long, customers may forget the first part of the question.
  • Avoid double negatives:  I find sentences with double negatives often very confusing.  When in a survey, they are even more so.  The following illustrates my point.  Does it seem impossible that the planned service never happened?  This type of phrasing takes time to decode, time that most survey respondents will not take.
  • Avoid using jargon:  Every company uses jargon.  These words or phrases are generally understood within the company but sometimes not outside.  When we first started our ExperienceConnect process, we asked questions for one client about time and mileage charges.  The client used the term T & M, which we used in our survey.  We found that some customers did not understand the term.  We replaced T & M with Time and Mileage.

The German architect used the phrase “God is in the details” to highlight the importance of getting little things right.  In his view, the little things separate great buildings from merely good ones.  Getting the details right is especially true in survey design.

2. Keep the survey short for good survey question design

Most of the surveys we conduct are 8 to 10 questions.  Nearly all of our surveys are by phone, so the survey completion rate is not a factor.  However, in digital surveys, length can dramatically affect overall response and completion rate (respondents who leave before completing the survey).  Below is a chart comparing the number of survey questions versus the completion rate.

3. Let the customer speak

One of the benefits of a well-designed survey is the opportunity to gather unstructured customer feedback.  But all this unstructured feedback comes with a problem—how do we structure it to identify clear customer signals?  It is challenging to read through comments from even a relatively small number of surveys and determine the signals the customers send.  We looked for ways to structure this unstructured but highly valuable data.

If you are doing phone or digital surveys, consider one way to (1) allow the customer to provide additional feedback and (2) structure the feedback to be effectively analyzed.  In our company, after a scale response question (e.g., Overall Satisfaction), we ask, “What went well?” (transaction) and (2) “What could have been better?  In a digital survey, we provide predefined choices for customers to click on and add comments if desired.  In a phone survey, our researchers tag the verbatim comments with predefined categories.  Our business intelligence tool (Power BI) lets us quickly and easily identify critical customer messages.

Some may ask, “Why not use an AI tool to identify messages and sentiment?”  While the AI tools are improving, they are still not good enough to be deployed in an industrial, business-to-business environment.  The terms are often more technical and/or specific to a market or industry, making these tools less effective.  However, I expect AI tools to continue to improve and, one day, allow users to apply structure to the unstructured effectively.

3. Include questions that focus on actual customer behavior

It is helpful to ask customers what they think (or feel) about an experience or product.  I have found it very powerful to ask about customers’ behaviors.  Actions always speak louder than words.  While the results from our question are simple, you can learn even more with a business intelligence tool.  We use the NPS question on all of our surveys, which is likelihood to recommend.  We have asked customers if they have referred the provider to someone else in the last six months.  The surprising results helped our clients better understand the service drivers that drove actual referrals.

Good survey design is both an art and a science.  The important things to remember are:

• • Start with the outcomes in mind.
  • Keep things short.  Get the information you want and do it quickly.
  • Always provide the opportunity for the customer to speak.

These are the keys to designing an effective survey.  Complete the form below receive a more detailed white paper on good survey question design and execution.

Lynn Daniel
Founder & CEO