Professional blog

Things to know about surveys:

Length

Length is the easiest way to tell the difference between a good and bad survey question.  If it is long and takes several mins to read… guess what?  It is a bad question.  Sure, there may be that rare case where a long explanation is needed – but I caution you against this.  Keep your survey questions as short as possible.  If they are too long, your respondents will lose interest and drop off.  Long paragraph style test makes bad survey questions.

Simplicity is King

This might seem like a no brainer, but the minute you let a little complexity into your survey you will lose your respondents attention and the value in their data.  Most often complexity is introduced accidentally.  Most survey creators do not intend to make their surveys overly complex.  Industry jargon is a great example.  Too often jargon that has been common for you is inserted but it is not known to the audience.  This small and simple slip up causes the respondent to be confused and loose interest in your survey.

Examples of Bad Survey Questions

1. The Leading Question

Leading questions are those that use biased language. This language influences the survey taker’s selection. The problem with a leading question is that it can seem innocuous, but actually be fishing for a certain answer.

It’s not only leading questions that you have to watch out for either, but also the imbalanced response options. If you populate the response options with bias, you’re not going to get the data you need.

When you use leading questions, you’re not going to get actionable, accurate data that can help guide business decisions. Instead, you may be encouraging customers to answer in a certain way.

What to do instead: Make sure that questions are clear and simple—and refrain from using adjectives like “amazing” that are highly subjective and likely to influence responses.

2. Loaded Questions

Loaded questions are similar to leading questions but are different in one key area.  They cater to making the user feel better.  If you make a respondent feel better, they will be more likely to answer in a positive fashion.  However, all you have done is altered their mood and the data you have collected is not truly valuable or accurate.  Avoid loading questions with comments or words that will make the user feel better and thus answer your survey with inaccurate data.

3. The Assumptive Question

Assumptive questions make assumptions about what the survey-taker knows and feels without taking a step back and considering where the survey-taker actually stands. For example, an assumptive question might ask someone which email marketing software they use, even though many of the survey-takers don’t use email marketing software at all.

Assumptive questions leave out essential information that is necessary to understand the survey-taker. They are similar to leading questions in that they inadvertently encourage survey-takers to respond in a certain way.

What to do instead: Don’t assume anything about your survey-taker. You don’t know if they drink alcohol, work out, or are a native English speaker. Be sure to create big picture questions that set context.

4. The Pushy Question

A pushy question forces survey-takers to make a choice and pushes respondents toward particular favored answers. Usually these questions require a respondent to choose from a list with two few categories, preventing them from answering accurately.

For example, if you ask survey-takers “Ice cream is good on a cold day” and offer only the ability to “agree” or “disagree” you force respondents to answer, whether or not their answer affects their true feelings.

What to do instead: As much as possible, try to put yourself in your respondent’s shoes. Consider whether the question is open-ended, or may push the survey-taker in a certain, favorable direction.

5. The Confusing Question

Without meaning to, you may be creating questions that are confusing to survey-takers. These confusing questions may be poorly worded questions and/or responses, be illogically formatted, or be the wrong question type for the matter at hand.

What to do instead: Avoid double negatives at all costs. In general, using negatives can be confusing. Try to use clear, concise statements and questions to get your meaning across.

6. The Random Question

Random questions are out-of-context and don’t focus on topics that are important or relevant to the survey-taker. Usually, random questions are clearly driven by the survey maker’s interests.

What to do instead: Make sure that all questions you ask are relevant to the survey and its goals?

7. The Double-Barreled Question

Double-barreled questions squeeze too much into one question, making it difficult for a survey-taker to answer accurately. Sometimes, these double-barreled questions ask respondents to rate/rank two or more things in one question, or combine two different ideas into one question.  Double-Barreled questions are those that split answers into two parts.  Most often this is done accidentally.  The problem this creates is that users are then confused as to how they should answer.  If the answers to both sides of the question oppose each other then they get lost and frustrated.  They quickly lose[AH1]  interest and often abandon the survey

What to do instead: Make sure your questions are asking for one answer to one main idea.

8. The Ambiguous Question

Ambiguous questions are far too broad. The questions and/or responses leave room for interpretation, which leads survey-takers to guess or default to whatever answer makes the most sense.

What to do instead: Consider adding an “other” field to multiple choice survey questions to capture survey-takers true feelings. Yes, the feedback will be qualitative, but it will be more accurate.

9. Double Negative Expressions

Do you not like this post?  Double negative questions are just confusing.  They distract the respondent and often cause them to focus on re-reading the question several times to make sure they understood it.  Double Negatives make bad survey questions and should be avoided.  After all – the simplest survey is usually the one that gets the most responses.

Conclusion

Avoiding bad survey questions will help you increase the quality of your data as well as the number of respondents who complete the survey.  The heart of good versus bad survey questions is at their core wording.  Are they simple and short?  Do they avoid leading and loaded statements?  Do they avoid double-barreled or double negative expressions?   These tips will help you write good survey questions and avoid bad survey questions.