Questionnaire surveys 1: some common pitfalls

When I undertook a research project of Dgroups while I was working at KIT in 2006-2007, I sent out questionnaire to thousands – literally – Dgroup facilitators in three languages (English, French and Spanish). This was quite frightening but it was a baptism of fire as far as questionnaires are concerned.

I was using Surveymonkey which is a great tool and it allows you to send and analyse many questionnaires – if you have the paid account, that is. Since then, I have been helping colleagues and some fellow students with their questionnaire surveys and I thought it would be fun to share some common pitfalls, relevant for practitioners and researchers.

  1. Dont do it just because you can
    Online surveys – like Surveymonkey- allow you to send out questionnaires to huge numbers of people. But you still need to be realistic and not send out questionnaires indiscriminately because, not only are you reducing the chance that anyone will ever answer your next questionnaire, you are also spoiling things for others. Just because you can send a questionnaire, doesn’t mean that you should.
  2. Make the punishment fit the crime
    If you are using the questionnaire to evaluate a day-long workshop, for example, only use a small number of questions. So often, I have seen that colleagues have the tendency to ask endless questions – because they can – which can take up to 30 minutes to fill in but this is totally out of proportion with the workshop – and will mean that not many respondents will be inclined to answer. As a rule of thumb, if you are evaluating a day-long activity, don’t ask more than 10 fairly simple questions. This means that you need to think about what you really need to know… Always a good thing anyway.
  3. Be careful with forced choices
    It’s important to be very, very careful with forced choices in questionnaire questions – because this is one of the moments when your respondent will leave the questionnaire and never return. This always really irritates me, particularly when it I am forced to choose between answers which are either not applicable or to which I don’t know the answer.
  4. Avoid obligatory answers
    You should keep obligatory answers to the minimum. Your respondents will answer relevant, clear questions so there is no need to make them obligatory. Only use this option when you really have to have an answer to a particular question. For example, something like country of residence might be key to the results. If you make all answers obligatory, respondents will give up when a question is not totally clear or when they don’t know the answer. Giving respondents the option to skip a question means that they will forgive you an unclear or badly formulated question. Which takes me on to my next point…
  5. Don’t know or no applicable
    When you are asking respondents to choose between different possibilities, you need to give them the option – and it can be a combined option – to say that something is not applicable or that they don’t know the answer. This means that you will know why they didn’t reply to any of the options that you offered. Otherwise, you might think that they missed the question by accident.
This entry was posted in pitfalls, questionnaires, surveys and tagged , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s