Designing a Questionnaire That Produces Usable Data
A questionnaire can look finished and still produce data that is hard to analyse or interpret. Many problems only appear once the responses are in, when it is too late to fix them. In this blog, we'll look at how to plan a questionnaire around your analysis so that the data you collect is usable from the start. For tips on wording individual questions, see our post on writing survey questions that don't lead the respondent.
Start from Your Research Questions:
- List your research questions and, for each one, the variables you need to measure.
- Every question in the questionnaire should link to at least one variable. If it does not, consider removing it.
- Plan your analysis at this stage. Knowing which statistical tests you will use tells you what type of data each question must produce.
Use Established Scales Where Possible:
- Validated scales from published studies save time and make your results easier to compare with others.
- Cite the original source, check whether permission is needed, and report any changes you make.
- If you translate a scale, use an accepted process such as forward and back translation.
Choose the Right Question Format:
- Closed questions, such as multiple choice and rating scales, are easier to analyse statistically.
- Open questions give richer detail but take longer to code. Use them sparingly and for a clear purpose.
- Decide the level of measurement you need. For example, asking for exact age gives more analysis options than asking for age bands.
Structure and Length:
- Group related questions into sections with short headings and clear instructions.
- Put easy, non-sensitive questions first and demographic questions near the end.
- Keep it as short as possible. Long questionnaires increase drop-out and careless answers.
Prepare for Analysis:
- Decide how each response will be coded before you collect data, and create a codebook.
- Include a clear option such as "Not applicable" where needed, so that missing answers can be told apart from genuine responses.
- Identify any reverse-worded items so you remember to recode them.
Pilot and Refine:
- Test the questionnaire with a small group from your target population.
- Check how long it takes, whether any questions are confusing, and whether the responses can be analysed as planned.
Usable data starts with careful design. Link every question to your research questions, use established scales where you can, choose formats that suit your planned analysis, and pilot before you launch. A little extra planning at this stage will save you many hours during analysis.