Writing Neutral Survey Questions
A survey is only as good as its questions. If the wording nudges people towards a particular answer, your data will reflect your assumptions rather than your respondents' views, and no amount of analysis can fix that later. In this blog, we'll look at how leading questions creep in and how to write questions that let respondents answer freely.
Recognise a Leading Question:
- A leading question signals the "right" answer. For example, "How much did you enjoy our excellent workshop?" pushes people towards a positive reply. "How would you rate the workshop?" does not.
- A loaded question builds in an assumption. "How often do you use the library's online databases?" assumes the respondent uses them at all. Add a filter question first, such as "Have you used the library's online databases in the past month?"
Keep the Wording Neutral:
- Remove emotionally charged words and praising or critical adjectives such as "excellent", "harmful" or "unfair".
- Avoid appeals to authority or to the majority, such as "Most experts agree that... Do you agree?" These put pressure on respondents to go along.
- Use balanced phrasing that names both sides, for example "To what extent do you agree or disagree with the following statement?"
Ask One Thing at a Time:
- Avoid double-barrelled questions such as "Was the course useful and well organised?" A respondent may feel differently about each part. Split it into two questions.
- Avoid double negatives, such as "Do you disagree that the library should not open on Sundays?" They confuse respondents and produce unreliable answers.
- Use simple, familiar words and define any technical term your respondents may not know.
Balance the Response Options:
- Give an equal number of positive and negative options, for example a five-point scale from "Strongly disagree" to "Strongly agree".
- Make sure the options do not overlap and cover every likely answer. Add "Other" or "Not applicable" where needed.
- Decide deliberately whether to include a neutral midpoint, and apply that decision consistently across similar questions.
Watch the Order of Questions:
- Earlier questions can shape how people answer later ones. As a general rule, ask broad questions before specific ones.
- Place sensitive questions later in the survey, once respondents are more comfortable, and remind them how their answers will be kept confidential.
Pilot Before You Launch:
- Test the survey with a small group similar to your target respondents.
- Ask a few of them to explain in their own words what each question means. If their explanations differ from what you intended, revise the wording.
- Share the draft with your supervisor or a colleague who can spot bias you may have missed.
Neutral questions take more effort to write, but they give you data you can trust and defend. By spotting leading and loaded wording, asking one thing at a time, balancing your response options, thinking about question order and piloting your survey, you give your respondents a fair chance to tell you what they really think.