How do I establish content and construct validity for my questionnaire?

Establishing Content and Construct Validity

If you develop or adapt a questionnaire, examiners will want evidence that it measures what it claims to measure. Two of the most commonly reported forms of evidence are content validity and construct validity. In this blog, we'll look at what each one means and the practical steps you can take to establish them.

Content Validity:

  • Content validity asks whether your items cover all the important aspects of the concept, and nothing irrelevant.
  • It is established mainly through careful definition and expert judgement, before data collection.

Steps for Content Validity:

  • Define your construct clearly using the literature, and list its key dimensions.
  • Write or select items that cover each dimension.
  • Ask a panel of subject experts to rate each item for relevance and clarity.
  • Quantify their ratings if appropriate, for example with a content validity index or Lawshe's content validity ratio, and revise or remove weak items.
  • Pilot the revised questionnaire with people from your target population to check that items are understood as intended.

Construct Validity:

  • Construct validity asks whether your measure behaves as theory says it should.
  • It is usually assessed statistically, after you have collected data.

Steps for Construct Validity:

  • Use exploratory factor analysis (EFA) to see whether items group into the expected factors.
  • Use confirmatory factor analysis (CFA) to test whether a pre-specified factor structure fits your data.
  • Check convergent validity: items measuring the same construct should be strongly related. Average variance extracted (AVE) is often used, with 0.50 as a common benchmark.
  • Check discriminant validity: different constructs should be distinct. Common approaches include the Fornell-Larcker criterion and the HTMT ratio.

Reporting Your Evidence:

  • Describe your expert review process and any changes you made.
  • Report the factor analysis results, including loadings and fit indices where relevant, and the validity statistics you used.
  • Cite the methodological sources for the thresholds you applied.

Content validity is built through careful design and expert review, while construct validity is tested through analysis of your data. Plan for both from the start, document each step, and report the evidence clearly in your methodology and results chapters.