Reliability vs Validity Explained
Reliability and validity are two of the most important ideas in research quality, and they are often confused. Put simply, reliability is about consistency and validity is about accuracy. In this blog, we'll explain both, look at the main types, and show how they relate to each other.
What Reliability Means:
- Reliability is the consistency of a measure. A reliable instrument gives similar results under similar conditions.
- If a questionnaire measuring stress gives very different scores to the same person on two calm days, it is not reliable.
Main Types of Reliability:
- Internal consistency: whether items in a scale measure the same thing, often assessed with Cronbach's alpha.
- Test-retest reliability: whether scores are stable when the same people complete the measure at two points in time.
- Inter-rater reliability: whether different observers or coders give consistent ratings, often assessed with statistics such as Cohen's kappa.
What Validity Means:
- Validity is whether a measure actually captures what it claims to measure, and whether your conclusions are justified.
- A bathroom scale that is always two kilograms too heavy is reliable, because it gives consistent readings, but it is not valid.
Main Types of Validity:
- Content validity: whether the items cover all relevant aspects of the concept.
- Construct validity: whether the measure behaves as theory predicts, including convergent and discriminant validity.
- Criterion validity: whether scores relate to an external standard or outcome.
- Internal and external validity: whether a study's causal conclusions are sound, and whether findings apply beyond the study setting.
How They Relate:
- A measure can be reliable without being valid, but it cannot be valid without being reasonably reliable.
- You need to address both in your methodology chapter and report the evidence for each.
In Qualitative Research:
- Qualitative researchers often use the idea of trustworthiness instead, with criteria such as credibility, transferability, dependability and confirmability.
Reliability tells you whether your measure is consistent; validity tells you whether it measures the right thing. Check both, report the evidence clearly, and you will be well prepared for questions about the quality of your data.