Imagine you run a survey where the same employees rate their manager's leadership and their own job satisfaction, all in one questionnaire, at one point in time. You find a strong positive correlation. Is this because good leadership really increases satisfaction? Or is part of that correlation created simply because the same people answered both sets of questions in the same way, at the same moment, using the same type of scale?
That second possibility is called common method bias. It is one of the most frequently raised concerns in reviews of survey-based research, especially in management, psychology, education, marketing and information systems. If you are writing a thesis with cross-sectional survey data, there is a good chance an examiner or reviewer will ask about it.
This post explains what common method bias is, where it comes from, how to reduce it through study design, how to assess it statistically, and how to discuss it honestly in your thesis.
What is common method bias?
Common method variance refers to variance in responses that comes from the measurement method rather than from the constructs you intend to measure. When this shared method variance inflates or deflates the relationships between your variables, the result is common method bias.
In simple terms, some of the correlation between two variables may be caused by how you measured them, not by a real relationship between the underlying constructs.
The "method" can include:
- The same respondent providing data for both predictor and outcome.
- The same time of measurement.
- The same response format, such as 7-point agreement scales for every item.
- The same questionnaire context and item wording style.
Where does common method bias come from?
Methodological literature, most notably a widely cited review by Podsakoff and colleagues in 2003, groups the sources into several categories. Here are the most important ones in plain language.
1. Common rater effects
These arise when the same person reports both the predictor and the outcome:
- Consistency motif: people try to appear consistent, so they answer related questions in a similar way.
- Social desirability: respondents present themselves favourably, for example by over-reporting positive behaviours.
- Mood and affect: a person in a good mood may rate everything more positively.
- Implicit theories: respondents may believe that certain things "go together" and answer to match that belief.
2. Item characteristic effects
- Ambiguous items encourage respondents to rely on general impressions.
- Common scale formats make it easy to fall into response patterns.
- Acquiescence: some respondents tend to agree with statements regardless of content.
3. Item context effects
- Priming: earlier questions influence how later ones are interpreted.
- Item embeddedness: neutral items placed among positive items may be rated more positively.
- Scale length and fatigue: long questionnaires encourage careless or patterned responding.
4. Measurement context effects
- Measuring predictor and outcome at the same time and place.
- Using the same medium, such as the same online form.
Why it matters
Common method bias can inflate correlations, making relationships look stronger than they are. It can also, in some situations, deflate them. Either way, your conclusions about relationships between constructs become less certain. This is particularly important for studies that test theoretical models using regression or structural equation modelling on single-source, cross-sectional data.
At the same time, it is worth knowing that methodologists do not all agree on how large the problem is in practice. Some argue it is often overstated; others argue it is under-recognised. The sensible position for a PhD student is to take it seriously, reduce it where possible, assess it, and discuss it openly.
Procedural remedies: reducing bias through design
The most effective way to deal with common method bias is to reduce it before collecting data. Statistical tests after the fact can only detect or partly adjust for it.
1. Use different sources
Where possible, obtain the predictor and the outcome from different sources. For example, employees rate leadership, while managers rate employee performance. Or combine survey data with objective records such as sales figures, absenteeism or grades.
2. Separate measurement in time
Collect predictor and outcome variables at different time points, such as two or three weeks apart. This reduces the influence of mood, memory and consistency motives. Temporal separation also strengthens the logic of your model, since causes should come before effects.
3. Create psychological separation
If you must collect all data in one survey, you can still separate constructs within the questionnaire. Use different sections with clear introductions, place unrelated filler questions between key constructs, or present the questions as belonging to different parts of the study.
4. Vary scale formats and anchors
Use different response formats for different constructs where this is compatible with the original scales. For example, use an agreement scale for one construct and a frequency scale for another. Be careful not to modify validated scales without justification.
5. Protect anonymity and reduce evaluation apprehension
Assure respondents that there are no right or wrong answers, that responses are anonymous or confidential, and that honest answers are most useful. This can reduce socially desirable responding.
6. Improve item clarity
Write clear, concrete, simple items. Avoid vague terms and double-barrelled questions. Pre-test the questionnaire to find ambiguous wording.
7. Counterbalance question order
In online surveys, you can randomise the order of blocks or items to reduce order effects. Check that this does not break logical flow.
8. Keep the questionnaire reasonably short
Fatigue increases patterned responses. Measure what your model needs and remove unnecessary questions.
Statistical approaches: assessing common method bias
After data collection, several statistical techniques are used to assess the possible influence of common method variance. None is perfect, and each has critics. Many researchers report more than one.
Harman's single-factor test
All measurement items are entered into an exploratory factor analysis, and the researcher checks whether a single unrotated factor explains the majority of the variance. A common rule of thumb is that a single factor explaining more than 50% of the variance suggests a problem.
This test is easy and very commonly reported, but it is widely considered weak. It can detect only severe bias and cannot rule out moderate method effects. If you use it, do not rely on it alone, and acknowledge its limitations.
Marker variable technique
You include a "marker" variable in your survey that is theoretically unrelated to your main constructs but is measured with the same method. If the marker correlates with your constructs, the size of that correlation gives an estimate of method variance, which can then be partialled out. The key is choosing and including the marker variable at the design stage.
Unmeasured latent method factor
In confirmatory factor analysis or structural equation modelling, you add a latent factor on which all items load, alongside their theoretical constructs. You then compare models with and without this common factor. This approach is popular but has technical limitations, and results can be difficult to interpret.
Measured method factors
If you have measured a likely source of bias directly, such as social desirability or negative affectivity, you can include it in the model to control for its influence.
Full collinearity assessment
Particularly in PLS-SEM research, some researchers compute variance inflation factors for all latent variables in a full collinearity test. Values above a suggested threshold (commonly 3.3 in this literature) are treated as a sign of possible common method bias. As with other thresholds, use it as a guide, cite the source, and interpret with care.
Reporting common method bias in your thesis
A strong section on common method bias usually includes:
- A brief explanation of why it may be relevant to your design (for example, single-source, cross-sectional survey).
- The procedural remedies you used, described specifically.
- The statistical tests you ran, with results.
- An honest statement about remaining limitations.
For example: "Because all variables were self-reported in a single survey, we took several procedural steps to reduce common method bias, including assuring anonymity, separating predictor and outcome measures into different sections, and using different response formats. Statistically, Harman's single-factor test showed that the first factor accounted for 28% of the variance, and a marker variable analysis indicated that correlations changed only marginally after adjustment. However, these tests cannot rule out method bias completely, and future research should use multi-source or longitudinal designs." (Figures are illustrative.)
A design checklist for your next survey
Use this list while planning your questionnaire, before any data is collected:
- Can I obtain the outcome variable from a different source, such as supervisor ratings or organisational records?
- Can I measure predictors and outcomes at two different time points?
- Have I separated key constructs into clearly introduced sections?
- Have I used different response formats where the original scales allow it?
- Have I included a marker variable that is theoretically unrelated to my model?
- Have I measured social desirability or another likely source of bias, if relevant to my topic?
- Are my instructions clear that there are no right or wrong answers?
- Is anonymity or confidentiality clearly explained?
- Have I pre-tested the items for ambiguity?
- Is the questionnaire short enough to avoid fatigue?
- Have I planned which statistical checks I will report?
You will not always be able to say yes to every question. Practical constraints, such as access to organisations or time limits, often make multi-source or multi-wave designs difficult in a PhD. That is acceptable, as long as you do what is feasible and explain the remaining risk honestly in your limitations section.
Common mistakes to avoid
- Relying only on Harman's test and claiming that common method bias "is not a problem".
- Ignoring procedural remedies during design and hoping statistics will fix everything later.
- Adding a marker variable after data collection. A marker must be planned and included in the questionnaire.
- Over-claiming causality from single-source, cross-sectional data.
- Not discussing it at all, which often draws examiner questions.
Final thoughts
Common method bias is not a reason to avoid survey research, but it is a reason to design surveys carefully. The best protection comes from design choices: multiple sources, separated measurement, clear items, anonymity and varied formats. Statistical tests can then help you assess how much of a concern remains. When you combine both and report them honestly, your findings become much more credible, and you will be well prepared when an examiner asks, "How do you know your results are not just method effects?"
Have questions about common method bias in your own study design? Leave a comment below.