Most business and management studies rely on surveys and interviews. These methods are useful, but they are limited when you want to show that one factor actually causes another. If you want to know whether a new training programme improves performance, whether a price message changes purchase intention, or whether a leadership style affects employee motivation, an experiment is often the strongest design available.
This post explains what an experiment is, the main types used in business and management research, the steps for designing one, and how to deal with common threats to validity.
What is an experiment?
An experiment is a research design in which the researcher deliberately changes one or more factors (the independent variables or treatments), controls other conditions, and measures the effect on an outcome (the dependent variable). The key features are:
- Manipulation: the researcher creates different conditions rather than simply observing them.
- Control: other factors that could influence the outcome are held constant or accounted for.
- Comparison: outcomes are compared between groups or conditions.
- Random assignment: in a true experiment, participants are assigned to conditions by chance.
Random assignment is what makes experiments so powerful. It spreads individual differences evenly across groups on average, so differences in outcomes can be attributed more confidently to the treatment.
Types of experiments in business and management
Laboratory experiments
Conducted in a controlled setting, often with students or online panel participants. They offer high internal validity but may have lower realism. They are common in marketing, consumer behaviour, organisational behaviour and behavioural finance.
Online and survey experiments
Participants are randomly shown different versions of a scenario, advertisement, message or product description within an online questionnaire. These are efficient and widely used, for example in vignette or scenario-based studies.
Field experiments
Conducted in real organisations or markets, such as testing two versions of a sales email or introducing a new incentive scheme in some branches but not others. They offer high realism but less control, and they require organisational cooperation.
Quasi-experiments
Used when random assignment is not possible, for example when an intervention is introduced to whole departments. Researchers compare treated and untreated groups and use techniques such as pre-tests, matched groups or difference-in-differences analysis to reduce bias.
Natural experiments
Researchers study the effects of events or policy changes that happen outside their control, such as a new regulation that affects some firms but not others.
Steps for designing an experiment
1. Define a clear causal question and hypotheses
State exactly what you expect to cause what. For example: "Employees who receive weekly recognition will report higher work engagement than those who do not." Ground your hypotheses in theory.
2. Define and operationalise your variables
Decide how you will manipulate the independent variable and how you will measure the dependent variable. Use validated measurement scales where possible, and decide which control variables you will measure.
3. Choose your design
- Between-subjects design: each participant experiences only one condition. This avoids carry-over effects but requires more participants.
- Within-subjects design: each participant experiences all conditions. This needs fewer participants but requires counterbalancing to control order effects.
- Factorial design: two or more independent variables are manipulated at the same time, for example a 2 x 2 design testing message type and price level. This allows you to test interaction effects.
- Pre-test/post-test control group design: both groups are measured before and after the intervention, which helps detect change and check that groups were similar at the start.
4. Develop your manipulation and materials
Create realistic scenarios, stimuli or interventions. Keep everything identical across conditions except the factor you are manipulating. Pilot test your materials to make sure participants understand them.
5. Include a manipulation check
A manipulation check is a question that confirms participants noticed or perceived the manipulation as intended. For example, if you manipulated perceived fairness, ask participants how fair they thought the decision was. Without a manipulation check, a non-significant result is hard to interpret.
6. Decide on sample size
Use a power analysis, for example with G*Power, to estimate how many participants you need to detect the expected effect size with adequate power. Underpowered experiments are a common reason for inconclusive findings.
7. Randomise properly
Use a random number generator or the randomisation feature in your survey platform to assign participants to conditions. Record the assignment process so you can report it.
8. Plan your analysis in advance
Common analyses include independent samples t-tests, ANOVA, ANCOVA when including covariates, and regression with dummy-coded conditions. Consider preregistering your hypotheses and analysis plan, for example on the Open Science Framework or AsPredicted.
Threats to validity and how to manage them
Internal validity
- Selection bias: groups differ before the treatment. Use random assignment or matching.
- Demand characteristics: participants guess the hypothesis and change their behaviour. Use cover stories where ethically acceptable and avoid revealing hypotheses.
- Experimenter effects: researchers unintentionally influence results. Standardise instructions and, where possible, keep researchers blind to conditions.
- Attrition: participants drop out unevenly across groups. Monitor and report dropout by condition.
External validity
- Unrepresentative samples: student samples may not reflect managers or consumers. Use relevant participants where possible.
- Artificial settings: lab tasks may not reflect real decisions. Use realistic scenarios or combine lab and field studies.
Construct validity
- Make sure your manipulation and measures capture the concepts you intend. Use manipulation checks and validated scales.
Ethical considerations
Experiments in organisations can affect people's work, pay or wellbeing. Obtain informed consent, avoid harmful manipulations, and debrief participants at the end, especially if you used deception or a cover story. If one condition is expected to be beneficial, consider offering it to the control group after the study. Always obtain ethics approval before starting.
Common mistakes to avoid
- Changing more than one thing between conditions, so you cannot tell which change caused the effect.
- Skipping the pilot test.
- Forgetting the manipulation check.
- Running too few participants per condition.
- Describing a quasi-experiment as a true experiment.
- Over-generalising results from a single lab study to all organisations.
Final thoughts
Experiments allow business and management researchers to test cause-and-effect relationships with a level of confidence that surveys alone cannot provide. Start with a clear causal question, manipulate only what you need to, randomise properly, check your manipulation, and plan an adequate sample size. Combine strong internal validity with realistic settings where you can, and report your design transparently so others can evaluate and replicate your work.