STATISTICAL METHODS · BEGINNER GUIDE
Bonferroni Correction After ANOVA: A Worked Example
See why follow-up comparisons need multiplicity control and how the Bonferroni rule changes a significance threshold.
Quick answer: An ANOVA F test can show that not all group means are equal, but it does not identify the differing pairs. If you run several follow-up tests, a correction such as Bonferroni can control the family-wise error rate by using a smaller per-test threshold.
What this method answers
Repeated tests create more chances to flag a difference by chance. Bonferroni is simple: divide the planned family-wide alpha level by the number of comparisons. It is valid under broad dependence conditions, but it can be conservative, especially with many tests.
Choose the comparisons that answer your research question. Do not run every possible comparison and then report only the smallest p-value. A post-hoc method designed for all pairwise comparisons may be more suitable when that is the goal.
Before running a calculation, write down the population and the exact quantity you want to estimate. A method name alone does not define the question, the comparison or the assumptions.
Steps to follow
- State which comparisons belong to one family before interpreting them.
- Choose a family-wise error rate, such as 0.05, based on the study plan.
- For Bonferroni, divide that rate by the number of comparisons.
- Compare each p-value with the adjusted threshold and report estimates as well as decisions.
Keep a record of exclusions, recoding decisions and software settings. That makes the analysis easier to check and repeat.
Worked example
After an overall analysis, a researcher planned three comparisons. The family-wide alpha is 0.05. The example p-values below are illustrative.
| Comparison | Example p-value | Below 0.0167? |
|---|---|---|
| A vs B | 0.010 | Yes |
| A vs C | 0.030 | No |
| B vs C | 0.400 | No |
With three planned tests, the Bonferroni threshold is 0.05 ÷ 3 ≈ 0.0167 for each test. Only the illustrative p-value 0.010 is below that threshold. The conclusion is limited to this prespecified family and these model assumptions.
Interpret the result carefully
The adjustment reduces the chance of at least one false positive across the family, at the cost of lower power. A non-significant adjusted test does not prove that two means are identical.
Limit to keep in mind: Do not confuse an adjusted p-value with the threshold method; software may report either adjusted values or unadjusted values. Check the function documentation and say which one you used.
When reporting an analysis, include the sample size, estimate, units and uncertainty. Explain why the method fits the sampling and measurement design. Separate planned tests from exploratory checks, and avoid saying that association proves a cause.
Frequently asked questions
Should I use Bonferroni for every ANOVA?
No. Use it when it matches the planned family of comparisons. Other procedures can be more powerful for specific goals.
Do I apply it to the omnibus F test?
The usual Bonferroni example here adjusts a family of follow-up tests. The overall test and the follow-up family should be described separately.
What if I planned 10 comparisons?
The threshold would be 0.05/10 = 0.005 under this rule. Consider whether all 10 comparisons are scientifically needed.
Does an adjusted non-significant result prove no effect?
No. It means the evidence did not cross the chosen adjusted threshold; estimate size and uncertainty still matter.