Chapter 14: Analysis of Variance
2026-04-07
This presentation is based on the following books. The references are coming from these books unless otherwise specified.
Main sources:
ClassShare App
You may be asked in class to go to the ClassShare App to answer questions.
SPSS Tutorial
By the end of this chapter, you should be able to:
| Symbol | Name | Pronunciation | Definition |
|---|---|---|---|
| \(\mu\) | Population Mean | “mu” | True mean in the entire population |
| \(k\) | Number of Groups | “k” | Total number of treatment levels or groups |
| \(N\) | Total Sample Size | “N” | Total number of observations across all groups |
| \(SS_B\) | Between-Groups SS | “S-S between” | Sum of squares reflecting variability of group means around the grand mean |
| \(SS_W\) | Within-Groups SS | “S-S within” | Sum of squares reflecting variability of scores around their group mean |
| \(MS\) | Mean Square | “M-S” | Variance estimate; SS divided by its degrees of freedom |
| \(F\) | F-ratio | “F” | Ratio of between-groups to within-groups variance |
| \(\eta^2\) | Eta-squared | “eta squared” | Proportion of total variance explained by the independent variable |
| \(\omega^2\) | Omega-squared | “omega squared” | Less biased estimate of the population effect size |
| \(\alpha\) | Alpha level | “\(\alpha\)” | Significance level (probability of a Type I error) |
When comparing three or more groups, the temptation is to run pairwise t-tests for every combination[1,2].
Why multiple t-tests are wrong:
With \(c\) independent comparisons each at \(\alpha = .05\), the familywise error rate is:
\[ \alpha_{\text{FW}} = 1 - (1 - \alpha)^c \]
| Groups | Comparisons | Familywise α |
|---|---|---|
| 3 | 3 | ≈ .14 |
| 4 | 6 | ≈ .26 |
| 5 | 10 | ≈ .40 |
The ANOVA solution:
ANOVA tests one omnibus null hypothesis—that all group means are equal—in a single analysis that maintains the stated α[3].
\[ H_0: \mu_1 = \mu_2 = \mu_3 = \ldots = \mu_k \]
If the omnibus F is significant, post hoc tests then make pairwise comparisons while controlling the familywise error rate.
Never substitute multiple t-tests for ANOVA
Running uncorrected pairwise t-tests when comparing three or more groups is not just a stylistic choice—it is a methodological error that produces misleading conclusions[1].
The F-ratio is the core of ANOVA — a signal-to-noise ratio[3,4]:
\[ F = \frac{MS_{\text{between}}}{MS_{\text{within}}} = \frac{\text{Signal}}{\text{Noise}} \]
When H₀ is true (all means equal):
Both \(MS_B\) and \(MS_W\) estimate the same population variance.
\[F \approx 1\]
The F-distribution is always positive and right-skewed. Its shape is determined by \((df_B, df_W)\).
When H₁ is true (at least one mean differs):
\(MS_B\) is inflated by real group differences; \(MS_W\) is unaffected.
\[F > 1\]
Large F values are unlikely under H₀ and provide strong evidence to reject it.
What F does NOT tell you
A significant F only tells you that somewhere among the groups a difference exists. It does not tell you which specific pairs of groups differ. Post hoc tests are required for that[2].
One-way between-subjects ANOVA is appropriate when[5–7]:
Design requirements:
If only two groups: use an independent t-test.
If same participants across conditions: use Repeated Measures ANOVA (Chapter 15).
Movement science examples:
| IV (Groups) | DV | Test |
|---|---|---|
| Training type (3) | VO₂max | One-way ANOVA |
| Age category (4) | Balance score | One-way ANOVA |
| Sport level (3) | Grip strength | One-way ANOVA |
| Rehab protocol (3) | ROM (degrees) | One-way ANOVA |
Real Example: Comparing VO₂max Across Training Groups
A researcher randomly assigns 60 students to one of three 12-week programs: endurance training (n = 20), resistance training (n = 20), or no-exercise control (n = 20). VO₂max is measured at post-training. With one factor (training group) at three levels and a continuous DV, one-way ANOVA is appropriate[5].
Null hypothesis (H₀):
All population group means are equal — the independent variable has no effect on the dependent variable:
\[ H_0: \mu_1 = \mu_2 = \mu_3 = \ldots = \mu_k \]
Alternative hypothesis (H₁):
At least one population mean differs from the others:
\[ H_1: \text{at least one } \mu_j \neq \mu_{j'} \]
Note: H₁ does not specify which groups differ — only that some difference exists somewhere among the means.
Steps for conducting a one-way ANOVA:
| Step | Action |
|---|---|
| 1 | State H₀ and H₁; set α (typically .05) |
| 2 | Check assumptions (independence, normality, homogeneity of variance) |
| 3 | Run the ANOVA in SPSS and examine the source table |
| 4 | If F is significant, run post hoc tests |
| 5 | Compute effect sizes (η² or ω²) |
| 6 | Report results in APA format |
Before computing F, verify three key assumptions[1,2,4]:
| Assumption | How to check | What to do if violated |
|---|---|---|
| 1. Independence | Validated via research design (random assignment, no repeated measures). | Use a different model (e.g., repeated measures ANOVA for within-subjects designs). |
| 2. Normality | Check within each group separately using histograms, Q-Q plots, and the Shapiro-Wilk test (p > .05 = normal). ANOVA is robust with n ≥ 30 per group due to the CLT[8]. | Log/square-root transformations or the nonparametric Kruskal-Wallis test (Chapter 19). |
| 3. Homogeneity of variance | Use Levene’s test (p > .05 = equal variances). Practical rule: largest SD < twice the smallest SD. | Use Welch’s ANOVA or Brown-Forsythe ANOVA; switch post hoc tests to Games-Howell[9,10]. |
When Levene’s test is significant (p < .05)
Switch from the standard F-test to Welch’s ANOVA (available under Analyze → Compare Means → One-Way ANOVA → Options → Welch). Use Games-Howell instead of Tukey HSD for post hoc comparisons[2].
A researcher compares VO₂max (mL·kg⁻¹·min⁻¹) in 60 students randomly assigned to three 12-week programs: endurance (n = 20), resistance (n = 20), and control (n = 20).
Step 1: State hypotheses
Step 2: Check assumptions
Independence confirmed by design; Shapiro-Wilk p > .05 in each group; Levene’s p = .312 (equal variances assumed).
Step 3: Run ANOVA in SPSS; interpret source table
| Source | SS | df | MS | F | p |
|---|---|---|---|---|---|
| Between groups | 1,248.6 | 2 | 624.3 | 14.87 | < .001 |
| Within groups | 2,391.4 | 57 | 41.95 | ||
| Total | 3,640.0 | 59 |
With F(2, 57) = 14.87, p < .001, we reject H₀. Post hoc tests are needed to identify which pairs differ.
SPSS: Run One-Way ANOVA
Analyze → Compare Means → One-Way ANOVA. Add DV to Dependent List and grouping variable to Factor. Under Options, check Descriptive and Homogeneity of variance test. See the SPSS Tutorial: One-Way ANOVA.
Answer: Because Levene’s test is significant (p = .011 < .05), the homogeneity of variance assumption is violated. The researcher should use Welch’s ANOVA (or Brown-Forsythe ANOVA) instead of the standard F-test, and apply Games-Howell post hoc comparisons rather than Tukey HSD. In SPSS: Analyze → Compare Means → One-Way ANOVA → Options → check Welch and Brown-Forsythe.
Open the ClassShare App to answer this question:
A significant F-test tells us something is different — post hoc tests tell us what[1,11].
Why post hoc tests are needed:
Never run post hoc tests on a non-significant F
Applying post hoc tests when the omnibus F is not significant — “fishing” — inflates Type I error and produces unreliable conclusions[2].
Choosing the right post hoc test:
| Test | Equal variances? | Best for |
|---|---|---|
| Tukey HSD | Yes | Most common; good balance of Type I/II error |
| Bonferroni | Yes | Small number of comparisons; conservative |
| Games-Howell | No | When Levene’s is significant |
| Scheffé | Yes | most conservative |
Rule: Use Tukey HSD when assumptions are met. Switch to Games-Howell when Levene’s is significant[2,10].
Following the VO₂max example, Tukey HSD post hoc comparisons reveal which specific pairs differ[11]:
| Comparison | Mean Difference | SE | p (adjusted)† | 95% CI |
|---|---|---|---|---|
| Endurance − Resistance | 7.2 | 2.05 | .003 | [2.1, 12.3] |
| Endurance − Control | 11.8 | 2.05 | < .001 | [6.7, 16.9] |
| Resistance − Control | 4.6 | 2.05 | .082 | [−0.5, 9.7] |
† Why “adjusted”? Each individual t-test runs at α = .05. With three comparisons, the chance of at least one false positive rises to ~14%. Tukey HSD corrects each p-value upward so that the risk of any false positive across all comparisons combined stays at .05.
Based on the Tukey HSD output below, which training groups differed significantly in VO₂max?
| Comparison | Mean Difference | SE | p (adjusted) | 95% CI |
|---|---|---|---|---|
| Endurance − Resistance | 7.2 | 2.05 | .003 | [2.1, 12.3] |
| Endurance − Control | 11.8 | 2.05 | < .001 | [6.7, 16.9] |
| Resistance − Control | 4.6 | 2.05 | .082 | [−0.5, 9.7] |
Answer: Endurance vs. Resistance (p = .003) and Endurance vs. Control (p < .001) were both significant. Resistance vs. Control (p = .082) was not significant.
Open the ClassShare App to answer this question:
SPSS: Post Hoc Tests
In the One-Way ANOVA dialog, click Post Hoc. Select Tukey (if equal variances) or Games-Howell (if unequal). See the SPSS Tutorial: Post Hoc Tests.
A one-way ANOVA comparing balance scores across three groups (n = 16 per group, N = 48) produces F(2, 45) = 5.83, p = .006. Levene’s test is non-significant. A student concludes that all three groups differ from each other. Is this conclusion correct?
Hint: df₁ = k − 1 = 3 − 1 = 2 (Between-groups row); df₂ = N − k = 48 − 3 = 45 (Within-groups row)Answer: No. A significant omnibus F only tells us that at least one group mean differs — it does not specify which pairs. The student must run Tukey HSD post hoc tests (since Levene’s is non-significant) to determine which specific group pairs are significantly different. Some comparisons may be significant and others may not.
A significant F-test answers whether groups differ — effect sizes answer how much[12–14].
Eta-squared (η²)
\[ \eta^2 = \frac{SS_{\text{between}}}{SS_{\text{total}}} \]
Omega-squared (ω²) — preferred for reporting
\[ \omega^2 = \frac{SS_{\text{between}} - (k-1) \cdot MS_{\text{within}}}{SS_{\text{total}} + MS_{\text{within}}} \]
Cohen’s (1988) benchmarks (η² and ω²):
| Effect | Value |
|---|---|
| Small | .01 |
| Medium | .06 |
| Large | .14 |
| Source | SS | df | MS | F | p |
|---|---|---|---|---|---|
| Between groups | 1,248.6 | 2 | 624.3 | 14.87 | < .001 |
| Within groups | 2,391.4 | 57 | 41.95 | ||
| Total | 3,640.0 | 59 |
Cohen’s f — for power analysis (G*Power):
\[ f = \sqrt{\frac{\eta^2}{1 - \eta^2}} \]
| Effect | f |
|---|---|
| Small | .10 |
| Medium | .25 |
| Large | .40 |
Effective visualizations communicate both central tendency and variability across all groups[16,17].
Reading CI Error Bars
Clear gap between two groups’ bars → strong visual evidence of a significant difference. Bars overlap → ambiguous — CIs can overlap by up to ~50% and the difference can still be significant. Always run the formal test; the chart is a guide, not a decision rule.
Use this sequence whenever you compare means across three or more independent groups[2,4]:
1. Identify the design — One factor, 3+ independent groups → One-Way ANOVA
2. State hypotheses — H₀: all μ equal; H₁: at least one μ differs
3. Screen your data — Histograms, Q-Q plots, boxplots per group
4. Check assumptions — Shapiro-Wilk (normality), Levene’s test (homogeneity)
5. Run ANOVA in SPSS — Analyze → Compare Means → One-Way ANOVA
6. Interpret omnibus F — Significant? → proceed to post hoc tests
7. Run post hoc tests — Tukey HSD (equal variances) or Games-Howell (unequal)
8. Compute effect sizes — η² = SS_B / SS_T; report ω² as primary
9. Report in APA format — F(df_B, df_W), p, η²/ω², M, SD per group, post hoc results
The Goal Is Not Just Numbers
Always ask: “Is the effect detectable (F, p)? Which groups drive it (post hoc)? How large is it (η², ω²)? Does it matter in practice?”
APA-style reporting includes: group descriptive statistics, omnibus F, degrees of freedom, p, effect sizes (both η² and ω²), and post hoc comparisons[17,18]. See the SMS Appendix: APA Reporting for full guidelines.
Omnibus F template:
“A one-way ANOVA revealed a [significant/non-significant] effect of [factor] on [DV], F([df_B], [df_W]) = [F-value], p = [p-value], η² = [value], ω² = [value].”
Full example (from SMS, Ch.14):
A one-way ANOVA revealed a significant effect of training group on VO₂max, F(2, 57) = 14.87, p < .001, η² = .34, ω² = .32. Post hoc comparisons using Tukey HSD indicated that the endurance group (M = 52.4, SD = 6.8 mL·kg⁻¹·min⁻¹) had significantly higher VO₂max than both the resistance group (M = 45.2, SD = 6.2), p = .003, 95% CI [2.1, 12.3], and the control group (M = 40.6, SD = 6.4), p < .001, 95% CI [6.7, 16.9]. Resistance and control groups did not differ significantly, p = .082, 95% CI [−0.5, 9.7].
Key reporting checklist
Include: M and SD per group • F(df_B, df_W) • exact p • η² and ω² • post hoc test name • pairwise p-values and 95% CIs. See the SPSS Tutorial: One-Way ANOVA for full write-up examples.
Statistical power in one-way ANOVA depends on effect size (Cohen’s f), sample size, α, and the number of groups[12,19].
Example: Medium effect (f = .25), 80% power, α = .05, 3 groups → ≈ 159 total participants (~53 per group)[12,19].
Converting η² to Cohen’s f:
| η² | f | Interpretation |
|---|---|---|
| .01 | .10 | Small |
| .06 | .25 | Medium |
| .14 | .40 | Large |
| .34 | .72 | Very large |
SPSS Power Analysis
SPSS Statistics 31+ includes built-in power analysis: Analyze → Power Analysis → One-Way ANOVA. Useful for both a priori (planning) and post hoc (achieved power) analyses.
Avoid these frequent mistakes when conducting and reporting one-way ANOVA[1,2,15]:
| # | Pitfall | Solution |
|---|---|---|
| 1 | Running multiple t-tests instead of ANOVA | Always use ANOVA as the omnibus test; post hoc tests for pairwise follow-up |
| 2 | Running post hoc tests after a non-significant F | Post hoc tests are only valid after a significant omnibus result |
| 3 | Reporting η² without acknowledging its bias | Report ω² as primary; note η²’s upward bias when including it |
| 4 | Treating non-significant F as proof of no difference | A non-significant result ≠ groups are equal; may reflect low power |
| 5 | Ignoring unequal group sizes | Aim for balanced designs; report group-specific descriptives |
Core Principle
Always ask: “Are the groups different (F, p)? Which ones (post hoc)? How much (η², ω²)? Does it matter in practice?”
Your instructor will provide you with a link to the activity in Canvas