15. Factorial ANOVA

Independent factorial

Author
Affiliation

Johnny van Doorn

University of Amsterdam

Published

October 8, 2025

In this lecture we aim to:

  • ANOVA with 2 categorical predictors
  • Discuss interaction
  • Demonstration in JASP

Reading: Chapter 13

Independent factorial ANOVA

Two or more independent variables with two or more categories. One dependent variable.

Independent factorial ANOVA

The independent factorial ANOVA analyses the variance of multiple independent variables (Factors) with two or more categories.

Effects and interactions:

  • 1 dependent/outcome variable
  • 2 or more independent/predictor variables
    • 2 or more cat./levels

Assumptions

  • Continuous variable
  • Random sample
  • Normally distributed
    • Q-Q plot
  • Equal variance within groups
    • Ratio of observed sd’s
    • Welch correction only exists for one-way ANOVA

Formulas

Variance Sum of squares df Mean squares F-ratio
Model \(\text{SS}_{\text{model}} = \sum{n_k(\bar{X}_k-\bar{X})^2}\) \(k_{model}-1\) \(\frac{\text{SS}_{\text{model}}}{\text{df}_{\text{model}}}\) \(\frac{\text{MS}_{\text{model}}}{\text{MS}_{\text{error}}}\)
\(\hspace{2ex}A\) \(\text{SS}_{\text{A}} = \sum{n_k(\bar{X}_k-\bar{X})^2}\) \(k_A-1\) \(\frac{\text{SS}_{\text{A}}}{\text{df}_{\text{A}}}\) \(\frac{\text{MS}_{\text{A}}}{\text{MS}_{\text{error}}}\)
\(\hspace{2ex}B\) \(\text{SS}_{\text{B}} = \sum{n_k(\bar{X}_k-\bar{X})^2}\) \(k_B-1\) \(\frac{\text{SS}_{\text{B}}}{\text{df}_{\text{B}}}\) \(\frac{\text{MS}_{\text{B}}}{\text{MS}_{\text{error}}}\)
\(\hspace{2ex}AB\) \(\text{SS}_{A \times B} = \text{SS}_{\text{model}} - \text{SS}_{\text{A}} - \text{SS}_{\text{B}}\) \(df_A \times df_B\) \(\frac{\text{SS}_{\text{AB}}}{\text{df}_{\text{AB}}}\) \(\frac{\text{MS}_{\text{AB}}}{\text{MS}_{\text{error}}}\)
Error \(\text{SS}_{\text{error}} = \sum{s_k^2(n_k-1)}\) \(N-k_{model}\) \(\frac{\text{SS}_{\text{error}}}{\text{df}_{\text{error}}}\)
Total \(\text{SS}_{\text{total}} = \text{SS}_{\text{model}} + \text{SS}_{\text{error}}\) \(N-1\) \(\frac{\text{SS}_{\text{total}}}{\text{df}_{\text{total}}}\)

Variance in factorial designs

Example

In this example we will look at the amount of accidents in a car driving simulator while subjects where given varying doses of speed and alcohol.

  • Dependent variable
    • Accidents
  • Independent variables
    • Speed dose
      • None / Small / Large
    • Alcohol dose
      • None / small / large

Data

SS full model

Variance Sum of squares df Mean squares F-ratio
Model \(\text{SS}_{\text{model}} = \sum{n_k(\bar{X}_k-\bar{X})^2}\) \(k_{model}-1\) \(\frac{\text{SS}_{\text{model}}}{\text{df}_{\text{model}}}\) \(\frac{\text{MS}_{\text{model}}}{\text{MS}_{\text{error}}}\)


Predicts group means for each cell of the design:

     speed  alcohol accidents  n
1 none (S) none (A)    2.1060 20
2 some (S) none (A)    2.9445 20
3 much (S) none (A)    3.8880 20
4 none (S) some (A)    3.4435 20
5 some (S) some (A)    4.7625 20
6 much (S) some (A)    5.5790 20
7 none (S) much (A)    5.2970 20
8 some (S) much (A)    6.5125 20
9 much (S) much (A)    7.5720 20

SS full model visual

SS error

Variance Sum of squares df Mean squares F-ratio
Error \(\text{SS}_{\text{error}} = \sum{s_k^2(n_k-1)}\) \(N-k\) \(\frac{\text{SS}_{\text{error}}}{\text{df}_{\text{error}}}\)


SS error visual

SS A Speed

Variance Sum of squares df Mean squares F-ratio
\(\hspace{2ex}A\) \(\text{SS}_{\text{A}} = \sum{n_k(\bar{X}_k-\bar{X})^2}\) \(k_A-1\) \(\frac{\text{SS}_{\text{A}}}{\text{df}_{\text{A}}}\) \(\frac{\text{MS}_{\text{A}}}{\text{MS}_{\text{error}}}\) 


none (S) some (S) much (S) 
3.615500 4.739833 5.679667

SS A Speed Visual

SS A Speed Error Visual

SS B Alcohol

Variance Sum of squares df Mean squares F-ratio
\(\hspace{2ex}B\) \(\text{SS}_{\text{B}} = \sum{n_k(\bar{X}_k-\bar{X})^2}\) \(k_B-1\) \(\frac{\text{SS}_{\text{B}}}{\text{df}_{\text{B}}}\) \(\frac{\text{MS}_{\text{B}}}{\text{MS}_{\text{error}}}\) 
none (A) some (A) much (A) 
  2.9795   4.5950   6.4605

SS B Alcohol Visual

SS B Alcohol Error Visual

SS AB Alcohol x Speed

Variance Sum of squares df Mean squares F-ratio
\(\hspace{2ex}AB\) \(\text{SS}_{A \times B} = \text{SS}_{\text{model}} - \text{SS}_{\text{A}} - \text{SS}_{\text{B}}\) \(df_A \times df_B\) \(\frac{\text{SS}_{\text{AB}}}{\text{df}_{\text{AB}}}\) \(\frac{\text{MS}_{\text{AB}}}{\text{MS}_{\text{error}}}\) 


# Sums of squares for the interaction between speed and alcohol
ss.speed.alcohol <- ss.model - ss.speed - ss.alcohol
ss.speed.alcohol
[1] 1.910727

Using mean squares to compute \(F\)

For every \(F\)-statistic, we use the \(MS_{error}\) of the full model:

\[\begin{aligned} F_{Speed} &= \frac{{MS}_{Speed}}{{MS}_{error}} \\ F_{Alcohol} &= \frac{{MS}_{Alcohol}}{{MS}_{error}} \\ F_{Alcohol \times Speed} &= \frac{{MS}_{Alcohol \times Speed}}{{MS}_{error}} \\ \end{aligned}\]

Interaction

\[F_{Alcohol \times Speed}\]

N          <- length(accidents)
k.speed    <- 3
k.alcohol  <- 3
k.model    <- 9
df.speed   <- k.speed   - 1
df.alcohol <- k.alcohol - 1
df.speed.alcohol <- df.speed * df.alcohol

ms.speed.alcohol <- ss.speed.alcohol / df.speed.alcohol

df.error <- N - k.model
ms.error <- ss.error / df.error

\(P\)-value

Partition the explained variance (BONUS)

Partition the explained variance (BONUS)

In reality, we do not know exactly how the shared variance is distributed across these three sources (IV1, IV2, shared)

Partition the explained variance (BONUS)

  • Fit a model with only IV1, then use that to compute the explained variance for IV2: \(SS_{M1} = 70 \rightarrow SS_{IV2} = 130 - 70 = 60\)
  • Fit a model with only IV2, then use that to compute the explained variance for IV1: \(SS_{M2} = 50 \rightarrow SS_{IV1} = 130 - 50 = 80\)
  • See also Oliver Twisted from Chapter 13

Contrast

Planned comparisons

  • Exploring differences of theoretical interest
  • Higher precision
  • Higher power

Post-Hoc

Unplanned comparisons

  • Exploring all possible differences
  • Adjust p-value for inflated type 1 error
    • P(1 type-I error) = \(0.05\)
    • P(not 1 type-I error) = \(0.95\)
    • P(not 5 type-I errors) = \(0.95^5 = 0.774\)
  • Bonferroni more conservative, Tukey more permissive

Exam note: know where the options are, exam question will inform you which correction to use

Effect size in ANOVA

General effect size measure: Partial omega squared \(\omega_p^2\)

How to interpret?

\(\omega^2\) and \(\omega_p^2\) Magnitude of Effect Interpretation
0.00–0.009 Very small Trivial practical importance
0.01–0.05 Small Small but meaningful variance explained
0.06–0.13 Medium Moderate variance explained
≥ 0.14 Large Substantial variance explained

Effect size in follow-up tests

Effect sizes of contrasts or post-hoc comparisons: Cohen’s \(d\)

How to interpret?

Cohen’s d Magnitude of Effect Interpretation
0.00–0.19 Very small Likely negligible in most contexts
0.20–0.49 Small Noticeable but modest difference
0.50–0.79 Medium Moderate, practically meaningful difference
≥ 0.80 Large Substantial, easily noticeable difference

JASP

Closing

Recap

With multiple predictor variables:

  • We are still explaining variance
  • We check the interaction
    • Plots!
    • F-value of interaction effect
    • Conditional post-hoc tests

Contact

CC BY-NC-SA 4.0