The analyses below illustrate the equivalence between various tests that we have so far discussed in the SSR course. The data you find here is presened both in long format (left 3 columns) and wide format (right 3 columns).
First, the t-tests are basically a special case of ANOVA: while ANOVA allows for 2 or more categories for its categorical predictors, the t-test is limited to two categories. The t-test equivalent of the regular ANOVA is the independent samples t-test, while for the RM ANOVA this is the paired samples t-test.
The difference between ANOVA and RM ANOVA (and therefore independent t-test and paired t-test) is that the latter takes into account baseline differences. In doing so, variability that first was categorized as "noise", becomes accounted for and is no longer noise/error. This makes for a more powerful test, and allows us to look for the effect of our categorical predictor, while accounting for individual (baseline) differences. In the analyses below, we have a pre and post-test design (so within subject design). If we were to ignore the fact that we are measuring within person, we find no effect between pre and post-test. This is shown by the ANOVA and independent t-test, where there is no meaningful difference between "TimePoint". Note that these two tests are doing exactly the same: they have the same p-value, and even test statistic: the F is simply the t-value squared: F = t2 = 0.5412 = 0.293 - magic!!
Next, we can account for individual differences by adding participant number as a predictor variable. Note that we can only meaningfully differentiate "within subject variance" and "between subject variance" when we have multiple measurements per participants (so whenever we would use a RM ANOVA, or paired t-test). We can account for these differences in various ways:
Now that we are able to explain a lot of additional variability in the data by accounting for individual differences, we are able to detect an effect of the "TimePoint" variable - in other words, detect a difference between pre- and post-test. Each of the three analyses now has a significant effect of "TimePoint". Again, the paired t-test is completely equivalent to the RM ANOVA - just with a t-statistic instead of an F (but you convert a t to F by squaring it).
Lastly, I show how the one-sample t-test conducted on the difference scores is exactly the same as a paired t-test.
| t | df | p | |||||
|---|---|---|---|---|---|---|---|
| y | -0.541 | 38 | 0.591 | ||||
| Note. Student's t-test. | |||||||
| Measure 1 | Measure 2 | t | df | p | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| preTest | - | postTest | -5.603 | 19 | < .001 | ||||||
| Note. Student's t-test. | |||||||||||
We can do exactly the same type of analysis, but using linear regression. First, we can see what happens when we just have a single categorical predictor, timePoint. When specifying a categorical predictor (in the Factors box), the first occuring category is used as the reference group, and all subsequent categories receive a regression weight that expresses the difference between that group, and the reference group. Here, the regression weight for timePoint (post) equals 2.03, so that is the observed difference between the pre- and post-test. The standard error is pretty large though, so the corresponding t- and p-values are insignificant and we fail to detect a group difference.
The next regression analysis includes ID as a second predictor variable, and shows how including baseline differences in the model, allows for us to detect the difference between pre- and post-test with greater accuracy: the standard error has decreased a lot, so even though the regression weight for timePoint (post) stays the same, we now have quite some evidence against the null hypothesis!