Results

The analyses below illustrate the equivalence between various tests that we have so far discussed in the SSR course. The data you find here is presened both in long format (left 3 columns) and wide format (right 3 columns).

First, the t-tests are basically a special case of ANOVA: while ANOVA allows for 2 or more categories for its categorical predictors, the t-test is limited to two categories. The t-test equivalent of the regular ANOVA is the independent samples t-test, while for the RM ANOVA this is the paired samples t-test.

The difference between ANOVA and RM ANOVA (and therefore independent t-test and paired t-test) is that the latter takes into account baseline differences. In doing so, variability that first was categorized as "noise", becomes accounted for and is no longer noise/error. This makes for a more powerful test, and allows us to look for the effect of our categorical predictor, while accounting for individual (baseline) differences. In the analyses below, we have a pre and post-test design (so within subject design). If we were to ignore the fact that we are measuring within person, we find no effect between pre and post-test. This is shown by the ANOVA and independent t-test, where there is no meaningful difference between "TimePoint". Note that these two tests are doing exactly the same: they have the same p-value, and even test statistic: the F is simply the t-value squared: F = t² = 0.541² = 0.293 - magic!!

Next, we can account for individual differences by adding participant number as a predictor variable. Note that we can only meaningfully differentiate "within subject variance" and "between subject variance" when we have multiple measurements per participants (so whenever we would use a RM ANOVA, or paired t-test). We can account for these differences in various ways:

• Conduct a RM ANOVA where we specify time points (this analysis automatically adds participant number as a predictor) - this analysis uses the wide data format
• Conduct an ANOVA where we include a categorical column that denotes participant number (longPP in this data) - this analysis uses the long data format
• Conduct a paired t-test, which also automatically assumes that the observations in each row are paired - this analysis uses the wide data format

Now that we are able to explain a lot of additional variability in the data by accounting for individual differences, we are able to detect an effect of the "TimePoint" variable - in other words, detect a difference between pre- and post-test. Each of the three analyses now has a significant effect of "TimePoint". Again, the paired t-test is completely equivalent to the RM ANOVA - just with a t-statistic instead of an F (but you convert a t to F by squaring it).

Lastly, I show how the one-sample t-test conducted on the difference scores is exactly the same as a paired t-test.