| Time | Chapter |
|---|---|
| 17:23 | WelcomeFor the final statistics lecture of SSR, today's topic is Bayesian ANOVA. Throughout the course we got very close to how statistics is being done in the real world, talking about assumptions, interactions and dependence between predictors. |
| 19:17 | Frequentist and Bayesian frameworksThe frequentist framework is all about asking what happens in the long run to my statistical test, in terms of type 1 and type 2 error rates. If you want to do a hypothesis test or parameter estimation, you are not limited to this framework, and the Bayesian framework also allows you to do inferential statistics, with some different metrics. |
| 22:08 | Goals and readingWe repeat the core Bayesian concepts, apply these to ANOVA, and do a Bayesian ANOVA in JASP, which is the analysis that you need to know for the exam. Chapters one through six of the booklet reintroduce the core concepts, and the new chapter for today is chapter 7 about ANOVA. |
| 23:09 | Beer tastingMany years ago we did a beer tasting test in our lab, where people tasted 2 unmarked cups of beer and had to say which of the two was the alcoholic one. If people cannot taste the difference, they are guessing, and θ, the probability of a correct response, is 0.5. |
| 25:55 | Bayesian updatingThe Bayesian framework is all based on this continuous updating of knowledge. We start with prior knowledge, derive predictions, test those against the observed data, and use the prediction error to update our knowledge, so what we end up with today is our starting knowledge of tomorrow. |
| 28:45 | Bayes' rule for knowledge updatingWe start with some prior beliefs and update them with the data, and that updating step is completely based on how well every individual value of the parameter predicted the data. Together they form our posterior beliefs, what we have learned after observing the data. |
| 30:33 | What is a model?In the Bayesian framework we look at different models, and each model reflects its beliefs about the parameter by a probability distribution. Sarah believes θ is exactly 0.5 and Paul is really sure it is 0.8, and because they go all in on a single value we call these point models. |
| 32:48 | What does a model predict?Based on what a model says about the parameter, we can derive the likely outcomes of an experiment with 10 tasters, which is no different from the sampling distribution. Asking each model how likely our observed 8 out of 10 is (the yellow bar) and taking the ratio of the two yellow bars gives exactly the Bayes factor. |
| 38:09 | Model with multiple valuesModels can also state multiple values, like Alex's open-minded model, which believes that every value of the proportion correct is equally likely before seeing the data. Any model that makes a statement about the parameter gives a sampling distribution, which shows what outcomes to expect if this model is the true model. |
| 40:10 | One-sided modelBetty's model puts probability mass on all values between 0.5 and 1 and none on values lower than 0.5, so it claims that people can distinguish between the alcoholic and non-alcoholic beers. Averaging the sampling distributions of all values Betty claims gives the average sampling distribution, and its yellow bar shows how likely 8 out of 10 is under her model. |
| 43:27 | Prior distributionThese statements of the models are prior distributions, and prior distributions are always probability distributions. A prior for a proportion needs the domain zero to one, and one convenient distribution that follows those criteria is the beta distribution. |
| 45:02 | The dataLet's say we observe 8 successes out of 10. To update the prior distributions with this data, we look at the likelihood of the observed data for every value in the model. |
| 45:40 | Likelihood functionThe likelihood function is not a probability distribution, it maps every value of θ onto the likelihood of the observed data, around 0.3 for θ = 0.8. It is always the same, whichever model you are updating, because it only depends on the type of data, here the binomial likelihood. |
| 47:48 | Marginal likelihoodThe marginal likelihood is the average likelihood of the observed data for all values in the model, so unlike the likelihood it depends on the model. For Alex's model it is around 0.09, and it quite literally sets the bar for what we consider better than average predictions. |
| 52:50 | Updating beliefsThe likelihood divided by the marginal likelihood is the predictive updating factor, which is greater than one exactly when that value of the parameter predicted the data better than average. Those values see a boost in plausibility, and values of 0.55 and lower and 0.95 and higher, which predicted the data worse than average, suffer a penalty. |
| 55:50 | Posterior distributionCompared to Alex's prior distribution, the posterior density has increased for all values that had a higher than average likelihood. Models with a single value, like Sarah's, don't learn from the data, because no amount of data is going to change their beliefs. |
| 1:01:09 | Bayesian estimationA point estimate such as the posterior median is our best guess, but an interval estimate also reflects uncertainty, and the Bayesian version is the credible interval. If Alex's model is the true model, there is a 95% probability that the true value of the proportion is between 0.48 and 0.94. |
| 1:04:41 | Break |
| 1:17:42 | Marginal likelihood, revisitedThe marginal likelihood for Alex's model, 0.09, is exactly the height of the yellow bar in Alex's sampling distribution, how likely 8 out of 10 is on average under that model. It differs for the different models (around 0.17 for Betty), and we use exactly those numbers to do model comparison. |
| 1:20:46 | Bayesian hypothesis testingHypothesis testing compares different models to each other, and null and alternative are really just labels. It is more important to pay attention to how you translate your research question into a statistical statement, such as θ = 0.5 if people cannot taste the difference between the beers. |
| 1:21:59 | Prior and posterior oddsIn the same way that we update beliefs about parameter values, we can update beliefs about which of two models did a better job in predicting the data. The prior odds tell you how many times more likely one model is than the other before seeing the data, and updating them with the data gives the posterior odds. |
| 1:23:18 | Bayes factorThe updating factor from prior odds to posterior odds is the Bayes factor, the ratio of the marginal likelihoods of two models. A BF10 of 3 means that the data are three times more likely under the alternative hypothesis than under the null hypothesis, and a BF01 flips that interpretation, so always report the subscript. |
| 1:27:01 | Sarah versus AlexThe marginal likelihood of 8 out of 10 is 0.04 under Sarah's model and 0.09 under Alex's model. Their ratio, the Bayes factor comparing Sarah to Alex, is 0.48, which means the data are about twice as likely (2.07 times) under Alex's model as under Sarah's model. |
| 1:29:16 | Heuristics for BFThere exist certain words you can associate with values of the Bayes factor, for instance a Bayes factor between 1 and 3 is anecdotal evidence. You don't need to memorize these labels, but you do need to know the definition of the Bayes factor, and an exam question that asks for a decision will tell you which Bayes factor threshold to use. |
| 1:30:56 | BF pizzaIn JASP we represent Bayes factors with pizza plots, where the proportion of mozzarella and tomato sauce shows the evidence for each of the two hypotheses. A Bayes factor of 1 means the data are as likely under both models, and we can even get evidence in favor of the null hypothesis, which is not possible with p-values. |
| 1:33:24 | Advantages of the Bayes factorThe Bayes factor assesses a model's performance in a continuous way, you can look at the data as you are gathering it and stop when you have decisive evidence, and it distinguishes evidence of absence from absence of evidence. A Bayes factor of 1.5, for instance, is not very decisive, which means the data are not informative enough to decide between the two models. |
| 1:36:39 | ANOVA in regression formulaANOVA is just a regression, also in Bayesian ANOVA, so we still represent the model with dummy variables. The regression slope b1 indicates the group difference between our two groups, and if b1 equals 0, there is no difference between the groups. |
| 1:38:17 | Two modelsThe null model states that the regression coefficient is 0, a single spike at 0, while the alternative model puts a Cauchy distribution on it, with most mass around 0 but also quite some mass on other values. In the Bayesian framework we need to make explicit what the alternative predicts, more than just the direction, by choosing its prior distribution. |
| 1:42:29 | Marginal likelihoods and the Bayes factorFor both models we can ask how well they predicted the data on average, and the data are very unlikely under the null model that states the coefficient equals 0. The ratio of the two marginal likelihoods, the Bayes factor, is around 10,000, so the data are 10,000 times more likely under the alternative model than under the null model. |
| 1:44:30 | Bayesian ANOVA in JASPThe Bayesian ANOVA sits under ANOVA in JASP, and here we predict people's confidence rating from whether they correctly identified the alcoholic beer. By default, the BF10 column compares every model to the best model at the top, and the inclusion Bayes factor (here 1.3) compares the models with a predictor to the models without it. |
| 1:54:13 | Two predictors in JASPWith a second predictor, whether people tasted the alcoholic beer first, JASP lists five models, up to the full model with both main effects and their interaction. The two main outputs to focus on for the exam are the BF10 column in the model comparison table and the inclusion Bayes factor from the effects table. |
| 1:58:19 | RecapBayesian statistics is still a thing, and you can do Bayesian versions of all the analyses we've seen in SSR. I would like you to primarily understand the Bayesian ANOVA, which is explained in the booklet. |
| 1:58:50 | Bayes and open science in the wildA large-scale investigation of the psychology literature over the past 20 years shows a nice increase in how many articles use Bayesian statistics. Open data, preregistration and reporting the statistical software have also improved, making for a little more robust science. |
| 2:01:31 | Looking back on SSROur central SSR formula, outcome equals model plus error, holds three things we are now getting a grip on. Statistics is a very relative endeavor, the assumptions are a very important part of robust statistics, and complete reporting means a number for accuracy and graphs for some vibes. |
| 2:02:52 | Metrics and alternativesDifferent metrics answer different research questions, such as whether we have evidence for an effect (a p-value or a Bayes factor) or which model predicted the best. Multiverse analysis, which is not part of the exam, helps you get a grip on sometimes arbitrary choices. |
| 2:03:39 | ClosingThe exam notes from the previous lectures of block 3 are at the end of the slides. Thank you for all your energy, I wish you all the best in your future studies, and see you next week at the exam. |
29. Bayesian ANOVA
Summary and chapters of the recording
13 November 2026 · Recording · Slides · Handout
In the final statistics lecture of SSR we repeat the core Bayesian concepts with a subset of the beer tasting data, in which 8 out of 10 tasters identified the alcoholic beer. Different models (Sarah, Paul, Alex and Betty) make different statements about θ, the probability of a correct response, and we update these prior distributions with the data into posterior distributions, based on how well each value of θ predicted the data. After the break we compare models with the Bayes factor, the ratio of their marginal likelihoods, and apply all of this to ANOVA, with a spike at 0 for the null model and a Cauchy prior for the alternative. In JASP we run a Bayesian ANOVA, where the BF10 column of the model comparison table and the inclusion Bayes factor of the effects table are the two outputs to understand for the exam. We close with Bayes and open science in the wild and a look back on SSR.
Open the recording and skip to the chapter you need. The chapters come from the lecture transcript, so one may start a few seconds early or late.