Every group comparison you will ever run is the same ratio with different bookkeeping.

The z-test and the one-sample t-test

The simplest inferential question: is my sample mean different from some stated value? A clinic's mean anxiety score against the published norm, a class's mean against the national average, a reaction time against a theoretical baseline. Everything more complicated is an elaboration of this.

Use z only when σ is genuinely known from a large normative sample — which in practice means almost never. Otherwise s estimates σ, the extra uncertainty fattens the tails, and the t distribution is the correct reference. At n above about 100 the two give nearly identical answers, which is why large-sample work is often described in z terms.

The independent-samples t-test

Two unrelated groups, one continuous outcome. The signal is the difference between means; the noise is the standard error of that difference, which — as day 14 warned — is built from both groups' variances added together.

Note the identity from day 21: with two groups, t² equals the F from a one-way ANOVA on the same data. They are one test. Which you report is a matter of convention and of how many groups you have.

The paired-samples t-test

When the two measurements come from the same participant — before and after, two conditions, matched pairs — you can subtract within person first. That single move removes all stable between-person variability from the error term, which is why within-subject designs are so much more powerful.

The power advantage depends entirely on the correlation between the two measurements. When pre and post correlate at .80, the SD of the differences is far smaller than either raw SD and the design is dramatically more efficient. When the correlation is near zero, pairing buys you nothing and costs you degrees of freedom.

Effect sizes and intervals for mean differences

A t-test gives you a decision. The three quantities that make it a finding are the raw difference, the standardised difference, and an interval around both. Journals now expect all of them, and entrance papers increasingly ask you to compute the second.

One-way ANOVA: partitioning variance

With three or more groups, running all pairwise t-tests inflates the familywise error rate (day 31). ANOVA answers a single omnibus question instead — do these means differ anywhere? — with one test at one α. The way it does so is the most elegant idea in classical statistics: split the total variability into the part explained by group membership and the part left over.

Total variability, measured as a sum of squares around the grand mean, splits exactly into between-groups SS (how far each group mean sits from the grand mean) and within-groups SS (how far each score sits from its own group mean). Divide each by its df to get mean squares, which are variance estimates, and take their ratio. Under H₀ both estimate error variance alone, so F hovers near 1. When group means genuinely differ, the numerator grows.

Push difference between means up and watch the between bar and F grow together. Now, holding that difference fixed, push within-group SD up: the same real effect becomes undetectable. That is the whole argument for careful measurement — reducing noise is mathematically equivalent to increasing the effect, and usually cheaper than recruiting.

The ANOVA source table, computed by hand

Filling in a partially completed source table is the single most common computational question in psychology entrance papers, and it is entirely mechanical once you know that the rows add up. Do it by hand once and you will never be troubled by it again.

Post hoc tests after a significant F

The omnibus F says something differs. Post hoc tests say what, while keeping the familywise error rate at the level you claimed. They are called post hoc because they are chosen after seeing that something is there — which is precisely why they carry a multiplicity penalty that planned contrasts do not.

Planned contrasts and trend analysis

If your hypothesis is more specific than 'something differs', you should not be running an omnibus test and then fishing. A planned contrast tests a specific, theory-derived comparison — two treatments combined against a control, or a linear increase across doses — with more power and a cleaner interpretation.

A contrast is a set of weights, one per group, that sum to zero. Weights of (1, −1, 0) compare groups 1 and 2 and ignore group 3. Weights of (1, 1, −2) compare the average of the first two against the third. Two contrasts are orthogonal — statistically independent — when the products of their corresponding weights sum to zero, and with k groups you can construct k − 1 mutually orthogonal contrasts that partition the between-groups SS completely.

Factorial ANOVA: two factors, three questions

A factorial design crosses two or more factors so that every combination of levels appears. A 2 × 3 design with therapy type (CBT, control) and severity (mild, moderate, severe) has six cells, and it answers three questions for the price of one study: the main effect of therapy, the main effect of severity, and their interaction.

The partitioning logic extends exactly as you would hope. Total SS splits into SS_A, SS_B, SS_AxB and SS_error, each with its own df, each mean square divided by MS_error to give its own F. The efficiency gain is real: testing two factors in one study takes far fewer participants than two separate studies, and only a factorial design can detect an interaction at all.

Interactions, simple effects, and how to read a plot

An interaction means the effect of one factor depends on the level of another. It is usually the most theoretically interesting result in a factorial design, because moderation — 'it works, but only for these people' — is what most psychological theory actually predicts.

Reading the plot: parallel lines mean no interaction; non-parallel lines suggest one; crossing lines suggest a disordinal interaction. But eyeballing is not testing — a small departure from parallel in a low-powered design is noise, and interactions typically need roughly four times the sample size to detect than main effects of the same raw magnitude.

Repeated-measures ANOVA and sphericity

When every participant experiences every condition, the same logic as the paired t-test scales up. Total variability now splits three ways: between-subjects (stable individual differences), the effect of condition, and the residual condition-by-subject interaction which becomes the error term. Because between-person variability is pulled out of the error, the design is substantially more powerful than its between-subjects equivalent.

With only two conditions sphericity cannot be violated (there is only one difference), which is why a paired t-test never needs it. The modern alternative for anything more complex is a mixed-effects model (volume 9), which handles the covariance structure explicitly and copes with missing data instead of deleting the whole participant.

Mixed designs: one between factor, one within

The commonest design in intervention research: participants are randomised to treatment or control (between) and measured before and after (within). It is powerful, it is efficient, and its output confuses people because it has two different error terms.

A word on the alternative. For pre-post data, ANCOVA on the post-test score with the pre-test as covariate is generally more powerful than the mixed ANOVA on change, and it is what most methodologists now recommend for randomised designs. Day 47 explains why — and why the same recommendation is dangerous in non-randomised ones.

ANCOVA, and what you must never control for

Analysis of covariance removes variance associated with a continuous covariate from the error term before testing group differences. Done properly, it increases power substantially and adjusts for chance baseline differences. Done improperly, it manufactures effects that do not exist.

Mechanically, ANCOVA is a regression with both a categorical predictor and a continuous one. The group means are adjusted to what they would be if every group had the same covariate mean, and the error term shrinks by however much the covariate explains. With a covariate correlating .60 with the outcome, error variance drops by 36 per cent — a free increase in power.

MANOVA: several dependent variables at once

When a study has several conceptually related outcomes — the five subscales of a wellbeing measure, say — running five separate ANOVAs inflates the familywise error rate and ignores the correlations among the outcomes. MANOVA tests them jointly: does the group membership predict the combination of dependent variables?

It works by forming the linear combination of the DVs that maximally separates the groups, and testing that. Four multivariate test statistics appear in output — Wilks' lambda (most commonly reported), Pillai's trace (most robust to assumption violations), Hotelling's trace and Roy's largest root. Report Pillai's when assumptions are shaky; Wilks' otherwise, and mention which.

Assumptions are the multivariate versions of the familiar ones: multivariate normality, homogeneity of covariance matrices (Box's M, which is notoriously oversensitive), independence, and no severe multicollinearity among the DVs. When the DVs correlate above about .90 you are measuring the same thing twice and should combine them instead.

Chi-square: goodness of fit and independence

When the outcome is a count of cases in categories rather than a score, the comparison is between observed frequencies and the frequencies a hypothesis predicts. Both chi-square tests use the same statistic and differ only in where the expected frequencies come from.

Nonparametric alternatives, and what they really test

Rank-based tests replace scores with their ranks and work with those. They require no distributional assumption about the population, which makes them useful for small samples, ordinal outcomes and severely skewed data. They are not, however, assumption-free — and they do not test the same hypothesis.

And the standard reason to reach for these tests is weaker than it looks. With moderate n the CLT protects the t-test well against non-normality; where rank tests genuinely win is with ordinal outcomes, tiny samples, or extreme outliers that a mean cannot survive. For heavy-tailed continuous data, modern robust methods — trimmed means with bootstrap intervals, day 110 — usually outperform rank tests while keeping an interpretable effect size.

Which test, for which design

The volume in one table. Take your design across the row; the parametric test, its rank-based alternative and the required effect size follow.

Reporting sentences for every test in this volume

Fill the numbers, keep the structure. Every one includes df, exact p, an effect size and an interval — which is what current reporting standards require.

Ten questions before you move on

A mix of computation, choice-of-test and interpretation — the three things group-comparison questions are built from. Answer without looking back.