Now last time we examined the core question of ‘what is a statistical test asking’ with the answer ‘what’s the likelihood that the results I’ve got from these two groups are different only because of the play of chance?’
Now depending on what sort of data you have will depend on what ‘the results from these two groups’ looks like. It might be that the data are continuous and Normal (so we need parametric tests), or continuous and non-Normal, or categorical (so we need non-parametric tests).
The choice of test, usually, falls out like this for continuous variables:
Repeated tests in same individuals Tests in different individuals Testing >2 groups
Parametric paired t-test (unpaired) t-test ANOVA
Non-Parametric Wilcoxon rank-sum test Mann-Whitney U test Kruskal-Wallis
With these tests you are essentially asking ‘how likely is it that the middleish value of the groups is different only because of the play of chance?’
For categorical variables you’ll be wanting the chi-squared test. (Note; χ2 is not just for squares – 2×2 tables – but can be 3×2, 4×6 or whatever.) What you’re asking here is “how likely is it that the proportion of items in each column, if we look at each row in turn, different only because of the play of chance?’
And, basically, that’s that.