By Ruben Geert van den Berg on March 28, 2017 under Other Essential Basics.

Simple Overview Statistical Comparison Tests

The vast majority of statistical tests really just

Now, if we add to this comparison

then choosing the appropriate statistical test suddenly becomes remarkably easy.The main limitation here is that correlations and association measures don't fit into this scheme. We'll soon come up with a similar overview for those.
The overviews below also nicely visualize how basic tests relate to each other. Click the links in the tables for a simple visualization and some key points for each test.

A. Comparing Groups of Cases

Outcome Variable 1 group of cases 2 groups of cases 3(+) groups of cases
Dichotomous Z-Test for One Proportion or
Binomial Test
Z-Test for 2 Independent Proportions or
Chi-Square Independence Test
Chi-Square Independence Test
Nominal Chi-Square Goodness-of-Fit Test Chi-Square Independence Test Chi-Square Independence Test
Ordinal Sign Test for One Median Mann-Whitney Test or
Median Test for 2(+) Independent Medians
Kruskal-Wallis test
Metric (“scale”) One Sample T-Test Independent Samples T-Test One-Way ANOVA

B. Comparing Variables

Outcome Variable(s) 1 variable 2 variables 3(+) variables
Dichotomous Z-Test for One Proportion or
Binomial Test
McNemar Test Cochran Q test
Nominal Chi-Square Goodness-of-Fit Test (None) (None)
Ordinal Sign Test for One Median Wilcoxon Signed-Ranks Test or
Sign Test for 2 Related Medians
Friedman Test
Metric (“scale”) One Sample T-Test Paired Samples T-Test Repeated Measures ANOVA

Z Test for One Proportion

Z-Test One Proportion - What Is It?

Null Hypothesis

The population proportion for some value is equal to some hypothesized proportion, p0.

Assumptions

Where in SPSS?

Not available. However, see below.

Notes

Further Reading

Z-Test and Confidence Interval Proportion Tool

Binomial Test

Binomial Test - What Is It?

Null Hypothesis

The population proportion for some value is equal to some hypothesized proportion, p0.

Assumptions

Where in SPSS?

On recent SPSS versions, we prefer using Analyze SPSS Menu Arrow Nonparametric Tests SPSS Menu Arrow Legacy Dialogs SPSS Menu Arrow Binomial.

Notes

The binomial test has the same null hypothesis but can't come up with a confidence interval for a proportion or a 2-tailed p-value unless p0 = 0.5. On the other hand, the binomial test always yields an exact p-value and does not require any minimum sample size.

Further Reading

Chi-Square Goodness-of-Fit Test

Chi-Square Goodness-of-Fit Test - What Is It?

Null Hypothesis

The population distribution of a categorical variable is identical to some hypothesized distribution.

Assumptions

Where in SPSS?

On recent SPSS versions, we prefer using Analyze SPSS Menu Arrow Nonparametric Tests SPSS Menu Arrow Legacy Dialogs SPSS Menu Arrow Chi-Square.

Notes

Further Reading

SPSS One Sample Chi-Square Test

Sign Test for One Median

Sign Test for One Median - What Is It?

Null Hypothesis

The population median of some variable is equal to a hypothesized value, η0.

Assumptions

Where in SPSS?

Not available. However, you can RECODE the test variable into signs (plus and minus) that indicate if each data value is larger or smaller than η0 and use a binomial test with p0 = 0.5.
Alternatively, add η0 as a new variable (or actually a constant) to your data and use a sign test for 2 related medians.

Further Reading

SPSS Sign Test for One Median - Simple Example

One Sample T-Test

One-Sample T-Test - What Is It?

Null Hypothesis

The population mean of some variable is equal to a hypothesized value, μ0.

Assumptions

Where in SPSS?

The easiest option is Analyze SPSS Menu Arrow Compare Means SPSS Menu Arrow One-Sample T Test.

Further Reading

SPSS One Sample T-Test

Z-Test for 2 Independent Proportions

Z-Test Difference Independent Proportions - What Is It?

Null Hypothesis

The difference between two independent proportions, p1 and p2 is equal to some hypothesized value, δ0. “Independent proportions” refers to proportions of one variable in two (disjoint) populations. Since δ0 is often zero, we'll simplify this to both proportions being equal.

Assumptions

Where in SPSS?

Not available. However, see below.

Further Reading

Z-Test and Confidence Intervals Independent Proportions Tool

Chi-Square Independence Test

Chi-Square Independence Test - What Is It?

Null Hypothesis

Two categorical variables are statistically independent in some population. Or, equivalently, the population distributions of some categorical variable are identical for a number of subpopulations defined by a second categorical variable.

Assumptions

Where in SPSS?

Analyze SPSS Menu Arrow Descriptive Statistics SPSS Menu Arrow Crosstabs SPSS Menu Arrow Statistics SPSS Menu Arrow Chi-Square
Alternatively, for 2 variables named var1 and var2, just type crosstabs var1 by var2/statistics chisq. into your Syntax Editor window and run it.

Notes

Further Reading

Mann-Whitney Test

Mann-Whitney Test - What Is It?

Null Hypothesis

The mean ranks on some variable are equal for two populations. Some textbooks propose that “the population distribution of some variable is identical in 2 populations except for the central tendency”3. We don't like this formulation because “central tendency” is vague (it certainly isn't a mean or a median) and therefore unmeasurable.

Assumptions

Independent and identically distributed variables.

Where in SPSS?

On recent SPSS versions, we prefer using Analyze SPSS Menu Arrow Nonparametric Tests SPSS Menu Arrow Legacy Dialogs SPSS Menu Arrow 2 Independent Samples SPSS Menu Arrow Mann-Whitney U

Notes

Further Reading

SPSS Mann-Whitney Test - Simple Example

Median Test for 2(+) Independent Medians

Median Test Independent Medians - What Is It?

Null Hypothesis

Two or more (disjoint) populations all have identical medians on some variable.

Assumptions

Independent and identically distributed variables (sometimes referred to as “independent observations”).

Where in SPSS?

On recent SPSS versions, we prefer using Analyze SPSS Menu Arrow Nonparametric Tests SPSS Menu Arrow Legacy Dialogs SPSS Menu Arrow K Independent Samples SPSS Menu Arrow Median

Notes

Further Reading

SPSS Median Test for 2 Independent Medians

Independent Samples T-Test

Null Hypothesis

Some variable has identical means in two (disjoint) populations.

Assumptions

Where in SPSS?

The easiest option is Analyze SPSS Menu Arrow Compare Means SPSS Menu Arrow Independent Samples T Test

Notes

Levene Test for Homogeneity

P < 0.05 for Levene’s test so the assumption of equal variances is not met.
We therefore report the second row of the t-test results denoted by “equal variances not assumed”.
Note how df has been adjusted according to the Welch correction.
We'll report something like “a t-test did not show the means to be statistically significantly different, t(29) = 1.55, p = 0.133”.

Further Reading

SPSS Independent Samples T Test

Null Hypothesis

The population proportion for some value is equal to some hypothesized proportion, p0.

Assumptions

Where in SPSS?

Not available. However, see below.

Notes

Further Reading

Z-Test and Confidence Interval Proportion Tool

Kruskal-Wallis Test

Kruskal-Wallis Test - What Is It?

Null Hypothesis

The mean ranks of some variable are all equal over 3 or more (disjoint) subpopulations.1
Other authors propose that the population distributions of some variable are identical over 3 or more (disjoint) subpopulations1 or even that the medians are identical over subpopulations.3 We feel the first formulation is preferable but opinions differ.

Assumptions

Independent and identically distributed variables (or, less precisely, independent observations).

Our preferred dialog is under Analyze SPSS Menu Arrow Nonparametric Tests SPSS Menu Arrow Legacy Dialogs SPSS Menu Arrow K Independent Samples SPSS Menu Arrow Kruskall-Wallis H

Notes

Further Reading

SPSS Kruskal-Wallis Test - Simple Tutorial with Example

One-Way ANOVA

One-Way ANOVA - What Is It?

Null Hypothesis

The population means for some variable are all equal over 3 or more (disjoint) subpopulations.

Assumptions

Where in SPSS?

SPSS offers many way for running a one-way ANOVA. For a very basic test, try Analyze SPSS Menu Arrow Compare Means SPSS Menu Arrow One-Way ANOVA
If you'd like some more detailed output, including partial eta squared (a measure for effect size), go for Analyze SPSS Menu Arrow General Linear Model SPSS Menu Arrow Univariate

Notes

Further Reading

McNemar Test

McNemar Test - What Is It?

Null Hypothesis

The population proportions with which some value occurs in two variables are equal.

Assumptions

Independent and identically distributed variables or, less precisely, independent observations.

Where in SPSS?

We prefer using the dialog under Analyze SPSS Menu Arrow Nonparametric Tests SPSS Menu Arrow Legacy Dialogs SPSS Menu Arrow 2 Related Samples SPSS Menu Arrow McNemar

Further Reading

SPSS McNemar Test

Wilcoxon Signed-Ranks Test

Wilcoxon Signed-Ranks Test - What Is It?

Null Hypothesis

Two variables are identically distributed in some population.3 Or, alternatively, the population median of difference scores is zero.1

Assumptions

Independent and identically distributed variables.

Where in SPSS?

We prefer using the dialog under Analyze SPSS Menu Arrow Nonparametric Tests SPSS Menu Arrow Legacy Dialogs SPSS Menu Arrow 2 Related Samples SPSS Menu Arrow Wilcoxon

Notes

Further Reading

SPSS Wilcoxon Signed-Ranks Test - Simple Example

Sign Test 2 Related Medians - What Is It?

Null Hypothesis

The population medians for two variables are equal.

Assumptions

Independent and identically distributed variables.

Where in SPSS?

The simplest option is Analyze SPSS Menu Arrow Nonparametric Tests SPSS Menu Arrow Legacy Dialogs SPSS Menu Arrow 2 Related Samples SPSS Menu Arrow Sign

Notes

Like all tests on medians, this test is not very powerful. It only takes into account if difference scores are larger or smaller than zero but it ignores how much larger or smaller. Therefore, the Wilcoxon signed-ranks test is usually a better alternative for the sign test.

Further Reading

SPSS Sign Test for Two Medians - Simple Example

Paired Samples T-Test

Paired-Samples T-Test - What Is It?

Null Hypothesis

Two variables have equal population means.

Assumptions

Where in SPSS?

The standard approach is Analyze SPSS Menu Arrow Compare Means SPSS Menu Arrow Paired-Samples T Test

For sample sizes < 25 or so, perhaps COMPUTE difference scores manually and first run a histogram on them to check if the normality assumption has been satisfied. If you'd like to test many variables, speed things up with DO REPEAT. You can now test if each variable holding difference scores has a population mean of zero with Analyze SPSS Menu Arrow Compare Means SPSS Menu Arrow One-Sample T Test

Notes

Further Reading

SPSS Paired Samples T Test

Cochran’s Q Test

Cochran Q Test - What Is It?

Null Hypothesis

The population proportions with which some value occurs in 3 or more variables are all equal.7

Assumptions

Independent and identically distributed variables.

Where in SPSS?

Our preferred approach is Analyze SPSS Menu Arrow Nonparametric Tests SPSS Menu Arrow Legacy Dialogs SPSS Menu Arrow K Related Samples SPSS Menu Arrow Cochran’s Q

Further Reading

SPSS Cochran Q Test

Friedman Test

Friedman Test - What Is It?

Null Hypothesis

Three or more variables have identical population distributions. However, the Friedman test mainly detects if the scores of all test variables are equally large. It won't detect other differences such as unequal standard deviations or skewnesses over variables.

Assumptions

Independent and identically distributed variables.

Where in SPSS?

Our preferred approach is Analyze SPSS Menu Arrow Nonparametric Tests SPSS Menu Arrow Legacy Dialogs SPSS Menu Arrow K Related Samples SPSS Menu Arrow Friedman

Notes

Further Reading

SPSS Friedman Test - Simple Example

Repeated Measures ANOVA

Repeated-Measures ANOVA - What Is It?

Null Hypothesis

The population means of 3 or more variables are all equal.

Assumptions

Where in SPSS?

Analyze SPSS Menu Arrow General Linear Model SPSS Menu Arrow Repeated Measures
Repeated measures ANOVA is only available if you have SPSS’ advanced statistics module installed.

Notes

Further Reading

SPSS Repeated Measures ANOVA

References

  1. Agresti, A. & Franklin, C. (2014). Statistics. The Art & Science of Learning from Data. Essex: Pearson Education Limited.
  2. Field. Discovering Statistics with IBM SPSS Newbury Park, CA: Sage.
  3. Howell, D.C. Statistical Methods for Psychology (5th ed.). Pacific Grove CA: Duxbury.
  4. Siegel, S. & Castellan, N.J. (1989). Nonparametric Statistics for the Behavioral Sciences (2nd ed.). Singapore: McGraw-Hill.
  5. Van den Brink, W.P. & Koele, P. (1998). Statistiek, deel 2 [Statistics, part 2]. Amsterdam: Boom.
  6. Van den Brink, W.P. & Koele, P. (2002). Statistiek, deel 3 [Statistics, part 3]. Amsterdam: Boom.
  7. Wijnen, K., Janssens, W., De Pelsmacker, P. & Van Kenhove, P. (2002). Marktonderzoek met SPSS: statistische verwerking en interpretatie [Market Research with SPSS: statistical processing and interpretation]. Leuven: Garant Uitgevers.

Comment on this Tutorial

*Required field. Your comment will show up after approval from a moderator.

This Tutorial has 6 Comments