Skip to main content

Non-Parametric Tests: Chi-Square, Median, Mann-Whitney U, Sign Test, and Wilcoxon

Introduction​

A researcher wants to know whether the distribution of anxiety levels (categorised as low/medium/high) differs between urban and rural school students. Her data are counts in categories β€” not continuous measurements. She cannot use a t-test. She needs a non-parametric test.

Non-parametric tests are the essential toolkit for research situations where parametric assumptions cannot be met: small samples, ordinal or categorical data, non-normal distributions. They are not fallback methods for when the "real" tests fail β€” they are the appropriate tests for a large proportion of real psychological research, particularly in India where many studies use Likert scales, ranking tasks, or work with small clinical samples.

This unit covers five key non-parametric tests across two contexts: independent samples (chi-square, median test, Mann-Whitney U) and related/dependent samples (sign test, Wilcoxon matched-pairs).

πŸ“„ Source PDF: Block-1/Unit-1.pdf
πŸ“š MPC-006 Statistics in Psychology β€” Block 1: Introduction to Statistics

Overview: Which Non-Parametric Test to Use?​

Tests for Independent Samples​

1. The Chi-Square (χ²) Test​

Purpose: Tests whether an observed distribution of categorical (counted) data differs significantly from an expected distribution. Tests for independence between two categorical variables.

Data type: Nominal (categorical, counted data β€” not measured)

Key principle: Chi-square evaluates the probability that an observed relationship between categorical variables results from chance. It is not a measure of the strength of the relationship β€” only of whether a relationship exists.

Formula:

χ² = Ξ£ [(f_o - f_e)Β² / f_e]

Where:

  • f_o = Observed frequency (actually counted in data)
  • f_e = Expected frequency (what we'd expect if there were no relationship)
  • Ξ£ = Sum across all cells

Assumptions:

  • Data are discrete counts (not continuous measurements)
  • Observations are randomly sampled and independent
  • Expected frequencies in each cell should be β‰₯ 5 (rule of thumb)

Example β€” Chi-square test of independence:

A researcher studies whether treatment preference (medication/therapy/both) differs between male and female patients:

MedicationTherapyBothTotal
Male30201060
Female15351060
Total455520120

Calculate expected frequencies (e.g., E for Male-Medication = 60 Γ— 45/120 = 22.5), then compute χ² = Ξ£[(O-E)Β²/E] and compare to critical value from chi-square table at appropriate degrees of freedom.

Example from the PDF: Analysing FM Radio programming β€” counting how many programs mention antiterrorism topics and testing whether the observed frequency differs from expected (chance) distribution.

2. The Median Test​

Purpose: Tests whether two independent samples have been drawn from populations with the same median. Tests for differences in central tendency.

Data type: Ordinal (or data that cannot meet t-test assumptions)

Procedure:

  1. Calculate the combined median of all scores from both samples together
  2. Dichotomise both sets of scores at the combined median: count how many fall above and below the median in each sample
  3. Create a 2Γ—2 contingency table:
Sample 1Sample 2
Above combined medianAB
Below combined medianCD
  1. Apply chi-square to the 2Γ—2 table

When to use: When the measurement for both samples is expressed in an ordinal scale and parametric t-test assumptions cannot be met.

Limitation: Less powerful than Mann-Whitney U because it only uses whether scores are above or below the median, discarding information about how far above or below.

3. The Mann-Whitney U Test​

Purpose: Tests whether two independent samples come from the same population. It is the most powerful alternative to the t-test when parametric assumptions cannot be met.

Data type: Ordinal scale or continuous data that violates normality

Key advantage over Median test: Uses the full rank-order information of both samples, not just above/below the median β€” therefore more powerful.

Procedure:

  1. Combine all scores from both samples and rank them from lowest to highest
  2. Sum the ranks for each group separately (W₁ and Wβ‚‚)
  3. Compute U using the formula:

U₁ = N₁Nβ‚‚ + N₁(N₁+1)/2 - W₁

Uβ‚‚ = N₁Nβ‚‚ + N₁(N₁+1)/2 - Wβ‚‚

  1. The smaller of U₁ and Uβ‚‚ is the test statistic
  2. Compare to critical U values in Mann-Whitney tables

Note: U₁ + Uβ‚‚ = N₁ Γ— Nβ‚‚ (useful check)

Example: Comparing coping strategy scores (rated on an ordinal scale 1–20) between students who received counselling (N=12) and those who didn't (N=11), when normality cannot be assumed with these small samples.

4. The Sign Test​

Purpose: The simplest non-parametric significance test. Used when quantitative measurement is impossible but paired comparisons can determine which of two conditions produces superior performance.

Data type: Only requires direction of difference (+/-) β€” not magnitude

Procedure:

  1. For each matched pair, determine whether condition A is better (+) or worse (-) than condition B
  2. Ties receive no sign and are excluded
  3. Count the number of pluses and minuses
  4. Under the null hypothesis, pluses and minuses should be equally frequent (50/50)
  5. Use the binomial distribution (or a z-approximation for large N) to test significance

Assumption: Only that the variable has a continuous underlying distribution β€” no normality or equal intervals required.

When to use:

  • Qualitative comparisons possible but measurement difficult or inconvenient
  • Ordinal data where only "better/worse" is known
  • Single sample measured under two conditions

Example: 15 pairs of therapists are rated on client outcome. For each pair (matched on experience level), we know which therapist had better outcomes but not by how much. Sign test uses this direction information.

Limitation: Ignores the size of differences β€” less powerful than Wilcoxon.

5. The Wilcoxon Matched-Pairs Signed-Ranks Test​

Purpose: A more powerful test than the sign test because it considers not only the direction but also the magnitude of differences within matched pairs.

Data type: At least ordinal, with meaningful differences between scores

Key principle: Uses both the sign (direction) and rank of differences β€” therefore uses more information than the sign test.

Procedure:

  1. For each matched pair, compute the difference (D = Score₁ - Scoreβ‚‚)
  2. Rank the absolute values of differences (ignoring sign)
  3. Assign the sign of the original difference to each rank
  4. Sum the positive ranks (W+) and negative ranks (W-)
  5. The smaller of W+ or W- is the test statistic T
  6. Compare T to critical values in Wilcoxon tables

Assumptions:

  • Dependent groups β€” matched pairs or repeated measures
  • Variable has a continuous distribution
  • Differences can be ranked (ordinal measurement of differences)
  • Not applicable to independent groups

Null hypothesis: The direction and magnitude of pair differences are approximately equal in both directions (no systematic tendency one way).

When to use instead of sign test:

  • When you can determine not just which condition is better, but how much better
  • When differences are meaningful and can be ranked
  • When you want more statistical power than the sign test offers

Example: Pre/post anxiety scores for 12 patients in a mindfulness programme. We know both the direction (decrease/increase) and magnitude (how many points) of change β€” Wilcoxon uses both.

Advantages of Non-Parametric Tests: Summary​

AdvantageExplanation
Small samplesValid even with N = 5 or 6
No distribution assumptionWorks when population is non-normal
Ordinal/nominal dataAppropriate for rank data, categorises
Mixed populationsHandles samples from different populations
Ease of computationSimpler formulas and procedures
Direct interpretationResults often more intuitively understandable
RobustnessValid even when data have outliers or skew

Disadvantages of Non-Parametric Tests​

DisadvantageExplanation
Lower powerLess efficient than parametric when assumptions are met
Power-efficiency costNon-parametric test with 90% efficiency needs 10% more cases
Table availabilityCritical value tables scattered across sources
Less precisionDiscards information by converting to ranks

Parametric vs Non-Parametric: Test Equivalents​

Research SituationParametric TestNon-Parametric Equivalent
Two independent groupsIndependent t-testMann-Whitney U / Median test
Two related groupsPaired t-testWilcoxon / Sign test
Three+ independent groupsOne-way ANOVAKruskal-Wallis H
CorrelationPearson's rSpearman's ρ
Categorical associationβ€”Chi-square test

🌐 External Resources​

Wikipedia​

Educational Videos​

Research Papers​

Calculators and Tools​

🧠 Memory Aid: "CMMWS" β€” The Five Non-Parametric Tests​

For independent samples (unrelated groups):

C β€” Chi-square: Counts/categorical data
M β€” Median test: Above/below combined median
M β€” Mann-Whitney U: Full ranked data, most powerful

For dependent/related samples (matched/repeated):

W β€” Wilcoxon: Direction + magnitude of differences
S β€” Sign test: Direction only (+/-)

Power ranking (low β†’ high): Sign test < Median test < Wilcoxon < Mann-Whitney β‰ˆ parametric equivalents

βœ… Self-Assessment Questions​

Level 1 β€” Recall​

  1. What type of data does the chi-square test require? What does it test for?
  2. What is the key difference between the sign test and the Wilcoxon matched-pairs signed-ranks test?

Level 2 β€” Understanding​

  1. Why is the Mann-Whitney U test preferred over the Median test for independent samples? What additional information does it use?
  2. A researcher has 10 matched pairs. For 7 pairs, the experimental condition produced better scores; for 3 pairs, the control was better. How would the Sign Test approach this? What additional data would the Wilcoxon test require?

Level 3 β€” Application & Analysis​

  1. A researcher in India studies whether type of school (government/private/aided) is associated with level of academic stress (low/medium/high). She has data from 90 students (30 per school type). (a) Which non-parametric test is appropriate? (b) What would the contingency table look like? (c) How many degrees of freedom does the chi-square have? (d) If χ² = 12.4, is this significant at p = .05 (critical value = 9.49)?

Summary​

Non-parametric tests provide valid, powerful methods for situations where parametric assumptions cannot be met. The five key tests covered here serve distinct purposes: chi-square for categorical data associations, median test for simple central tendency comparison, Mann-Whitney U for the most powerful comparison of two independent ranked samples, sign test for the simplest case of paired direction comparison, and Wilcoxon matched-pairs signed-ranks for paired comparisons that incorporate both direction and magnitude. Together with their parametric equivalents, these tests form the complete toolkit for inferential statistics in psychological research. The choice between them should always be driven by the data's properties and the research question β€” not by habit or convenience.

Previous ← Parametric Tests: t, F, and Pearson's r