Skip to main content

Reliability — Previous Year Questions

📊 Topic Overview​

MetricValue
Papers Appeared12 of 28 (42.9%)
Priority🔴 HIGH — dominates June papers
Section DistributionSec A: 7
Most RepeatedMethods of estimating reliability (10 times)
Session PatternHeavily June
ProbabilityVery High in June

🎯 Must-Know Concepts​

  1. Definition of reliability and its importance
  2. Methods of estimating reliability: test-retest, parallel forms, split-half, internal consistency (Cronbach's alpha, KR-20)
  3. Inter-rater reliability
  4. Relationship between reliability and validity
  5. Factors affecting reliability

🔥 Most Repeated Question: Methods of Estimating Reliability (10 appearances)​

Answer (Section A — ~450 words):​

Reliability refers to the consistency, stability, and repeatability of a measurement. A reliable measure yields the same result on repeated occasions, under the same conditions, when the true score has not changed. Reliability is necessary but not sufficient for validity — a measure can be reliably wrong.

Methods of Estimating Reliability:

1. Test-Retest Reliability (Coefficient of Stability) The same test is administered to the same group on two separate occasions. The correlation coefficient between the two sets of scores is the reliability estimate.

Advantages: Direct measure of temporal stability. Limitations: Carry-over effects (memory of previous answers); practice effects; scores may genuinely change if the interval is long; requires two administrations.

Appropriate when: Measuring stable traits (intelligence, personality) with a moderate time interval (2–4 weeks).

2. Parallel Forms Reliability (Coefficient of Equivalence) Two equivalent versions of the test (same content, difficulty, format) are administered to the same group, either simultaneously or with an interval. Correlation between the two forms is the reliability estimate.

Advantages: Eliminates carry-over effects; provides two usable test forms. Limitations: Difficult and expensive to construct genuinely parallel forms; still susceptible to practice effects if given simultaneously.

3. Split-Half Reliability (Internal Consistency) The test is administered once and split into two halves (odd/even items). The correlation between the two halves is corrected for test length using the Spearman-Brown prophecy formula: r_corrected = 2r / (1+r).

Advantages: Single administration; no carry-over effects. Limitations: Different splits produce different estimates; assumes both halves measure the same construct.

4. Internal Consistency: Cronbach's Alpha (α) Generalisation of split-half reliability that considers all possible split-half combinations. Ranges from 0 to 1; α ≥ 0.7 is conventionally acceptable. Equivalent to the average of all split-half correlations.

KR-20 (Kuder-Richardson Formula 20): Special case of Cronbach's α for dichotomously scored items (correct/incorrect).

5. Inter-Rater (Inter-Observer) Reliability Used when scoring involves judgement. Two or more raters independently score the same responses; reliability is the correlation or agreement coefficient (Cohen's Kappa for categorical ratings) between their scores.

Important when: Using observational methods, projective tests, essay scoring, or coding qualitative data.

Factors Affecting Reliability: Test length (longer tests are more reliable — Spearman-Brown); item difficulty (very easy or very hard items reduce variance and reliability); sample homogeneity; test administration conditions; time interval (for test-retest).

Relationship Between Reliability and Validity: A test cannot be valid without being reliable. However, a reliable test need not be valid (a scale that consistently measures weight is reliable but not a valid measure of intelligence). The reliability of a test sets an upper bound on its validity.

🧠 Memory Aids​

Methods (TPSIA): Test-retest, Parallel forms, Split-half, Internal consistency (Cronbach), Inter-rater

Spearman-Brown formula: r_corrected = 2r / (1 + r) — "double it, divide by one more"

📚 Related Content​

✅ Self-Assessment​

  1. What is the Spearman-Brown formula used for and how is it applied?
  2. How does Cronbach's alpha relate to split-half reliability?
  3. When is inter-rater reliability particularly important?
  4. What factors affect the reliability of a test?
  5. Can a test be valid without being reliable? Can it be reliable without being valid?

💡 Exam Tips​

  • Section A (June staple): Define reliability → 5 methods (each with procedure, formula where relevant, advantages, limitations) → Factors → Reliability-validity relationship.
  • Cronbach's alpha with the formula (or at least the concept) is a mark-earner.
  • Always state: reliability is necessary but not sufficient for validity.