Statistical Testing
High-Yield Summary
- The null hypothesis (H0) assumes no effect; the alternative hypothesis (Ha) proposes one exists.
- A p-value below the significance level (α, usually 0.05) leads to rejecting H0 as statistically significant.
- Type I error = false positive (rejecting a true H0); Type II error = false negative (failing to reject a false H0).
- Statistical power (raised by larger sample/effect size) reduces the risk of Type II errors.
- A confidence interval that excludes the null value indicates statistical significance.
Hypothesis-Testing Decision Framework
| Decision | If H0 is true / if H0 is false |
|---|---|
| Reject H0 | Type I error (false positive) / Correct rejection |
| Fail to reject H0 | Correct acceptance / Type II error (false negative) |
Key Terms
- Null hypothesis (H0)
- Statement of no effect/difference — any observed variation is due to chance.
- Alternative hypothesis (Ha)
- Proposes an effect/difference exists; can be directional or non-directional.
- p-value
- Probability of observing the results (or more extreme) if H0 were true.
- Significance level (α)
- Predefined threshold (commonly 0.05) the p-value is compared against.
- Statistical power
- Probability of correctly rejecting H0 when it's actually false.
- Confidence interval (CI)
- Range expected to contain the true population parameter; excludes the null value → significant.
Common MCAT Trap
- Type I (α) and Type II (β) errors are easy to swap — Type I is a false positive (rejecting a true H0); Type II is a false negative (missing a real effect).
- A low p-value tells you the result is unlikely under H0 — it does NOT by itself tell you the effect is practically meaningful (that's practical significance, a separate concept).
Quick Recall
A study concludes a drug works when it actually doesn't. Which error is this?
p-value = 0.03, α = 0.05. Reject or fail to reject H0?