Research in the Real World
High-Yield Summary
- A population's measurements are parameters; a representative sample's measurements are statistics used to estimate them.
- Generalizability depends on internal validity (did the variable really cause the outcome, within the study?) and external validity (do results apply beyond the study?).
- Statistical significance (unlikely due to chance, e.g., p < 0.05) is not the same as clinical significance (a meaningful real-world effect) — an intervention needs both to truly matter.
Population vs. Sample
| Population | Sample |
|---|---|
| All individuals sharing a set of characteristics (e.g., all adults with diabetes in a country) | A smaller, representative subset (e.g., a study group of 500 diabetic adults) |
| Its measurements are called parameters | Its measurements are called statistics — used to estimate the population's parameters |
Internal vs. External Validity
| Internal validity | External validity |
|---|---|
| Did the independent variable actually cause the outcome, within this study? | Do these results apply beyond this study's sample (generalizability)? |
| Requires rigorous control of variables, careful design | Requires a representative sample and diverse population characteristics |
Statistical vs. Clinical Significance
| Statistical significance | Clinical significance |
|---|---|
| Is this result likely more than random chance (e.g., p < 0.05)? | Does this result meaningfully improve patient outcomes? |
| Limitation: doesn't guarantee the effect is large enough to matter | Limitation: a study can be underpowered to detect it even if the real-world effect exists |
Common MCAT Trap
- A statistically significant result (low p-value) is not automatically meaningful in practice — a tiny blood-sugar drop can be statistically significant yet clinically irrelevant.
- Internal validity is about causality WITHIN the study; external validity is about applying findings OUTSIDE it — a study can have one without the other.
Quick Recall
A drug lowers blood sugar with p < 0.05, but the drop is too small to change patient quality of life. Statistically significant, clinically significant, both, or neither?
A study only enrolls participants from one city. Which type of validity is most at risk?