Study design and the hierarchy of evidence
Recognise every major study design from a two-line abstract, know which question each design answers best, and place it in the hierarchy of evidence, GRADE, NICE and ESC.
Reviewed and kept current
Last editorial review: 28 September 2026 · Next scheduled refresh: March 2027.
Clinical vignette
Begin with the patient. Read the case, weigh the options, then work through the unit.
Mrs A, aged 68, has taken a proton pump inhibitor for several years. She brings a newspaper article to her review. It describes a study that used QResearch, a database of GP records. The researchers identified 400,000 adults prescribed a PPI between 2010 and 2015 and 400,000 matched non-users. They compared the incidence of hip fracture up to 2025.
Mrs A has two questions. Does this study prove that her tablet will cause a hip fracture? Should she stop it today?
Before you answer, you need to know what kind of study this was, and what that kind of study can show.
Options to weigh
- Tell her that the study proves the tablet causes hip fracture.
- Explain that nobody allocated the tablet: it was prescribed in routine care, so users and non-users may differ in other ways.
- Explain where this design sits in the hierarchy of evidence, and what stronger evidence would look like.
- Advise her to stop the tablet today on the strength of the article.
- Ask why she takes the tablet, and weigh the article against that reason before she decides.
Reflect
How would you explain what this kind of study can and cannot show, so that Mrs A can decide about her tablet, and what would you be careful not to say?
What this unit covers
Pocket-sized and self-contained: three AKT-style questions, the answers explained, an expanded discussion with hyperlinked guidance, a decision algorithm and a certificate.
By the end of this unit you will be able to
Five steps, in order
Answer first, then learn. Each section unlocks as you progress, and your progress is saved in this browser.
1Three single best answer questions
Commit to an answer for all three single best answer questions, then submit. Sections 2–4 open on submission; your certificate opens at 2/3 or better.
Question 1 of 3
Researchers recruit 2,400 secondary-school pupils aged 13–14. At baseline they record whether each pupil has used e-cigarettes. They follow all pupils for five years and record who starts smoking tobacco cigarettes.
Which study design has been used?
Question 2 of 3
A GP registrar wants to investigate whether occupational exposure to wood dust is associated with sinonasal adenocarcinoma, a very rare tumour. She has access to a regional cancer registry and can recruit matched controls from general practice lists.
Which study design is the most efficient for answering this question?
Question 3 of 3
A new point-of-care troponin test is being evaluated for use in general practice. The investigators want to establish its sensitivity and specificity.
Which design is most appropriate?
2Answers explained
Question 1
Correct answer D. Prospective cohort study
Participants are grouped by exposure (vaping or not) and followed forwards in time to see who develops the outcome (smoking). That is the definition of a prospective cohort study. The investigators did not allocate the exposure, so it is not a trial.
- Case–control: starts with the outcome (smokers vs non-smokers) and looks back for exposure.
- Cross-sectional: exposure and outcome measured at one point in time; no follow-up.
- Ecological: unit of analysis is a population (e.g. countries), not individuals.
Question 2
Correct answer E. Case–control study
For a rare outcome, a cohort would need to follow enormous numbers of people for decades to accumulate enough cases. A case–control study starts with the cases (from the registry), selects comparable controls and compares past exposure. It is quick, cheap and efficient for rare diseases and diseases with long latency.
Its weaknesses — recall bias and difficulty choosing controls — are the price of that efficiency. An RCT would be unethical (you cannot randomise people to a carcinogen).
Question 3
Correct answer D. A cross-sectional study applying both the new test and the reference standard to consecutive patients with chest pain
Diagnostic accuracy is measured by applying the index test and the reference (gold) standard to a representative, consecutive series of the patients in whom the test will actually be used — a cross-sectional design.
- Comparing confirmed MI cases against healthy volunteers (option E) produces spectrum bias and overestimates accuracy — the classic diagnostic case–control trap.
- An RCT (option B) answers a different question: whether using the test improves patient outcomes.
3Expanded discussion, worked examples & guidance
1. The one question that sorts every design
Ask “Did the investigators assign the exposure?” If yes, it is an experimental study (trial). If no, it is observational — then ask which way time runs.
| Design | Starts with… | Direction | Best for | Main measure | Key weakness |
|---|---|---|---|---|---|
| Randomised controlled trial | Participants randomly allocated to intervention | Forwards | Therapy / prevention | RR, ARR, NNT, HR | Cost; generalisability; rare harms missed |
| Cohort (prospective or retrospective) | Exposure status | Forwards (from exposure to outcome) | Prognosis, harm, aetiology, incidence | RR, incidence, HR | Confounding; loss to follow-up; slow for rare outcomes |
| Case–control | Outcome status (cases vs controls) | Backwards | Rare diseases, long latency, outbreaks | Odds ratio only | Recall and selection bias; no incidence |
| Cross-sectional | A population at one time | Snapshot | Prevalence; diagnostic accuracy | Prevalence, sens/spec | Cannot show temporal sequence |
| Ecological | Groups/populations | Either | Hypothesis generation | Correlation | Ecological fallacy |
| Case report / series | Individual patients | Descriptive | New diseases, rare adverse effects | None | No comparison group |
| Qualitative | Experiences, beliefs | — | “Why?” and “how?” questions | Themes | Not generalisable statistically |
Retrospective cohort studies still run from exposure to outcome, but both occurred in the past and are extracted from records (e.g. CPRD or QResearch GP databases). Do not confuse “retrospective” with “case–control”.
2. Match the clinical question to the design
| Question type | Best primary design | Best overall evidence |
|---|---|---|
| Therapy / prevention | RCT | Systematic review of RCTs |
| Diagnosis (accuracy) | Cross-sectional comparison with reference standard | Systematic review of such studies |
| Prognosis | Inception cohort | Systematic review of cohorts |
| Harm / aetiology | Cohort (or case–control if rare) | Systematic review; RCT if ethical |
| Patient experience | Qualitative interviews / focus groups | Qualitative evidence synthesis |
3. Worked examples — name that design
Worked example 1
“Using QResearch, we identified 400,000 adults prescribed a PPI between 2010 and 2015 and 400,000 matched non-users, and compared the incidence of hip fracture up to 2025.”
- Exposure assigned by investigators? No — prescribed in routine care → observational.
- Grouped by exposure (PPI use) and outcome counted afterwards → cohort.
- Data already existed → retrospective cohort.
Answer. Retrospective cohort study; the measure will be RR or HR for hip fracture.
Worked example 2
“We compared 150 children with Kawasaki disease with 450 age-matched children from the same practices, asking parents about carpet shampoo use in the preceding three months.”
- Grouped by outcome (Kawasaki vs not) → case–control.
- Exposure established by parental recall → high risk of recall bias (parents of sick children search their memory harder).
Answer. Case–control study; report an odds ratio.
Worked example 3
“On 1 March 2026 we searched every record in 40 Hertfordshire practices for a coded diagnosis of atrial fibrillation.”
Answer. Cross-sectional study measuring point prevalence.
4. Hierarchy of evidence and certainty
The traditional pyramid, from strongest to weakest for questions of therapy: systematic review/meta-analysis of RCTs → individual RCT → cohort → case–control → case series → expert opinion. It is a guide, not a law: a large, well-conducted cohort can be more informative than a small, biased RCT.
GRADE rates the certainty of a body of evidence as high, moderate, low or very low. RCT evidence starts high and is downgraded for risk of bias, inconsistency, indirectness, imprecision or publication bias; observational evidence starts low and can be upgraded (e.g. large effect, dose–response). NICE uses GRADE-informed methods and words recommendations as “offer” (strong) or “consider” (weaker).
ESC guidelines give each recommendation a Class (I = is recommended; IIa = should be considered; IIb = may be considered; III = is not recommended) and a Level of evidence (A = multiple RCTs or meta-analyses; B = a single RCT or large non-randomised studies; C = consensus/small studies).
5. Reliability, validity and generalisability
- Reliability — repeatability: the same result on repeated measurement or between observers (inter-rater agreement, often reported as Cohen’s kappa).
- Internal validity — whether the study measured what it set out to measure without bias in its own participants.
- External validity / generalisability — whether the results apply to your patients (e.g. trial excluded over-75s or people with CKD).
A measure can be reliable but not valid (a scale that always reads 2 kg heavy); it cannot be valid without being reliable.
References and further reading
Guidance checked at the last editorial review (28 September 2026). Always use the current version.
- RCGP — MRCGP Applied Knowledge Test (AKT)
- RCGP — Data interpretation topics in the AKT (PDF)
- RCGP GP Curriculum 2025 — Evidence in practice topic guide
- Oxford Centre for Evidence-Based Medicine — Levels of Evidence
- GRADE Working Group
- NICE — Developing NICE guidelines: the manual (PMG20)
- ESC — Clinical Practice Guidelines: classes of recommendations and levels of evidence
4Decision algorithm
Which study design am I looking at?
Start
Read the methods: who decided who got the exposure?
Decision
Did the investigators assign the intervention/exposure?
Decision
Was allocation random (with concealment)?
Decision
Observational: is there a comparison group?
Decision
Were participants selected by outcome (cases vs controls)?
Decision
Were participants selected by exposure and followed to outcome?
Outcome
Finally: is the unit of analysis a population rather than individuals? If so → ecological study (beware the ecological fallacy).
Revision summary; apply clinical judgement and local pathways.
5Certificate of completion
You have met the pass mark. Enter your details to issue your personalised certificate of completion, then save it as a PDF for your appraisal or revalidation portfolio.
Your certificate is issued immediately as a PDF. Your details stay in your browser and are not sent to MD Acumen.
Your completion and score have also been recorded by your organisation’s learning platform.
MRCGP AKT · CAS Programmes · For institutions · Legal & regulatory · enquiry@mdacumen.com
Educational content for healthcare professionals; apply clinical judgement and current local guidance. MRCGP is a registered trademark of the Royal College of General Practitioners. These units are independent, curriculum-aligned revision resources and are not endorsed by the RCGP.
© 2026 MD Acumen Ltd · Registered in England and Wales · Company no. 16538952 · ICO registration ZC105897