Bias, confounding and appraising a trial
Name the bias from the scenario, tell confounding from bias, and appraise a randomised trial in five questions — randomisation, concealment, blinding, follow-up and intention-to-treat.
Reviewed and kept current
Last editorial review: 28 September 2026 · Next scheduled refresh: March 2027.
Clinical vignette
Begin with the patient. Read the case, weigh the options, then work through the unit.
A registrar runs a search of the practice records. It shows that patients prescribed a new hypnotic have more than twice the fracture rate of non-users. The registrar proposes, at the practice meeting, that the hypnotic be stopped for everyone.
That afternoon Mr B, aged 79, asks for a repeat prescription of the same hypnotic. He has heard about the search from the practice pharmacist. He asks whether the tablet has doubled his risk of breaking a bone.
The search compared all users with all non-users. It did not look at the age of either group.
Options to weigh
- Tell Mr B that the hypnotic doubles his risk of fracture.
- Support stopping the hypnotic for all patients on the strength of the crude comparison.
- Ask whether users and non-users differ in age before drawing any conclusion.
- Split the data by age group and compare fracture rates within each group before advising Mr B.
Reflect
What would you tell Mr B about his risk of fracture from this search, and what would you be careful not to say until the comparison is fair?
What this unit covers
Pocket-sized and self-contained: three AKT-style questions, the answers explained, an expanded discussion with hyperlinked guidance, a decision algorithm and a certificate.
By the end of this unit you will be able to
Five steps, in order
Answer first, then learn. Each section unlocks as you progress, and your progress is saved in this browser.
1Three single best answer questions
Commit to an answer for all three single best answer questions, then submit. Sections 2–4 open on submission; your certificate opens at 2/3 or better.
Question 1 of 3
In a trial of a new asthma inhaler, allocation to treatment or placebo is by a list pinned to the practice-nurse office wall. The nurse, who recruits patients, notices that the next allocation is ‘active’ and delays recruiting a patient with poorly controlled asthma until the following slot.
Which type of bias is most likely to result?
Question 2 of 3
A cohort study reports that people who drink more than four cups of coffee a day have a higher incidence of lung cancer. After adjustment for smoking, the association disappears.
What best explains the original association?
Question 3 of 3
An RCT of a weight-management programme randomises 600 adults. 90 participants in the intervention arm and 40 in the control arm stop attending. The investigators analyse all 600 participants in the groups to which they were originally randomised.
What is the main advantage of this approach?
2Answers explained
Question 1
Correct answer B. Selection bias
Knowing the next allocation lets the recruiter influence who enters each arm, so the groups differ systematically at baseline — selection bias. The safeguard is allocation concealment (e.g. central web-based randomisation or sealed opaque sequentially numbered envelopes).
- Allocation concealment happens before allocation and protects the randomisation; blinding happens after allocation and protects against performance and detection bias. They are different things.
- Detection bias arises when outcome assessors know the allocation; attrition bias from unequal drop-out.
Question 2
Correct answer B. Confounding
Smoking is associated with the exposure (heavy coffee drinkers smoke more) and independently causes the outcome, and it is not on the causal pathway between them. It is therefore a confounder. Adjusting for it removes the spurious association.
- Effect modification is a genuine difference in effect between subgroups (e.g. a drug that works in men but not women); it is reported, not removed.
- Reverse causation would mean lung cancer caused coffee drinking.
Question 3
Correct answer A. It preserves the balance of known and unknown confounders created by randomisation
This is intention-to-treat (ITT) analysis. Because every randomised participant is analysed in the arm to which they were allocated, the groups remain comparable (randomisation is preserved) and the result reflects real-world effectiveness, including non-adherence. It tends to be conservative.
- Option C describes per-protocol analysis (efficacy under ideal adherence) — which is vulnerable to attrition bias.
- In non-inferiority trials ITT can make two treatments look artificially similar, so both ITT and per-protocol results should agree before non-inferiority is accepted.
3Expanded discussion, worked examples & guidance
1. Bias — systematic error that pushes results in one direction
Bias is not reduced by making a study bigger; chance (random error) is. The AKT usually gives you a scenario and asks you to name the bias.
| Bias | What happens | Typical scenario | Protection |
|---|---|---|---|
| Selection | Groups differ at baseline because of how people were chosen or allocated | Recruiter knows next allocation; volunteers healthier than population | Random allocation with concealment; representative sampling |
| Performance | Groups receive different care apart from the intervention | Unblinded clinicians give intervention patients extra attention | Blinding of participants and clinicians; standardised protocols |
| Detection (measurement / observer) | Outcomes measured differently between groups | Assessor who knows allocation interprets X-rays | Blinded outcome assessment; objective outcomes |
| Attrition | Unequal or outcome-related drop-out | Side-effects cause drop-outs in one arm | ITT; minimise loss to follow-up; sensitivity analyses |
| Recall | Cases remember past exposure differently from controls | Mothers of babies with anomalies recall medicines more fully | Use records rather than memory |
| Reporting / publication | Positive results more likely to be reported or published | Asymmetric funnel plot | Trial registration; searching grey literature |
| Healthy-worker / volunteer | Participants healthier than the general population | Occupational cohort mortality lower than population | Choose appropriate comparison group |
| Lead-time & length-time | Apparent survival gains from screening | Earlier diagnosis “lengthens” survival without delaying death | Compare mortality, not survival from diagnosis (see Unit 10) |
The Hawthorne effect — people change their behaviour because they know they are being observed — often appears in audit and quality-improvement questions.
2. Confounding
A confounder is associated with the exposure, is an independent cause of the outcome, and is not on the causal pathway. Age and smoking are the commonest confounders in AKT questions.
Analysis stage: stratification, multivariable regression (adjustment).
Worked example — how confounding creates an association
A practice finds that patients prescribed a new hypnotic have more than twice the fracture rate of non-users. When the data are split by age:
| Age group | Fracture rate: users | Fracture rate: non-users | % of users in age group | % of non-users in age group |
|---|---|---|---|---|
| <75 | 1% | 1% | 20% | 80% |
| ≥75 | 5% | 5% | 80% | 20% |
- Within each age stratum the fracture rate is identical in users and non-users.
- Crude rate in users = 0.2×1% + 0.8×5% = 4.2%; in non-users = 0.8×1% + 0.2×5% = 1.8% — an apparent 2.3-fold risk.
- Users are mostly older, and older people fracture more, so the crude association is produced by age.
Answer. Age confounds the association; stratified (age-adjusted) analysis shows no drug effect.
3. Appraising an RCT in five questions
- Randomised properly? Computer-generated sequence, not alternate days or date of birth.
- Allocation concealed? Central randomisation or opaque sealed envelopes.
- Groups similar at baseline? Check Table 1.
- Blinded? Participants, clinicians and outcome assessors. If blinding is impossible (e.g. surgery vs physiotherapy), were outcomes objective and assessors blinded?
- Follow-up complete and analysed by ITT? Losses above about 20%, or unequal losses, threaten validity.
Then ask: were the results important (effect size and confidence interval — Units 5 and 6), and do they apply to my patient (inclusion and exclusion criteria, setting)? Trial reports should follow CONSORT; the Cochrane Risk of Bias tool (RoB 2) formalises the same questions.
- “Double-blind” does not mean allocation was concealed.
- A cluster RCT randomises practices, not patients — useful when the intervention is delivered at practice level, but it needs a larger sample.
- A crossover trial suits stable chronic conditions and short-acting treatments; it needs a wash-out period.
References and further reading
Guidance checked at the last editorial review (28 September 2026). Always use the current version.
4Decision algorithm
Appraising a randomised controlled trial
Start
Is the question clearly focused (PICO: population, intervention, comparator, outcome)?
Decision
Was the allocation sequence truly random?
Decision
Was allocation concealed from recruiters?
Decision
Were participants, clinicians and outcome assessors blinded?
Decision
Was follow-up ≥80%, balanced between arms, and analysed by ITT?
Action
Results: effect size (ARR, NNT, HR) and precision (95% CI). Clinically important?
Outcome
Applicability: were the participants like my patient? Are the benefits worth the harms and costs?
Revision summary; apply clinical judgement and local pathways.
5Certificate of completion
You have met the pass mark. Enter your details to issue your personalised certificate of completion, then save it as a PDF for your appraisal or revalidation portfolio.
Your certificate is issued immediately as a PDF. Your details stay in your browser and are not sent to MD Acumen.
Your completion and score have also been recorded by your organisation’s learning platform.
MRCGP AKT · CAS Programmes · For institutions · Legal & regulatory · enquiry@mdacumen.com
Educational content for healthcare professionals; apply clinical judgement and current local guidance. MRCGP is a registered trademark of the Royal College of General Practitioners. These units are independent, curriculum-aligned revision resources and are not endorsed by the RCGP.
© 2026 MD Acumen Ltd · Registered in England and Wales · Company no. 16538952 · ICO registration ZC105897