07. Plan a transparent evidence synthesis
Lesson
LEARNING OBJECTIVES
Distinguish a systematic review from a preliminary search, plan study selection and extraction, and justify a synthesis approach without automatically pooling incompatible studies.
START WITH A REVIEW PROTOCOL
Define the question and eligibility criteria before examining which results seem attractive. Specify populations, interventions or exposures, comparisons, outcomes, designs, setting and time as appropriate. A qualitative or other review may need a question framework different from an intervention PICO.
Plan information sources, reproducible searches, screening, extraction, design-appropriate risk-of-bias assessment and synthesis. Record amendments with reasons and dates. A convenient collection of papers is not a systematic review merely because it uses a search engine.
SEARCH AND SELECT TRANSPARENTLY
Choose databases and additional sources relevant to the question. Document full strategies, dates and restrictions. Manage duplicate records, then assess reports against criteria with a planned process for independent checks and disagreement resolution. Record reasons for full-text exclusions and track the selection flow.
A study may have multiple reports. Link them so the same participants are not counted as separate studies. Plan how conflicting information across reports or missing details will be handled without choosing whichever source supports a preferred conclusion.
WORKED FICTIONAL EXTRACTION PROBLEM
Three reports concern reminder programmes. One is a randomized clinic study, another is a later report from that same trial, and the third is a cross-sectional comparison with a different outcome definition. Treat the first two as linked reports of one study. Do not average their estimates as independent evidence, and do not pool the cross-sectional association with the trial effect merely because both use the word “attendance.”
Use an extraction form capturing design, population, context, intervention/exposure, comparison, outcome definition and timing, analysis denominator, estimate, uncertainty, adjustment, missingness and relevant bias judgments. Pilot the form and preserve links to source locations.
SYNTHESIZE WHAT CAN BE COMPARED
Meta-analysis is not a compulsory final step. Consider whether study questions, designs, outcomes, measures and assumptions support a meaningful pooled quantity. If pooling is inappropriate or data are incomplete, use a prespecified, transparent structured synthesis with tables and explicit limitations.
Do not count how many studies have p<0.05 and label the majority conclusion “the evidence.” Study precision, direction, magnitude, bias, applicability and missing evidence matter. A well-conducted review also cannot erase serious limitations in the underlying studies.
YOUR TASK
Draft a two-page review protocol and an extraction form for a narrow question. Write eligibility decisions for three hypothetical reports, including two from the same study. State what would make pooling defensible and what would lead you to a structured non-pooled synthesis instead.
SELF-CHECK AND FEEDBACK
Is a paper always the unit of a review? No; distinguish studies and their reports. Does a pooled estimate automatically increase validity? No. Is “narrative synthesis” sufficient description of methods? No; explain how studies were grouped, compared and interpreted.
PORTFOLIO OUTPUT
A review-protocol outline, screening rules, extraction template and justified synthesis plan. This exercise does not constitute a completed systematic review.
FURTHER READING
Cochrane Handbook, Chapters 4, 5 and 12.
https://training.cochrane.org/handbook/current/chapter-04
https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-05
https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-12
EQUATOR reporting guidelines, including PRISMA.
https://www.equator-network.org/reporting-guidelines/
Distinguish a systematic review from a preliminary search, plan study selection and extraction, and justify a synthesis approach without automatically pooling incompatible studies.
START WITH A REVIEW PROTOCOL
Define the question and eligibility criteria before examining which results seem attractive. Specify populations, interventions or exposures, comparisons, outcomes, designs, setting and time as appropriate. A qualitative or other review may need a question framework different from an intervention PICO.
Plan information sources, reproducible searches, screening, extraction, design-appropriate risk-of-bias assessment and synthesis. Record amendments with reasons and dates. A convenient collection of papers is not a systematic review merely because it uses a search engine.
SEARCH AND SELECT TRANSPARENTLY
Choose databases and additional sources relevant to the question. Document full strategies, dates and restrictions. Manage duplicate records, then assess reports against criteria with a planned process for independent checks and disagreement resolution. Record reasons for full-text exclusions and track the selection flow.
A study may have multiple reports. Link them so the same participants are not counted as separate studies. Plan how conflicting information across reports or missing details will be handled without choosing whichever source supports a preferred conclusion.
WORKED FICTIONAL EXTRACTION PROBLEM
Three reports concern reminder programmes. One is a randomized clinic study, another is a later report from that same trial, and the third is a cross-sectional comparison with a different outcome definition. Treat the first two as linked reports of one study. Do not average their estimates as independent evidence, and do not pool the cross-sectional association with the trial effect merely because both use the word “attendance.”
Use an extraction form capturing design, population, context, intervention/exposure, comparison, outcome definition and timing, analysis denominator, estimate, uncertainty, adjustment, missingness and relevant bias judgments. Pilot the form and preserve links to source locations.
SYNTHESIZE WHAT CAN BE COMPARED
Meta-analysis is not a compulsory final step. Consider whether study questions, designs, outcomes, measures and assumptions support a meaningful pooled quantity. If pooling is inappropriate or data are incomplete, use a prespecified, transparent structured synthesis with tables and explicit limitations.
Do not count how many studies have p<0.05 and label the majority conclusion “the evidence.” Study precision, direction, magnitude, bias, applicability and missing evidence matter. A well-conducted review also cannot erase serious limitations in the underlying studies.
YOUR TASK
Draft a two-page review protocol and an extraction form for a narrow question. Write eligibility decisions for three hypothetical reports, including two from the same study. State what would make pooling defensible and what would lead you to a structured non-pooled synthesis instead.
SELF-CHECK AND FEEDBACK
Is a paper always the unit of a review? No; distinguish studies and their reports. Does a pooled estimate automatically increase validity? No. Is “narrative synthesis” sufficient description of methods? No; explain how studies were grouped, compared and interpreted.
PORTFOLIO OUTPUT
A review-protocol outline, screening rules, extraction template and justified synthesis plan. This exercise does not constitute a completed systematic review.
FURTHER READING
Cochrane Handbook, Chapters 4, 5 and 12.
https://training.cochrane.org/handbook/current/chapter-04
https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-05
https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-12
EQUATOR reporting guidelines, including PRISMA.
https://www.equator-network.org/reporting-guidelines/
My study notes
Stored only in this browser. Do not enter patient information.Saved in this browser. No account needed.