Learning · Sources

How to read a study

Also called read a paper · absolute risk · p-hacking · preprint · CONSORT · GRADE

Named sources. May contain inaccuracies or be incomplete. Not medical, legal, financial, or other professional advice.

Read a study as an answer to a particular question. Identify who was studied, what was compared, what was measured and for how long. Then examine the size and uncertainty of the result, how the comparison could have been distorted, and how it fits the wider evidence. A striking headline, a small p-value or a respected journal cannot do those jobs for you.

Find the question behind the headline

You do not need to understand every equation before you can start. First establish what kind of answer this paper can provide.

Rewrite the headline as a question about specific people

When a headline says a treatment works, find the question in the paper itself. For a treatment comparison, PICO means Population, Intervention, Comparator and Outcome: who participated, what they received, what it was compared with and what changed. Add the follow-up period. A study of adults with a particular condition over six months does not automatically answer a question about healthy teenagers over ten years. Cochrane's question-setting guidance also distinguishes estimating how common something is from testing a treatment or understanding people's experiences. Keep the conclusion attached to the question the researchers actually investigated.

Cochrane Handbook, Chapter 3: Defining the criteria for including studies and how they will be grouped for the synthesis

Ask how people came to receive the exposure or treatment

Suppose people who choose an exercise programme later report better health. Exercise may help, but those participants may also differ in income, starting health or access to care. Such a third factor is a confounder when it influences both the exposure and the outcome. An observational study records choices or circumstances; a randomized trial assigns a comparison by chance. Adjustment can address measured differences, while unmeasured differences may remain. Cochrane's guidance on non-randomized studies therefore asks readers to inspect the causal comparison and its assumptions. An association deserves investigation, but its design and measurements limit the conclusion it supports.

Cochrane Handbook, Chapter 24: Including non-randomized studies on intervention effects

Give a percentage a starting risk and a time period

A headline announcing a 50 percent reduction leaves out information needed to understand the result. In an invented example, a fall from 2 events per 100 people to 1 per 100 halves the risk and prevents 1 event per 100. A fall from 50 to 25 per 100 also halves it, but prevents 25 events per 100. These are absolute reductions of 1 and 25 percentage points, respectively. Neither example describes a real treatment. Find the actual groups, event definition and observation period in the paper; if the headline omits them, do not invent them. The same relative change can mean very different practical benefits.

Cochrane Handbook, Chapter 6: Choosing effect measures and computing estimates of effect

Half as many: start with the original count

These invented comparisons each use 100 people per group over the same hypothetical one-year period. One square represents one person in the denominator. Gold marks the comparison-group event count; stripes mark the difference from the starting count. Unfilled squares complete the denominator.

2 in 100 → 1 in 100

1 fewer per 100
2% − 1% = 1 percentage point

Relative decrease:
(2 − 1) ÷ 2 = 50%

50 in 100 → 25 in 100

25 fewer per 100
50% − 25% = 25 percentage points

Relative decrease:
(50 − 25) ÷ 50 = 50%

The same relative decrease describes 1 fewer event per 100 in the first comparison and 25 fewer in the second.

This overlays counts to explain arithmetic; it does not identify particular people whose events were prevented. These are not treatment results or forecasts. A real interpretation still needs the event definition, study design, uncertainty, harms and follow-up period.

Check a preprint's version and later review

A preprint makes a manuscript public before it completes journal peer review; its experiment may already be complete. That can make new work available sooner, but availability does not establish that its methods or conclusions are sound. JAMA's discussion of medical preprints explains both the opportunity and the danger of acting on an early report. Read the methods, identify the version, and look for a later journal article, correction or other evidence on the same question. Journal peer review adds scrutiny; it does not certify truth. A preprint is not, by itself, a reason to change medical practice, and a journal label does not replace reading the study.

Preprints Involving Medical Research—Do the Benefits Outweigh the Challenges?

Inspect how the comparison was made

Separate random assignment, concealed allocation and blinding

Random assignment helps create comparable groups. Allocation concealment prevents someone enrolling participants from knowing the next assignment and influencing who enters which group. Blinding keeps participants, care providers or outcome assessors unaware of assignments after enrolment, where feasible. These safeguards address different problems. CONSORT 2025 asks authors to explain them separately. A paper calling itself randomized still needs an account of how assignment worked, who knew it, and whether departures from the intended treatment could distort the comparison.

CONSORT 2025 statement: updated guideline for reporting randomised trials Cochrane Handbook, Chapter 8: Assessing risk of bias in a randomized trial

Follow participants from assignment to the reported result

Read the flow diagram alongside the results: how many people entered each group, received the intervention, were followed and contributed measurements? Missing outcomes matter especially when their absence is related to how people were doing, and the problem may differ between groups. Cochrane's trial-bias guidance does not treat a fixed percentage of missing data as a universal safety threshold. An intention-to-treat approach aims to analyse people according to their assigned group, but that label cannot recover an outcome that was never measured. Look for reasons for missingness, assumptions and analyses testing those assumptions.

Cochrane Handbook, Chapter 8: Assessing risk of bias in a randomized trial

Distinguish a meaningful outcome from a convenient marker

An outcome is what the study measures. A clinical outcome can concern how someone feels, functions or survives; a surrogate is a marker used to stand in for a clinical benefit. A laboratory value can be useful without proving that people live longer or feel better. FDA guidance distinguishes surrogates supported by validation from those only reasonably likely to predict benefit. Ask what links this particular marker to the outcome you care about, for this use. Also check whether the paper measured that outcome directly or inferred it.

FDA: Surrogate Endpoint Resources for Drug and Biologic Development

Read harms with the same attention as benefits

A short trial with few participants may detect a benefit yet miss an uncommon or delayed harm. Cochrane's adverse-effects guidance explains why other designs, including observational studies, can contribute information that trials do not supply. Check how harms were sought, defined and counted, how long people were observed, and whether both groups had the same opportunities for detection. A report of no observed events is different from evidence that an event cannot occur. Spontaneous reports can identify a concern but, without a suitable denominator, cannot establish its frequency.

Cochrane Handbook, Chapter 19: Adverse effects

Use reporting guidelines to locate information, then assess bias separately

A reporting checklist asks whether readers have the information needed to understand a study. CONSORT 2025 has 30 items for randomized trials; STROBE covers major observational designs, and PRISMA 2020 covers systematic reviews. EQUATOR helps locate the relevant guideline. Reporting completeness and risk of bias are different judgments: a clearly described method can still systematically distort a result, while an omitted detail may leave its risk unclear. STROBE explicitly says its checklist is not a study-quality scoring tool. Find the methods and supplements before treating a filled checklist as reassurance.

EQUATOR Network: Enhancing the QUAlity and Transparency Of health Research EQUATOR Network: CONSORT 2025 Statement CONSORT 2025 statement: updated guideline for reporting randomised trials STROBE: Strengthening the Reporting of Observational Studies in Epidemiology PRISMA 2020 statement Cochrane Handbook, Chapter 7: Considering bias and conflicts of interest among the included studies

Read the size and uncertainty of the result

Translate a result into an amount people can understand

For an event outcome, record the absolute difference as well as the relative effect. In the invented examples above, preventing 1 event per 100 corresponds to a number needed to treat of 100; preventing 25 per 100 corresponds to 4. This calculation is one divided by the absolute risk reduction expressed as a proportion. It describes an average additional benefit over the stated period, compared with the specified alternative. Cochrane cautions that baseline risk, follow-up and uncertainty affect this interpretation. Keep possible harms beside the benefit rather than using one attractive number as a decision rule.

Cochrane Handbook, Chapter 15: Interpreting results and drawing conclusions

Keep the effect measure and its units visible

A risk ratio compares event probabilities; an odds ratio compares odds, which are the probability of an event divided by the probability of no event. They are different measures and can differ substantially when events are common. For a numerical outcome, a mean difference retains the measurement scale, such as points on a symptom questionnaire. Check what direction means improvement and whether that amount is meaningful. A ratio of 1 or a difference of 0 represents no contrast on those respective scales; neither tells you how uncertain the estimate is.

Cochrane Handbook, Chapter 6: Choosing effect measures and computing estimates of effect

Read the interval before declaring an effect present or absent

A confidence interval shows uncertainty around an estimate under the statistical method's assumptions. For a conventional 95 percent interval, the procedure is designed to cover the true value in 95 percent of repeated applications; it does not assign a 95 percent probability to the fixed true value in this particular interval. Look at whether the interval includes effects that would matter, including benefit and harm. An estimate near zero with a wide interval may leave an important effect unresolved. Precision does not protect against a biased comparison.

Cochrane Handbook, Chapter 15: Interpreting results and drawing conclusions Statement of the ASA President's Task Force on Statistical Significance and Replicability

Interpret a p-value in relation to the tested model

A p-value asks how unusual results at least as extreme as those observed would be if the specified statistical model, including its null hypothesis, were correct. It is not the probability that the hypothesis is true, or the probability that chance alone caused the findings. The American Statistical Association's 2016 statement also warns that statistical significance does not measure effect size or importance. A small p-value cannot settle whether the comparison was fair, the measurement useful or the result applicable. A large one does not establish an exact zero effect.

American Statistical Association releases statement on statistical significance and p-values

Ask what the study was capable of detecting

Power is a study's probability of meeting a specified statistical test criterion when an assumed effect really exists. It depends on the effect, variability, sample and test. A small, imprecise study can miss an important effect; a very large one can detect an effect too small to matter. Ioannidis's modelling also connects low power, prior plausibility, bias and repeated testing to the reliability of positive findings. Read the sample-size assumptions and the actual interval, rather than treating the participant count as a universal grade.

Why Most Published Research Findings Are False American Statistical Association releases statement on statistical significance and p-values

Trace what was planned and what became visible

Compare the dated plan with the published outcome

A registration records a study; a protocol describes its design; a statistical analysis plan specifies how results will be analysed. The primary outcome addresses the main planned question; secondary outcomes address additional questions. Check dates and versions rather than assuming every document preceded knowledge of the results. Preregistration helps distinguish planned tests from analyses chosen after seeing data. Changes may be legitimate, but their timing and reasons should be visible. If a paper highlights a secondary or exploratory result after an uncertain primary result, keep that distinction in your conclusion. CONSORT 2025 requests protocols, analysis plans and explanations of important changes.

Center for Open Science: Preregistration CONSORT 2025 statement: updated guideline for reporting randomised trials

Compare the abstract's conclusion with the primary result

Boutron and colleagues examined 72 published parallel-group randomized trials with statistically nonsignificant primary results, drawn from a defined 2006 publication sample. They identified presentation that emphasized favourable results or drew conclusions beyond the primary result, which they called spin. This is evidence about a selected set of reports, not the rate of spin in every field or evidence of each author's motive. Follow the abstract's claim into the primary-outcome table and its uncertainty. Useful secondary findings can remain useful when their status is clear; they do not silently replace the original question.

Reporting and Interpretation of Randomized Controlled Trials With Statistically Nonsignificant Results for Primary Outcomes

Look for evidence that did not reach the visible papers

The studies you can find may be a selected subset of the studies conducted. Entire studies can remain unpublished, and particular results can be omitted within a published report. Cochrane describes how selection related to results can distort a review's conclusion. Registries, protocols and attempts to obtain unavailable results help reviewers investigate the gap. None guarantees a complete record. When a review says it searched for missing evidence, read where it looked and what remained unavailable rather than assuming publication alone makes the evidence representative.

Cochrane Handbook, Chapter 13: Assessing risk of bias due to missing evidence in a synthesis Cochrane Handbook, Chapter 4: Searching for and selecting studies

Examine who could influence the study and its reporting

Lundh, Bero and colleagues' 2017 Cochrane review included 75 empirical papers examining industry sponsorship. Industry-sponsored drug and device studies more often had favourable efficacy results and conclusions than studies with other sponsorship. The finding is an association across the included evidence, not proof that a particular sponsored paper is false. Inspect the comparator, outcomes, access to data, control of analysis and ability to publish unfavourable results. Funding disclosure starts that inquiry. Non-financial interests and investigators' own commitments can also matter; skipping the methods would remove the information needed to assess them.

Industry sponsorship and research outcome Cochrane Handbook, Chapter 7: Considering bias and conflicts of interest among the included studies

Place the paper beside the wider evidence

Look for a systematic review that answers the same question

A systematic review uses an explicit question, eligibility rules stating which studies qualify, a reproducible search and an assessment of the included evidence. A meta-analysis is a statistical combination of results; some systematic reviews appropriately do not perform one. PRISMA 2020 helps readers see how a review was conducted. Check the last search date, relevant populations, comparisons and outcomes, and reasons for including or excluding studies. A review can be rigorous yet answer a different question from yours. Its title or size cannot replace that match.

The PRISMA 2020 statement: An updated guideline for reporting systematic reviews Cochrane Handbook, Chapter 4: Searching for and selecting studies Cochrane Handbook, Chapter 3: Defining the criteria for including studies and how they will be grouped for the synthesis

Read the studies behind a forest plot's summary

A forest plot commonly displays each study's estimate with a line for its confidence interval; a summary diamond shows a pooled estimate and interval. Larger symbols often indicate greater statistical weight. Read the axis and legend, because displays vary. Cochrane's meta-analysis guidance asks whether it makes sense to combine the studies and how differences between them affect interpretation. A precise pooled result cannot repair biased input studies. If populations, treatments or outcomes differ materially, ask whether the summary obscures the comparison you need.

Cochrane Handbook, Chapter 10: Analysing data and undertaking meta-analyses

Read a funnel plot with its possible explanations

A funnel plot places study estimates against a measure of their size or precision. An asymmetric pattern may reflect missing results, differences between smaller and larger studies, other bias or chance. Cochrane therefore cautions against treating asymmetry as a diagnosis of publication bias. A symmetric plot does not prove that nothing is missing either. These assessments are especially limited when few studies are available; tests are generally not recommended with fewer than ten. Read the explanation and search methods alongside the picture.

Cochrane Handbook, Chapter 13: Assessing risk of bias due to missing evidence in a synthesis

Read certainty separately for each important outcome

GRADE describes certainty in an evidence body for a particular outcome as high, moderate, low or very low. Cochrane's GRADE guidance considers risk of bias, inconsistency between studies, indirectness to the question, imprecision and publication bias. These judgments require explanations, not merely counting studies. One treatment can have relatively certain evidence about symptom improvement and uncertain evidence about a rare harm. Read both before repeating a single label as if it covered every consequence.

Cochrane Handbook, Chapter 14: Completing Summary of findings tables and grading the certainty of the evidence

Distinguish confidence in the evidence from a recommendation to act

Certainty concerns how confidently an effect is known. Recommendation strength also considers the balance of benefits and harms, people's values and decision factors such as resources, equity and feasibility. The GRADE Working Group treats these as connected but distinct judgments. A strong recommendation is not simply a synonym for a large effect or high certainty. Read the reasoning, applicable setting and options. Your circumstances may differ from those used by a guideline panel, so one label is not a personal instruction.

GRADE Working Group

Understand where reasonable judgments differ

These are specific choices about evidence and decisions. They do not make every interpretation equally well supported.

Recognize the debate about significance thresholds

Some statistical guidance urges readers to avoid splitting findings into significant and nonsignificant boxes. The ASA president's 2021 task force instead defended carefully used significance tests and noted that thresholds can serve explicit decisions. It also distinguished its position from a 2019 editorial proposing abandonment of statistical significance. The shared practical requirement is to explain the estimate, uncertainty, assumptions and consequences. If a decision needs a cutoff, say why that cutoff fits the decision; crossing 0.05 alone does not determine scientific or practical importance.

Statement of the ASA President's Task Force on Statistical Significance and Replicability Cochrane Handbook, Chapter 15: Interpreting results and drawing conclusions

Choose the evidence design for the question and its constraints

Randomization can strengthen a treatment-effect comparison, but it is not available or sufficient for every question. Rare or delayed harms may require long observational follow-up; deliberately assigning a harmful exposure may be unacceptable. Cochrane describes a role for non-randomized evidence while requiring careful attention to confounding and other biases. Readers and reviewers can differ over how much confidence to place in that evidence for a particular decision. Identify the question, why this design was used, and which unresolved differences could change the answer.

Cochrane Handbook, Chapter 24: Including non-randomized studies on intervention effects Cochrane Handbook, Chapter 19: Adverse effects

Make the tradeoff between earlier access and confirmed benefit visible

Waiting for patient outcomes and acting on an earlier surrogate involve different risks, especially in serious disease with few options. FDA's accelerated-approval framework permits some reasonably likely surrogates while requiring further work to confirm clinical benefit. That is a conditional decision under uncertainty. Prasad and colleagues documented reversals of established practices in selected articles from one major journal during 2001–2010. Their denominator does not represent all medical care. Together these sources support asking what evidence justified adoption and what later evidence could confirm or overturn it, rather than assuming every established practice is either proven or worthless.

FDA: Surrogate Endpoint Resources for Drug and Biologic Development A decade of reversal: an analysis of 146 contradicted medical practices

Give planned confirmation and open exploration their proper jobs

Exploration can discover an unexpected pattern; a planned test can provide a more interpretable check of a specified claim. Preregistration makes that distinction visible. Registered Reports go further: journals evaluate a proposal before the results are known and can offer acceptance in principle, subject to meeting agreed conditions. The Center for Open Science acknowledges constraints involving timing, resources and concerns about flexibility; it also explains that exploratory analyses can still be reported. Readers should not treat exploration as wrongdoing or as completed confirmation. Neither format guarantees a good design, faithful execution or a useful result.

Center for Open Science: Preregistration Center for Open Science: Registered Reports

Turn broad scepticism into a specific methods question

Ioannidis's 2005 essay uses a mathematical model to examine how prior plausibility, power, bias and multiple testing can undermine positive findings. Its conclusions depend on assumptions about those conditions; it is not a census establishing that all published studies are false or fabricated. The journal also records a 2022 correction to parentheses in a table equation. You do not need to accept or reject all research at once. Ask which feature makes this claim fragile and what independent evidence could strengthen it. Specific criticism gives you something to check and something that could change your mind.

Why Most Published Research Findings Are False Correction: Why Most Published Research Findings Are False

Keep a record you can use later

A short note helps you remember the study's answer without detaching it from its limits.

Keep the question attached to your conclusion

Write who was studied, the comparison, outcome and duration before your one-sentence conclusion. If your own question concerns a different group or outcome, name that gap. This preserves the difference between what the paper measured and what you hope it means.

Cochrane Handbook, Chapter 3: Defining the criteria for including studies and how they will be grouped for the synthesis

Save the result with its denominator and uncertainty

Keep the original outcome units, group results, effect estimate, interval and time period together. If absolute effects are unavailable, record that limitation. A saved percentage without its starting risk becomes a different claim when you repeat it.

Here is an invented result to carry into a reading note: over one year, an event occurs in 20 of 1,000 people in one group and 10 of 1,000 in the comparison group. The observed risks are 2% and 1%. The difference is 1 percentage point, or 10 fewer events per 1,000 people over that year. Relative to the 2% starting risk, the observed reduction is 50%. All of those descriptions use the same numbers; '50% lower' alone conceals the starting risk.

The reading note still needs the allocation method, missing-data information and uncertainty interval before a strong interpretation. These toy counts do not supply those details or establish that the intervention caused the difference. If the headline concerns lifetime benefit, the one-year result cannot settle it. Keeping denominator, duration and limits beside the estimate prevents the more impressive wording from quietly becoming a broader claim.

Cochrane Handbook, Chapter 6: Choosing effect measures and computing estimates of effect Cochrane Handbook, Chapter 15: Interpreting results and drawing conclusions

Record the method concern that could change the answer

Use a concrete statement such as 'missing outcomes may differ between groups' or 'the comparator does not match my decision.' Separate an unreported detail from an observed methodological problem. A reporting checklist can locate the information; a bias assessment explains why a problem might distort the result.

Cochrane Handbook, Chapter 7: Considering bias and conflicts of interest among the included studies STROBE: Strengthening the Reporting of Observational Studies in Epidemiology

Record where you looked for missing evidence

Save the registration or protocol link and the relevant review's last search date. Note unavailable studies or results and how the review tried to find them. This lets a later reader check what may be missing rather than mistake your collection for everything studied.

Cochrane Handbook, Chapter 4: Searching for and selecting studies Cochrane Handbook, Chapter 13: Assessing risk of bias due to missing evidence in a synthesis

Update the question when the evidence changes

Record the paper's version, a matching review or guideline, and the outcome-specific certainty where available. Revisit the note when a correction, additional study or updated review appears. Preserve the earlier uncertainty in your record so that a changed conclusion is understandable rather than hidden.

The PRISMA 2020 statement: An updated guideline for reporting systematic reviews Cochrane Handbook, Chapter 14: Completing Summary of findings tables and grading the certainty of the evidence Preprints Involving Medical Research—Do the Benefits Outweigh the Challenges?

Respect the limits of use and access

Bring practice changes to the responsible professional

This is a reading companion about methods, not medical, legal, or financial advice. A headline is not a reason to start or stop a medicine. Bring practice changes to a qualified clinician or the person who owns the decision.

Preprints Involving Medical Research—Do the Benefits Outweigh the Challenges?GRADE Working Group

Ask for transparent access conditions while protecting participants

Open materials can make scrutiny easier, but research participants' identifiable data are not automatically suitable for public release. PLOS's data policy recognizes privacy, consent and ethical restrictions and asks authors to explain restrictions and an appropriate access route. A justified limit is different from unexplained concealment. Look for the availability statement, what can be checked, who controls access and why a restriction exists.

PLOS ONE: Data Availability

Check the evidence behind a consequential choice

Named sources. Honest paraphrase of the finding. Not medical, legal, or financial advice.

GRADE Working Group Preprints Involving Medical Research—Do the Benefits Outweigh the Challenges?

Read one claim all the way through

Start with one paper and a five-part note

Choose a study behind a claim you encountered. Write its question, design, result in the original units, uncertainty and follow-up period. Leave a blank wherever you cannot find the answer. The point is to establish what you can responsibly say, then identify the missing information that would help most.

Informed Health Choices Key Concepts for assessing claims about treatment effects, 2022 Statement of the ASA President's Task Force on Statistical Significance and Replicability

Trace the headline into the methods and results

Read the primary outcome, its group results and interval in the full paper and supplements. Follow participant flow and missing outcomes. Then compare the dated registration, protocol or analysis plan with the published report, including disclosed amendments. Write whether the headline describes the primary result or an exploratory finding, and what methodological uncertainty remains.

CONSORT 2025 statement: updated guideline for reporting randomised trials Cochrane Handbook, Chapter 7: Considering bias and conflicts of interest among the included studies Reporting and Interpretation of Randomized Controlled Trials With Statistically Nonsignificant Results for Primary Outcomes

Find the wider evidence before making a practical decision

Look for a systematic review or relevant guideline answering the same question. Compare its population, outcomes, search date, missing-evidence assessment and reasoning about certainty. If the decision concerns treatment, legal action or money, take the question and your note to the qualified professional or responsible decision-maker. Use the information, learning and attention topics to continue practising how you find, retain and inspect a claim.

Cochrane Handbook, Chapter 4: Searching for and selecting studies GRADE Working Group Informed Health Choices Key Concepts for assessing claims about treatment effects, 2022

References for this topic

Sources and context

Cochrane Handbook, Chapter 3: Defining the criteria for including studies and how they will be grouped for the synthesis ↗
id
read-a-study--cochrane-questions — Cochrane Handbook, Chapter 3: Defining the criteria for including studies and how they will be grouped for the synthesis
kind
guideline
accessed at
2026-10-01
supports
Opened methods chapter: defining populations, interventions, comparisons, outcomes, eligible designs and the question a review can answer.
limitations
Systematic-review methodology; PICO suits intervention questions and is not imposed on all research questions.
original id
cochrane-questions
Cochrane Handbook, Chapter 24: Including non-randomized studies on intervention effects ↗
id
read-a-study--cochrane-nonrandomized — Cochrane Handbook, Chapter 24: Including non-randomized studies on intervention effects
kind
guideline
accessed at
2026-10-01
supports
Opened causal questions, confounding, selection and the role and limits of non-randomized intervention evidence.
limitations
Illustrative exercise example is authored, not a finding from an exercise study; the chapter does not authorize causal claims from any association.
original id
cochrane-nonrandomized
Cochrane Handbook, Chapter 6: Choosing effect measures and computing estimates of effect ↗
id
read-a-study--cochrane-effect-measures — Cochrane Handbook, Chapter 6: Choosing effect measures and computing estimates of effect
kind
guideline
accessed at
2026-10-01
supports
Opened definitions of risk, odds, relative and absolute measures, continuous outcomes and null values. Article arithmetic examples are invented illustrations.
limitations
Technical methods guidance; an effect measure alone supplies neither clinical importance nor a personal forecast.
original id
cochrane-effect-measures
Preprints Involving Medical Research—Do the Benefits Outweigh the Challenges? ↗
id
read-a-study--jama-medical-preprints — Preprints Involving Medical Research—Do the Benefits Outweigh the Challenges?
kind
professional
accessed at
2026-10-01
supports
Opened 2020 editorial defining medical preprints, describing review status and discussing cautions about use for clinical decisions.
limitations
A professional editorial rather than a quantitative study; it explains the distinction between a preprint and a peer-reviewed paper.
original id
jama-medical-preprints
CONSORT 2025 statement: updated guideline for reporting randomised trials ↗
id
read-a-study--consort-2025 — CONSORT 2025 statement: updated guideline for reporting randomised trials
kind
guideline
accessed at
2026-10-01
supports
Opened official 12-page statement PDF, including the 30-item checklist, concealment versus blinding, flow, harms, registration, protocol, analysis plan and amendments.
limitations
A reporting standard, not a study-quality score or guarantee that a trial was conducted soundly.
original id
consort-2025
Cochrane Handbook, Chapter 8: Assessing risk of bias in a randomized trial ↗
id
read-a-study--cochrane-trial-bias — Cochrane Handbook, Chapter 8: Assessing risk of bias in a randomized trial
kind
guideline
accessed at
2026-10-01
supports
Opened randomization, deviations, missing outcome data and intention-to-treat limitations; no universal safe missing-data percentage.
limitations
RoB 2 guidance requires study-specific information and judgment, not a checklist score.
original id
cochrane-trial-bias
FDA: Surrogate Endpoint Resources for Drug and Biologic Development ↗
id
read-a-study--fda-surrogate-endpoints — FDA: Surrogate Endpoint Resources for Drug and Biologic Development
kind
official
accessed at
2026-10-01
supports
Opened definitions of clinical outcomes, validated versus reasonably likely surrogates and further confirmation of clinical benefit in accelerated approval.
limitations
US drug/biologic regulatory context; a surrogate's validity is specific to its proposed use and this article gives no treatment recommendation.
original id
fda-surrogate-endpoints
Cochrane Handbook, Chapter 19: Adverse effects ↗
id
read-a-study--cochrane-harms — Cochrane Handbook, Chapter 19: Adverse effects
kind
guideline
accessed at
2026-10-01
supports
Opened adverse-effect detection, rare and delayed harms, contributions of observational designs and spontaneous-report denominator limits.
limitations
Harms evidence has design- and outcome-specific limitations; no observed event does not prove no risk.
original id
cochrane-harms
EQUATOR Network: Enhancing the QUAlity and Transparency Of health Research ↗
id
read-a-study--equator-library — EQUATOR Network: Enhancing the QUAlity and Transparency Of health Research
kind
official
accessed at
2026-10-01
supports
Opened reporting-guideline library and study-type navigation.
limitations
Finding an applicable reporting guideline is different from assessing bias in a study.
original id
equator-library
EQUATOR Network: CONSORT 2025 Statement ↗
id
read-a-study--equator-consort — EQUATOR Network: CONSORT 2025 Statement
kind
official
accessed at
2026-10-01
supports
Opened current registry entry identifying the 2025 randomized-trial reporting guideline and its supporting materials.
limitations
Guideline registry metadata; the full statement supplies methodological detail.
original id
equator-consort
STROBE: Strengthening the Reporting of Observational Studies in Epidemiology ↗
id
read-a-study--strobe — STROBE: Strengthening the Reporting of Observational Studies in Epidemiology
kind
guideline
accessed at
2026-10-01
supports
Opened official scope for cohort, case-control and cross-sectional reporting and explicit warning against use as a quality-assessment instrument.
limitations
Reporting recommendations are not design prescriptions or an observational-study quality score.
original id
strobe
PRISMA 2020 statement ↗
id
read-a-study--prisma-2020 — PRISMA 2020 statement
kind
guideline
accessed at
2026-10-01
supports
Opened official 27-item checklist and supporting links for systematic-review reporting.
limitations
Transparent reporting is not sufficient to establish review validity; PRISMA and CONSORT address different report types.
original id
prisma-2020
Cochrane Handbook, Chapter 7: Considering bias and conflicts of interest among the included studies ↗
id
read-a-study--cochrane-bias — Cochrane Handbook, Chapter 7: Considering bias and conflicts of interest among the included studies
kind
guideline
accessed at
2026-10-01
supports
Opened distinction between reporting information and evaluating systematic distortion, including conflicts of interest.
limitations
Risk-of-bias judgments are tied to a result and design; disclosure or reporting completeness is not an automatic validity verdict.
original id
cochrane-bias
Cochrane Handbook, Chapter 15: Interpreting results and drawing conclusions ↗
id
read-a-study--cochrane-interpretation — Cochrane Handbook, Chapter 15: Interpreting results and drawing conclusions
kind
guideline
accessed at
2026-10-01
supports
Opened absolute effects, baseline risk, time-bound NNT, uncertainty, confidence intervals and cautions about binary significance interpretation.
limitations
Interpretation depends on outcome, assumptions and applicability; example NNT calculations are teaching arithmetic, not observed treatment results.
original id
cochrane-interpretation
Statement of the ASA President's Task Force on Statistical Significance and Replicability ↗
id
read-a-study--asa-task-force — Statement of the ASA President's Task Force on Statistical Significance and Replicability
kind
professional
accessed at
2026-10-01
supports
Opened 2021 task-force discussion of tests, intervals, decision thresholds and its distinction from the 2019 abandonment editorial.
limitations
Task-force position; it does not establish that all statisticians agree on significance terminology or threshold use.
original id
asa-task-force
American Statistical Association releases statement on statistical significance and p-values ↗
id
read-a-study--asa-p-values — American Statistical Association releases statement on statistical significance and p-values
kind
professional
accessed at
2026-10-01
supports
Opened official 2016 statement summary and its six principles: model compatibility, no hypothesis-probability interpretation, no effect-size interpretation and no threshold-only reasoning.
limitations
Principles guide interpretation rather than prescribe a single statistical analysis for every study.
original id
asa-p-values
Why Most Published Research Findings Are False ↗
id
read-a-study--ioannidis-2005 — Why Most Published Research Findings Are False
kind
professional
accessed at
2026-10-01
supports
Opened 2005 methods essay and modelling of prior odds, power, bias and multiple testing; distinguished model conclusions from an empirical census.
limitations
Assumption-dependent model, not a measurement of the fraction of all science that is false or fabricated; no equations or model-derived prevalence reproduced.
original id
ioannidis-2005
Center for Open Science: Preregistration ↗
id
read-a-study--cos-preregistration — Center for Open Science: Preregistration
kind
official
accessed at
2026-10-01
supports
Opened explanation of separating planned confirmatory tests from exploratory analyses and transparently reporting deviations.
limitations
An initiative's methods guidance, not evidence that registration alone fixes design or execution.
original id
cos-preregistration
Reporting and Interpretation of Randomized Controlled Trials With Statistically Nonsignificant Results for Primary Outcomes ↗
id
read-a-study--boutron-spin — Reporting and Interpretation of Randomized Controlled Trials With Statistically Nonsignificant Results for Primary Outcomes
kind
study
accessed at
2026-10-01
supports
Opened 2010 primary study: 72 eligible parallel-group trials with nonsignificant primary outcomes from a specified December 2006 publication sample; spin in presentation and interpretation.
limitations
Selected sample and interpretive coding; not a prevalence estimate for all papers, a diagnosis of author motives or proof that a nonsignificant effect is exactly zero.
original id
boutron-spin
Cochrane Handbook, Chapter 13: Assessing risk of bias due to missing evidence in a synthesis ↗
id
read-a-study--cochrane-missing-evidence — Cochrane Handbook, Chapter 13: Assessing risk of bias due to missing evidence in a synthesis
kind
guideline
accessed at
2026-10-01
supports
Opened publication/selective-reporting bias, alternatives explaining funnel asymmetry, and limited power of assessments with few studies.
limitations
Neither plot symmetry nor asymmetry settles whether evidence is missing; the ten-study guidance is a rule for tests, not proof of safety above ten.
original id
cochrane-missing-evidence
Cochrane Handbook, Chapter 4: Searching for and selecting studies ↗
id
read-a-study--cochrane-searching — Cochrane Handbook, Chapter 4: Searching for and selecting studies
kind
guideline
accessed at
2026-10-01
supports
Opened reproducible searching, selection, registry sources and the importance of a review's search dates and scope.
limitations
A comprehensive search reduces but cannot guarantee elimination of unavailable or unreported evidence.
original id
cochrane-searching
Industry sponsorship and research outcome ↗
id
read-a-study--lundh-sponsorship — Industry sponsorship and research outcome
kind
review
accessed at
2026-10-01
supports
Opened 2017 Cochrane review summary: 75 empirical papers and more favourable efficacy results/conclusions associated with industry sponsorship in drug/device studies.
limitations
The search ran through February 2015, so these findings are not a current 2026 sponsorship rate. An association across studies does not establish the validity of an individual paper.
original id
lundh-sponsorship
The PRISMA 2020 statement: An updated guideline for reporting systematic reviews ↗
id
read-a-study--prisma-explanation — The PRISMA 2020 statement: An updated guideline for reporting systematic reviews
kind
guideline
accessed at
2026-10-01
supports
Opened primary guideline publication explaining transparent review methods and the scope of reviews with or without statistical synthesis.
limitations
A reporting guideline does not guarantee an exhaustive search or an appropriate pooled analysis.
original id
prisma-explanation
Cochrane Handbook, Chapter 10: Analysing data and undertaking meta-analyses ↗
id
read-a-study--cochrane-meta-analysis — Cochrane Handbook, Chapter 10: Analysing data and undertaking meta-analyses
kind
guideline
accessed at
2026-10-01
supports
Opened forest-plot conventions, pooling decisions, heterogeneity and the limits of synthesis when studies are biased or different.
limitations
Visual conventions vary; pooled precision does not establish validity or applicability.
original id
cochrane-meta-analysis
Cochrane Handbook, Chapter 14: Completing Summary of findings tables and grading the certainty of the evidence ↗
id
read-a-study--cochrane-grade — Cochrane Handbook, Chapter 14: Completing Summary of findings tables and grading the certainty of the evidence
kind
guideline
accessed at
2026-10-01
supports
Opened outcome-specific certainty levels and risk of bias, inconsistency, indirectness, imprecision and publication bias considerations.
limitations
GRADE includes additional considerations, including circumstances for rating up; this introduction does not replace a full assessment.
original id
cochrane-grade
GRADE Working Group ↗
id
read-a-study--grade-working-group — GRADE Working Group
kind
official
accessed at
2026-10-01
supports
Opened GRADE criteria and Evidence to Decision framework; outcome-specific certainty is distinct from recommendation strength and decision considerations.
limitations
The criteria include a May 2023 update; documents linked from the page have their own publication dates.
original id
grade-working-group
A decade of reversal: an analysis of 146 contradicted medical practices ↗
id
read-a-study--prasad-reversal — A decade of reversal: an analysis of 146 contradicted medical practices
kind
study
accessed at
2026-10-01
supports
Opened Duke's coauthor institutional record and primary abstract: established-practice evaluations in selected original articles from one journal during 2001–2010 documented reversals.
limitations
Abstract and bibliographic record inspected, not full publisher manuscript; journal selection and established-practice denominator prevent extrapolation to all medical care.
original id
prasad-reversal
Center for Open Science: Registered Reports ↗
id
read-a-study--cos-registered-reports — Center for Open Science: Registered Reports
kind
official
accessed at
2026-10-01
supports
Opened pre-results peer review, conditional acceptance, exploratory-analysis provisions, practical constraints and public professional explainers.
limitations
Acceptance in principle remains conditional on agreed standards; the format does not guarantee useful or unbiased findings.
original id
cos-registered-reports
Correction: Why Most Published Research Findings Are False ↗
id
read-a-study--ioannidis-correction — Correction: Why Most Published Research Findings Are False
kind
official
accessed at
2026-10-01
supports
Opened August 25, 2022 correction identifying missing parentheses in one Table 2 equation.
limitations
Specific equation correction, not a retraction or a new empirical test of the original model.
original id
ioannidis-correction
PLOS ONE: Data Availability ↗
id
read-a-study--plos-data-access — PLOS ONE: Data Availability
kind
official
accessed at
2026-10-01
supports
Opened policy recognizing privacy, consent and ethical restrictions while requesting clear availability statements and appropriate access mechanisms.
limitations
One publisher's policy illustrates a legitimate transparency boundary; it is not universal law or permission to disclose participant data.
original id
plos-data-access
Informed Health Choices Key Concepts for assessing claims about treatment effects, 2022 ↗
id
read-a-study--ihc-key-concepts — Informed Health Choices Key Concepts for assessing claims about treatment effects, 2022
kind
professional
accessed at
2026-10-01
supports
Opened public teaching framework PDF and development credits; fair comparisons and thoughtful interpretation of treatment claims.
limitations
Educational framework, not individual clinical advice; article's five-part reading note is an authored application, not a validated intervention.
original id
ihc-key-concepts
AskStatistics reader question about interpreting statistical results ↗
id
read-a-study--reader-p-value-question — AskStatistics reader question about interpreting statistical results
kind
question_signal
accessed at
2026-10-01
supports
Opened public thread as a signal that readers ask about binary significance and what statistical results mean.
limitations
Anecdotal reader question only; comments, votes and apparent agreement supply no factual guidance or prevalence claim.
original id
reader-p-value-question
PhD reader question about PRISMA and CONSORT diagrams ↗
id
read-a-study--reader-reporting-question — PhD reader question about PRISMA and CONSORT diagrams
kind
question_signal
accessed at
2026-10-01
supports
Opened public thread as a signal of confusion between reporting frameworks for reviews and trials.
limitations
Question discovery only; guidance is supported by the opened official reporting standards, not forum answers.
original id
reader-reporting-question

Professional perspectives

John P. A. Ioannidis
MD, DSc; Stanford professor of medicine and epidemiology and population health; METRICS co-director
Public usefulness
His official profile links public talks and teaching about evidence reliability and meta-research.
Contribution
The 2005 model helps explain why plausibility, power, bias and repeated testing matter together.
Scope and limits
The model depends on assumptions and is not a census declaring all studies false.
Reviewed
2026-10-01
Gordon Guyatt
Physician; McMaster professor in Health Research Methods, Evidence, and Impact with a joint medicine appointment
Public usefulness
Users' Guides teaching and evidence-based medicine explanations help clinicians and readers move from findings to decisions; McMaster's official biography confirms his clinical background.
Contribution
GRADE and clinical evidence interpretation distinguish certainty from recommendation strength.
Scope and limits
A framework cannot choose an individual's preferred tradeoff or replace context-specific professional judgment.
Reviewed
2026-10-01
Julian P. T. Higgins
PhD; University of Bristol professor of evidence synthesis
Public usefulness
The freely available Cochrane Handbook and methodological teaching explain synthesis and bias in detail.
Contribution
Handbook editing, heterogeneity methods and risk-of-bias tools support reading evidence bodies rather than isolated results.
Scope and limits
Statistical synthesis does not repair biased input studies or answer a mismatched question.
Reviewed
2026-10-01
Jonathan A. C. Sterne
PhD; University of Bristol professor of medical statistics and epidemiology
Public usefulness
Public Cochrane methods chapters and risk-of-bias resources explain technical issues that readers often mistake for simple scores.
Contribution
Missing-evidence, funnel-plot and causal-methods guidance informs assessment of fragile comparisons.
Scope and limits
Funnel asymmetry has several explanations and cannot diagnose publication bias by itself.
Reviewed
2026-10-01
Brian Nosek
PhD; Center for Open Science co-founder and executive director; University of Virginia psychology professor
Public usefulness
COS's public explainers, initiatives and training make preregistration and reproducibility accessible.
Contribution
Separating planned tests from exploration and increasing transparent research practices.
Scope and limits
Registration supplies a record; it does not make a weak design sound or guarantee replication.
Reviewed
2026-10-01
David Moher
PhD in clinical epidemiology and biostatistics; Ottawa Hospital Research Institute senior scientist and Journalology director; University of Ottawa professor
Public usefulness
CONSORT, PRISMA and Journalology resources give readers practical ways to locate essential reporting information.
Contribution
Leadership in developing and evaluating reporting guidance for trials and systematic reviews.
Scope and limits
Reporting completeness is separate from the validity of the reported methods.
Reviewed
2026-10-01
Holger J. Schünemann
MD, PhD; McMaster professor emeritus in Health Research Methods, Evidence, and Impact
Public usefulness
GRADE guidance, Cochrane interpretation chapters and clinical-epidemiology education explain how evidence informs recommendations.
Contribution
Outcome-specific certainty assessment and transparent evidence-to-decision methods.
Scope and limits
Certainty judgments concern specified outcomes; a guideline recommendation is not automatically an individual's preferred decision.
Reviewed
2026-10-01
Lisa Bero
PhD; University of Colorado professor working in evidence synthesis, research integrity and conflicts of interest
Public usefulness
Invited public speaking, media explanation and accessible sponsorship reviews help readers examine influence on evidence.
Contribution
Research on industry sponsorship, publication and factors shaping research conclusions.
Scope and limits
An association across sponsored studies is not proof that any named study or author is fraudulent.
Reviewed
2026-10-01
Isabelle Boutron
MD, PhD; professor of epidemiology and Cochrane Bias Methods Group convenor
Public usefulness
Public Cochrane methods work and reporting research explain bias and spin beyond the abstract's headline.
Contribution
Primary empirical study of spin in reports with nonsignificant primary trial outcomes.
Scope and limits
The 72-trial study is a defined historical sample; presentation coding does not establish author motives.
Reviewed
2026-10-01
Gerd Gigerenzer
Psychologist; Max Planck Institute for Human Development director emeritus; Harding Center for Risk Literacy director
Public usefulness
Risk Savvy, Calculated Risks and professional risk-literacy teaching explain numbers using understandable event frequencies.
Contribution
Risk communication helps readers see what an impressive relative percentage leaves out.
Scope and limits
Clearer numerical presentation cannot eliminate uncertainty or predict an individual's outcome.
Reviewed
2026-10-01
Andrew D. Oxman
Evidence-methods researcher and co-lead developer credited in the Informed Health Choices Key Concepts framework
Public usefulness
The public Key Concepts framework supplies plain teaching about treatment claims and fair comparisons.
Contribution
A structured educational approach to asking what evidence supports a claim and which comparison is fair.
Scope and limits
The framework develops reading judgment and does not provide individual clinical advice; no unverified current institutional title or degree is assigned.
Reviewed
2026-10-01
Christopher D. Chambers
PhD; Chartered Psychologist and Cardiff University professor of cognitive neuroscience
Public usefulness
Public writing, The Seven Deadly Sins of Psychology, workshops and Registered Reports explanations make reproducibility choices understandable.
Contribution
Development and advocacy of pre-results review and Registered Reports.
Scope and limits
The format changes incentives and scrutiny; it cannot guarantee execution quality or a result's importance.
Reviewed
2026-10-01
Daniel J. Simons
PhD; University of Illinois psychology professor
Public usefulness
COS Registered Reports explanations and an officially documented public lecture on Nobody's Fool help readers question apparently persuasive claims.
Contribution
Registered replication publishing and methodological communication encourage scrutiny independent of a striking outcome.
Scope and limits
Replication and a transparent publication format still require interpretation of populations, measures and real-world meaning.
Reviewed
2026-10-01
Matthew J. Page
PhD in epidemiology; Monash University associate professor and evidence-synthesis methods researcher
Public usefulness
PRISMA 2020 materials and accessible synthesis-methods guidance help readers inspect review searches and reporting.
Contribution
PRISMA 2020 development, risk-of-bias methods and research on missing evidence.
Scope and limits
A reporting standard does not establish that a particular review searched adequately or pooled appropriate studies.
Reviewed
2026-10-01
Paul Glasziou
Physician and evidence-based medicine researcher at Bond University
Public usefulness
Public evidence-based medicine resources and clinical translation work connect research findings to useful, understandable choices.
Contribution
Cochrane interpretation guidance and work on evidence use emphasize clinical meaning and practical applicability.
Scope and limits
Useful evidence interpretation remains specific to the patient-important outcome, comparison and decision; no current director title is asserted.
Reviewed
2026-10-01

Keep this topic in view

Preserved original record

The earlier public profile is preserved separately from the current article. It may contain superseded wording and is not newly checked material.

Read the full original profile

slug

read-a-study

name

How to read a study

browse Line

Read the question, the comparison, and the absolute numbers before the headline.

photo

/learn/assets/topics/read-a-study.jpg

photo Alt

An adult at a desk with a printed paper and a laptop, comparing a headline to a methods section

aliases

read a paper
absolute risk
p-hacking
preprint
CONSORT
GRADE

family

learning

heat

mid

talk

Sources

most Discussed

true

first Line

A research study investigates a defined question using stated methods and data. Reading it well means identifying what was studied, what the results show, and which uncertainties limit the conclusions.

why Line

feel Words

headline
miracle
breakthrough
press release
randomized
control
observational
confounding

common Picture

lead
Read the question the study actually asked before the headline
why
Start with the actual question: who or what was studied, what was measured and over what period. A treatment trial usually compares alternatives, while a descriptive study may estimate a frequency without testing a treatment. Cochrane methods help match the design to the question. Boutron’s research shows how an abstract can emphasize favorable details beyond the primary result. Compare the headline with the measured outcome before deciding whether the paper addresses the claim that drew your attention.
lead
Check who was compared with whom, and whether assignment was random
why
For an intervention question, PICO means Population, Intervention, Comparator and Outcome. Then identify how groups were formed: random assignment differs from people choosing a treatment or being observed in routine care. CONSORT 2025 asks trial authors to report allocation, blinding and participant flow, while risk-of-bias assessment asks whether design or conduct could distort the result. Check missing outcomes and departures from the assigned treatment too. Randomization strengthens causal inference, but the word randomized alone does not settle all of those questions.
lead
Translate a relative change into an absolute number of people
why
A 50 percent relative reduction could mean 2 people in 100 having an event instead of 1, or 50 instead of 25, over the same follow-up period. The absolute differences are 1 and 25 percentage points, which describe very different magnitudes. Cochrane recommends reading the baseline risk, comparison group and time period with the result. Also look for uncertainty, such as a confidence interval, and for harms. If the headline omits those details, find the paper or review before treating the percentage as a complete basis for a decision.
lead
Check a preprint’s methods, version and later review
why
A preprint is a publicly shared research manuscript that has not yet completed journal peer review; it is not necessarily an unfinished experiment. medRxiv warns that its reports should not directly guide clinical decisions. Check the posted version, methods, registered plan and whether a later journal article or correction exists. Peer review is another layer of scrutiny, not proof of correctness. A useful reading question is how this finding fits with other evidence, rather than whether one publication label makes it a practice-changing result.

situation Quick Answers

- label
  Find the question
  value
  Identify who was studied, what was measured, the design and follow-up.
- label
  Check the headline
  value
  Compare its claim with the primary outcome and reported result, including uncertainty.
- label
  Identify the design
  value
  Random assignment, observation and a before-and-after comparison have different strengths and limits.
- label
  Inspect the conduct
  value
  Look for the comparison, missing outcomes, participant flow and changes to the registered plan.
- label
  Translate the percentage
  value
  Find the event risks in both groups using the same denominator and follow-up period.
- label
  Complete the result
  value
  Read the absolute difference, uncertainty and harms before applying the relative percentage.
- label
  Know the status
  value
  A preprint is a public manuscript that has not completed journal peer review.
- label
  Check what came later
  value
  Look for new versions, a journal article, corrections and other evidence; do not change a medicine from one early report.

picture Lead

Read the question the study actually asked before the headline

watch For

what People Say

Match the design to the question. Cochrane’s intervention framework identifies the population, intervention, comparator and outcome before judging a result. Other designs can describe experiences, estimate frequency or investigate associations. Naming the actual job of the study prevents a useful descriptive finding from being mistaken for proof that a treatment caused a change.
Inspect how a randomized trial was actually conducted. Random assignment helps balance competing explanations, but missing data, deviations and selective outcome reporting can still bias the estimate. CONSORT asks authors to report the necessary detail; risk-of-bias tools assess its implications. Read the participant flow and analysis population rather than treating randomization as a certificate that all later steps were sound.
Put absolute effects beside relative effects. A relative percentage describes change in proportion to the starting risk, while an absolute difference shows how many events differ on a common scale. Cochrane links that number to the population and follow-up period. Use the same denominator and time interval for both groups, then compare benefits, harms and uncertainty.
Look for results that may be missing or selectively emphasized. Publication bias can occur when the availability of results depends on their direction or statistical significance. Boutron also documented spin within published reports. Trial registries, protocols and systematic reviews help reveal these problems, though none guarantees that every missing result has been found. Read what was planned as well as what was highlighted.
Separate evidence certainty from a recommendation. GRADE rates confidence in the evidence for a particular outcome, not the enthusiasm of a speaker. A recommendation also weighs the balance of benefits and harms and the context of a decision. Ask which outcome was rated and why certainty was reduced before applying a single high or low label to the entire topic.
Use reporting checklists to find information. CONSORT addresses randomized trial reports, PRISMA addresses systematic reviews, and STROBE addresses observational reports. EQUATOR provides the relevant guidelines. A checklist helps locate what authors should disclose; it is not a numerical quality score or proof that the design and analysis avoided bias. Good reporting makes judgment possible rather than unnecessary.

disputed Notes

Read a methods model within its assumptions. Ioannidis’s 2005 paper models how sample size, bias, analytical flexibility and prior odds can affect the credibility of findings. It is not an empirical count establishing that every study is false. The useful response is to inspect the design and seek corroborating evidence, while keeping the model’s assumptions separate from the quality of one specific paper.
Ask whether a claimed benefit was measured directly. Prasad’s work on medical reversal examines established practices contradicted by later research. It illustrates the cost of adopting a practice on weak evidence, not a reason to assume all care is ineffective. When a paper measures a surrogate marker, ask what connects it to the patient-important outcome and whether that connection has been tested.
Use funding disclosure to sharpen the methods questions. Research synthesized by Lundh, Bero and colleagues associates industry sponsorship with more favorable findings and conclusions in drug and device research. That pattern cannot diagnose the cause of an individual result. Examine the comparator, outcomes, analysis access and sponsor’s role, and apply the same scrutiny to noncommercial interests.
Check when the analysis plan was recorded. Preregistration documents intended questions and methods before the relevant results are known. CONSORT 2025 asks for registration dates, protocol access and explanations of important amendments. A change can be legitimate if reported transparently; the key is distinguishing the original confirmatory analysis from later exploratory work. Registration alone does not repair a weak design.

talk Notes

Translate the headline percentage before comparing options. A large relative change may correspond to a small or large absolute difference depending on baseline risk. Cochrane’s approach asks for the outcome, denominator and period first. If those are missing from the article, follow the source to the paper or a review rather than invent the unreported number.
Check the status of an early public report. medRxiv makes research available before journal review and labels that status explicitly. Look for later versions, peer-reviewed publication and relevant criticism alongside the actual methods. A preprint can contribute information without supporting an immediate medication change, and a journal logo cannot replace the same critical reading.
Turn skepticism into a specific question. Ioannidis’s work identifies conditions under which results become less reliable. Use that to ask about sample size, prespecified outcomes, missing observations or replication. A blanket statement that all researchers lie explains none of those features and cannot distinguish a stronger result from a weaker one.
Compare the primary outcome with the registered plan. An abstract may highlight a favorable secondary outcome when the primary analysis is inconclusive. Boutron studied this form of spin, and CONSORT asks authors to explain changes to the plan. Check when and why a change occurred; a documented amendment is different from quietly presenting an exploratory result as the original test.

how People Run It

Before sharing or changing a plan, write the question, design and result in its original units. Write who was studied, what the design can answer and the principal result with its uncertainty and follow-up. For event risks, include counts or an absolute difference; for a measured scale, name the units and size of the change. Find a relevant review or guideline before deciding what the new paper adds. Discuss consequential practice changes with the qualified professional or person responsible for the decision.
Follow one headline to the actual result. Find the paper’s primary outcome and read the group results, estimate and uncertainty. If the abstract lacks the detail, check the full report, tables or supplement before concluding that the study itself omitted it. Boutron’s work on spin shows why the headline and abstract conclusion should be checked against the reported analysis.
Trace participants from enrollment to analysis. CONSORT’s flow diagram distinguishes people assessed, assigned, followed and analyzed. Large losses or different reasons for loss between groups can change how a result should be interpreted. Ask how missing outcomes were handled; an intention-to-treat label alone does not recover information that was never measured.
Find the plan and the existing body of evidence. Look for the registration number, protocol, analysis plan and any explanations of changes. Then check whether a systematic review already addresses the same population and outcome. This separates what the new study contributes from what was already known and makes disagreement easier to locate.

how People Keep It

Keep the question attached to the conclusion. Record the population, design, comparison when relevant, outcome and follow-up. Cochrane’s framework makes these boundaries visible. A claim about a different population or longer-term outcome requires additional evidence rather than a more confident paraphrase of the same study.
Keep units, denominators and time intervals visible. A percentage, risk ratio or mean change is incomplete without knowing what was counted or measured. Cochrane explains why absolute event risks depend on baseline risk and duration. Preserve those details when summarizing so the same result is not transformed into a larger or broader claim.
Keep reporting and risk of bias as separate judgments. A clearly written study can reveal a serious design limitation; an unclear report can leave the risk uncertain. CONSORT and other reporting guidelines help find the needed information. Risk-of-bias assessment then asks how the conduct and analysis could distort the specific result.
Keep the search for missing evidence in the review. A published set of studies may omit unreported trials or selected outcomes. Cochrane recommends considering that risk rather than treating the available papers as a complete census. Registry records and protocols can help, but an absence of detected bias is not proof that nothing is missing.
Keep one paper in the context of the whole question. A new result may agree with, refine or conflict with earlier work. A systematic review can show that pattern while assessing the quality and relevance of its included studies. Read its search date and population too; being a review does not automatically make it current or applicable.

claims Notes

Start with the question, the people, and the comparison. Claim: “A study’s design and measurements limit the conclusions it supports” — established. Cochrane methods connect the question to an appropriate design. A treatment comparison, a frequency estimate and an account of experience answer different kinds of question. Check the actual population and outcome before extending the conclusion to a broader headline.
Demand absolute numbers beside relative ones. Claim: “A 50 percent risk reduction is enough information to decide” — not what the research found. Cochrane recommends interpreting relative effects alongside baseline risk, absolute effects and a stated time period. The same relative reduction can correspond to very different numbers of events. Uncertainty and harms also matter before drawing a practical conclusion.
Read Ioannidis 2005 as a methods warning, not as all science is fake. Claim: “All published research findings are fake” — not what the research found. Ioannidis models conditions that make positive findings less reliable; the paper does not establish that every published result is fabricated or false. Study design, transparent reporting and corroborating evidence help distinguish stronger conclusions from fragile ones. Criticism should identify those features in the actual paper.
Check a preprint’s methods, version and later review. Claim: “Preprints are the same as peer-reviewed practice changers” — not what the research found. medRxiv preprints are public manuscripts that have not completed journal peer review and are not intended to directly guide clinical decisions. Check methods, versions, later publication and the wider evidence. Peer review is scrutiny rather than a guarantee of truth.
Treat unpublished and spun results as a known bias. Claim: “Unpublished negative studies can bias what we think we know” — established. Cochrane describes how nonpublication or selective reporting can distort a synthesis of the available studies. Boutron also documented spin within published reports. A review should discuss these risks and its search for missing evidence rather than imply that every visible result represents everything studied.
Use GRADE to separate certainty from enthusiasm. Claim: “GRADE ratings separate how sure we are from how strong a recommendation is” — established. GRADE assesses certainty for each important outcome. Recommendation strength also considers the balance of benefits and harms and decision factors such as values and resources. Read the supporting explanation; one label does not replace the judgment needed for a specific decision.
Weigh industry funding as a risk of bias, not as an automatic fake stamp. Claim: “Industry funding means you can skip reading the methods” — not what the research found. Funding is a reason to inspect the comparator, outcomes, sponsor’s role and publication access more closely. The association reported by Lundh, Bero and colleagues does not establish that an individual sponsored study is false. Skipping methods would remove the information needed to assess that concern.

deep Dive

Use the Ioannidis model to identify fragility. The 2005 essay explains how prior plausibility, study power, bias and repeated testing affect the chance that a positive finding reflects a real relationship. It is a model with assumptions, not a survey proving that a fixed fraction of all science is wrong. For a surprising small study, ask whether the hypothesis and analysis were set in advance, how uncertain the estimate is and whether independent evidence points in the same direction.
Use the current CONSORT checklist as a reporting map. CONSORT 2025 supersedes the 2010 statement and contains 30 items for reporting randomized trials. Its expanded checklist asks for the trial registration, protocol and analysis plan, harms, changes to prespecified methods and the flow of participants, among other details. Use these to locate missing information. A completed checklist is not a risk-of-bias score, and information reported elsewhere in a protocol or supplement should be read before declaring it absent.
Read a GRADE rating outcome by outcome. GRADE can lower certainty for risk of bias, inconsistency, indirectness, imprecision or publication bias. Thus a treatment’s evidence for short-term symptoms may be more certain than its evidence for serious long-term harms. Recommendation strength is a further judgment involving benefits, harms, values, resources and feasibility. Look for the explanation under each outcome and the decision framework instead of assuming one certainty label answers whether everyone should choose the intervention.
Ask what makes a review systematic. A systematic review uses a stated question, explicit eligibility criteria, a reproducible search and an assessment of included evidence. Meta-analysis is an optional statistical combination when studies are suitable; a review can be systematic without pooling. A forest plot displays estimates but cannot correct poor eligibility choices or biased studies by itself. Check the search date, study selection and reasons for combining or separating results before relying on the summary.
Distinguish preregistration from a Registered Report. Preregistration records a plan before results are known. In the Center for Open Science’s Registered Reports model, peer reviewers assess the question and methods before outcomes are observed, and a journal can grant in-principle acceptance independent of whether the result is positive. Both improve transparency, but neither guarantees careful execution. Look for amendments and adherence, and recognize that privacy or consent can limit public data sharing without by itself proving concealment.
Interpret funnel plots as clues with alternatives. A funnel plot compares study estimates with a measure of their precision or size. Cochrane explains that asymmetry can reflect missing results, genuine differences between studies, methodological problems or chance. It is therefore not a diagnosis of publication bias, and tests have important limits when few studies exist. Read the review’s explanation and registry searches alongside the plot rather than counting a symmetrical shape as proof that all evidence is present.
Check whether the conclusion follows the primary analysis. Boutron and colleagues examined 72 published trials with statistically nonsignificant primary outcomes and identified ways reports could still emphasize benefit. Their definition of spin concerned presentation, not an assumption about the authors’ motives. A nonsignificant result also does not prove an effect is exactly zero. Compare the estimate, confidence interval and prespecified outcome with the abstract’s claim, and distinguish an exploratory secondary result from the main test.
Read medical reversal with its selection criteria in view. Prasad and colleagues studied a selected set of published evaluations of medical practice and found examples where later evidence contradicted established use. That denominator is not all treatments or all clinical decisions. The lesson is to ask what evidence supported adoption and whether later trials measured outcomes that matter to patients. A surrogate improvement, such as a laboratory marker, needs evidence connecting it to the clinical benefit being claimed.
Ask what control a sponsor had over the study. Lundh, Bero and colleagues’ review associates industry sponsorship with more favorable results and conclusions in drug and device research. For a specific report, inspect who chose the comparator, held the data, performed the analysis and could influence publication. CONSORT asks for the funder’s role and conflicts of interest because disclosure enables this assessment. Funding is relevant context, but it does not remove the need to judge methods and results.

related

information
learning
attention

watch For Long

Good to know. This is a reading companion about methods, not medical, legal, or financial advice. A headline is not a reason to start or stop a medicine. Bring practice changes to a qualified clinician or the person who owns the decision.
Not advice. Named sources. Honest paraphrase of the finding. Not medical, legal, or financial advice.

sources

John Ioannidis. physician and meta-researcher. Relevant work: why many published findings are less robust than headlines.
Gordon Guyatt. physician; GRADE and evidence-based medicine. Relevant work: GRADE certainty ratings.
Julian Higgins. statistician; Cochrane Handbook. Relevant work: Cochrane methods for reviews.
Douglas Altman. statistician; reporting and bias. Relevant work: better reporting as a public good.
Kenneth Schulz. epidemiologist; CONSORT and allocation concealment. Relevant work: randomized-trial reporting.
Brian Nosek. psychologist; Center for Open Science. Relevant work: reproducibility and preregistration.
Ben Goldacre. physician; unreported trials and better evidence. Relevant work: missing trials and outcome switching.
Vinay Prasad. physician; medical reversal. Relevant work: when later better trials overturn practice.
David Moher. epidemiologist; CONSORT and PRISMA. Relevant work: reporting guidelines.
Andrew Oxman. health-services researcher; GRADE and informed health choices. Relevant work: helping non-specialists use evidence.
Holger Schünemann. physician; GRADE methods. Relevant work: certainty and recommendation strength.
Lisa Bero. research-integrity scholar; industry bias. Relevant work: funding and spin as methods issues.
Jonathan Sterne. statistician; risk-of-bias tools. Relevant work: RoB tools for randomized and nonrandomized studies.
Isabelle Boutron. epidemiologist; spin in abstracts. Relevant work: how abstracts overstate results.
Kay Dickersin. epidemiologist; publication bias. Relevant work: unpublished negative studies.

source Links

IOANNIDIS-2005

name

Ioannidis JPA. Why Most Published Research Findings Are False. PLoS Med. 2005;2(8):e124.

kind

evidence

links

label
Source
href
https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124

COCHRANE-HB

name

Higgins JPT, Thomas J, Chandler J, et al. Cochrane Handbook for Systematic Reviews of Interventions.

kind

evidence

links

label
Source
href
https://training.cochrane.org/handbook

CONSORT

name

Schulz KF, Altman DG, Moher D. CONSORT 2010 statement.

kind

evidence

links

label
Source
href
https://www.consort-prism.org/

GRADE

name

GRADE Working Group. GRADE handbook and papers.

kind

evidence

links

label
Source
href
https://www.gradeworkinggroup.org/

EQUATOR

name

EQUATOR Network. Library of reporting guidelines.

kind

evidence

links

label
Source
href
https://www.equator-network.org/

COS-OSF

name

Nosek BA and Center for Open Science. Open Science Framework.

kind

evidence

links

label
Source
href
https://www.cos.io/

DICKERSIN-BIAS

name

Dickersin K. Publication bias papers.

kind

evidence

links

label
Source
href
https://pubmed.ncbi.nlm.nih.gov/?term=Dickersin+publication+bias

BOUTRON-SPIN

name

Boutron I, Dutton S, Ravaud P, Altman DG. Reporting and interpretation of randomized trials with statistically nonsignificant results.

kind

evidence

links

label
Source
href
https://pubmed.ncbi.nlm.nih.gov/?term=Boutron+spin+randomized

PRASAD-REVERSAL

name

Prasad V, Cifu A. Medical reversal papers and Ending Medical Reversal.

kind

evidence

links

label
Source
href
https://pubmed.ncbi.nlm.nih.gov/?term=Prasad+medical+reversal

GIGERENZER-ABS

name

Gigerenzer G. Risk-communication papers on absolute risk.

kind

evidence

links

label
Source
href
https://pubmed.ncbi.nlm.nih.gov/?term=Gigerenzer+absolute+risk

IOANNIDIS-STD-2026

name

John Ioannidis

kind

professional

links

label
Source
href
https://pubmed.ncbi.nlm.nih.gov/16060722/

GUYATT-STD-2026

name

Gordon Guyatt

kind

professional

links

label
Source
href
https://pubmed.ncbi.nlm.nih.gov/?term=Guyatt+GH+GRADE

HIGGINS-STD-2026

name

Julian Higgins

kind

professional

links

label
Source
href
https://training.cochrane.org/handbook

ALTMAN-STD-2026

name

Douglas Altman

kind

professional

links

label
Source
href
https://pubmed.ncbi.nlm.nih.gov/?term=Altman+DG+reporting

SCHULZ-STD-2026

name

Kenneth Schulz

kind

professional

links

label
Source
href
https://pubmed.ncbi.nlm.nih.gov/?term=Schulz+KF+CONSORT

NOSEK-STD-2026

name

Brian Nosek

kind

professional

links

label
Source
href
https://www.cos.io/

GOLDACRE-STD-2026

name

Ben Goldacre

kind

professional

links

label
Source
href
https://pubmed.ncbi.nlm.nih.gov/?term=Goldacre+B+trials

PRASAD-STD-2026

name

Vinay Prasad

kind

professional

links

label
Source
href
https://pubmed.ncbi.nlm.nih.gov/?term=Prasad+V+medical+reversal

MOHER-STD-2026

name

David Moher

kind

professional

links

label
Source
href
https://pubmed.ncbi.nlm.nih.gov/?term=Moher+D+CONSORT+PRISMA

OXMAN-STD-2026

name

Andrew Oxman

kind

professional

links

label
Source
href
https://pubmed.ncbi.nlm.nih.gov/?term=Oxman+AD+GRADE

SCHUNEMANN-STD-2026

name

Holger Schünemann

kind

professional

links

label
Source
href
https://pubmed.ncbi.nlm.nih.gov/?term=Schunemann+HJ+GRADE

BERO-STD-2026

name

Lisa Bero

kind

professional

links

label
Source
href
https://pubmed.ncbi.nlm.nih.gov/?term=Bero+L+industry+bias

STERNE-STD-2026

name

Jonathan Sterne

kind

professional

links

label
Source
href
https://pubmed.ncbi.nlm.nih.gov/?term=Sterne+JAC+risk+of+bias

BOUTRON-STD-2026

name

Isabelle Boutron

kind

professional

links

label
Source
href
https://pubmed.ncbi.nlm.nih.gov/?term=Boutron+I+spin+abstracts

DICKERSIN-STD-2026

name

Kay Dickersin

kind

professional

links

label
Source
href
https://pubmed.ncbi.nlm.nih.gov/?term=Dickersin+K+publication+bias

BOUTRON-SPIN-2010-EXACT

name

Boutron et al: Reporting and interpretation of RCTs with nonsignificant primary outcomes. Source

kind

evidence

links

label
Source
href
https://pubmed.ncbi.nlm.nih.gov/20501928/

CONSORT-2025-EXACT

name

CONSORT 2025 expanded checklist. Source

kind

evidence

links

label
Source
href
https://www.consort-spirit.org/_files/ugd/b5740e_a6856e5e2cf94a1db5a8005853404160.pdf

COCHRANE-15-EXACT

name

Cochrane Handbook chapter 15: Interpreting results. Source

kind

evidence

links

label
Source
href
https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-15

MEDRXIV-ABOUT-EXACT

name

medRxiv: About preprints. Source

kind

evidence

links

label
Source
href
https://www.medrxiv.org/about-biorxiv

COCHRANE-13-EXACT

name

Cochrane Handbook chapter 13: Missing evidence. Source

kind

evidence

links

label
Source
href
https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-13

GRADE-CURRENT-EXACT

name

GRADE Working Group: Criteria for using GRADE. Source

kind

evidence

links

label
Source
href
https://www.gradeworkinggroup.org/

PRASAD-REVERSAL-2013-EXACT

name

Prasad et al: A decade of reversal. Source

kind

evidence

links

label
Source
href
https://pubmed.ncbi.nlm.nih.gov/23871230/

LUNDH-FUNDING-2017-EXACT

name

Lundh et al: Industry sponsorship and research outcome. Source

kind

evidence

links

label
Source
href
https://pubmed.ncbi.nlm.nih.gov/28207928/

CONSORT-2025-STATEMENT

name

CONSORT 2025 statement. Source

kind

evidence

links

label
Source
href
https://www.bmj.com/content/389/bmj-2024-081123

COS-REGISTERED-REPORTS-EXACT

name

Center for Open Science: Registered Reports. Source

kind

evidence

links

label
Source
href
https://www.cos.io/initiatives/registered-reports

source Claims

watch For

IOANNIDIS-2005
COCHRANE-HB
IOANNIDIS-STD-2026

situations.headline

IOANNIDIS-2005
BOUTRON-SPIN
IOANNIDIS-STD-2026
BOUTRON-STD-2026
COCHRANE-HB
BOUTRON-SPIN-2010-EXACT

situations.methods

COCHRANE-HB
CONSORT
HIGGINS-STD-2026
SCHULZ-STD-2026
STERNE-STD-2026
CONSORT-2025-EXACT

situations.absolute-risk

GIGERENZER-ABS
GRADE
GUYATT-STD-2026
OXMAN-STD-2026
COCHRANE-15-EXACT

situations.preprint

COS-OSF
NOSEK-STD-2026
GOLDACRE-STD-2026
EQUATOR
MEDRXIV-ABOUT-EXACT
CONSORT-2025-EXACT

split.established.0

COCHRANE-HB
HIGGINS-STD-2026

split.established.1

CONSORT
SCHULZ-STD-2026
STERNE-STD-2026
ALTMAN-STD-2026
CONSORT-2025-EXACT
COCHRANE-HB

split.established.2

GIGERENZER-ABS
GRADE
GUYATT-STD-2026
OXMAN-STD-2026
COCHRANE-15-EXACT

split.established.3

DICKERSIN-BIAS
BOUTRON-SPIN
GOLDACRE-STD-2026
DICKERSIN-STD-2026
COCHRANE-13-EXACT
BOUTRON-SPIN-2010-EXACT

split.established.4

GRADE
GUYATT-STD-2026
SCHUNEMANN-STD-2026
OXMAN-STD-2026
GRADE-CURRENT-EXACT

split.established.5

EQUATOR
CONSORT
MOHER-STD-2026
ALTMAN-STD-2026
CONSORT-2025-EXACT

split.disputed.0

IOANNIDIS-2005
IOANNIDIS-STD-2026
COS-OSF
NOSEK-STD-2026

split.disputed.1

PRASAD-REVERSAL
PRASAD-STD-2026
GRADE
GUYATT-STD-2026
PRASAD-REVERSAL-2013-EXACT

split.disputed.2

BERO-STD-2026
COCHRANE-HB
HIGGINS-STD-2026
LUNDH-FUNDING-2017-EXACT

split.disputed.3

COS-OSF
NOSEK-STD-2026
GOLDACRE-STD-2026
CONSORT-2025-EXACT

split.talk.0

GIGERENZER-ABS
GRADE
COCHRANE-15-EXACT

split.talk.1

COS-OSF
EQUATOR
NOSEK-STD-2026
MEDRXIV-ABOUT-EXACT

split.talk.2

IOANNIDIS-2005
COCHRANE-HB
HIGGINS-STD-2026

split.talk.3

GOLDACRE-STD-2026
CONSORT
BOUTRON-SPIN
BOUTRON-SPIN-2010-EXACT
CONSORT-2025-EXACT

move

COCHRANE-HB
GIGERENZER-ABS
GRADE
COCHRANE-15-EXACT
GRADE-CURRENT-EXACT

try.0

BOUTRON-SPIN
GIGERENZER-ABS
BOUTRON-SPIN-2010-EXACT

try.1

CONSORT
STERNE-STD-2026
CONSORT-2025-EXACT
COCHRANE-HB

try.2

GOLDACRE-STD-2026
COCHRANE-HB
CONSORT-2025-EXACT

keep.0

COCHRANE-HB
HIGGINS-STD-2026

keep.1

GIGERENZER-ABS
GRADE
COCHRANE-15-EXACT

keep.2

STERNE-STD-2026
CONSORT
COCHRANE-HB
CONSORT-2025-EXACT

keep.3

DICKERSIN-BIAS
BOUTRON-SPIN
GOLDACRE-STD-2026
COCHRANE-13-EXACT

keep.4

EQUATOR
COCHRANE-HB

deep.0

IOANNIDIS-2005
IOANNIDIS-STD-2026

deep.1

CONSORT
SCHULZ-STD-2026
ALTMAN-STD-2026
MOHER-STD-2026
CONSORT-2025-EXACT
CONSORT-2025-STATEMENT

deep.2

GRADE
GUYATT-STD-2026
SCHUNEMANN-STD-2026
OXMAN-STD-2026
GRADE-CURRENT-EXACT

deep.3

COCHRANE-HB
HIGGINS-STD-2026

deep.4

COS-OSF
NOSEK-STD-2026
COS-REGISTERED-REPORTS-EXACT
CONSORT-2025-EXACT

deep.5

DICKERSIN-BIAS
DICKERSIN-STD-2026
GOLDACRE-STD-2026
COCHRANE-13-EXACT

deep.6

BOUTRON-SPIN
BOUTRON-STD-2026
ALTMAN-STD-2026
BOUTRON-SPIN-2010-EXACT

deep.7

PRASAD-REVERSAL
PRASAD-STD-2026
PRASAD-REVERSAL-2013-EXACT

deep.8

BERO-STD-2026
COCHRANE-HB
LUNDH-FUNDING-2017-EXACT
CONSORT-2025-EXACT

claims.0

COCHRANE-HB
IOANNIDIS-2005
BOUTRON-SPIN

claims.1

GIGERENZER-ABS
GRADE
COCHRANE-15-EXACT

claims.2

IOANNIDIS-2005
COCHRANE-HB

claims.3

COS-OSF
NOSEK-STD-2026
MEDRXIV-ABOUT-EXACT

claims.4

DICKERSIN-BIAS
GOLDACRE-STD-2026
BOUTRON-SPIN
COCHRANE-13-EXACT
BOUTRON-SPIN-2010-EXACT

claims.5

GRADE
GUYATT-STD-2026
GRADE-CURRENT-EXACT

claims.6

BERO-STD-2026
COCHRANE-HB
LUNDH-FUNDING-2017-EXACT
CONSORT-2025-EXACT

source Situation Keys

situations.headline
situations.methods
situations.absolute-risk
situations.preprint

professional Source Ids

IOANNIDIS-STD-2026
GUYATT-STD-2026
HIGGINS-STD-2026
ALTMAN-STD-2026
SCHULZ-STD-2026
NOSEK-STD-2026
GOLDACRE-STD-2026
PRASAD-STD-2026
MOHER-STD-2026
OXMAN-STD-2026
SCHUNEMANN-STD-2026
BERO-STD-2026
STERNE-STD-2026
BOUTRON-STD-2026
DICKERSIN-STD-2026

house Pointers

href
/learn/
label
STUDlearn. For checking online claims, see Making sense of online information.

positive Guidance

status: DONE
reviewed At: 2026-09-18
professional Count: 15
scope: content-and-positive-titles-only

learn Article

article

format version

presentation

complete-card

overview

sections

id
find-the-question
role
explanation
title
Find the question behind the headline
intro
You do not need to understand every equation before you can start. First establish what kind of answer this paper can provide.
items
- label
  Rewrite the headline as a question about specific people
  text
  When a headline says a treatment works, find the question in the paper itself. For a treatment comparison, PICO means Population, Intervention, Comparator and Outcome: who participated, what they received, what it was compared with and what changed. Add the follow-up period. A study of adults with a particular condition over six months does not automatically answer a question about healthy teenagers over ten years. Cochrane's question-setting guidance also distinguishes estimating how common something is from testing a treatment or understanding people's experiences. Keep the conclusion attached to the question the researchers actually investigated.
  source ids
  cochrane-questions
- label
  Ask how people came to receive the exposure or treatment
  text
  Suppose people who choose an exercise programme later report better health. Exercise may help, but those participants may also differ in income, starting health or access to care. Such a third factor is a confounder when it influences both the exposure and the outcome. An observational study records choices or circumstances; a randomized trial assigns a comparison by chance. Adjustment can address measured differences, while unmeasured differences may remain. Cochrane's guidance on non-randomized studies therefore asks readers to inspect the causal comparison and its assumptions. An association deserves investigation, but its design and measurements limit the conclusion it supports.
  source ids
  cochrane-nonrandomized
- label
  Give a percentage a starting risk and a time period
  text
  A headline announcing a 50 percent reduction leaves out information needed to understand the result. In an invented example, a fall from 2 events per 100 people to 1 per 100 halves the risk and prevents 1 event per 100. A fall from 50 to 25 per 100 also halves it, but prevents 25 events per 100. These are absolute reductions of 1 and 25 percentage points, respectively. Neither example describes a real treatment. Find the actual groups, event definition and observation period in the paper; if the headline omits them, do not invent them. The same relative change can mean very different practical benefits.
  source ids
  cochrane-effect-measures
- label
  Check a preprint's version and later review
  text
  A preprint makes a manuscript public before it completes journal peer review; its experiment may already be complete. That can make new work available sooner, but availability does not establish that its methods or conclusions are sound. JAMA's discussion of medical preprints explains both the opportunity and the danger of acting on an early report. Read the methods, identify the version, and look for a later journal article, correction or other evidence on the same question. Journal peer review adds scrutiny; it does not certify truth. A preprint is not, by itself, a reason to change medical practice, and a journal label does not replace reading the study.
  source ids
  jama-medical-preprints
id
inspect-the-comparison
role
findings
title
Inspect how the comparison was made
items
- label
  Separate random assignment, concealed allocation and blinding
  text
  Random assignment helps create comparable groups. Allocation concealment prevents someone enrolling participants from knowing the next assignment and influencing who enters which group. Blinding keeps participants, care providers or outcome assessors unaware of assignments after enrolment, where feasible. These safeguards address different problems. CONSORT 2025 asks authors to explain them separately. A paper calling itself randomized still needs an account of how assignment worked, who knew it, and whether departures from the intended treatment could distort the comparison.
  source ids
  consort-2025
  cochrane-trial-bias
- label
  Follow participants from assignment to the reported result
  text
  Read the flow diagram alongside the results: how many people entered each group, received the intervention, were followed and contributed measurements? Missing outcomes matter especially when their absence is related to how people were doing, and the problem may differ between groups. Cochrane's trial-bias guidance does not treat a fixed percentage of missing data as a universal safety threshold. An intention-to-treat approach aims to analyse people according to their assigned group, but that label cannot recover an outcome that was never measured. Look for reasons for missingness, assumptions and analyses testing those assumptions.
  source ids
  cochrane-trial-bias
- label
  Distinguish a meaningful outcome from a convenient marker
  text
  An outcome is what the study measures. A clinical outcome can concern how someone feels, functions or survives; a surrogate is a marker used to stand in for a clinical benefit. A laboratory value can be useful without proving that people live longer or feel better. FDA guidance distinguishes surrogates supported by validation from those only reasonably likely to predict benefit. Ask what links this particular marker to the outcome you care about, for this use. Also check whether the paper measured that outcome directly or inferred it.
  source ids
  fda-surrogate-endpoints
- label
  Read harms with the same attention as benefits
  text
  A short trial with few participants may detect a benefit yet miss an uncommon or delayed harm. Cochrane's adverse-effects guidance explains why other designs, including observational studies, can contribute information that trials do not supply. Check how harms were sought, defined and counted, how long people were observed, and whether both groups had the same opportunities for detection. A report of no observed events is different from evidence that an event cannot occur. Spontaneous reports can identify a concern but, without a suitable denominator, cannot establish its frequency.
  source ids
  cochrane-harms
- label
  Use reporting guidelines to locate information, then assess bias separately
  text
  A reporting checklist asks whether readers have the information needed to understand a study. CONSORT 2025 has 30 items for randomized trials; STROBE covers major observational designs, and PRISMA 2020 covers systematic reviews. EQUATOR helps locate the relevant guideline. Reporting completeness and risk of bias are different judgments: a clearly described method can still systematically distort a result, while an omitted detail may leave its risk unclear. STROBE explicitly says its checklist is not a study-quality scoring tool. Find the methods and supplements before treating a filled checklist as reassurance.
  source ids
  equator-library
  equator-consort
  consort-2025
  strobe
  prisma-2020
  cochrane-bias
id
read-size-and-uncertainty
role
findings
title
Read the size and uncertainty of the result
items
- label
  Translate a result into an amount people can understand
  text
  For an event outcome, record the absolute difference as well as the relative effect. In the invented examples above, preventing 1 event per 100 corresponds to a number needed to treat of 100; preventing 25 per 100 corresponds to 4. This calculation is one divided by the absolute risk reduction expressed as a proportion. It describes an average additional benefit over the stated period, compared with the specified alternative. Cochrane cautions that baseline risk, follow-up and uncertainty affect this interpretation. Keep possible harms beside the benefit rather than using one attractive number as a decision rule.
  source ids
  cochrane-interpretation
- label
  Keep the effect measure and its units visible
  text
  A risk ratio compares event probabilities; an odds ratio compares odds, which are the probability of an event divided by the probability of no event. They are different measures and can differ substantially when events are common. For a numerical outcome, a mean difference retains the measurement scale, such as points on a symptom questionnaire. Check what direction means improvement and whether that amount is meaningful. A ratio of 1 or a difference of 0 represents no contrast on those respective scales; neither tells you how uncertain the estimate is.
  source ids
  cochrane-effect-measures
- label
  Read the interval before declaring an effect present or absent
  text
  A confidence interval shows uncertainty around an estimate under the statistical method's assumptions. For a conventional 95 percent interval, the procedure is designed to cover the true value in 95 percent of repeated applications; it does not assign a 95 percent probability to the fixed true value in this particular interval. Look at whether the interval includes effects that would matter, including benefit and harm. An estimate near zero with a wide interval may leave an important effect unresolved. Precision does not protect against a biased comparison.
  source ids
  cochrane-interpretation
  asa-task-force
- label
  Interpret a p-value in relation to the tested model
  text
  A p-value asks how unusual results at least as extreme as those observed would be if the specified statistical model, including its null hypothesis, were correct. It is not the probability that the hypothesis is true, or the probability that chance alone caused the findings. The American Statistical Association's 2016 statement also warns that statistical significance does not measure effect size or importance. A small p-value cannot settle whether the comparison was fair, the measurement useful or the result applicable. A large one does not establish an exact zero effect.
  source ids
  asa-p-values
- label
  Ask what the study was capable of detecting
  text
  Power is a study's probability of meeting a specified statistical test criterion when an assumed effect really exists. It depends on the effect, variability, sample and test. A small, imprecise study can miss an important effect; a very large one can detect an effect too small to matter. Ioannidis's modelling also connects low power, prior plausibility, bias and repeated testing to the reliability of positive findings. Read the sample-size assumptions and the actual interval, rather than treating the participant count as a universal grade.
  source ids
  ioannidis-2005
  asa-p-values
id
trace-plans-and-publication
role
findings
title
Trace what was planned and what became visible
items
- label
  Compare the dated plan with the published outcome
  text
  A registration records a study; a protocol describes its design; a statistical analysis plan specifies how results will be analysed. The primary outcome addresses the main planned question; secondary outcomes address additional questions. Check dates and versions rather than assuming every document preceded knowledge of the results. Preregistration helps distinguish planned tests from analyses chosen after seeing data. Changes may be legitimate, but their timing and reasons should be visible. If a paper highlights a secondary or exploratory result after an uncertain primary result, keep that distinction in your conclusion. CONSORT 2025 requests protocols, analysis plans and explanations of important changes.
  source ids
  cos-preregistration
  consort-2025
- label
  Compare the abstract's conclusion with the primary result
  text
  Boutron and colleagues examined 72 published parallel-group randomized trials with statistically nonsignificant primary results, drawn from a defined 2006 publication sample. They identified presentation that emphasized favourable results or drew conclusions beyond the primary result, which they called spin. This is evidence about a selected set of reports, not the rate of spin in every field or evidence of each author's motive. Follow the abstract's claim into the primary-outcome table and its uncertainty. Useful secondary findings can remain useful when their status is clear; they do not silently replace the original question.
  source ids
  boutron-spin
- label
  Look for evidence that did not reach the visible papers
  text
  The studies you can find may be a selected subset of the studies conducted. Entire studies can remain unpublished, and particular results can be omitted within a published report. Cochrane describes how selection related to results can distort a review's conclusion. Registries, protocols and attempts to obtain unavailable results help reviewers investigate the gap. None guarantees a complete record. When a review says it searched for missing evidence, read where it looked and what remained unavailable rather than assuming publication alone makes the evidence representative.
  source ids
  cochrane-missing-evidence
  cochrane-searching
- label
  Examine who could influence the study and its reporting
  text
  Lundh, Bero and colleagues' 2017 Cochrane review included 75 empirical papers examining industry sponsorship. Industry-sponsored drug and device studies more often had favourable efficacy results and conclusions than studies with other sponsorship. The finding is an association across the included evidence, not proof that a particular sponsored paper is false. Inspect the comparator, outcomes, access to data, control of analysis and ability to publish unfavourable results. Funding disclosure starts that inquiry. Non-financial interests and investigators' own commitments can also matter; skipping the methods would remove the information needed to assess them.
  source ids
  lundh-sponsorship
  cochrane-bias
id
place-in-wider-evidence
role
findings
title
Place the paper beside the wider evidence
items
- label
  Look for a systematic review that answers the same question
  text
  A systematic review uses an explicit question, eligibility rules stating which studies qualify, a reproducible search and an assessment of the included evidence. A meta-analysis is a statistical combination of results; some systematic reviews appropriately do not perform one. PRISMA 2020 helps readers see how a review was conducted. Check the last search date, relevant populations, comparisons and outcomes, and reasons for including or excluding studies. A review can be rigorous yet answer a different question from yours. Its title or size cannot replace that match.
  source ids
  prisma-explanation
  cochrane-searching
  cochrane-questions
- label
  Read the studies behind a forest plot's summary
  text
  A forest plot commonly displays each study's estimate with a line for its confidence interval; a summary diamond shows a pooled estimate and interval. Larger symbols often indicate greater statistical weight. Read the axis and legend, because displays vary. Cochrane's meta-analysis guidance asks whether it makes sense to combine the studies and how differences between them affect interpretation. A precise pooled result cannot repair biased input studies. If populations, treatments or outcomes differ materially, ask whether the summary obscures the comparison you need.
  source ids
  cochrane-meta-analysis
- label
  Read a funnel plot with its possible explanations
  text
  A funnel plot places study estimates against a measure of their size or precision. An asymmetric pattern may reflect missing results, differences between smaller and larger studies, other bias or chance. Cochrane therefore cautions against treating asymmetry as a diagnosis of publication bias. A symmetric plot does not prove that nothing is missing either. These assessments are especially limited when few studies are available; tests are generally not recommended with fewer than ten. Read the explanation and search methods alongside the picture.
  source ids
  cochrane-missing-evidence
- label
  Read certainty separately for each important outcome
  text
  GRADE describes certainty in an evidence body for a particular outcome as high, moderate, low or very low. Cochrane's GRADE guidance considers risk of bias, inconsistency between studies, indirectness to the question, imprecision and publication bias. These judgments require explanations, not merely counting studies. One treatment can have relatively certain evidence about symptom improvement and uncertain evidence about a rare harm. Read both before repeating a single label as if it covered every consequence.
  source ids
  cochrane-grade
- label
  Distinguish confidence in the evidence from a recommendation to act
  text
  Certainty concerns how confidently an effect is known. Recommendation strength also considers the balance of benefits and harms, people's values and decision factors such as resources, equity and feasibility. The GRADE Working Group treats these as connected but distinct judgments. A strong recommendation is not simply a synonym for a large effect or high certainty. Read the reasoning, applicable setting and options. Your circumstances may differ from those used by a guideline panel, so one label is not a personal instruction.
  source ids
  grade-working-group
id
understand-judgment-calls
role
perspectives
title
Understand where reasonable judgments differ
intro
These are specific choices about evidence and decisions. They do not make every interpretation equally well supported.
items
- label
  Recognize the debate about significance thresholds
  text
  Some statistical guidance urges readers to avoid splitting findings into significant and nonsignificant boxes. The ASA president's 2021 task force instead defended carefully used significance tests and noted that thresholds can serve explicit decisions. It also distinguished its position from a 2019 editorial proposing abandonment of statistical significance. The shared practical requirement is to explain the estimate, uncertainty, assumptions and consequences. If a decision needs a cutoff, say why that cutoff fits the decision; crossing 0.05 alone does not determine scientific or practical importance.
  source ids
  asa-task-force
  cochrane-interpretation
- label
  Choose the evidence design for the question and its constraints
  text
  Randomization can strengthen a treatment-effect comparison, but it is not available or sufficient for every question. Rare or delayed harms may require long observational follow-up; deliberately assigning a harmful exposure may be unacceptable. Cochrane describes a role for non-randomized evidence while requiring careful attention to confounding and other biases. Readers and reviewers can differ over how much confidence to place in that evidence for a particular decision. Identify the question, why this design was used, and which unresolved differences could change the answer.
  source ids
  cochrane-nonrandomized
  cochrane-harms
- label
  Make the tradeoff between earlier access and confirmed benefit visible
  text
  Waiting for patient outcomes and acting on an earlier surrogate involve different risks, especially in serious disease with few options. FDA's accelerated-approval framework permits some reasonably likely surrogates while requiring further work to confirm clinical benefit. That is a conditional decision under uncertainty. Prasad and colleagues documented reversals of established practices in selected articles from one major journal during 2001–2010. Their denominator does not represent all medical care. Together these sources support asking what evidence justified adoption and what later evidence could confirm or overturn it, rather than assuming every established practice is either proven or worthless.
  source ids
  fda-surrogate-endpoints
  prasad-reversal
- label
  Give planned confirmation and open exploration their proper jobs
  text
  Exploration can discover an unexpected pattern; a planned test can provide a more interpretable check of a specified claim. Preregistration makes that distinction visible. Registered Reports go further: journals evaluate a proposal before the results are known and can offer acceptance in principle, subject to meeting agreed conditions. The Center for Open Science acknowledges constraints involving timing, resources and concerns about flexibility; it also explains that exploratory analyses can still be reported. Readers should not treat exploration as wrongdoing or as completed confirmation. Neither format guarantees a good design, faithful execution or a useful result.
  source ids
  cos-preregistration
  cos-registered-reports
- label
  Turn broad scepticism into a specific methods question
  text
  Ioannidis's 2005 essay uses a mathematical model to examine how prior plausibility, power, bias and multiple testing can undermine positive findings. Its conclusions depend on assumptions about those conditions; it is not a census establishing that all published studies are false or fabricated. The journal also records a 2022 correction to parentheses in a table equation. You do not need to accept or reject all research at once. Ask which feature makes this claim fragile and what independent evidence could strengthen it. Specific criticism gives you something to check and something that could change your mind.
  source ids
  ioannidis-2005
  ioannidis-correction
id
keep-a-reading-record
role
application
title
Keep a record you can use later
intro
A short note helps you remember the study's answer without detaching it from its limits.
items
- label
  Keep the question attached to your conclusion
  text
  Write who was studied, the comparison, outcome and duration before your one-sentence conclusion. If your own question concerns a different group or outcome, name that gap. This preserves the difference between what the paper measured and what you hope it means.
  source ids
  cochrane-questions
- label
  Save the result with its denominator and uncertainty
  text
  Keep the original outcome units, group results, effect estimate, interval and time period together. If absolute effects are unavailable, record that limitation. A saved percentage without its starting risk becomes a different claim when you repeat it. Here is an invented result to carry into a reading note: over one year, an event occurs in 20 of 1,000 people in one group and 10 of 1,000 in the comparison group. The observed risks are 2% and 1%. The difference is 1 percentage point, or 10 fewer events per 1,000 people over that year. Relative to the 2% starting risk, the observed reduction is 50%. All of those descriptions use the same numbers; '50% lower' alone conceals the starting risk. The reading note still needs the allocation method, missing-data information and uncertainty interval before a strong interpretation. These toy counts do not supply those details or establish that the intervention caused the difference. If the headline concerns lifetime benefit, the one-year result cannot settle it. Keeping denominator, duration and limits beside the estimate prevents the more impressive wording from quietly becoming a broader claim.
  source ids
  cochrane-effect-measures
  cochrane-interpretation
- label
  Record the method concern that could change the answer
  text
  Use a concrete statement such as 'missing outcomes may differ between groups' or 'the comparator does not match my decision.' Separate an unreported detail from an observed methodological problem. A reporting checklist can locate the information; a bias assessment explains why a problem might distort the result.
  source ids
  cochrane-bias
  strobe
- label
  Record where you looked for missing evidence
  text
  Save the registration or protocol link and the relevant review's last search date. Note unavailable studies or results and how the review tried to find them. This lets a later reader check what may be missing rather than mistake your collection for everything studied.
  source ids
  cochrane-searching
  cochrane-missing-evidence
- label
  Update the question when the evidence changes
  text
  Record the paper's version, a matching review or guideline, and the outcome-specific certainty where available. Revisit the note when a correction, additional study or updated review appears. Preserve the earlier uncertainty in your record so that a changed conclusion is understandable rather than hidden.
  source ids
  prisma-explanation
  cochrane-grade
  jama-medical-preprints
id
respect-use-and-access-limits
role
cautions
title
Respect the limits of use and access
items
- label
  Bring practice changes to the responsible professional
  text
  This is a reading companion about methods, not medical, legal, or financial advice. A headline is not a reason to start or stop a medicine. Bring practice changes to a qualified clinician or the person who owns the decision.
  source ids
  jama-medical-preprints
  grade-working-group
- label
  Ask for transparent access conditions while protecting participants
  text
  Open materials can make scrutiny easier, but research participants' identifiable data are not automatically suitable for public release. PLOS's data policy recognizes privacy, consent and ethical restrictions and asks authors to explain restrictions and an appropriate access route. A justified limit is different from unexplained concealment. Look for the availability statement, what can be checked, who controls access and why a restriction exists.
  source ids
  plos-data-access
- label
  Check the evidence behind a consequential choice
  text
  Named sources. Honest paraphrase of the finding. Not medical, legal, or financial advice.
  source ids
  grade-working-group
  jama-medical-preprints
id
read-one-claim-through
role
next_steps
title
Read one claim all the way through
items
- label
  Start with one paper and a five-part note
  text
  Choose a study behind a claim you encountered. Write its question, design, result in the original units, uncertainty and follow-up period. Leave a blank wherever you cannot find the answer. The point is to establish what you can responsibly say, then identify the missing information that would help most.
  source ids
  ihc-key-concepts
  asa-task-force
- label
  Trace the headline into the methods and results
  text
  Read the primary outcome, its group results and interval in the full paper and supplements. Follow participant flow and missing outcomes. Then compare the dated registration, protocol or analysis plan with the published report, including disclosed amendments. Write whether the headline describes the primary result or an exploratory finding, and what methodological uncertainty remains.
  source ids
  consort-2025
  cochrane-bias
  boutron-spin
- label
  Find the wider evidence before making a practical decision
  text
  Look for a systematic review or relevant guideline answering the same question. Compare its population, outcomes, search date, missing-evidence assessment and reasoning about certainty. If the decision concerns treatment, legal action or money, take the question and your note to the qualified professional or responsible decision-maker. Use the information, learning and attention topics to continue practising how you find, retain and inspect a claim.
  source ids
  cochrane-searching
  grade-working-group
  ihc-key-concepts

related

information
learning
attention

sources

id
cochrane-questions
title
Cochrane Handbook, Chapter 3: Defining the criteria for including studies and how they will be grouped for the synthesis
url
https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-03
kind
guideline
accessed at
2026-10-01
supports
Opened methods chapter: defining populations, interventions, comparisons, outcomes, eligible designs and the question a review can answer.
limitations
Systematic-review methodology; PICO suits intervention questions and is not imposed on all research questions.
id
cochrane-effect-measures
title
Cochrane Handbook, Chapter 6: Choosing effect measures and computing estimates of effect
url
https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-06
kind
guideline
accessed at
2026-10-01
supports
Opened definitions of risk, odds, relative and absolute measures, continuous outcomes and null values. Article arithmetic examples are invented illustrations.
limitations
Technical methods guidance; an effect measure alone supplies neither clinical importance nor a personal forecast.
id
cochrane-bias
title
Cochrane Handbook, Chapter 7: Considering bias and conflicts of interest among the included studies
url
https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-07
kind
guideline
accessed at
2026-10-01
supports
Opened distinction between reporting information and evaluating systematic distortion, including conflicts of interest.
limitations
Risk-of-bias judgments are tied to a result and design; disclosure or reporting completeness is not an automatic validity verdict.
id
cochrane-trial-bias
title
Cochrane Handbook, Chapter 8: Assessing risk of bias in a randomized trial
url
https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-08
kind
guideline
accessed at
2026-10-01
supports
Opened randomization, deviations, missing outcome data and intention-to-treat limitations; no universal safe missing-data percentage.
limitations
RoB 2 guidance requires study-specific information and judgment, not a checklist score.
id
cochrane-meta-analysis
title
Cochrane Handbook, Chapter 10: Analysing data and undertaking meta-analyses
url
https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-10
kind
guideline
accessed at
2026-10-01
supports
Opened forest-plot conventions, pooling decisions, heterogeneity and the limits of synthesis when studies are biased or different.
limitations
Visual conventions vary; pooled precision does not establish validity or applicability.
id
cochrane-missing-evidence
title
Cochrane Handbook, Chapter 13: Assessing risk of bias due to missing evidence in a synthesis
url
https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-13
kind
guideline
accessed at
2026-10-01
supports
Opened publication/selective-reporting bias, alternatives explaining funnel asymmetry, and limited power of assessments with few studies.
limitations
Neither plot symmetry nor asymmetry settles whether evidence is missing; the ten-study guidance is a rule for tests, not proof of safety above ten.
id
cochrane-grade
title
Cochrane Handbook, Chapter 14: Completing Summary of findings tables and grading the certainty of the evidence
url
https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-14
kind
guideline
accessed at
2026-10-01
supports
Opened outcome-specific certainty levels and risk of bias, inconsistency, indirectness, imprecision and publication bias considerations.
limitations
GRADE includes additional considerations, including circumstances for rating up; this introduction does not replace a full assessment.
id
cochrane-interpretation
title
Cochrane Handbook, Chapter 15: Interpreting results and drawing conclusions
url
https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-15
kind
guideline
accessed at
2026-10-01
supports
Opened absolute effects, baseline risk, time-bound NNT, uncertainty, confidence intervals and cautions about binary significance interpretation.
limitations
Interpretation depends on outcome, assumptions and applicability; example NNT calculations are teaching arithmetic, not observed treatment results.
id
cochrane-searching
title
Cochrane Handbook, Chapter 4: Searching for and selecting studies
url
https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-04
kind
guideline
accessed at
2026-10-01
supports
Opened reproducible searching, selection, registry sources and the importance of a review's search dates and scope.
limitations
A comprehensive search reduces but cannot guarantee elimination of unavailable or unreported evidence.
id
cochrane-harms
title
Cochrane Handbook, Chapter 19: Adverse effects
url
https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-19
kind
guideline
accessed at
2026-10-01
supports
Opened adverse-effect detection, rare and delayed harms, contributions of observational designs and spontaneous-report denominator limits.
limitations
Harms evidence has design- and outcome-specific limitations; no observed event does not prove no risk.
id
cochrane-nonrandomized
title
Cochrane Handbook, Chapter 24: Including non-randomized studies on intervention effects
url
https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-24
kind
guideline
accessed at
2026-10-01
supports
Opened causal questions, confounding, selection and the role and limits of non-randomized intervention evidence.
limitations
Illustrative exercise example is authored, not a finding from an exercise study; the chapter does not authorize causal claims from any association.
id
consort-2025
title
CONSORT 2025 statement: updated guideline for reporting randomised trials
url
https://www.consort-spirit.org/_files/ugd/b5740e_a6856e5e2cf94a1db5a8005853404160.pdf
kind
guideline
accessed at
2026-10-01
supports
Opened official 12-page statement PDF, including the 30-item checklist, concealment versus blinding, flow, harms, registration, protocol, analysis plan and amendments.
limitations
A reporting standard, not a study-quality score or guarantee that a trial was conducted soundly.
id
equator-consort
title
EQUATOR Network: CONSORT 2025 Statement
url
https://www.equator-network.org/reporting-guidelines/consort/
kind
official
accessed at
2026-10-01
supports
Opened current registry entry identifying the 2025 randomized-trial reporting guideline and its supporting materials.
limitations
Guideline registry metadata; the full statement supplies methodological detail.
id
equator-library
title
EQUATOR Network: Enhancing the QUAlity and Transparency Of health Research
url
https://www.equator-network.org/
kind
official
accessed at
2026-10-01
supports
Opened reporting-guideline library and study-type navigation.
limitations
Finding an applicable reporting guideline is different from assessing bias in a study.
id
strobe
title
STROBE: Strengthening the Reporting of Observational Studies in Epidemiology
url
https://www.strobe-statement.org/
kind
guideline
accessed at
2026-10-01
supports
Opened official scope for cohort, case-control and cross-sectional reporting and explicit warning against use as a quality-assessment instrument.
limitations
Reporting recommendations are not design prescriptions or an observational-study quality score.
id
prisma-2020
title
PRISMA 2020 statement
url
https://www.prisma-statement.org/prisma-2020-statement
kind
guideline
accessed at
2026-10-01
supports
Opened official 27-item checklist and supporting links for systematic-review reporting.
limitations
Transparent reporting is not sufficient to establish review validity; PRISMA and CONSORT address different report types.
id
prisma-explanation
title
The PRISMA 2020 statement: An updated guideline for reporting systematic reviews
url
https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1003583
kind
guideline
accessed at
2026-10-01
supports
Opened primary guideline publication explaining transparent review methods and the scope of reviews with or without statistical synthesis.
limitations
A reporting guideline does not guarantee an exhaustive search or an appropriate pooled analysis.
id
grade-working-group
title
GRADE Working Group
url
https://www.gradeworkinggroup.org/
kind
official
accessed at
2026-10-01
supports
Opened GRADE criteria and Evidence to Decision framework; outcome-specific certainty is distinct from recommendation strength and decision considerations.
limitations
The criteria include a May 2023 update; documents linked from the page have their own publication dates.
id
asa-p-values
title
American Statistical Association releases statement on statistical significance and p-values
url
https://www.amstat.org/asa/files/pdfs/p-valuestatement.pdf
kind
professional
accessed at
2026-10-01
supports
Opened official 2016 statement summary and its six principles: model compatibility, no hypothesis-probability interpretation, no effect-size interpretation and no threshold-only reasoning.
limitations
Principles guide interpretation rather than prescribe a single statistical analysis for every study.
id
asa-task-force
title
Statement of the ASA President's Task Force on Statistical Significance and Replicability
url
https://magazine.amstat.org/blog/2021/08/01/task-force-statement-p-value/
kind
professional
accessed at
2026-10-01
supports
Opened 2021 task-force discussion of tests, intervals, decision thresholds and its distinction from the 2019 abandonment editorial.
limitations
Task-force position; it does not establish that all statisticians agree on significance terminology or threshold use.
id
cos-preregistration
title
Center for Open Science: Preregistration
url
https://www.cos.io/initiatives/prereg
kind
official
accessed at
2026-10-01
supports
Opened explanation of separating planned confirmatory tests from exploratory analyses and transparently reporting deviations.
limitations
An initiative's methods guidance, not evidence that registration alone fixes design or execution.
id
cos-registered-reports
title
Center for Open Science: Registered Reports
url
https://www.cos.io/initiatives/registered-reports
kind
official
accessed at
2026-10-01
supports
Opened pre-results peer review, conditional acceptance, exploratory-analysis provisions, practical constraints and public professional explainers.
limitations
Acceptance in principle remains conditional on agreed standards; the format does not guarantee useful or unbiased findings.
id
ioannidis-2005
title
Why Most Published Research Findings Are False
url
https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124
kind
professional
accessed at
2026-10-01
supports
Opened 2005 methods essay and modelling of prior odds, power, bias and multiple testing; distinguished model conclusions from an empirical census.
limitations
Assumption-dependent model, not a measurement of the fraction of all science that is false or fabricated; no equations or model-derived prevalence reproduced.
id
ioannidis-correction
title
Correction: Why Most Published Research Findings Are False
url
https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1004085
kind
official
accessed at
2026-10-01
supports
Opened August 25, 2022 correction identifying missing parentheses in one Table 2 equation.
limitations
Specific equation correction, not a retraction or a new empirical test of the original model.
id
boutron-spin
title
Reporting and Interpretation of Randomized Controlled Trials With Statistically Nonsignificant Results for Primary Outcomes
url
https://jamanetwork.com/journals/jama/fullarticle/185952
kind
study
accessed at
2026-10-01
supports
Opened 2010 primary study: 72 eligible parallel-group trials with nonsignificant primary outcomes from a specified December 2006 publication sample; spin in presentation and interpretation.
limitations
Selected sample and interpretive coding; not a prevalence estimate for all papers, a diagnosis of author motives or proof that a nonsignificant effect is exactly zero.
id
prasad-reversal
title
A decade of reversal: an analysis of 146 contradicted medical practices
url
https://scholars.duke.edu/publication/1407557
kind
study
accessed at
2026-10-01
supports
Opened Duke's coauthor institutional record and primary abstract: established-practice evaluations in selected original articles from one journal during 2001–2010 documented reversals.
limitations
Abstract and bibliographic record inspected, not full publisher manuscript; journal selection and established-practice denominator prevent extrapolation to all medical care.
id
lundh-sponsorship
title
Industry sponsorship and research outcome
url
https://www.cochrane.org/evidence/MR000033_industry-sponsorship-and-research-outcome
kind
review
accessed at
2026-10-01
supports
Opened 2017 Cochrane review summary: 75 empirical papers and more favourable efficacy results/conclusions associated with industry sponsorship in drug/device studies.
limitations
The search ran through February 2015, so these findings are not a current 2026 sponsorship rate. An association across studies does not establish the validity of an individual paper.
id
jama-medical-preprints
title
Preprints Involving Medical Research—Do the Benefits Outweigh the Challenges?
url
https://jamanetwork.com/journals/jama/fullarticle/2772743
kind
professional
accessed at
2026-10-01
supports
Opened 2020 editorial defining medical preprints, describing review status and discussing cautions about use for clinical decisions.
limitations
A professional editorial rather than a quantitative study; it explains the distinction between a preprint and a peer-reviewed paper.
id
fda-surrogate-endpoints
title
FDA: Surrogate Endpoint Resources for Drug and Biologic Development
url
https://www.fda.gov/drugs/development-resources/surrogate-endpoint-resources-drug-and-biologic-development
kind
official
accessed at
2026-10-01
supports
Opened definitions of clinical outcomes, validated versus reasonably likely surrogates and further confirmation of clinical benefit in accelerated approval.
limitations
US drug/biologic regulatory context; a surrogate's validity is specific to its proposed use and this article gives no treatment recommendation.
id
plos-data-access
title
PLOS ONE: Data Availability
url
https://journals.plos.org/plosone/s/data-availability
kind
official
accessed at
2026-10-01
supports
Opened policy recognizing privacy, consent and ethical restrictions while requesting clear availability statements and appropriate access mechanisms.
limitations
One publisher's policy illustrates a legitimate transparency boundary; it is not universal law or permission to disclose participant data.
id
ihc-key-concepts
title
Informed Health Choices Key Concepts for assessing claims about treatment effects, 2022
url
https://informedhealthchoices.org/wp-content/uploads/2023/01/Informed-Health-Choices-Key-Concepts-2022.pdf
kind
professional
accessed at
2026-10-01
supports
Opened public teaching framework PDF and development credits; fair comparisons and thoughtful interpretation of treatment claims.
limitations
Educational framework, not individual clinical advice; article's five-part reading note is an authored application, not a validated intervention.
id
reader-p-value-question
title
AskStatistics reader question about interpreting statistical results
url
https://www.reddit.com/r/AskStatistics/comments/rt4lax/
kind
question_signal
accessed at
2026-10-01
supports
Opened public thread as a signal that readers ask about binary significance and what statistical results mean.
limitations
Anecdotal reader question only; comments, votes and apparent agreement supply no factual guidance or prevalence claim.
id
reader-reporting-question
title
PhD reader question about PRISMA and CONSORT diagrams
url
https://www.reddit.com/r/PhD/comments/1de0epu/
kind
question_signal
accessed at
2026-10-01
supports
Opened public thread as a signal of confusion between reporting frameworks for reviews and trials.
limitations
Question discovery only; guidance is supported by the opened official reporting standards, not forum answers.

professional panel

name
John P. A. Ioannidis
credential
MD, DSc; Stanford professor of medicine and epidemiology and population health; METRICS co-director
official url
https://profiles.stanford.edu/john-ioannidis
public usefulness
His official profile links public talks and teaching about evidence reliability and meta-research.
contribution
The 2005 model helps explain why plausibility, power, bias and repeated testing matter together.
boundary
The model depends on assumptions and is not a census declaring all studies false.
reviewed at
2026-10-01
name
Gordon Guyatt
credential
Physician; McMaster professor in Health Research Methods, Evidence, and Impact with a joint medicine appointment
official url
https://experts.mcmaster.ca/people/guyatt
public usefulness
Users' Guides teaching and evidence-based medicine explanations help clinicians and readers move from findings to decisions; McMaster's official biography confirms his clinical background.
contribution
GRADE and clinical evidence interpretation distinguish certainty from recommendation strength.
boundary
A framework cannot choose an individual's preferred tradeoff or replace context-specific professional judgment.
reviewed at
2026-10-01
name
Julian P. T. Higgins
credential
PhD; University of Bristol professor of evidence synthesis
official url
https://www.bristol.ac.uk/person/Julian-Higgins-cddd7691-abca-46cb-8fa6-3b01745c2b98/
public usefulness
The freely available Cochrane Handbook and methodological teaching explain synthesis and bias in detail.
contribution
Handbook editing, heterogeneity methods and risk-of-bias tools support reading evidence bodies rather than isolated results.
boundary
Statistical synthesis does not repair biased input studies or answer a mismatched question.
reviewed at
2026-10-01
name
Jonathan A. C. Sterne
credential
PhD; University of Bristol professor of medical statistics and epidemiology
official url
https://www.bristol.ac.uk/people/person/Jonathan-Sterne-a3e0a405-8096-424d-a966-e082d3dfcf67/
public usefulness
Public Cochrane methods chapters and risk-of-bias resources explain technical issues that readers often mistake for simple scores.
contribution
Missing-evidence, funnel-plot and causal-methods guidance informs assessment of fragile comparisons.
boundary
Funnel asymmetry has several explanations and cannot diagnose publication bias by itself.
reviewed at
2026-10-01
name
Brian Nosek
credential
PhD; Center for Open Science co-founder and executive director; University of Virginia psychology professor
official url
https://www.cos.io/team/brian-nosek
public usefulness
COS's public explainers, initiatives and training make preregistration and reproducibility accessible.
contribution
Separating planned tests from exploration and increasing transparent research practices.
boundary
Registration supplies a record; it does not make a weak design sound or guarantee replication.
reviewed at
2026-10-01
name
David Moher
credential
PhD in clinical epidemiology and biostatistics; Ottawa Hospital Research Institute senior scientist and Journalology director; University of Ottawa professor
official url
https://ohri.ca/en/find-researcher/david-moher?page=1
public usefulness
CONSORT, PRISMA and Journalology resources give readers practical ways to locate essential reporting information.
contribution
Leadership in developing and evaluating reporting guidance for trials and systematic reviews.
boundary
Reporting completeness is separate from the validity of the reported methods.
reviewed at
2026-10-01
name
Holger J. Schünemann
credential
MD, PhD; McMaster professor emeritus in Health Research Methods, Evidence, and Impact
official url
https://experts.mcmaster.ca/people/schuneh
public usefulness
GRADE guidance, Cochrane interpretation chapters and clinical-epidemiology education explain how evidence informs recommendations.
contribution
Outcome-specific certainty assessment and transparent evidence-to-decision methods.
boundary
Certainty judgments concern specified outcomes; a guideline recommendation is not automatically an individual's preferred decision.
reviewed at
2026-10-01
name
Lisa Bero
credential
PhD; University of Colorado professor working in evidence synthesis, research integrity and conflicts of interest
official url
https://medschool.cuanschutz.edu/general-internal-medicine/research/research-programs/cbh-research-group
public usefulness
Invited public speaking, media explanation and accessible sponsorship reviews help readers examine influence on evidence.
contribution
Research on industry sponsorship, publication and factors shaping research conclusions.
boundary
An association across sponsored studies is not proof that any named study or author is fraudulent.
reviewed at
2026-10-01
name
Isabelle Boutron
credential
MD, PhD; professor of epidemiology and Cochrane Bias Methods Group convenor
official url
https://methods.cochrane.org/bias/convenors
public usefulness
Public Cochrane methods work and reporting research explain bias and spin beyond the abstract's headline.
contribution
Primary empirical study of spin in reports with nonsignificant primary trial outcomes.
boundary
The 72-trial study is a defined historical sample; presentation coding does not establish author motives.
reviewed at
2026-10-01
name
Gerd Gigerenzer
credential
Psychologist; Max Planck Institute for Human Development director emeritus; Harding Center for Risk Literacy director
official url
https://www.mpib-berlin.mpg.de/staff/gerd-gigerenzer
public usefulness
Risk Savvy, Calculated Risks and professional risk-literacy teaching explain numbers using understandable event frequencies.
contribution
Risk communication helps readers see what an impressive relative percentage leaves out.
boundary
Clearer numerical presentation cannot eliminate uncertainty or predict an individual's outcome.
reviewed at
2026-10-01
name
Andrew D. Oxman
credential
Evidence-methods researcher and co-lead developer credited in the Informed Health Choices Key Concepts framework
official url
https://informedhealthchoices.org/wp-content/uploads/2023/01/Informed-Health-Choices-Key-Concepts-2022.pdf
public usefulness
The public Key Concepts framework supplies plain teaching about treatment claims and fair comparisons.
contribution
A structured educational approach to asking what evidence supports a claim and which comparison is fair.
boundary
The framework develops reading judgment and does not provide individual clinical advice; no unverified current institutional title or degree is assigned.
reviewed at
2026-10-01
name
Christopher D. Chambers
credential
PhD; Chartered Psychologist and Cardiff University professor of cognitive neuroscience
official url
https://profiles.cardiff.ac.uk/staff/chambersc1
public usefulness
Public writing, The Seven Deadly Sins of Psychology, workshops and Registered Reports explanations make reproducibility choices understandable.
contribution
Development and advocacy of pre-results review and Registered Reports.
boundary
The format changes incentives and scrutiny; it cannot guarantee execution quality or a result's importance.
reviewed at
2026-10-01
name
Daniel J. Simons
credential
PhD; University of Illinois psychology professor
official url
https://experts.illinois.edu/en/persons/daniel-j-simons/
public usefulness
COS Registered Reports explanations and an officially documented public lecture on Nobody's Fool help readers question apparently persuasive claims.
contribution
Registered replication publishing and methodological communication encourage scrutiny independent of a striking outcome.
boundary
Replication and a transparent publication format still require interpretation of populations, measures and real-world meaning.
reviewed at
2026-10-01
name
Matthew J. Page
credential
PhD in epidemiology; Monash University associate professor and evidence-synthesis methods researcher
official url
https://research.monash.edu/en/persons/matthew-page/
public usefulness
PRISMA 2020 materials and accessible synthesis-methods guidance help readers inspect review searches and reporting.
contribution
PRISMA 2020 development, risk-of-bias methods and research on missing evidence.
boundary
A reporting standard does not establish that a particular review searched adequately or pooled appropriate studies.
reviewed at
2026-10-01
name
Paul Glasziou
credential
Physician and evidence-based medicine researcher at Bond University
official url
https://research.bond.edu.au/en/persons/paul-glasziou/
public usefulness
Public evidence-based medicine resources and clinical translation work connect research findings to useful, understandable choices.
contribution
Cochrane interpretation guidance and work on evidence use emphasize clinical meaning and practical applicability.
boundary
Useful evidence interpretation remains specific to the patient-important outcome, comparison and decision; no current director title is asserted.
reviewed at
2026-10-01

Download original public profile Download complete article

End of How to read a study