STUDboardsPlay · Research · Self · Fit · Money · Learn · Study · Plan · Token STUDtokenOptional

STUDlearn · Learning

How to read a study

Also known as

read a paper · absolute risk · p-hacking · preprint · CONSORT · GRADE

Named sources. May contain inaccuracies or be incomplete. Not medical, legal, financial, or other professional advice.

Open in the directory ↗

Read the question, the comparison, and the absolute numbers before the headline.

An adult at a desk with a printed paper and a laptop, comparing a headline to a methods section

What would help today?

Read the question the study actually asked before the headline
Start here
Read the question the study actually asked before the headline.
Next
Bring the pattern to a qualified clinician or the named official source; this page is a reading companion.

A study can only answer the comparison it ran, in the people it enrolled, for the outcomes it measured. Ioannidis showed how flexible questions and small samples produce fragile claims. Boutron documented spin in abstracts that makes a null trial sound like a win. If the headline names a different disease or a bigger promise than the methods, stop there. That finding is a group result, not a personal promise. A qualified clinician still has to apply it to one history.

Sources 4
Check who was compared with whom, and whether assignment was random
Start here
Check who was compared with whom, and whether assignment was random.
Next
Bring the pattern to a qualified clinician or the named official source; this page is a reading companion.

Cochrane methods, with Higgins as a named editor, start with the PICO question and then with risk of bias. CONSORT, from Schulz, Altman, and Moher, asks whether a trial actually randomized and followed everyone. Sterne's risk-of-bias tools are how reviewers judge that without a vibe. A before-and-after story is not the same design. That finding is a group result, not a personal promise. A qualified clinician still has to apply it to one history.

Sources 5
Translate a relative change into an absolute number of people
Start here
Translate a relative change into an absolute number of people.
Next
Bring the pattern to a qualified clinician or the named official source; this page is a reading companion.

Gigerenzer's risk-literacy work shows that a 50 percent reduction can mean 2 in 100 going to 1 in 100, or 50 in 100 going to 25. Guyatt's GRADE teaching wants the absolute effect in the summary of findings. Oxman has spent years putting that in language non-specialists can use. If the story will not give you the raw counts, you do not yet have the result. That finding is a group result, not a personal promise.

Sources 4
Treat a preprint as an early draft until it is reviewed and consistent
Start here
Treat a preprint as an early draft until it is reviewed and consistent.
Next
Bring the pattern to a qualified clinician or the named official source; this page is a reading companion.

Nosek's open-science work wants reports out in the open, which is good, and still wants preregistration and review. Goldacre's missing-trial work is a reminder that what you can see is a biased sample. PRISMA and EQUATOR exist because reporting quality varies. A PDF with a logo is not automatically a practice changer. That finding is a group result, not a personal promise. A qualified clinician still has to apply it to one history.

Sources 4

Name the question a study can answer, see what the methods field found about bias and headlines, then take one next step before you change a plan.

Good to know. This is a reading companion about methods, not medical, legal, or financial advice. A headline is not a reason to start or stop a medicine. Bring practice changes to a qualified clinician or the person who owns the decision.

What the research found

  • Start with the question, the people, and the comparison. Cochrane PICO. Higgins. A study cannot answer a question it did not ask. Headlines often swap in a bigger question. That finding is a group result, not a personal promise. A qualified clinician still has to apply it to one history. This page is a reading companion, not a clinic visit. That finding is a group result, not a personal promise.
    Sources 2
  • Prefer randomized trials for treatment effects, with bias checked. CONSORT reporting. Sterne risk of bias. Schulz on concealment. Observational studies still matter for harms and rare events, with different limits. That finding is a group result, not a personal promise. A qualified clinician still has to apply it to one history. This page is a reading companion, not a clinic visit. That finding is a group result, not a personal promise.
    Sources 4
  • Demand absolute numbers beside relative ones. Gigerenzer. GRADE summaries. Guyatt and Oxman. A huge percent of a tiny risk is still a tiny risk. That finding is a group result, not a personal promise. A qualified clinician still has to apply it to one history. This page is a reading companion, not a clinic visit. That finding is a group result, not a personal promise.
    Sources 4
  • Treat unpublished and spun results as a known bias. Dickersin publication bias. Boutron spin. Goldacre missing trials. The visible literature is not the whole literature. That finding is a group result, not a personal promise. A qualified clinician still has to apply it to one history. This page is a reading companion, not a clinic visit. That finding is a group result, not a personal promise.
    Sources 4
  • Use GRADE to separate certainty from enthusiasm. Guyatt, Schünemann, Oxman. High certainty is not the same as a strong recommendation. Low-certainty evidence can still matter if the harm is large. A press release has no GRADE. That finding is a group result, not a personal promise. A qualified clinician still has to apply it to one history. This page is a reading companion, not a clinic visit.
    Sources 4
  • Use reporting guidelines as a checklist, not as a quality halo. EQUATOR library. CONSORT, PRISMA, STROBE. Moher and Altman. A completed checklist is necessary reporting, not proof the study is true. That finding is a group result, not a personal promise. A qualified clinician still has to apply it to one history. This page is a reading companion, not a clinic visit. That finding is a group result, not a personal promise.
    Sources 4

Where experts still disagree

  • Read Ioannidis 2005 as a methods warning, not as 'all science is fake'. The paper models how many claims can be false under small samples and bias. It is not a license to ignore vaccines or gravity. Nosek's reproducibility work is the constructive follow-up. Use it to slow down, not to drop the lights.
    Sources 4
  • Treat medical reversal as a reason to wait on weak evidence, not as nihilism. Prasad documents practices that later better trials overturned. Guyatt would still act on high-certainty important effects. The disagreement is how much to adopt on surrogate outcomes. Ask whether anyone measured the outcome you care about. That finding is a group result, not a personal promise.
    Sources 4
  • Weigh industry funding as a risk of bias, not as an automatic fake stamp. Bero's work documents how funding and spin associate with friendlier results. Cochrane still includes industry trials when methods are clear. Disclosure is a clue, not a verdict. Read the methods anyway. That finding is a group result, not a personal promise.
    Sources 3
  • Treat preregistration as strong medicine that is still unevenly used. Nosek and COS built the rails. Many trials still change outcomes after seeing data, which Goldacre flags. A preregistered trial can still be small or unblinded. It is a plus, not a halo. That finding is a group result, not a personal promise.
    Sources 3

Just talk

  • Read past a miracle percent. Ads love relative risk. Gigerenzer wants the absolute number. GRADE summaries show both. Ask 50 percent of what. That finding is a group result, not a personal promise. A qualified clinician still has to apply it to one history.
    Sources 2
  • Treat a preprint headline as a draft. Open posting is good. Review and replication still matter. Nosek and EQUATOR. Do not change a medicine on a first PDF. That finding is a group result, not a personal promise. A qualified clinician still has to apply it to one history.
    Sources 3
  • Keep 'they all lie' from replacing actual methods. Ioannidis is a warning about fragility. Higgins still teaches how to read a review. The constructive move is better questions. Cynicism is not a methods skill. That finding is a group result, not a personal promise.
    Sources 3
  • Look for the outcome that was switched. Goldacre and CONSORT want the prespecified outcome. Boutron's spin papers show how abstracts wander. If the headline uses a different endpoint than the methods, pause. That pause is the skill. That finding is a group result, not a personal promise.
    Sources 3

What to try

  • Before sharing or changing a plan, write the question, the comparison, and one absolute number. Higgins would start with PICO. Gigerenzer wants the absolute risk. If those three lines cannot be filled from the paper, you do not yet have a finding. Bring practice changes to a clinician or the decision owner. That finding is a group result, not a personal promise. A qualified clinician still has to apply it to one history.
    Sources 3
  • On the next health headline, find the abstract's numbers, not only the quote. Boutron spin. Absolute counts. If they are missing, stop. That finding is a group result, not a personal promise. A qualified clinician still has to apply it to one history.
    Sources 2
  • Ask whether the study was randomized and whether everyone was followed. CONSORT. Sterne. A survey is a different design. That finding is a group result, not a personal promise. A qualified clinician still has to apply it to one history. This page is a reading companion, not a clinic visit.
    Sources 2
  • Search whether the trial was registered and whether a review already exists. Goldacre missing trials. Cochrane. A single new PDF is rarely the whole field. That finding is a group result, not a personal promise. A qualified clinician still has to apply it to one history.
    Sources 2

How to keep it

  • Keep the question in view. PICO. Higgins. A study cannot answer a swapped headline. That finding is a group result, not a personal promise. A qualified clinician still has to apply it to one history.
    Sources 2
  • Keep absolute numbers beside percents. Gigerenzer. GRADE. 50 percent of what. That finding is a group result, not a personal promise. A qualified clinician still has to apply it to one history. This page is a reading companion, not a clinic visit.
    Sources 2
  • Keep bias tools, not vibes. Sterne. CONSORT. Cochrane. That finding is a group result, not a personal promise. A qualified clinician still has to apply it to one history. This page is a reading companion, not a clinic visit.
    Sources 3
  • Keep unpublished and spun work in mind. Dickersin. Boutron. Goldacre. That finding is a group result, not a personal promise. A qualified clinician still has to apply it to one history. This page is a reading companion, not a clinic visit.
    Sources 3
  • Keep this distinct from Information's lateral reading. Information checks claims across sites. This page opens the methods. Use both. Neither is a conspiracy story. That finding is a group result, not a personal promise. A qualified clinician still has to apply it to one history.
    Sources 2

Sayings people repeat

  • Start with the question, the people, and the comparison. Claim: “A study can only answer the comparison it actually ran” — established. Cochrane methods start with a defined question. Ioannidis and Boutron show how headlines and abstracts wander from that question. If the comparison is missing, you do not yet have a finding. A qualified clinician still has to apply the finding to one history.
    Sources 3
    • Higgins JPT, Thomas J, Chandler J, et al. Cochrane Handbook for Systematic Reviews of Interventions.Research or guidanceSource (opens in a new tab) ↗
    • Ioannidis JPA. Why Most Published Research Findings Are False. PLoS Med. 2005;2(8):e124.Research or guidanceSource (opens in a new tab) ↗
    • Boutron I, Dutton S, Ravaud P, Altman DG. Reporting and interpretation of randomized trials with statistically nonsignificant results.Research or guidanceSource (opens in a new tab) ↗
  • Demand absolute numbers beside relative ones. Claim: “A 50 percent risk reduction is enough information to decide” — not what the research found. Gigerenzer's risk-literacy work and GRADE summaries treat absolute numbers as required. Fifty percent of a tiny risk is still tiny. A qualified clinician still has to apply the finding to one history.
    Sources 2
  • Read Ioannidis 2005 as a methods warning, not as all science is fake. Claim: “All published research findings are fake” — not what the research found. Ioannidis modeled why many claims are fragile. Cochrane, CONSORT, and GRADE exist to read the ones that are stronger. The constructive move is better questions, not the lights out. A qualified clinician still has to apply the finding to one history.
    Sources 2
  • Treat a preprint as an early draft until it is reviewed and consistent. Claim: “Preprints are the same as peer-reviewed practice changers” — not what the research found. Nosek's open-science work wants reports visible and still wants review, preregistration, and replication. A first PDF is not a reason to change a medicine. A qualified clinician still has to apply the finding to one history.
    Sources 2
  • Treat unpublished and spun results as a known bias. Claim: “Unpublished negative studies can bias what we think we know” — established. Dickersin documented publication bias. Goldacre has campaigned about missing trials. Boutron showed spin in abstracts of nonsignificant trials. A qualified clinician still has to apply the finding to one history. A qualified clinician still has to apply the finding to one history.
    Sources 3
  • Use GRADE to separate certainty from enthusiasm. Claim: “GRADE ratings separate how sure we are from how strong a recommendation is” — established. Guyatt, Schünemann, and Oxman built GRADE so certainty of evidence and strength of recommendation are different sliders. A press release has neither. A qualified clinician still has to apply the finding to one history.
    Sources 2
  • Weigh industry funding as a risk of bias, not as an automatic fake stamp. Claim: “Industry funding means you can skip reading the methods” — disputed. Bero documents that funding associates with friendlier results. Cochrane still reads industry trials when methods are clear. Disclosure is a clue, not a substitute for methods. A qualified clinician still has to apply the finding to one history.
    Sources 2

The longer notes

  • Read Ioannidis 2005 as a probability warning. Small n, many analyses, bias, and low prior odds. Many published claims will not replicate. It is not 'science is fake'. Slow down when the claim is surprising and tiny. That finding is a group result, not a personal promise.
    Sources 2
  • Read CONSORT as a trial's table of contents. Schulz, Altman, Moher. Randomization, concealment, blinding, flow of participants. Missing pieces are clues. A beautiful graph is not a CONSORT diagram. That finding is a group result, not a personal promise. A qualified clinician still has to apply it to one history.
    Sources 4
  • Read GRADE as two sliders. Certainty of evidence. Strength of recommendation. Guyatt, Schünemann, Oxman. A loud doctor is not a GRADE rating. That finding is a group result, not a personal promise. A qualified clinician still has to apply it to one history. This page is a reading companion, not a clinic visit.
    Sources 4
  • Read Cochrane as a recipe for reviews. Question, search, select, bias, synthesize. Higgins. A narrative review by one expert is a different product. Look for the forest plot and the certainty. That finding is a group result, not a personal promise. A qualified clinician still has to apply it to one history.
    Sources 2
  • Read open science as infrastructure. Nosek and COS. Preregistration, open data, Registered Reports. It makes hiding harder. It does not make a small biased trial large. That finding is a group result, not a personal promise. A qualified clinician still has to apply it to one history.
    Sources 2
  • Read publication bias as missing mass. Dickersin. Goldacre unreported trials. Funnel plots in reviews. What you cannot see can still change the average. That finding is a group result, not a personal promise. A qualified clinician still has to apply it to one history. This page is a reading companion, not a clinic visit.
    Sources 3
  • Read spin as a writing problem with consequences. Boutron. Nonsignificant results described as promising. Abstracts travel farther than methods. Read past the first paragraph. That finding is a group result, not a personal promise. A qualified clinician still has to apply it to one history. This page is a reading companion, not a clinic visit.
    Sources 3
  • Read medical reversal as a caution about surrogate outcomes. Prasad. Adopted on weak evidence, later overturned. Ask for the outcome patients care about. It is not an argument against all treatment. That finding is a group result, not a personal promise. A qualified clinician still has to apply it to one history.
    Sources 2
  • Keep industry and advocacy funding as a methods clue. Bero. Friendlier results on average in some fields. Read the protocol anyway. A nonprofit can also spin. That finding is a group result, not a personal promise. A qualified clinician still has to apply it to one history. This page is a reading companion, not a clinic visit.
    Sources 2

Who this is drawing from

  • John Ioannidis. physician and meta-researcher. This packet uses why many published findings are less robust than headlines. A profile is not a clinic for the visitor.
    Sources 1
  • Gordon Guyatt. physician; GRADE and evidence-based medicine. This packet uses GRADE certainty ratings. A profile is not a clinic for the visitor.
    Sources 1
  • Julian Higgins. statistician; Cochrane Handbook. This packet uses Cochrane methods for reviews. A profile is not a clinic for the visitor.
    Sources 1
  • Douglas Altman. statistician; reporting and bias. This packet uses better reporting as a public good. A profile is not a clinic for the visitor.
    Sources 1
  • Kenneth Schulz. epidemiologist; CONSORT and allocation concealment. This packet uses randomized-trial reporting. A profile is not a clinic for the visitor.
    Sources 1
  • Brian Nosek. psychologist; Center for Open Science. This packet uses reproducibility and preregistration. A profile is not a clinic for the visitor.
    Sources 1
  • Ben Goldacre. physician; unreported trials and better evidence. This packet uses missing trials and outcome switching. A profile is not a clinic for the visitor.
    Sources 1
  • Vinay Prasad. physician; medical reversal. This packet uses when later better trials overturn practice. A profile is not a clinic for the visitor.
    Sources 1
  • David Moher. epidemiologist; CONSORT and PRISMA. This packet uses reporting guidelines. A profile is not a clinic for the visitor.
    Sources 1
  • Andrew Oxman. health-services researcher; GRADE and informed health choices. This packet uses helping non-specialists use evidence. A profile is not a clinic for the visitor.
    Sources 1
  • Holger Schünemann. physician; GRADE methods. This packet uses certainty and recommendation strength. A profile is not a clinic for the visitor.
    Sources 1
  • Lisa Bero. research-integrity scholar; industry bias. This packet uses funding and spin as methods issues. A profile is not a clinic for the visitor.
    Sources 1
  • Jonathan Sterne. statistician; risk-of-bias tools. This packet uses RoB tools for randomized and nonrandomized studies. A profile is not a clinic for the visitor.
    Sources 1
  • Isabelle Boutron. epidemiologist; spin in abstracts. This packet uses how abstracts overstate results. A profile is not a clinic for the visitor.
    Sources 1
  • Kay Dickersin. epidemiologist; publication bias. This packet uses unpublished negative studies. A profile is not a clinic for the visitor.
    Sources 1

Good to know

  • Good to know. This is a reading companion about methods, not medical, legal, or financial advice. A headline is not a reason to start or stop a medicine. Bring practice changes to a qualified clinician or the person who owns the decision.
    Sources 3
  • Not advice. Named sources. Honest paraphrase of the finding. Not medical, legal, or financial advice.

Related

Making sense of online informationLearning and rememberingAttention, phones, focus

Also on this house

STUDlearn. For checking online claims, see Making sense of online information. This page teaches how a study is built.

All STUDlearn topics →