{ "article": { "format_version": 5, "presentation": "complete-card", "overview": "Read a study as an answer to a particular question. Identify who was studied, what was compared, what was measured and for how long. Then examine the size and uncertainty of the result, how the comparison could have been distorted, and how it fits the wider evidence. A striking headline, a small p-value or a respected journal cannot do those jobs for you.", "sections": [ { "id": "find-the-question", "role": "explanation", "title": "Find the question behind the headline", "intro": "You do not need to understand every equation before you can start. First establish what kind of answer this paper can provide.", "items": [ { "label": "Rewrite the headline as a question about specific people", "text": "When a headline says a treatment works, find the question in the paper itself. For a treatment comparison, PICO means Population, Intervention, Comparator and Outcome: who participated, what they received, what it was compared with and what changed. Add the follow-up period. A study of adults with a particular condition over six months does not automatically answer a question about healthy teenagers over ten years. Cochrane's question-setting guidance also distinguishes estimating how common something is from testing a treatment or understanding people's experiences. Keep the conclusion attached to the question the researchers actually investigated.", "source_ids": [ "cochrane-questions" ] }, { "label": "Ask how people came to receive the exposure or treatment", "text": "Suppose people who choose an exercise programme later report better health. Exercise may help, but those participants may also differ in income, starting health or access to care. Such a third factor is a confounder when it influences both the exposure and the outcome. An observational study records choices or circumstances; a randomized trial assigns a comparison by chance. Adjustment can address measured differences, while unmeasured differences may remain. Cochrane's guidance on non-randomized studies therefore asks readers to inspect the causal comparison and its assumptions. An association deserves investigation, but its design and measurements limit the conclusion it supports.", "source_ids": [ "cochrane-nonrandomized" ] }, { "label": "Give a percentage a starting risk and a time period", "text": "A headline announcing a 50 percent reduction leaves out information needed to understand the result. In an invented example, a fall from 2 events per 100 people to 1 per 100 halves the risk and prevents 1 event per 100. A fall from 50 to 25 per 100 also halves it, but prevents 25 events per 100. These are absolute reductions of 1 and 25 percentage points, respectively. Neither example describes a real treatment. Find the actual groups, event definition and observation period in the paper; if the headline omits them, do not invent them. The same relative change can mean very different practical benefits.", "source_ids": [ "cochrane-effect-measures" ] }, { "label": "Check a preprint's version and later review", "text": "A preprint makes a manuscript public before it completes journal peer review; its experiment may already be complete. That can make new work available sooner, but availability does not establish that its methods or conclusions are sound. JAMA's discussion of medical preprints explains both the opportunity and the danger of acting on an early report. Read the methods, identify the version, and look for a later journal article, correction or other evidence on the same question. Journal peer review adds scrutiny; it does not certify truth. A preprint is not, by itself, a reason to change medical practice, and a journal label does not replace reading the study.", "source_ids": [ "jama-medical-preprints" ] } ] }, { "id": "inspect-the-comparison", "role": "findings", "title": "Inspect how the comparison was made", "items": [ { "label": "Separate random assignment, concealed allocation and blinding", "text": "Random assignment helps create comparable groups. Allocation concealment prevents someone enrolling participants from knowing the next assignment and influencing who enters which group. Blinding keeps participants, care providers or outcome assessors unaware of assignments after enrolment, where feasible. These safeguards address different problems. CONSORT 2025 asks authors to explain them separately. A paper calling itself randomized still needs an account of how assignment worked, who knew it, and whether departures from the intended treatment could distort the comparison.", "source_ids": [ "consort-2025", "cochrane-trial-bias" ] }, { "label": "Follow participants from assignment to the reported result", "text": "Read the flow diagram alongside the results: how many people entered each group, received the intervention, were followed and contributed measurements? Missing outcomes matter especially when their absence is related to how people were doing, and the problem may differ between groups. Cochrane's trial-bias guidance does not treat a fixed percentage of missing data as a universal safety threshold. An intention-to-treat approach aims to analyse people according to their assigned group, but that label cannot recover an outcome that was never measured. Look for reasons for missingness, assumptions and analyses testing those assumptions.", "source_ids": [ "cochrane-trial-bias" ] }, { "label": "Distinguish a meaningful outcome from a convenient marker", "text": "An outcome is what the study measures. A clinical outcome can concern how someone feels, functions or survives; a surrogate is a marker used to stand in for a clinical benefit. A laboratory value can be useful without proving that people live longer or feel better. FDA guidance distinguishes surrogates supported by validation from those only reasonably likely to predict benefit. Ask what links this particular marker to the outcome you care about, for this use. Also check whether the paper measured that outcome directly or inferred it.", "source_ids": [ "fda-surrogate-endpoints" ] }, { "label": "Read harms with the same attention as benefits", "text": "A short trial with few participants may detect a benefit yet miss an uncommon or delayed harm. Cochrane's adverse-effects guidance explains why other designs, including observational studies, can contribute information that trials do not supply. Check how harms were sought, defined and counted, how long people were observed, and whether both groups had the same opportunities for detection. A report of no observed events is different from evidence that an event cannot occur. Spontaneous reports can identify a concern but, without a suitable denominator, cannot establish its frequency.", "source_ids": [ "cochrane-harms" ] }, { "label": "Use reporting guidelines to locate information, then assess bias separately", "text": "A reporting checklist asks whether readers have the information needed to understand a study. CONSORT 2025 has 30 items for randomized trials; STROBE covers major observational designs, and PRISMA 2020 covers systematic reviews. EQUATOR helps locate the relevant guideline. Reporting completeness and risk of bias are different judgments: a clearly described method can still systematically distort a result, while an omitted detail may leave its risk unclear. STROBE explicitly says its checklist is not a study-quality scoring tool. Find the methods and supplements before treating a filled checklist as reassurance.", "source_ids": [ "equator-library", "equator-consort", "consort-2025", "strobe", "prisma-2020", "cochrane-bias" ] } ] }, { "id": "read-size-and-uncertainty", "role": "findings", "title": "Read the size and uncertainty of the result", "items": [ { "label": "Translate a result into an amount people can understand", "text": "For an event outcome, record the absolute difference as well as the relative effect. In the invented examples above, preventing 1 event per 100 corresponds to a number needed to treat of 100; preventing 25 per 100 corresponds to 4. This calculation is one divided by the absolute risk reduction expressed as a proportion. It describes an average additional benefit over the stated period, compared with the specified alternative. Cochrane cautions that baseline risk, follow-up and uncertainty affect this interpretation. Keep possible harms beside the benefit rather than using one attractive number as a decision rule.", "source_ids": [ "cochrane-interpretation" ] }, { "label": "Keep the effect measure and its units visible", "text": "A risk ratio compares event probabilities; an odds ratio compares odds, which are the probability of an event divided by the probability of no event. They are different measures and can differ substantially when events are common. For a numerical outcome, a mean difference retains the measurement scale, such as points on a symptom questionnaire. Check what direction means improvement and whether that amount is meaningful. A ratio of 1 or a difference of 0 represents no contrast on those respective scales; neither tells you how uncertain the estimate is.", "source_ids": [ "cochrane-effect-measures" ] }, { "label": "Read the interval before declaring an effect present or absent", "text": "A confidence interval shows uncertainty around an estimate under the statistical method's assumptions. For a conventional 95 percent interval, the procedure is designed to cover the true value in 95 percent of repeated applications; it does not assign a 95 percent probability to the fixed true value in this particular interval. Look at whether the interval includes effects that would matter, including benefit and harm. An estimate near zero with a wide interval may leave an important effect unresolved. Precision does not protect against a biased comparison.", "source_ids": [ "cochrane-interpretation", "asa-task-force" ] }, { "label": "Interpret a p-value in relation to the tested model", "text": "A p-value asks how unusual results at least as extreme as those observed would be if the specified statistical model, including its null hypothesis, were correct. It is not the probability that the hypothesis is true, or the probability that chance alone caused the findings. The American Statistical Association's 2016 statement also warns that statistical significance does not measure effect size or importance. A small p-value cannot settle whether the comparison was fair, the measurement useful or the result applicable. A large one does not establish an exact zero effect.", "source_ids": [ "asa-p-values" ] }, { "label": "Ask what the study was capable of detecting", "text": "Power is a study's probability of meeting a specified statistical test criterion when an assumed effect really exists. It depends on the effect, variability, sample and test. A small, imprecise study can miss an important effect; a very large one can detect an effect too small to matter. Ioannidis's modelling also connects low power, prior plausibility, bias and repeated testing to the reliability of positive findings. Read the sample-size assumptions and the actual interval, rather than treating the participant count as a universal grade.", "source_ids": [ "ioannidis-2005", "asa-p-values" ] } ] }, { "id": "trace-plans-and-publication", "role": "findings", "title": "Trace what was planned and what became visible", "items": [ { "label": "Compare the dated plan with the published outcome", "text": "A registration records a study; a protocol describes its design; a statistical analysis plan specifies how results will be analysed. The primary outcome addresses the main planned question; secondary outcomes address additional questions. Check dates and versions rather than assuming every document preceded knowledge of the results. Preregistration helps distinguish planned tests from analyses chosen after seeing data. Changes may be legitimate, but their timing and reasons should be visible. If a paper highlights a secondary or exploratory result after an uncertain primary result, keep that distinction in your conclusion. CONSORT 2025 requests protocols, analysis plans and explanations of important changes.", "source_ids": [ "cos-preregistration", "consort-2025" ] }, { "label": "Compare the abstract's conclusion with the primary result", "text": "Boutron and colleagues examined 72 published parallel-group randomized trials with statistically nonsignificant primary results, drawn from a defined 2006 publication sample. They identified presentation that emphasized favourable results or drew conclusions beyond the primary result, which they called spin. This is evidence about a selected set of reports, not the rate of spin in every field or evidence of each author's motive. Follow the abstract's claim into the primary-outcome table and its uncertainty. Useful secondary findings can remain useful when their status is clear; they do not silently replace the original question.", "source_ids": [ "boutron-spin" ] }, { "label": "Look for evidence that did not reach the visible papers", "text": "The studies you can find may be a selected subset of the studies conducted. Entire studies can remain unpublished, and particular results can be omitted within a published report. Cochrane describes how selection related to results can distort a review's conclusion. Registries, protocols and attempts to obtain unavailable results help reviewers investigate the gap. None guarantees a complete record. When a review says it searched for missing evidence, read where it looked and what remained unavailable rather than assuming publication alone makes the evidence representative.", "source_ids": [ "cochrane-missing-evidence", "cochrane-searching" ] }, { "label": "Examine who could influence the study and its reporting", "text": "Lundh, Bero and colleagues' 2017 Cochrane review included 75 empirical papers examining industry sponsorship. Industry-sponsored drug and device studies more often had favourable efficacy results and conclusions than studies with other sponsorship. The finding is an association across the included evidence, not proof that a particular sponsored paper is false. Inspect the comparator, outcomes, access to data, control of analysis and ability to publish unfavourable results. Funding disclosure starts that inquiry. Non-financial interests and investigators' own commitments can also matter; skipping the methods would remove the information needed to assess them.", "source_ids": [ "lundh-sponsorship", "cochrane-bias" ] } ] }, { "id": "place-in-wider-evidence", "role": "findings", "title": "Place the paper beside the wider evidence", "items": [ { "label": "Look for a systematic review that answers the same question", "text": "A systematic review uses an explicit question, eligibility rules stating which studies qualify, a reproducible search and an assessment of the included evidence. A meta-analysis is a statistical combination of results; some systematic reviews appropriately do not perform one. PRISMA 2020 helps readers see how a review was conducted. Check the last search date, relevant populations, comparisons and outcomes, and reasons for including or excluding studies. A review can be rigorous yet answer a different question from yours. Its title or size cannot replace that match.", "source_ids": [ "prisma-explanation", "cochrane-searching", "cochrane-questions" ] }, { "label": "Read the studies behind a forest plot's summary", "text": "A forest plot commonly displays each study's estimate with a line for its confidence interval; a summary diamond shows a pooled estimate and interval. Larger symbols often indicate greater statistical weight. Read the axis and legend, because displays vary. Cochrane's meta-analysis guidance asks whether it makes sense to combine the studies and how differences between them affect interpretation. A precise pooled result cannot repair biased input studies. If populations, treatments or outcomes differ materially, ask whether the summary obscures the comparison you need.", "source_ids": [ "cochrane-meta-analysis" ] }, { "label": "Read a funnel plot with its possible explanations", "text": "A funnel plot places study estimates against a measure of their size or precision. An asymmetric pattern may reflect missing results, differences between smaller and larger studies, other bias or chance. Cochrane therefore cautions against treating asymmetry as a diagnosis of publication bias. A symmetric plot does not prove that nothing is missing either. These assessments are especially limited when few studies are available; tests are generally not recommended with fewer than ten. Read the explanation and search methods alongside the picture.", "source_ids": [ "cochrane-missing-evidence" ] }, { "label": "Read certainty separately for each important outcome", "text": "GRADE describes certainty in an evidence body for a particular outcome as high, moderate, low or very low. Cochrane's GRADE guidance considers risk of bias, inconsistency between studies, indirectness to the question, imprecision and publication bias. These judgments require explanations, not merely counting studies. One treatment can have relatively certain evidence about symptom improvement and uncertain evidence about a rare harm. Read both before repeating a single label as if it covered every consequence.", "source_ids": [ "cochrane-grade" ] }, { "label": "Distinguish confidence in the evidence from a recommendation to act", "text": "Certainty concerns how confidently an effect is known. Recommendation strength also considers the balance of benefits and harms, people's values and decision factors such as resources, equity and feasibility. The GRADE Working Group treats these as connected but distinct judgments. A strong recommendation is not simply a synonym for a large effect or high certainty. Read the reasoning, applicable setting and options. Your circumstances may differ from those used by a guideline panel, so one label is not a personal instruction.", "source_ids": [ "grade-working-group" ] } ] }, { "id": "understand-judgment-calls", "role": "perspectives", "title": "Understand where reasonable judgments differ", "intro": "These are specific choices about evidence and decisions. They do not make every interpretation equally well supported.", "items": [ { "label": "Recognize the debate about significance thresholds", "text": "Some statistical guidance urges readers to avoid splitting findings into significant and nonsignificant boxes. The ASA president's 2021 task force instead defended carefully used significance tests and noted that thresholds can serve explicit decisions. It also distinguished its position from a 2019 editorial proposing abandonment of statistical significance. The shared practical requirement is to explain the estimate, uncertainty, assumptions and consequences. If a decision needs a cutoff, say why that cutoff fits the decision; crossing 0.05 alone does not determine scientific or practical importance.", "source_ids": [ "asa-task-force", "cochrane-interpretation" ] }, { "label": "Choose the evidence design for the question and its constraints", "text": "Randomization can strengthen a treatment-effect comparison, but it is not available or sufficient for every question. Rare or delayed harms may require long observational follow-up; deliberately assigning a harmful exposure may be unacceptable. Cochrane describes a role for non-randomized evidence while requiring careful attention to confounding and other biases. Readers and reviewers can differ over how much confidence to place in that evidence for a particular decision. Identify the question, why this design was used, and which unresolved differences could change the answer.", "source_ids": [ "cochrane-nonrandomized", "cochrane-harms" ] }, { "label": "Make the tradeoff between earlier access and confirmed benefit visible", "text": "Waiting for patient outcomes and acting on an earlier surrogate involve different risks, especially in serious disease with few options. FDA's accelerated-approval framework permits some reasonably likely surrogates while requiring further work to confirm clinical benefit. That is a conditional decision under uncertainty. Prasad and colleagues documented reversals of established practices in selected articles from one major journal during 2001–2010. Their denominator does not represent all medical care. Together these sources support asking what evidence justified adoption and what later evidence could confirm or overturn it, rather than assuming every established practice is either proven or worthless.", "source_ids": [ "fda-surrogate-endpoints", "prasad-reversal" ] }, { "label": "Give planned confirmation and open exploration their proper jobs", "text": "Exploration can discover an unexpected pattern; a planned test can provide a more interpretable check of a specified claim. Preregistration makes that distinction visible. Registered Reports go further: journals evaluate a proposal before the results are known and can offer acceptance in principle, subject to meeting agreed conditions. The Center for Open Science acknowledges constraints involving timing, resources and concerns about flexibility; it also explains that exploratory analyses can still be reported. Readers should not treat exploration as wrongdoing or as completed confirmation. Neither format guarantees a good design, faithful execution or a useful result.", "source_ids": [ "cos-preregistration", "cos-registered-reports" ] }, { "label": "Turn broad scepticism into a specific methods question", "text": "Ioannidis's 2005 essay uses a mathematical model to examine how prior plausibility, power, bias and multiple testing can undermine positive findings. Its conclusions depend on assumptions about those conditions; it is not a census establishing that all published studies are false or fabricated. The journal also records a 2022 correction to parentheses in a table equation. You do not need to accept or reject all research at once. Ask which feature makes this claim fragile and what independent evidence could strengthen it. Specific criticism gives you something to check and something that could change your mind.", "source_ids": [ "ioannidis-2005", "ioannidis-correction" ] } ] }, { "id": "keep-a-reading-record", "role": "application", "title": "Keep a record you can use later", "intro": "A short note helps you remember the study's answer without detaching it from its limits.", "items": [ { "label": "Keep the question attached to your conclusion", "text": "Write who was studied, the comparison, outcome and duration before your one-sentence conclusion. If your own question concerns a different group or outcome, name that gap. This preserves the difference between what the paper measured and what you hope it means.", "source_ids": [ "cochrane-questions" ] }, { "label": "Save the result with its denominator and uncertainty", "text": "Keep the original outcome units, group results, effect estimate, interval and time period together. If absolute effects are unavailable, record that limitation. A saved percentage without its starting risk becomes a different claim when you repeat it.\n\nHere is an invented result to carry into a reading note: over one year, an event occurs in 20 of 1,000 people in one group and 10 of 1,000 in the comparison group. The observed risks are 2% and 1%. The difference is 1 percentage point, or 10 fewer events per 1,000 people over that year. Relative to the 2% starting risk, the observed reduction is 50%. All of those descriptions use the same numbers; '50% lower' alone conceals the starting risk.\n\nThe reading note still needs the allocation method, missing-data information and uncertainty interval before a strong interpretation. These toy counts do not supply those details or establish that the intervention caused the difference. If the headline concerns lifetime benefit, the one-year result cannot settle it. Keeping denominator, duration and limits beside the estimate prevents the more impressive wording from quietly becoming a broader claim.", "source_ids": [ "cochrane-effect-measures", "cochrane-interpretation" ] }, { "label": "Record the method concern that could change the answer", "text": "Use a concrete statement such as 'missing outcomes may differ between groups' or 'the comparator does not match my decision.' Separate an unreported detail from an observed methodological problem. A reporting checklist can locate the information; a bias assessment explains why a problem might distort the result.", "source_ids": [ "cochrane-bias", "strobe" ] }, { "label": "Record where you looked for missing evidence", "text": "Save the registration or protocol link and the relevant review's last search date. Note unavailable studies or results and how the review tried to find them. This lets a later reader check what may be missing rather than mistake your collection for everything studied.", "source_ids": [ "cochrane-searching", "cochrane-missing-evidence" ] }, { "label": "Update the question when the evidence changes", "text": "Record the paper's version, a matching review or guideline, and the outcome-specific certainty where available. Revisit the note when a correction, additional study or updated review appears. Preserve the earlier uncertainty in your record so that a changed conclusion is understandable rather than hidden.", "source_ids": [ "prisma-explanation", "cochrane-grade", "jama-medical-preprints" ] } ] }, { "id": "respect-use-and-access-limits", "role": "cautions", "title": "Respect the limits of use and access", "items": [ { "label": "Bring practice changes to the responsible professional", "text": "This is a reading companion about methods, not medical, legal, or financial advice. A headline is not a reason to start or stop a medicine. Bring practice changes to a qualified clinician or the person who owns the decision.", "source_ids": [ "jama-medical-preprints", "grade-working-group" ] }, { "label": "Ask for transparent access conditions while protecting participants", "text": "Open materials can make scrutiny easier, but research participants' identifiable data are not automatically suitable for public release. PLOS's data policy recognizes privacy, consent and ethical restrictions and asks authors to explain restrictions and an appropriate access route. A justified limit is different from unexplained concealment. Look for the availability statement, what can be checked, who controls access and why a restriction exists.", "source_ids": [ "plos-data-access" ] }, { "label": "Check the evidence behind a consequential choice", "text": "Named sources. Honest paraphrase of the finding. Not medical, legal, or financial advice.", "source_ids": [ "grade-working-group", "jama-medical-preprints" ] } ] }, { "id": "read-one-claim-through", "role": "next_steps", "title": "Read one claim all the way through", "items": [ { "label": "Start with one paper and a five-part note", "text": "Choose a study behind a claim you encountered. Write its question, design, result in the original units, uncertainty and follow-up period. Leave a blank wherever you cannot find the answer. The point is to establish what you can responsibly say, then identify the missing information that would help most.", "source_ids": [ "ihc-key-concepts", "asa-task-force" ] }, { "label": "Trace the headline into the methods and results", "text": "Read the primary outcome, its group results and interval in the full paper and supplements. Follow participant flow and missing outcomes. Then compare the dated registration, protocol or analysis plan with the published report, including disclosed amendments. Write whether the headline describes the primary result or an exploratory finding, and what methodological uncertainty remains.", "source_ids": [ "consort-2025", "cochrane-bias", "boutron-spin" ] }, { "label": "Find the wider evidence before making a practical decision", "text": "Look for a systematic review or relevant guideline answering the same question. Compare its population, outcomes, search date, missing-evidence assessment and reasoning about certainty. If the decision concerns treatment, legal action or money, take the question and your note to the qualified professional or responsible decision-maker. Use the information, learning and attention topics to continue practising how you find, retain and inspect a claim.", "source_ids": [ "cochrane-searching", "grade-working-group", "ihc-key-concepts" ] } ] } ], "related": [ "information", "learning", "attention" ] }, "sources": [ { "id": "cochrane-questions", "title": "Cochrane Handbook, Chapter 3: Defining the criteria for including studies and how they will be grouped for the synthesis", "url": "https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-03", "kind": "guideline", "accessed_at": "2026-10-01", "supports": "Opened methods chapter: defining populations, interventions, comparisons, outcomes, eligible designs and the question a review can answer.", "limitations": "Systematic-review methodology; PICO suits intervention questions and is not imposed on all research questions." }, { "id": "cochrane-effect-measures", "title": "Cochrane Handbook, Chapter 6: Choosing effect measures and computing estimates of effect", "url": "https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-06", "kind": "guideline", "accessed_at": "2026-10-01", "supports": "Opened definitions of risk, odds, relative and absolute measures, continuous outcomes and null values. Article arithmetic examples are invented illustrations.", "limitations": "Technical methods guidance; an effect measure alone supplies neither clinical importance nor a personal forecast." }, { "id": "cochrane-bias", "title": "Cochrane Handbook, Chapter 7: Considering bias and conflicts of interest among the included studies", "url": "https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-07", "kind": "guideline", "accessed_at": "2026-10-01", "supports": "Opened distinction between reporting information and evaluating systematic distortion, including conflicts of interest.", "limitations": "Risk-of-bias judgments are tied to a result and design; disclosure or reporting completeness is not an automatic validity verdict." }, { "id": "cochrane-trial-bias", "title": "Cochrane Handbook, Chapter 8: Assessing risk of bias in a randomized trial", "url": "https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-08", "kind": "guideline", "accessed_at": "2026-10-01", "supports": "Opened randomization, deviations, missing outcome data and intention-to-treat limitations; no universal safe missing-data percentage.", "limitations": "RoB 2 guidance requires study-specific information and judgment, not a checklist score." }, { "id": "cochrane-meta-analysis", "title": "Cochrane Handbook, Chapter 10: Analysing data and undertaking meta-analyses", "url": "https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-10", "kind": "guideline", "accessed_at": "2026-10-01", "supports": "Opened forest-plot conventions, pooling decisions, heterogeneity and the limits of synthesis when studies are biased or different.", "limitations": "Visual conventions vary; pooled precision does not establish validity or applicability." }, { "id": "cochrane-missing-evidence", "title": "Cochrane Handbook, Chapter 13: Assessing risk of bias due to missing evidence in a synthesis", "url": "https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-13", "kind": "guideline", "accessed_at": "2026-10-01", "supports": "Opened publication/selective-reporting bias, alternatives explaining funnel asymmetry, and limited power of assessments with few studies.", "limitations": "Neither plot symmetry nor asymmetry settles whether evidence is missing; the ten-study guidance is a rule for tests, not proof of safety above ten." }, { "id": "cochrane-grade", "title": "Cochrane Handbook, Chapter 14: Completing Summary of findings tables and grading the certainty of the evidence", "url": "https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-14", "kind": "guideline", "accessed_at": "2026-10-01", "supports": "Opened outcome-specific certainty levels and risk of bias, inconsistency, indirectness, imprecision and publication bias considerations.", "limitations": "GRADE includes additional considerations, including circumstances for rating up; this introduction does not replace a full assessment." }, { "id": "cochrane-interpretation", "title": "Cochrane Handbook, Chapter 15: Interpreting results and drawing conclusions", "url": "https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-15", "kind": "guideline", "accessed_at": "2026-10-01", "supports": "Opened absolute effects, baseline risk, time-bound NNT, uncertainty, confidence intervals and cautions about binary significance interpretation.", "limitations": "Interpretation depends on outcome, assumptions and applicability; example NNT calculations are teaching arithmetic, not observed treatment results." }, { "id": "cochrane-searching", "title": "Cochrane Handbook, Chapter 4: Searching for and selecting studies", "url": "https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-04", "kind": "guideline", "accessed_at": "2026-10-01", "supports": "Opened reproducible searching, selection, registry sources and the importance of a review's search dates and scope.", "limitations": "A comprehensive search reduces but cannot guarantee elimination of unavailable or unreported evidence." }, { "id": "cochrane-harms", "title": "Cochrane Handbook, Chapter 19: Adverse effects", "url": "https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-19", "kind": "guideline", "accessed_at": "2026-10-01", "supports": "Opened adverse-effect detection, rare and delayed harms, contributions of observational designs and spontaneous-report denominator limits.", "limitations": "Harms evidence has design- and outcome-specific limitations; no observed event does not prove no risk." }, { "id": "cochrane-nonrandomized", "title": "Cochrane Handbook, Chapter 24: Including non-randomized studies on intervention effects", "url": "https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-24", "kind": "guideline", "accessed_at": "2026-10-01", "supports": "Opened causal questions, confounding, selection and the role and limits of non-randomized intervention evidence.", "limitations": "Illustrative exercise example is authored, not a finding from an exercise study; the chapter does not authorize causal claims from any association." }, { "id": "consort-2025", "title": "CONSORT 2025 statement: updated guideline for reporting randomised trials", "url": "https://www.consort-spirit.org/_files/ugd/b5740e_a6856e5e2cf94a1db5a8005853404160.pdf", "kind": "guideline", "accessed_at": "2026-10-01", "supports": "Opened official 12-page statement PDF, including the 30-item checklist, concealment versus blinding, flow, harms, registration, protocol, analysis plan and amendments.", "limitations": "A reporting standard, not a study-quality score or guarantee that a trial was conducted soundly." }, { "id": "equator-consort", "title": "EQUATOR Network: CONSORT 2025 Statement", "url": "https://www.equator-network.org/reporting-guidelines/consort/", "kind": "official", "accessed_at": "2026-10-01", "supports": "Opened current registry entry identifying the 2025 randomized-trial reporting guideline and its supporting materials.", "limitations": "Guideline registry metadata; the full statement supplies methodological detail." }, { "id": "equator-library", "title": "EQUATOR Network: Enhancing the QUAlity and Transparency Of health Research", "url": "https://www.equator-network.org/", "kind": "official", "accessed_at": "2026-10-01", "supports": "Opened reporting-guideline library and study-type navigation.", "limitations": "Finding an applicable reporting guideline is different from assessing bias in a study." }, { "id": "strobe", "title": "STROBE: Strengthening the Reporting of Observational Studies in Epidemiology", "url": "https://www.strobe-statement.org/", "kind": "guideline", "accessed_at": "2026-10-01", "supports": "Opened official scope for cohort, case-control and cross-sectional reporting and explicit warning against use as a quality-assessment instrument.", "limitations": "Reporting recommendations are not design prescriptions or an observational-study quality score." }, { "id": "prisma-2020", "title": "PRISMA 2020 statement", "url": "https://www.prisma-statement.org/prisma-2020-statement", "kind": "guideline", "accessed_at": "2026-10-01", "supports": "Opened official 27-item checklist and supporting links for systematic-review reporting.", "limitations": "Transparent reporting is not sufficient to establish review validity; PRISMA and CONSORT address different report types." }, { "id": "prisma-explanation", "title": "The PRISMA 2020 statement: An updated guideline for reporting systematic reviews", "url": "https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1003583", "kind": "guideline", "accessed_at": "2026-10-01", "supports": "Opened primary guideline publication explaining transparent review methods and the scope of reviews with or without statistical synthesis.", "limitations": "A reporting guideline does not guarantee an exhaustive search or an appropriate pooled analysis." }, { "id": "grade-working-group", "title": "GRADE Working Group", "url": "https://www.gradeworkinggroup.org/", "kind": "official", "accessed_at": "2026-10-01", "supports": "Opened GRADE criteria and Evidence to Decision framework; outcome-specific certainty is distinct from recommendation strength and decision considerations.", "limitations": "The criteria include a May 2023 update; documents linked from the page have their own publication dates." }, { "id": "asa-p-values", "title": "American Statistical Association releases statement on statistical significance and p-values", "url": "https://www.amstat.org/asa/files/pdfs/p-valuestatement.pdf", "kind": "professional", "accessed_at": "2026-10-01", "supports": "Opened official 2016 statement summary and its six principles: model compatibility, no hypothesis-probability interpretation, no effect-size interpretation and no threshold-only reasoning.", "limitations": "Principles guide interpretation rather than prescribe a single statistical analysis for every study." }, { "id": "asa-task-force", "title": "Statement of the ASA President's Task Force on Statistical Significance and Replicability", "url": "https://magazine.amstat.org/blog/2021/08/01/task-force-statement-p-value/", "kind": "professional", "accessed_at": "2026-10-01", "supports": "Opened 2021 task-force discussion of tests, intervals, decision thresholds and its distinction from the 2019 abandonment editorial.", "limitations": "Task-force position; it does not establish that all statisticians agree on significance terminology or threshold use." }, { "id": "cos-preregistration", "title": "Center for Open Science: Preregistration", "url": "https://www.cos.io/initiatives/prereg", "kind": "official", "accessed_at": "2026-10-01", "supports": "Opened explanation of separating planned confirmatory tests from exploratory analyses and transparently reporting deviations.", "limitations": "An initiative's methods guidance, not evidence that registration alone fixes design or execution." }, { "id": "cos-registered-reports", "title": "Center for Open Science: Registered Reports", "url": "https://www.cos.io/initiatives/registered-reports", "kind": "official", "accessed_at": "2026-10-01", "supports": "Opened pre-results peer review, conditional acceptance, exploratory-analysis provisions, practical constraints and public professional explainers.", "limitations": "Acceptance in principle remains conditional on agreed standards; the format does not guarantee useful or unbiased findings." }, { "id": "ioannidis-2005", "title": "Why Most Published Research Findings Are False", "url": "https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124", "kind": "professional", "accessed_at": "2026-10-01", "supports": "Opened 2005 methods essay and modelling of prior odds, power, bias and multiple testing; distinguished model conclusions from an empirical census.", "limitations": "Assumption-dependent model, not a measurement of the fraction of all science that is false or fabricated; no equations or model-derived prevalence reproduced." }, { "id": "ioannidis-correction", "title": "Correction: Why Most Published Research Findings Are False", "url": "https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1004085", "kind": "official", "accessed_at": "2026-10-01", "supports": "Opened August 25, 2022 correction identifying missing parentheses in one Table 2 equation.", "limitations": "Specific equation correction, not a retraction or a new empirical test of the original model." }, { "id": "boutron-spin", "title": "Reporting and Interpretation of Randomized Controlled Trials With Statistically Nonsignificant Results for Primary Outcomes", "url": "https://jamanetwork.com/journals/jama/fullarticle/185952", "kind": "study", "accessed_at": "2026-10-01", "supports": "Opened 2010 primary study: 72 eligible parallel-group trials with nonsignificant primary outcomes from a specified December 2006 publication sample; spin in presentation and interpretation.", "limitations": "Selected sample and interpretive coding; not a prevalence estimate for all papers, a diagnosis of author motives or proof that a nonsignificant effect is exactly zero." }, { "id": "prasad-reversal", "title": "A decade of reversal: an analysis of 146 contradicted medical practices", "url": "https://scholars.duke.edu/publication/1407557", "kind": "study", "accessed_at": "2026-10-01", "supports": "Opened Duke's coauthor institutional record and primary abstract: established-practice evaluations in selected original articles from one journal during 2001–2010 documented reversals.", "limitations": "Abstract and bibliographic record inspected, not full publisher manuscript; journal selection and established-practice denominator prevent extrapolation to all medical care." }, { "id": "lundh-sponsorship", "title": "Industry sponsorship and research outcome", "url": "https://www.cochrane.org/evidence/MR000033_industry-sponsorship-and-research-outcome", "kind": "review", "accessed_at": "2026-10-01", "supports": "Opened 2017 Cochrane review summary: 75 empirical papers and more favourable efficacy results/conclusions associated with industry sponsorship in drug/device studies.", "limitations": "The search ran through February 2015, so these findings are not a current 2026 sponsorship rate. An association across studies does not establish the validity of an individual paper." }, { "id": "jama-medical-preprints", "title": "Preprints Involving Medical Research—Do the Benefits Outweigh the Challenges?", "url": "https://jamanetwork.com/journals/jama/fullarticle/2772743", "kind": "professional", "accessed_at": "2026-10-01", "supports": "Opened 2020 editorial defining medical preprints, describing review status and discussing cautions about use for clinical decisions.", "limitations": "A professional editorial rather than a quantitative study; it explains the distinction between a preprint and a peer-reviewed paper." }, { "id": "fda-surrogate-endpoints", "title": "FDA: Surrogate Endpoint Resources for Drug and Biologic Development", "url": "https://www.fda.gov/drugs/development-resources/surrogate-endpoint-resources-drug-and-biologic-development", "kind": "official", "accessed_at": "2026-10-01", "supports": "Opened definitions of clinical outcomes, validated versus reasonably likely surrogates and further confirmation of clinical benefit in accelerated approval.", "limitations": "US drug/biologic regulatory context; a surrogate's validity is specific to its proposed use and this article gives no treatment recommendation." }, { "id": "plos-data-access", "title": "PLOS ONE: Data Availability", "url": "https://journals.plos.org/plosone/s/data-availability", "kind": "official", "accessed_at": "2026-10-01", "supports": "Opened policy recognizing privacy, consent and ethical restrictions while requesting clear availability statements and appropriate access mechanisms.", "limitations": "One publisher's policy illustrates a legitimate transparency boundary; it is not universal law or permission to disclose participant data." }, { "id": "ihc-key-concepts", "title": "Informed Health Choices Key Concepts for assessing claims about treatment effects, 2022", "url": "https://informedhealthchoices.org/wp-content/uploads/2023/01/Informed-Health-Choices-Key-Concepts-2022.pdf", "kind": "professional", "accessed_at": "2026-10-01", "supports": "Opened public teaching framework PDF and development credits; fair comparisons and thoughtful interpretation of treatment claims.", "limitations": "Educational framework, not individual clinical advice; article's five-part reading note is an authored application, not a validated intervention." }, { "id": "reader-p-value-question", "title": "AskStatistics reader question about interpreting statistical results", "url": "https://www.reddit.com/r/AskStatistics/comments/rt4lax/", "kind": "question_signal", "accessed_at": "2026-10-01", "supports": "Opened public thread as a signal that readers ask about binary significance and what statistical results mean.", "limitations": "Anecdotal reader question only; comments, votes and apparent agreement supply no factual guidance or prevalence claim." }, { "id": "reader-reporting-question", "title": "PhD reader question about PRISMA and CONSORT diagrams", "url": "https://www.reddit.com/r/PhD/comments/1de0epu/", "kind": "question_signal", "accessed_at": "2026-10-01", "supports": "Opened public thread as a signal of confusion between reporting frameworks for reviews and trials.", "limitations": "Question discovery only; guidance is supported by the opened official reporting standards, not forum answers." } ], "professional_panel": [ { "name": "John P. A. Ioannidis", "credential": "MD, DSc; Stanford professor of medicine and epidemiology and population health; METRICS co-director", "official_url": "https://profiles.stanford.edu/john-ioannidis", "public_usefulness": "His official profile links public talks and teaching about evidence reliability and meta-research.", "contribution": "The 2005 model helps explain why plausibility, power, bias and repeated testing matter together.", "boundary": "The model depends on assumptions and is not a census declaring all studies false.", "reviewed_at": "2026-10-01" }, { "name": "Gordon Guyatt", "credential": "Physician; McMaster professor in Health Research Methods, Evidence, and Impact with a joint medicine appointment", "official_url": "https://experts.mcmaster.ca/people/guyatt", "public_usefulness": "Users' Guides teaching and evidence-based medicine explanations help clinicians and readers move from findings to decisions; McMaster's official biography confirms his clinical background.", "contribution": "GRADE and clinical evidence interpretation distinguish certainty from recommendation strength.", "boundary": "A framework cannot choose an individual's preferred tradeoff or replace context-specific professional judgment.", "reviewed_at": "2026-10-01" }, { "name": "Julian P. T. Higgins", "credential": "PhD; University of Bristol professor of evidence synthesis", "official_url": "https://www.bristol.ac.uk/person/Julian-Higgins-cddd7691-abca-46cb-8fa6-3b01745c2b98/", "public_usefulness": "The freely available Cochrane Handbook and methodological teaching explain synthesis and bias in detail.", "contribution": "Handbook editing, heterogeneity methods and risk-of-bias tools support reading evidence bodies rather than isolated results.", "boundary": "Statistical synthesis does not repair biased input studies or answer a mismatched question.", "reviewed_at": "2026-10-01" }, { "name": "Jonathan A. C. Sterne", "credential": "PhD; University of Bristol professor of medical statistics and epidemiology", "official_url": "https://www.bristol.ac.uk/people/person/Jonathan-Sterne-a3e0a405-8096-424d-a966-e082d3dfcf67/", "public_usefulness": "Public Cochrane methods chapters and risk-of-bias resources explain technical issues that readers often mistake for simple scores.", "contribution": "Missing-evidence, funnel-plot and causal-methods guidance informs assessment of fragile comparisons.", "boundary": "Funnel asymmetry has several explanations and cannot diagnose publication bias by itself.", "reviewed_at": "2026-10-01" }, { "name": "Brian Nosek", "credential": "PhD; Center for Open Science co-founder and executive director; University of Virginia psychology professor", "official_url": "https://www.cos.io/team/brian-nosek", "public_usefulness": "COS's public explainers, initiatives and training make preregistration and reproducibility accessible.", "contribution": "Separating planned tests from exploration and increasing transparent research practices.", "boundary": "Registration supplies a record; it does not make a weak design sound or guarantee replication.", "reviewed_at": "2026-10-01" }, { "name": "David Moher", "credential": "PhD in clinical epidemiology and biostatistics; Ottawa Hospital Research Institute senior scientist and Journalology director; University of Ottawa professor", "official_url": "https://ohri.ca/en/find-researcher/david-moher?page=1", "public_usefulness": "CONSORT, PRISMA and Journalology resources give readers practical ways to locate essential reporting information.", "contribution": "Leadership in developing and evaluating reporting guidance for trials and systematic reviews.", "boundary": "Reporting completeness is separate from the validity of the reported methods.", "reviewed_at": "2026-10-01" }, { "name": "Holger J. Schünemann", "credential": "MD, PhD; McMaster professor emeritus in Health Research Methods, Evidence, and Impact", "official_url": "https://experts.mcmaster.ca/people/schuneh", "public_usefulness": "GRADE guidance, Cochrane interpretation chapters and clinical-epidemiology education explain how evidence informs recommendations.", "contribution": "Outcome-specific certainty assessment and transparent evidence-to-decision methods.", "boundary": "Certainty judgments concern specified outcomes; a guideline recommendation is not automatically an individual's preferred decision.", "reviewed_at": "2026-10-01" }, { "name": "Lisa Bero", "credential": "PhD; University of Colorado professor working in evidence synthesis, research integrity and conflicts of interest", "official_url": "https://medschool.cuanschutz.edu/general-internal-medicine/research/research-programs/cbh-research-group", "public_usefulness": "Invited public speaking, media explanation and accessible sponsorship reviews help readers examine influence on evidence.", "contribution": "Research on industry sponsorship, publication and factors shaping research conclusions.", "boundary": "An association across sponsored studies is not proof that any named study or author is fraudulent.", "reviewed_at": "2026-10-01" }, { "name": "Isabelle Boutron", "credential": "MD, PhD; professor of epidemiology and Cochrane Bias Methods Group convenor", "official_url": "https://methods.cochrane.org/bias/convenors", "public_usefulness": "Public Cochrane methods work and reporting research explain bias and spin beyond the abstract's headline.", "contribution": "Primary empirical study of spin in reports with nonsignificant primary trial outcomes.", "boundary": "The 72-trial study is a defined historical sample; presentation coding does not establish author motives.", "reviewed_at": "2026-10-01" }, { "name": "Gerd Gigerenzer", "credential": "Psychologist; Max Planck Institute for Human Development director emeritus; Harding Center for Risk Literacy director", "official_url": "https://www.mpib-berlin.mpg.de/staff/gerd-gigerenzer", "public_usefulness": "Risk Savvy, Calculated Risks and professional risk-literacy teaching explain numbers using understandable event frequencies.", "contribution": "Risk communication helps readers see what an impressive relative percentage leaves out.", "boundary": "Clearer numerical presentation cannot eliminate uncertainty or predict an individual's outcome.", "reviewed_at": "2026-10-01" }, { "name": "Andrew D. Oxman", "credential": "Evidence-methods researcher and co-lead developer credited in the Informed Health Choices Key Concepts framework", "official_url": "https://informedhealthchoices.org/wp-content/uploads/2023/01/Informed-Health-Choices-Key-Concepts-2022.pdf", "public_usefulness": "The public Key Concepts framework supplies plain teaching about treatment claims and fair comparisons.", "contribution": "A structured educational approach to asking what evidence supports a claim and which comparison is fair.", "boundary": "The framework develops reading judgment and does not provide individual clinical advice; no unverified current institutional title or degree is assigned.", "reviewed_at": "2026-10-01" }, { "name": "Christopher D. Chambers", "credential": "PhD; Chartered Psychologist and Cardiff University professor of cognitive neuroscience", "official_url": "https://profiles.cardiff.ac.uk/staff/chambersc1", "public_usefulness": "Public writing, The Seven Deadly Sins of Psychology, workshops and Registered Reports explanations make reproducibility choices understandable.", "contribution": "Development and advocacy of pre-results review and Registered Reports.", "boundary": "The format changes incentives and scrutiny; it cannot guarantee execution quality or a result's importance.", "reviewed_at": "2026-10-01" }, { "name": "Daniel J. Simons", "credential": "PhD; University of Illinois psychology professor", "official_url": "https://experts.illinois.edu/en/persons/daniel-j-simons/", "public_usefulness": "COS Registered Reports explanations and an officially documented public lecture on Nobody's Fool help readers question apparently persuasive claims.", "contribution": "Registered replication publishing and methodological communication encourage scrutiny independent of a striking outcome.", "boundary": "Replication and a transparent publication format still require interpretation of populations, measures and real-world meaning.", "reviewed_at": "2026-10-01" }, { "name": "Matthew J. Page", "credential": "PhD in epidemiology; Monash University associate professor and evidence-synthesis methods researcher", "official_url": "https://research.monash.edu/en/persons/matthew-page/", "public_usefulness": "PRISMA 2020 materials and accessible synthesis-methods guidance help readers inspect review searches and reporting.", "contribution": "PRISMA 2020 development, risk-of-bias methods and research on missing evidence.", "boundary": "A reporting standard does not establish that a particular review searched adequately or pooled appropriate studies.", "reviewed_at": "2026-10-01" }, { "name": "Paul Glasziou", "credential": "Physician and evidence-based medicine researcher at Bond University", "official_url": "https://research.bond.edu.au/en/persons/paul-glasziou/", "public_usefulness": "Public evidence-based medicine resources and clinical translation work connect research findings to useful, understandable choices.", "contribution": "Cochrane interpretation guidance and work on evidence use emphasize clinical meaning and practical applicability.", "boundary": "Useful evidence interpretation remains specific to the patient-important outcome, comparison and decision; no current director title is asserted.", "reviewed_at": "2026-10-01" } ] }