MétaCan
Menu
Back to cohort
Record W2124548530 · doi:10.1136/ebm.12.2.59

Simon S. Statistical evidence in medical trials: what do the data really tell us? Oxford: Oxford University Press, 2006.

2007· article· en· W2124548530 on OpenAlexaff
Janet Martin

Bibliographic record

VenueEvidence-Based Medicine · 2007
Typearticle
Languageen
FieldDecision Sciences
TopicMeta-analysis and systematic reviews
Canadian institutionsLondon Health Sciences Centre
Fundersnot available
KeywordsHistoryPsychology

Abstract

fetched live from OpenAlex

fter reading the first paragraph of this book, I was immediately drawn in, mesmerised, and entertained right through to the end of the book.That says a lot for a book about statistics!It reads better than some of my bedside novels, but is not at all fictional.The book begins with a statistical joke-a perfect introduction for both the serious statistician and the more lighthearted clinician alike-which sets the tone for the remainder of the book.Many chapters are prefaced with a comical pictorial or word sketch that hints at the content to follow, while making the reader hungry enough to read on.That the injection of humour was possible in a statistics book is in itself quite enlightening.This point alone makes the book a worthwhile read.But the worth does not stop there.The primary purpose of this book is to provide an understanding of how medical literature should be interpreted, despite its limitations.The concepts are useful for all levels, from beginner to expert.In the words of the author himself, this book is for consumers, not producers, of medical research.The book grew from the experience of the author, a statistician, who has provided training and expertise in interpreting medical literature in the hospital setting.The major thesis of the book is that ''you should worry more about how the data were collected rather than how it was analyzed.''Fittingly, then, to support this thesis the author writes the entire book about statistics without using numbers and formulas.Concepts are explained with words and pictorials so effectively that the absence of numbers goes unnoticed.As a result, one comes away with an understanding of the very essence-the conceptual core-of what is important for interpreting clinical trials.This is much more than can be said for many books about statistics in medicine.In fewer than 200 pages (7 chapters), the book covers a sizable range of topics necessary for discerning the literature.Chapter 1 discusses the risk of unfair (applesto-oranges) comparisons and how to detect them across a variety of study designs.Chapter 2 discusses the implications of selective recruitment, purposeful exclusions of troublemakers, and the vexing problem of incomplete follow up.Chapter 3 discusses how to detect whether trial results can be considered worthy enough to change your practice, or whether they are trivial.Chapter 4 addresses how studies should be interpreted in the context of whether other evidence (or its absence) corroborates or detracts from its message.Chapter 5 discusses ways to properly assess the totality of the evidence when more than one study exists.Chapter 6 provides explanation of concepts such as confidence interval, odds ratio, number needed to treat, correlation, without ever resorting to statistical jargon or complex formulae.Chapter 7 offers a simple strategy for finding clinical trials of highest quality in response to well built clinical questions.Each subtopic is accompanied by a short explanation and >1 example from the literature.Excerpts from open access journal articles are used as examples so that readers can access the full text freely.The author provides a good balance of pointcounterpoint discussions for areas that remain controversial (ie, blinding is important, but can be over-rated).At the end of each chapter, key points are summarised for easy reference.The author's website provides further examples and opportunity for more advanced learning.Two detractions in this book should be highlighted so that the reader is forewarned.Firstly, several flaws in the wordsmithing and grammar were missed during the editing process.Secondly, some of the medical terminology or clinical explanations are less than perfect, which is not entirely surprising given that Simon himself admits up front that he is not a clinician.I found that both of these limitations were easy to overlook, and other benefits far outweighed any detractions.Clearly this book is not ''just another statistics book.''Rather, it borders on the side of being revolutional-a statistics book without numbers!While this might be considered near sacrilege in the world of pure statistics, for the purposes of inciting balanced, practical, evidence-based clinical decision making, it is nearly a 5 star resource.The tasteful humour injected throughout the text is just the perfect spoonful of sugar to make the medicine go down.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.

How this classification was reachedexpand

Direct model labels (unvalidated)

Per-model category and study-design labels from the labeling rounds. They are machine output, unvalidated, and the disagreement between models ships as data. No study design here is MEDLINE-validated yet.

Model armCategoriesStudy designConfidence
gemmaMetaresearch
Domain: Methods · Genre: Other
About the Canadian research system: no · About a Canadian topic: no
Not applicablelow
gptInsufficient payload (model declined to judge)
Domain: not available · Genre: Other
About the Canadian research system: no · About a Canadian topic: no
Not applicablelow
models splitAgreement compares identical category sets and study designs across arms.

Full frame distilled prediction

Teacher imitation

Not calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.

metaresearch head score (Codex)0.539
metaresearch head score (Gemma)0.608
Version: codex-gemma-dda1882f352aValidation status: machine_predicted_unvalidated
Candidate categoriesMetaresearch, Open science, Insufficient payload (model declined to judge)
Consensus categoriesMetaresearch
DomainCandidate signal: none · Consensus signal: none
Study designCandidate signal: Not applicable · Consensus signal: Not applicable
GenreCandidate signal: Empirical · Consensus signal: none
Teacher disagreement score0.776
Threshold uncertainty score0.996

Codex and Gemma teacher scores by category

CategoryCodexGemma
Metaresearch0.5390.608
Meta-epidemiology (narrow)0.0010.000
Meta-epidemiology (broad)0.0050.001
Bibliometrics0.0010.003
Science and technology studies0.0000.001
Scholarly communication0.0010.003
Open science0.0090.001
Research integrity0.0000.001
Insufficient payload (model declined to judge)0.0200.000

Machine scores (provisional)

The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.

Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.

Opus teacher head0.789
GPT teacher head0.548
Teacher spread0.240 · how far apart the two teachers sit on this one work
Validation statusscore_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it

Classification

machine, unvalidated

Labeled directly by 2 models reading the full record.

MetaresearchInsufficient payload (model declined to judge)

The models disagree on parts of this classification; every voice is preserved in the section at the end of the page.

Study designNot applicable
DomainMethods
GenreOther

How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".

Quick stats

Citations0
Published2007
Admission routes1
Has abstractyes

Explore more

Same venueEvidence-Based MedicineSame topicMeta-analysis and systematic reviewsCategoryMetaresearchFrench-language works237,207