MétaCan
Menu
Back to cohort
Record W2733427969 · doi:10.1097/ede.0000000000000706

Re

2017· letter· war· W2733427969 on OpenAlexaffabout
Michael A. McIsaac

Bibliographic record

VenueEpidemiology · 2017
Typeletter
Languagewar
FieldComputer Science
TopicBayesian Modeling and Causal Inference
Canadian institutionsQueen's University
Fundersnot available
KeywordsPropositionBayes' theoremBelief revisionMistakeEpistemologySet (abstract data type)Point (geometry)Mathematical economicsComputer sciencePsychologyMathematicsPhilosophyArtificial intelligenceBayesian probabilityLawPolitical science

Abstract

fetched live from OpenAlex

To the Editor: In their letter, Cole et al1 set out to prove that a dogmatist (one whose beliefs cannot be influenced by observations) cannot learn (which they define as “having your beliefs influenced by observations”). Disregarding that this proposition is true by definition, the authors provided an amusing exploration into this idea using Bayes’ theorem. However, the authors made an important error in their calculations. This error does not change the authors’ point about dogmatists being unable to learn, but it does lead to the incorrect conclusion that it is good practice to allow any piece of information to sway your opinion, regardless of the quality of that information; this seems like a major oversight in this era of “fake news”2 and “alternative facts!”3,4 Cole et al1 consider an encrypted question that has two possible answers; we will denote these as or . The authors suppose that 12 people answer the question of interest and examine how prior belief in the correct answer is updated by these observations. The authors remind us of Bayes’ theorem, which says that belief in an answer after observing data [the posterior probability ] is a function of both prior belief and the likelihood of the observed data if the answer is . They define a dogmatic belief as one that is certain a priori [e.g., ] and observe that data do not influence this dogmatic belief . However, in their calculations, Cole et al1 conflated their answer to the question of interest with the probability that each person actually gave the right answer. In the remainder of this letter, I will correct the calculations and demonstrate the important finding that learning from bad information can be worse than not learning at all. As in reference (1), suppose that in absence of evidence, a nondogmatist believes the two possible answers to be equally likely . Cole et al1 wish to calculate the nondogmatist’s belief that after learning that five people think and seven think . That is, the goal is to find the posterior probability. The key point missed by Cole et al1 is that is a function of the probability (call it ) that each of these independent and exchangeable people gave the right answer. In their example with deterministic Q, If the observations came from 12 individuals who were each fairly likely to get the right answer, say = 2/3, then the posterior probabilities would be and , and as reported by Cole et al,1 a nondogmatist should be swayed to believe that the correct answer is . However, if the 12 “experts” were answering completely randomly (so = 1/2), then we can see that their information truly should not influence beliefs: and . Most importantly, if the information came from 12 people who were unlikely to give the right answer, say = 1/3, then beliefs should actually be influenced in the opposite direction: and . Without knowing the quality of the data, it is impossible to properly learn from it. If presented with bad data, the nondogmatist could easily be swayed to posterior beliefs that are further from the truth than the ones established a priori! In this light, I propose an addendum to the proposition by Cole et al1: it is true that dogmatists cannot learn, but learning may actually be detrimental to those who cannot separate truth from “alternative facts.” Michael A. McIsaac Department of Public Health Sciences Queen’s University Kingston, ON, Canada [email protected]

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.

How this classification was reachedexpand

Full frame machine prediction

Teacher imitation

Not calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.

metaresearch head score (Codex)0.002
metaresearch head score (Gemma)0.015
Version: metacan-v3-hybrid-931329e0061cValidation status: machine_predicted_unvalidated
Candidate categoriesInsufficient payload (model declined to judge)
Consensus categoriesnone
DomainCandidate signal: none · Consensus signal: none
Study designCandidate signal: Not applicable · Consensus signal: Not applicable
GenreCandidate signal: Other · Consensus signal: none
Teacher disagreement score0.600
Threshold uncertainty score0.000

Distilled classifier scores by category (both heads)

CategoryCodexGemma
Metaresearch0.0020.015
Meta-epidemiology (narrow)0.0010.000
Meta-epidemiology (broad)0.0010.001
Bibliometrics0.0010.001
Science and technology studies0.0020.001
Scholarly communication0.0060.003
Open science0.0030.002
Research integrity0.0090.006
Insufficient payload (model declined to judge)0.4000.308

Machine scores (provisional)

The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.

Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.

Opus teacher head0.197
GPT teacher head0.379
Teacher spread0.182 · how far apart the two teachers sit on this one work
Validation statusscore_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it

Classification

machine, unvalidated

Machine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.

Study designNot applicable
Domainnot available
GenreOther

How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".

Quick stats

Citations1
Published2017
Admission routes2
Has abstractyes

Explore more

Same venueEpidemiologySame topicBayesian Modeling and Causal InferenceFrench-language works237,207