Bibliographic record
Abstract
I read with interest the alternate analysis offered by Goertz et al1Goertz CM, Hurwitz E, Murphy B, Coulter I. Extrapolating beyond the data in a systematic review of spinal manipulation for nonmusculoskeletal disorders: a fall from the summit [e-pub ahead of print]. J Manipulative Physiol Ther. doi: https://doi.org/10.1016/j.jmpt.2021.02.003. Accessed April 17, 2021.Google Scholar and would like to add to the comments regarding the limitations of systematic reviews, including important limitations I have observed but that were not discussed. In particular, I point out the obvious, which is that the entire systematic review process is as much a social exercise as a scientific one, and so may be subject to strong and even destructive social influences.2Szanto T Collaborative irrationality, akrasia and groupthink: social disruptions of emotion regulation.Frontiers Psychol. 2017; 7: 8Crossref Scopus (5) Google Scholar My recollection is that the first invitation to the Global Summit referred to vitalism as the focus, not clinical practice, and, although later the purpose was reframed, they conflated management of nonmusculoskeletal conditions with disregard for evidence. Thus, notwithstanding whatever scientific merit the Summit claims, it appears, nonetheless, to be motivated by a desire to engineer the profession of chiropractic. This motivation in itself is not unjustified but appears to have modulated the methodology in a way that undermines the credibility of its rather far-reaching conclusions. Thus, I would like to draw attention to my observations of the social dynamics of the Summit. In the field of psychology, one will see discussion of the continuum between consensus, compliance, and coercion.3Galam S Moscovici S Towards a theory of collective phenomena: consensus and attitude changes in groups.Eur J Soc Psychol. 1991; 21: 49-74Crossref Scopus (266) Google Scholar In this regard, when we see a “purposive and snowball sampling” to solicit participant reviewers for the Summit, this causes us to ask what, then, was the purpose? Was it to listen to a diversity of expert opinion, or were there other considerations? Note that 42 of the 46 authors had previously coauthored articles with other members of the group, and in some cases prior coauthorships numbered in the dozens. Certainly, the chiropractic profession is a small pond to draw from; however, should one consider whether the social structure of this group likely modulated independent thought?4Bond R Smith PB Culture and conformity: a meta-analysis of studies using Asch's (1952b, 1956) line judgement task.Psychol Bull. 1996; 119: 111-137Crossref Scopus (874) Google Scholar Of equal concern, in some instances, participants were in positions of power over other participants. Did the organizers understand the pressure that the junior reviewers would be under to manifest consensus agreement with their supervisors or senior colleagues? Should it also be a concern that the authors included colleagues whose previously published opinions on the topic calls into question their ability to conduct an unbiased examination of the literature? The dynamics of the enterprise were further contaminated by having the Summit's work take place under the examination of observers including reviewers’ employers, insurers, and professional regulators who were given the opportunity to interact with the participant-reviewers in the course of their work. In sum, these conditions prime the collective for considerable bias in the execution of its deliberations. In a work that so scrupulously examined risk of bias in the literature under review, perhaps the Summit participants ought to have turned the mirror on themselves. Given human nature, no matter how contrived the ceremony, systematic reviews represent opinions, and consensus cannot avoid amalgamation with compliance or coercion. I believe that these concerns should also be taken into consideration with the analysis by Goertz et al. when interpreting the results and recommendations of the Summit.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.130 | 0.662 |
| Meta-epidemiology (narrow) | 0.002 | 0.002 |
| Meta-epidemiology (broad) | 0.004 | 0.004 |
| Bibliometrics | 0.008 | 0.007 |
| Science and technology studies | 0.003 | 0.007 |
| Scholarly communication | 0.010 | 0.013 |
| Open science | 0.005 | 0.007 |
| Research integrity | 0.014 | 0.016 |
| Insufficient payload (model declined to judge) | 0.022 | 0.011 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".