A novel computerized approach to scoping reviews using Synthesis software: the first 15 years of The American College of Surgeons National Surgical Quality Improvement Program
Bibliographic record
Abstract
Scoping reviews of innovations in health care characterized by large numbers and types of publications present a unique challenge. A novel software application, Synthesis, can efficiently scan the literature to map the evidence and inform practice. We applied Synthesis to the National Surgical Quality Improvement Program (NSQIP), a high-quality database designed to measure risk-adjusted 30-day surgical outcomes for national and international benchmarking. The scoping review describes the breadth of studies in the NSQIP literature. We performed a comprehensive electronic literature search using PubMed, MEDLINE, Web of Knowledge and Scopus to capture all NSQIP articles published between Jan. 1, 2000, and Dec. 31, 2020. Two reviewers independently reviewed articles to determine their relevance using predefined inclusion criteria. We imported references into Synthesis to semiautomate data management. Extracted data included surgical specialty, study type and year of publication. Of the 4661 NSQIP articles included, 3631 (77.9%) were published within the last 5 years. Among NSQIP-related articles, the most common study types were based on outcomes (46.7%) and association (41.7%), and the most common surgical specialties were general surgery and orthopedic surgery, representing 35.7% and 24.0% of the articles, respectively. Synthesis enabled a rapid review of thousands of NSQIP publications. The scoping review provided an overview of the articles in the NSQIP literature and suggested that the NSQIP is increasingly being described in publications of quality and safety in surgery.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.307 | 0.546 |
| Meta-epidemiology (narrow) | 0.005 | 0.006 |
| Meta-epidemiology (broad) | 0.007 | 0.008 |
| Bibliometrics | 0.048 | 0.045 |
| Science and technology studies | 0.003 | 0.003 |
| Scholarly communication | 0.018 | 0.010 |
| Open science | 0.005 | 0.013 |
| Research integrity | 0.003 | 0.005 |
| Insufficient payload (model declined to judge) | 0.028 | 0.006 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".