Assessing the Quality of Evidence Presented at Annual General Meetings: A 5-Year Retrospective Study
Bibliographic record
Abstract
INTRODUCTION: Health care professionals rely on annual general meetings (AGMs) to obtain up-to-date information and practice guidelines relevant to their specialty. The majority of such information at meetings is presented through abstract sessions. However, the quality of the evidence presented during such abstract sessions is unclear. Standardized measures were applied to assess the quality of evidence of abstracts presented at the Canadian Society of Nephrology AGM over a 5-year period. METHODS: Two authors independently reviewed all CSN AGM abstracts presented from 2012 to 2016. Using a schema published in 2011 by the Oxford Centre for Evidence-Based Medicine (OCEBM), each abstract was subsequently ranked based on the quality of evidence. Schema categories ranged from level I, representing the highest evidence quality, to level V, representing the lowest. The number of authors and the authors' institution affiliations were also collected from the abstracts, where available, or if affiliations were unclear, an internet search of the author was performed. RESULTS: Six hundred forty-two articles were screened. In total, 70% (n = 450) met the inclusion criteria. When assessed, 15% of articles were level I (highest quality), 17% level II, 53% level III, 12% level IV, and 3% level V (lowest quality). A Jonckheere-Terpstra test demonstrated a significant trend of increasing quality of evidence (P < .05) and collaboration (P < .005) over the 5-year study period. There was a significant correlation between level of evidence and collaboration across years reviewed in the study, rs(98) = -0.226, P < .001. DISCUSSION: The results indicate a consistent increase in quality of evidence and collaborative submissions over time. To the authors' knowledge, this is the first assessment and analysis of AGM presentation quality within internal medicine and its subspecialties. Documenting and monitoring changes in the quality of evidence with a standardized framework may offer valuable insight pertaining to the medical field and the research community.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.050 | 0.162 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.004 | 0.006 |
| Bibliometrics | 0.023 | 0.025 |
| Science and technology studies | 0.002 | 0.002 |
| Scholarly communication | 0.005 | 0.006 |
| Open science | 0.002 | 0.003 |
| Research integrity | 0.002 | 0.001 |
| Insufficient payload (model declined to judge) | 0.004 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".