Levels of Evidence at the Pediatric Orthopaedic Society of North America Annual Meetings
Bibliographic record
Abstract
BACKGROUND: Since 2003, levels of evidence have been used in the orthopaedic literature to inform the reader of the study quality and its relative significance. Our primary research question was to identify if, since their introduction, there has been an improvement in the levels of evidence of the scientific papers presented at Pediatric Orthopaedic Society of North America (POSNA) meetings. METHODS: The abstract of every paper presented orally at the POSNA annual meeting were identified for the years 2001, 2002 (pre-2003) and 2007, 2008 (post-2003). In all, 364 abstracts were identified by an independent reviewer who then excluded cadaver, animal, and basic science studies. The 307 included abstracts were then independently blinded and randomized. Two nonepidemiologically trained pediatric orthopaedic fellows independently assigned a study type and level of evidence to each abstract based on the primary research question. The first reviewer reanalyzed 50 randomly selected abstracts. Disagreement was resolved by consensus opinion with an epidemiologically trained pediatric orthopaedic surgeon. The interobserver and intraobserver reliability was calculated for the assignment of study type and levels of evidence. Changes in the study types and levels of evidence were analyzed to compare papers presented pre-2003 and post-2003. RESULTS: For study type and levels of evidence the interobserver reliability between the authors showed substantial agreement (kappa 0.755 and 0.647, respectively). The intraobserver reliability also showed substantial agreement (kappa 0.806 and 0.789, respectively). Comparing pre-2003 and post-2003 studies there were no significant differences between the types of study presented. An increase in the number of level III studies was identified. A decrease in level I and level IV studies was identified. CONCLUSIONS: There has been no significant improvement in the scientific quality of studies presented at POSNA Annual Meetings as measured by levels of evidence since their introduction. We have shown substantial agreement between nonepidemiologically trained orthopaedic surgeons when categorizing abstracts using this system. Study types and levels of evidence can be difficult to assign from the current abstract format due to their brevity and structure. LEVEL OF EVIDENCE: Not applicable.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.021 | 0.157 |
| Meta-epidemiology (narrow) | 0.003 | 0.002 |
| Meta-epidemiology (broad) | 0.006 | 0.004 |
| Bibliometrics | 0.014 | 0.011 |
| Science and technology studies | 0.002 | 0.004 |
| Scholarly communication | 0.012 | 0.005 |
| Open science | 0.007 | 0.003 |
| Research integrity | 0.013 | 0.008 |
| Insufficient payload (model declined to judge) | 0.090 | 0.040 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".