Longitudinal Evaluation of Interobserver and Intraobserver Agreement of Cervical Intraepithelial Neoplasia Diagnosis Among an Experienced Panel of Gynecologic Pathologists
Bibliographic record
Abstract
Histologic diagnoses of cervical intraepithelial neoplasia grades 2 and 3 (CIN 2/3) are the key end points in clinical trials that evaluate the efficacy of a prophylactic quadrivalent human papillomavirus vaccine against cervical cancer. Adjudication of end points uses a panel of 4 pathologists. Quality control slides (n=185) from a nonclinical trial study with preestablished gold standard CIN diagnoses were used to characterize the panel's agreement on CIN diagnoses and monitor performance longitudinally. At 3-month intervals over 2 years, 1 of 6 different batches of quality control slides (n=30-31) was included with clinical trial slides for independent review by each of the 4 panelists. Unweighted kappas (kappa) were estimated within each panelist pair by dichotomizing the diagnoses as CIN+ versus non-CIN+ (including normal, unsatisfactory, and atypical immature metaplasia) or CIN 2/3+ versus non-CIN 2/3+ (including normal, unsatisfactory, atypical immature metaplasia, and CIN 1). Quadratic weighted kappa was calculated within each panelist pair using 4 diagnostic categories: normal, CIN 1, CIN 2, and CIN 3 or worse. Substantial interobserver agreement was observed (weighted kappa=0.765 to 0.865). Agreement with weighted kappa=0.779 to 0.887 was observed between the individual panelists and the gold standard, which is almost perfect agreement by Landis-defined categories. Intraobserver agreement was very high (weighted kappa=0.756 to 0.883). Some fluctuation in intraobserver and interobserver agreement was observed over the study period but there was no decreasing time trend. These data indicate that the interpretation of histologic end points used in the quadrivalent vaccine clinical trial program is highly valid and reliable.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.030 | 0.038 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".