Safety Profile Assessment: An online tool to gauge safety-critical performance in radiation oncology
Bibliographic record
Abstract
PURPOSE: It is challenging for the radiation oncology practitioner to manage and implement the plethora of recently generated recommendations on quality and safety improvement. The online Safety Profile Assessment (SPA) tool uses an easy-to-use question-and-answer format to assess safety/quality within a clinic, provide a way to benchmark against peers, and facilitate improvement. This report describes the design and development of the SPA and experience from the first year of use. METHODS: Performance indicators for the SPA were derived from 4 foundations: the Agency for Healthcare Research and Quality, a review of 7 recent authoritative documents specific to radiation oncology, a recent American Association of Physicists in Medicine report on incident learning, and the American College of Radiology-American Society for Radiation Oncology accreditation system as of 2011. After pilot testing, the free-access tool was launched through the American Association of Physicists in Medicine website (http://spa.aapm.org) in July 2013. Questionnaire data were collected to assess the experience of users. RESULTS: The SPA tool consists of 92 indicators designed to probe safety and quality. A clinic's performance is benchmarked against all other responses in the database, and aided by a downloadable log, quality/safety improvement strategies can be developed and tracked over time. At the time this paper was written, 279 individuals had registered, and 107 had completed the SPA. On average, the SPA required 1.3 hours to complete. The majority of respondents to the questionnaire (56%) completed the SPA with a multidisciplinary group of 4 people on average. Respondents noted that the SPA was easy or very easy to use (70%) and that they would definitely or very probably complete it again (63%). CONCLUSIONS: SPA provides a straightforward means of gauging a clinic's performance in key safety-critical areas and has been evaluated favorably by the first cohort of users. The tool has been qualified by the American Board of Radiology (ABR) as meeting the criteria for Practice Quality Improvement requirements of the ABR Maintenance of Certification Program.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.001 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".