SU‐F‐T‐249: Application of Human Factors Methods: Usability Testing in the Radiation Oncology Environment
Bibliographic record
Abstract
Purpose: As a quality improvement measure, we undertook this work to incorporate usability testing into the implementation procedures for new electronic documents and forms used by four affiliated radiation therapy centers. Methods: A human factors specialist provided training in usability testing for a team of medical physicists, radiation therapists, and radiation oncologists from four radiotherapy centers. A usability testing plan was then developed that included controlled scenarios and standardized forms for qualitative and quantitative feedback from participants, including patients. Usability tests were performed by end users using the same hardware and viewing conditions that are found in the clinical environment. A pilot test of a form used during radiotherapy CT simulation was performed in a single department; feedback informed adaptive improvements to the electronic form, hardware requirements, resource accessibility and the usability testing plan. Following refinements to the testing plan, usability testing was performed at three affiliated cancer centers with different vault layouts and hardware. Results: Feedback from the testing resulted in the detection of 6 critical errors (omissions and inability to complete task without assistance), 6 non‐critical errors (recoverable), and multiple suggestions for improvement. Usability problems with room layout were detected at one center and problems with hardware were detected at one center. Upon amalgamation and summary of the results, three key recommendations were presented to the document's authors for incorporation into the electronic form. Documented inefficiencies and patient safety concerns related to the room layout and hardware were presented to administration along with a request for funding to purchase upgraded hardware and accessories to allow a more efficient workflow within the simulator vault. Conclusion: By including usability testing as part of the process when introducing any new document or procedure into clinical use, associated risks can be identified and mitigated before patient care and clinical workflow are impacted.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.038 | 0.068 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.002 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.002 | 0.002 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.004 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".