SU‐FF‐T‐406: Testing of ATC Method 2 for Supporting QA of Cooperative Group Advanced Technology Clinical Trials Requiring Digital Data Submission
Bibliographic record
Abstract
Purpose: To test the readiness of a system of software (“ATC Method 2‐v.2.3”) developed by the Resource Center for Emerging Technologies (RCET) for supporting QA of cooperative group clinical trials within the Advanced Technology QA Consortium (ATC). Method and Materials: ATC Method 2‐v.2.3 includes WebSys client and server for secure data upload/download/archiving of volumetric imaging and radiotherapy treatment planning data, a web‐based Rapid Image Viewer (RIV) tool, and web‐based tools for server administration. The software was implemented on a test server at the Image‐guided Therapy QA Center (ITC), and underwent rigorous testing by ITC personnel. Tests conducted included examination of user interface behavior, as well as systematic comparison of submitted/retrieved copies of 16 representative test data sets (in DICOM and RTOG Data Exchange format) from nine different treatment planning system vendors. Results: Evaluation tests of version 2.3 of the ATC Method 2 software identified improvements in the usability of software over the previous version, and provided general suggestions for further improvement. These tests also identified specific input that led to failure of the WebSys client, usability issues in the RIV tool, database changes needed to support case identifiers in ATC trails, and corrections needed in handling certain DICOM objects. These test results have contributed to improvements in version 2.4 of this software in preparation for its use to support clinical trials. Version 2.4 is expected to be ready for testing beginning March 2006. The National Cancer Institute of Canada (NCIC) Clinical Trials Group (CTG) will participate in the new round of testing. Conclusion: Since this software is intended to play a major role in QA of data submitted for future ATC‐supported clinical trials, rigorous testing is essential to its ongoing development. Future testing is expected to benefit from collaborative efforts of NCIC CTG. Supported by NIH U24 Grant CA81647.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.050 | 0.233 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.000 | 0.002 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.002 | 0.002 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; both teacher heads agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".