The PROMIS of Better Outcome Assessment: Responsiveness, Floor and Ceiling Effects, and Internet Administration
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
OBJECTIVE: Use of item response theory (IRT) and, subsequently, computerized adaptive testing (CAT), under the umbrella of the NIH-PROMIS initiative (National Institutes of Health-Patient-Reported Outcomes Measurement Information System), to bring strong new assets to the development of more sensitive, more widely applicable, and more efficiently administered patient-reported outcome (PRO) instruments. We present data on current progress in 3 crucial areas: floor and ceiling effects, responsiveness to change, and interactive computer-based administration over the Internet. METHODS: We examined nearly 1000 patients with rheumatoid arthritis and related diseases in a series of studies including a one-year longitudinal examination of detection of change; compared responsiveness of the Legacy SF-36 and HAQ-DI instruments with IRT-based instruments; performed a randomized head-to-head trial of 4 modes of item administration; and simulated the effect of lack of floor and ceiling items upon statistical power and sample sizes. RESULTS: IRT-based PROMIS instruments are more sensitive to change, resulting in the potential to reduce sample size requirements substantially by up to a factor of 4. The modes of administration tested did not differ from each other in any instance by more than one-tenth of a standard deviation. Floor and ceiling effects greatly reduce the number of available subjects, particularly at the ceiling. CONCLUSION: Failure to adequately address floor and ceiling effects, which determine the range of an instrument, can result in suboptimal assessment of many patients. Improved items, improved instruments, and computer-based administration improve PRO assessment and represent a fundamental advance in clinical outcomes research.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.013 | 0.028 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it