Bibliographic record
Abstract
[Editor's Note: Both the letter to the editor by Guccione and Mielenz and the response by Riddle and colleagues are commenting on the accepted but unedited author manuscript version of this article that was published ahead of print on February 7, 2013.] We appreciate the opportunity to respond to the letter by Guccione and Mielenz.1 Ours was a straightforward replication study of the work by Guccione and colleagues,2 with some additional analyses.3 We studied 1,030 outpatients with isolated musculoskeletal disorders, the same types of disorders as 77% of the 391 outpatients in the study by Guccione et al. The Outpatient Physical Therapy Improvement in Movement Assessment Log (OPTIMAL) is an American Physical Therapy Association–endorsed4 patient-reported outcome instrument that has undergone surprisingly little study since it was first introduced in 2003. Given the importance of replication in determining the extent to which findings reported in one study may generalize to other sites and patients, we thought the psychometric characteristics of OPTIMAL needed re-examination. We found that several of the measurement properties of OPTIMAL were disappointingly low and recommended that clinicians treating outpatients with musculoskeletal disorders consider other instruments that are more psychometrically sound. We focus our comments on issues identified by Guccione and Mielenz that relate to the interpretation of our data. One of our key findings was that patients' responses to the OPTIMAL Difficulty and OPTIMAL Confidence Scales demonstrated extensive overlap. That is, our factor analysis loadings grouped on anatomical site and not separately on difficulty and confidence constructs. In addition, the Pearson r association between the 2 scale scores was .89. Our data suggested that clinicians who use both OPTIMAL scales are not actually measuring difficulty and confidence constructs but rather are capturing essentially the same information with both scores. Guccione and Mielenz discussed findings from unpublished work that they claimed supported use of the OPTIMAL Difficulty and Confidence Scales. Unpublished data, in our view, should not be used to advocate for the utility of an instrument. Guccione and Mielenz questioned the use of region-specific scales as comparators for our convergent construct validity analysis. They contend that the region-specific scales we used have content and theoretical underpinnings that are substantially different from OPTIMAL. As we discussed in our article, the region-specific scales we selected are among the most studied and validated patient-report functional status measures available for patients with musculoskeletal disorders. The purpose of construct validation is to assess the extent to which measures with similar theoretical foundations are associated with one another.5 We agree that the region-specific scales do not capture the same phenomena as OPTIMAL. If they did, there would be no need for OPTIMAL. In our view, the overlap between OPTIMAL items and the region-specific scale items, as well as their conceptual foundations, is considerable and justifies a convergent construct validation approach. We are intrigued by OPTIMAL and see potential advantages to measuring the extent of limitations in the types of items included in the instrument. Unfortunately, our data did not support clinical application for outpatients with musculoskeletal disorders. We look forward to future publications designed to determine the extent to which valid inferences can be drawn from OPTIMAL scores.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".