Strengths and challenges of the COSMIN tools in outcome measures appraisal: A case example for speech–language therapy
Bibliographic record
Abstract
BACKGROUND: The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) is an international initiative that offers standardized and validated tools to guide the appraisal of patient-reported outcome measures in healthcare. AIMS: To explore the use of a new set of tools from the COSMIN to appraise studies on one outcome measure available to speech and language therapists (SLTs). METHODS & PROCEDURES: We used the COSMIN tools to appraise seven studies and a user manual that reported the measurement properties of the Focus on the Outcomes of Children Under Six (FOCUS), a validated measure of pre-schoolers' communicative participation that is used in various contexts around the world. OUTCOMES & RESULTS: Using COSMIN guidelines, the FOCUS was categorized as a 'category A' tool because there was a sufficient level of evidence to support its content validity and internal consistency. According to the COSMIN guidelines, this means that the FOCUS can be recommended for use. The quality of evidence supporting measurement properties of the FOCUS received a rating of 'moderate', meaning users can have moderate confidence in its measurement properties. Since these ratings from the COSMIN tools may be unclear to users of the FOCUS, we have provided more specific recommendations. CONCLUSIONS & IMPLICATIONS: The COSMIN tools offer detailed standards to support the appraisal of outcome measures available to SLTs. However, several limitations were observed, and recommendations to support the application of the COSMIN tools are provided. What this paper adds What is already known on the subject Collecting outcome data is essential to ensure speech and language therapy is effective. Until the development of COSMIN there was a lack of standards in the way the measurement properties of outcome measure instruments were appraised. What this paper adds to existing knowledge This paper used the FOCUS, a measure of pre-schoolers' communicative participation outcomes in speech and language therapy, as a case example to illustrate the applications of the COSMIN tools. In doing so, the strengths and limitations of the current COSMIN tools in appraising the quality of outcome measure instruments are emphasized. What are the potential or actual clinical implications of this work? The COSMIN tools offer a step-by-step, standardized approach to appraise various measurement properties in outcome instruments. Due to existing limitations of the COSMIN tools, appraisal should provide clear and specific recommendations so users of outcome measures (e.g., SLTs, researchers) can identify the appropriate uses of each instrument.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.001 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.001 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".