Strengths and challenges of the COSMIN tools in outcome measures appraisal: A case example for speech–language therapy
Bibliographic record
Abstract
BACKGROUND: The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) is an international initiative that offers standardized and validated tools to guide the appraisal of patient-reported outcome measures in healthcare. AIMS: To explore the use of a new set of tools from the COSMIN to appraise studies on one outcome measure available to speech and language therapists (SLTs). METHODS & PROCEDURES: We used the COSMIN tools to appraise seven studies and a user manual that reported the measurement properties of the Focus on the Outcomes of Children Under Six (FOCUS), a validated measure of pre-schoolers' communicative participation that is used in various contexts around the world. OUTCOMES & RESULTS: Using COSMIN guidelines, the FOCUS was categorized as a 'category A' tool because there was a sufficient level of evidence to support its content validity and internal consistency. According to the COSMIN guidelines, this means that the FOCUS can be recommended for use. The quality of evidence supporting measurement properties of the FOCUS received a rating of 'moderate', meaning users can have moderate confidence in its measurement properties. Since these ratings from the COSMIN tools may be unclear to users of the FOCUS, we have provided more specific recommendations. CONCLUSIONS & IMPLICATIONS: The COSMIN tools offer detailed standards to support the appraisal of outcome measures available to SLTs. However, several limitations were observed, and recommendations to support the application of the COSMIN tools are provided. What this paper adds What is already known on the subject Collecting outcome data is essential to ensure speech and language therapy is effective. Until the development of COSMIN there was a lack of standards in the way the measurement properties of outcome measure instruments were appraised. What this paper adds to existing knowledge This paper used the FOCUS, a measure of pre-schoolers' communicative participation outcomes in speech and language therapy, as a case example to illustrate the applications of the COSMIN tools. In doing so, the strengths and limitations of the current COSMIN tools in appraising the quality of outcome measure instruments are emphasized. What are the potential or actual clinical implications of this work? The COSMIN tools offer a step-by-step, standardized approach to appraise various measurement properties in outcome instruments. Due to existing limitations of the COSMIN tools, appraisal should provide clear and specific recommendations so users of outcome measures (e.g., SLTs, researchers) can identify the appropriate uses of each instrument.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.630 | 0.706 |
| Meta-epidemiology (narrow) | 0.003 | 0.002 |
| Meta-epidemiology (broad) | 0.004 | 0.005 |
| Bibliometrics | 0.011 | 0.010 |
| Science and technology studies | 0.008 | 0.019 |
| Scholarly communication | 0.022 | 0.026 |
| Open science | 0.007 | 0.016 |
| Research integrity | 0.012 | 0.015 |
| Insufficient payload (model declined to judge) | 0.003 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".