Measuring communication as a core outcome in aphasia trials: Results of the ROMA‐2 international core outcome set development meeting
Bibliographic record
Abstract
BACKGROUND: Evidence-based recommendations for a core outcome set (COS; minimum set of outcomes) for aphasia treatment research have been developed (the Research Outcome Measurement in Aphasia-ROMA, COS). Five recommended core outcome constructs: communication, language, quality of life, emotional well-being and patient-reported satisfaction/impact of treatment, were identified through three international consensus studies. Constructs were paired with outcome measurement instruments (OMIs) during an international consensus meeting (ROMA-1). Before the current study (ROMA-2), agreement had not been reached on OMIs for the constructs of communication or patient-reported satisfaction/impact of treatment. AIM: To establish consensus on a communication OMI for inclusion in the ROMA COS. METHODS & PROCEDURES: Research methods were based on recommendations from the Core Outcome Measures in Effectiveness Trials (COMET) Initiative. Participants with expertise in design and conduct of aphasia trials, measurement instrument development/testing and/or communication outcome measurement were recruited through an open call. Before the consensus meeting, participants agreed on a definition of communication, identified appropriate OMIs, extracted their measurement properties and established criteria for their quality assessment. During the consensus meeting they short-listed OMIs and participants without conflicts of interest voted on the two most highly ranked instruments. Consensus was defined a priori as agreement by ≥ 70% of participants. OUTCOMES & RESULTS: In total, 40 researchers from nine countries participated in ROMA-2 (including four facilitators and three-panel members who participated in pre-meeting activities only). A total of 20 OMIs were identified and evaluated. Eight short-listed communication measures were further evaluated for their measurement properties and ranked. Participants in the consensus meeting (n = 33) who did not have conflicts of interest (n = 29) voted on the top two ranked OMIs: The Scenario Test (TST) and the Communication Activities of Daily Living-3 (CADL-3). TST received 72% (n = 21) of 'yes' votes and the CADL-3 received 28% (n = 8) of 'yes' votes. CONCLUSIONS & IMPLICATIONS: Consensus was achieved that TST was the preferred communication OMI for inclusion in the ROMA COS. It is currently available in the original Dutch version and has been adapted into English, German and Greek. Further consideration must be given to the best way to measure communication in people with mild aphasia. Development of a patient-reported measure for satisfaction with/impact of treatment and multilingual versions of all OMIs of the COS is still required. Implementation of the ROMA COS would improve research outcome measurement and the quality, relevance, transparency, replicability and efficiency of aphasia treatment research. WHAT THIS PAPER ADDS: What is already known on this subject International consensus has been reached on five core constructs to be routinely measured in aphasia treatment studies. International consensus has also been established for OMIs for the three constructs of language, quality of life and emotional well-being. Before this study, OMIs for the constructs of communication and patient-reported satisfaction/impact of treatment were not established. What this paper adds to existing knowledge We gained international consensus on an OMI for the construct of communication. TST is recommended for inclusion in the ROMA COS for routine use in aphasia treatment research. What are the potential or actual clinical implications of this work? The ROMA COS recommends OMIs for a minimum set of outcomes for adults with post-stroke aphasia within phases I-IV aphasia treatment research. Although not intended for clinical use, clinicians may employ the instruments of the ROMA COS, considering the quality of their measurement properties. The systematic inclusion of a measure of communication, such as TST, in clinical practice could ultimately support the implementation of research evidence and best practices.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.439 | 0.494 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.008 | 0.015 |
| Bibliometrics | 0.004 | 0.005 |
| Science and technology studies | 0.001 | 0.002 |
| Scholarly communication | 0.007 | 0.004 |
| Open science | 0.003 | 0.007 |
| Research integrity | 0.004 | 0.005 |
| Insufficient payload (model declined to judge) | 0.004 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".