Redesigning a Web-Based Stakeholder Consensus Meeting About Core Outcomes for Clinical Trials: Formative Feedback Study
Bibliographic record
Abstract
BACKGROUND: Clinical trials that assess the benefits and harms of an intervention do so by measuring and reporting outcomes. Inconsistent selection and diversity in the choice of outcomes make it challenging to directly compare interventions. To achieve an agreed core set of outcomes, a consensus methodology is recommended, comprising a web-based Delphi survey and a face-to-face consensus meeting. However, UK government regulations to control the pandemic prohibited plans for a face-to-face consensus meeting as part of the Core Rehabilitation Outcome Set for Single-Sided Deafness (CROSSSD) study. OBJECTIVE: This study aims to evaluate the modifications made by the CROSSSD study team to achieve consensus using web-based methods, but with minimal deviation from the original study protocol. METHODS: The study team worked with health care users and professionals to translate the planned face-to-face consensus meeting in a web-based format, preserving the key elements of the nominal group technique. A follow-up survey gathered evaluation feedback on the experiences of the 22 participating members. Feedback covered premeeting preparation, the process of facilitated discussions and voting, ability to contribute, and perceived fairness of the outcome. RESULTS: Overall, 98% (53/54) of feedback responses agreed or strongly agreed with the statements given, indicating that the web-based meeting achieved its original goals of open discussion, debate, and voting to agree with a core outcome set for single-sided deafness. Hearing-impaired participants were fully engaged, but there were some methodological challenges. For the participants, challenges included building rapport, understanding, and delivering the tasks in hand. For the study team, challenges included the need for thorough preparation and management of the unpredictability of tasks on the day. CONCLUSIONS: Sharing our experiences and lessons learned can benefit future core outcome set developers. Overcoming the challenges of delivering a web-based consensus exercise in the face of the pandemic can be applied more generally to maximize inclusiveness, enhance geographical access, and reduce research costs.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.713 | 0.829 |
| Meta-epidemiology (narrow) | 0.003 | 0.002 |
| Meta-epidemiology (broad) | 0.003 | 0.005 |
| Bibliometrics | 0.007 | 0.005 |
| Science and technology studies | 0.003 | 0.004 |
| Scholarly communication | 0.008 | 0.010 |
| Open science | 0.006 | 0.010 |
| Research integrity | 0.005 | 0.008 |
| Insufficient payload (model declined to judge) | 0.007 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".