Lived experience-centred word clouds may improve research uncertainty gathering in priority setting partnerships
Bibliographic record
Abstract
Abstract Introduction AO Spine RECODE-DCM was a multi-stakeholder priority setting partnership (PSP) to define the top ten research priorities for degenerative cervical myelopathy (DCM). Priorities were generated and iteratively refined using a series of surveys administered to surgeons, other healthcare professionals (oHCP) and people with DCM (PwDCM). The aim of this work was to utilise word clouds to enable the perspectives of people with the condition to be heard earlier in the PSP process than is traditionally the case. The objective was to evaluate the added value of word clouds in the process of defining research uncertainties in National Institute for Health Research (NIHR) James Lind Alliance (JLA) Priority Setting Partnerships. Methods Patient-generated word clouds were created for the four survey subsections of the AO Spine RECODE-DCM PSP: diagnosis, treatment, long-term management and other issues. These were then evaluated as a nested methodological study. Word-clouds were created and iteratively refined by an online support group of people with DCM, before being curated by the RECODE-DCM management committee and expert healthcare professional representatives. The final word clouds were embedded within the surveys administered at random to 50% of participants. DCM research uncertainties suggested by participants were compared pre- and post-word cloud presentation. Results A total of 215 (50.9%) participants were randomised to the word cloud stream, including 118 (55%) spinal surgeons, 52 (24%) PwDCM and 45 (21%) oHCP. Participants submitted 434 additional uncertainties after word cloud review: word count was lower and more uniform across each survey subsections compared to pre-word cloud uncertainties. Twenty-three (32%) of the final 74 PSP summary questions did not have a post-word cloud contribution and no summary question was formed exclusively on post-word cloud uncertainties. There were differences in mapping of pre- and post-word cloud uncertainties to summary questions, with greater mapping of post-word cloud uncertainties to the number 1 research question priority: raising awareness. Five of the final summary questions were more likely to map to the research uncertainties suggested by participants after having reviewed the word clouds. Conclusions Word clouds may increase the perspective of underrepresented stakeholders in the research question gathering stage of priority setting partnerships. This may help steer the process towards research questions that are of highest priority for people with the condition.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.174 | 0.381 |
| Meta-epidemiology (narrow) | 0.002 | 0.002 |
| Meta-epidemiology (broad) | 0.002 | 0.003 |
| Bibliometrics | 0.007 | 0.006 |
| Science and technology studies | 0.006 | 0.008 |
| Scholarly communication | 0.016 | 0.018 |
| Open science | 0.004 | 0.028 |
| Research integrity | 0.003 | 0.003 |
| Insufficient payload (model declined to judge) | 0.024 | 0.003 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".