The development of lived experience-centered word clouds to support research uncertainty gathering in degenerative cervical myelopathy: results from an engagement process and protocol for their evaluation, via a nested randomized controlled trial
Bibliographic record
Abstract
OBJECTIVES: AO Spine REsearch objectives and Common Data Elements for Degenerative Cervical Myelopathy [RECODE-DCM] is a multi-stakeholder consensus process aiming to promote research efficiency in DCM. It aims to establish the top 10 research uncertainties, through a James Lind Alliance Priority Setting Partnership [PSP]. Through a consensus process, research questions are generated and ranked. The inclusion of people with cervical myelopathy [PwCM] is central to the process. We hypothesized that presenting PwCM experience through word cloud generation would stimulate other key stakeholders to generate research questions better aligned with PwCM needs. This protocol outlines our plans to evaluate this as a nested methodological study within our PSP. METHODS: An online poll asked PwCM to submit and vote on words associated with aspects of DCM. After review, a refined word list was re-polled for voting and word submission. Word clouds were generated and an implementation plan for AO Spine RECODE-DCM PSP surveys was subsequently developed. RESULTS: Seventy-nine terms were submitted after the first poll. Eighty-seven refined words were then re-polled (which added a further 39 words). Four word clouds were generated under the categories of diagnosis, management, long-term effects, and other. A 1:1 block randomization protocol to assess word cloud impact on the number and relevance of PSP research questions was generated. CONCLUSIONS: We have shown it is feasible to work with PwCM to generate a tool for the AO Spine RECODE-DCM nested methodological study. Once the survey stage is completed, we will be able to evaluate the impact of the word clouds. Further research will be needed to assess the value of any impact in terms of stimulating a more creative research agenda.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.165 | 0.297 |
| Meta-epidemiology (narrow) | 0.002 | 0.002 |
| Meta-epidemiology (broad) | 0.003 | 0.004 |
| Bibliometrics | 0.003 | 0.002 |
| Science and technology studies | 0.003 | 0.004 |
| Scholarly communication | 0.005 | 0.004 |
| Open science | 0.003 | 0.005 |
| Research integrity | 0.004 | 0.005 |
| Insufficient payload (model declined to judge) | 0.024 | 0.004 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".