Presenting evidence to patients online: what do web users think of consumer summaries of Cochrane musculoskeletal reviews?
Bibliographic record
Abstract
The Internet has the potential to be an effective medium for delivering health care knowledge to consumers. While computer usability research makes recommendations about how to present Web-based information generally, there remains no clear guidance on how to present specific forms of health care research evidence online in a way that facilitates understanding and good health care decision making. The two goals of this study were to describe the Cochrane Musculoskeletal Group's (CMSG's) process for developing online patient-focused summaries of systematic reviews and to evaluate the impressions of these summaries formed by users. A process for summarizing the results of systematic reviews via consumer summaries has evolved over 15 years. An evaluation of this approach took the form of Internet surveys on the Arthritis Society of Canada website and surveys of members of the Canadian Arthritis Patient Alliance (CAPA). Respondents provided information on background, relationship to the decision, their satisfaction with and preparation for decision making, and suggestions for improvements to the summaries. Survey data were collected between August 1, 2005, and February 28, 2006. A total of 261 respondents completed the survey. The majority (226/261 or 87%) of respondents reported having an arthritis-related condition. The consumer summary approach was generally reviewed favorably by respondents, with most agreeing that the summary provided appropriate information (177/261 or 68%), would be useful to others (160/261 or 61%), was well laid out (159/261 or 61%), was easy to learn from (157/261 or 60%), and was useful to the reader (153/261 or 59%). Areas of potential improvement were indicated by relatively fewer respondents agreeing that they could easily find all the information they wanted (118/261 or 45%), by a substantial proportion being unable to judge whether the providers of the information are reliable (80/261 or 31%), and by a similar proportion being unable to determine whether the information presented was the best available (68/261 or 26%). The CMSG has developed an approach to summarizing the results of often-technical systematic reviews into public-friendly consumer summaries. Our online survey showed that this approach was generally well liked but identified specific areas for improvement. Feedback from this survey will help to reshape and improve the current template for consumer summaries used by the CMSG.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.045 | 0.313 |
| Meta-epidemiology (narrow) | 0.000 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.004 | 0.004 |
| Science and technology studies | 0.002 | 0.003 |
| Scholarly communication | 0.011 | 0.013 |
| Open science | 0.001 | 0.004 |
| Research integrity | 0.005 | 0.005 |
| Insufficient payload (model declined to judge) | 0.004 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".