The impact of panel composition and topic on stakeholder perspectives: Generating hypotheses from online maternal and child health modified‐Delphi panels
Bibliographic record
Abstract
INTRODUCTION: Multistakeholder engagement is crucial for conducting health services research. Delphi-based methodologies combining iterative rounds of questions with feedback on and discussion of group results are a well-documented approach to multistakeholder engagement. This study develops hypotheses about the impact of panel composition and topic on the propensity and meaningfulness of response changes in multistakeholder modified-Delphi panels. METHODS: We conducted three online modified-Delphi (OMD) multistakeholder panels using the same protocol. We assigned 60 maternal and child health professionals to a homogeneous (professionals only) panel, 60 pregnant or postpartum women (patients) to a homogeneous panel, and 30 professionals and 30 patients to a mixed panel. In Round 1, participants rated the seriousness of 11 maternal and child health outcomes using a 0-100 scale and explained their ratings. In Round 2, participants saw their own and their panel's Round 1 results and discussed them using asynchronous, anonymous discussion boards moderated by the study investigators. In Round 3, participants revised their original ratings. Our outcome measures included binary indicators of response changes to ratings of the low, medium and high severity maternal and child health outcomes and their meaningfulness, measured by a change of 10 or more points. RESULTS: Participants changed 818 of 1491 (55%) of responses; the majority of response changes were meaningful. Patterns of response changes were different for patients and professionals and for different levels of outcome seriousness. Using study results and the literature, we developed three hypotheses. First, OMD participants, regardless of their stakeholder group, are more likely to change their responses on preference-sensitive topics where there is a range of viable alternatives or perspectives. Second, patients are more likely to change their responses and to do so meaningfully in mixed panels, whereas professionals are more likely to do so in homogeneous panels. Third, the association between panel composition and response change varies according to the topic (e.g., the level of outcome seriousness). CONCLUSIONS: Results of our work not only helped generate empirically derived hypotheses to be tested in future research but also offer practical recommendations for designing multistakeholder OMD panels. PATIENT OR PUBLIC CONTRIBUTION: Pregnant or postpartum women were involved in this study.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.301 | 0.509 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.003 |
| Bibliometrics | 0.003 | 0.002 |
| Science and technology studies | 0.003 | 0.008 |
| Scholarly communication | 0.005 | 0.006 |
| Open science | 0.002 | 0.009 |
| Research integrity | 0.002 | 0.002 |
| Insufficient payload (model declined to judge) | 0.005 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".