When piloting health services interventions, what predicts real world behaviours? A systematic concept mapping review
Bibliographic record
Abstract
BACKGROUND: Modeling studies to inform the design of complex health services interventions often involves elements that differ from the intervention's ultimate real-world use. These "hypothetical" elements include pilot participants, materials, and settings. Understanding the conditions under which studies with "hypothetical" elements can yield valid results would greatly help advance health services research. Our objectives are: 1) to conduct a systematic review of the literature to identify factors affecting the relationship between hypothetical decisions and real-world behaviours, and 2) to summarise and organize these factors into a preliminary framework. METHODS: We conducted an electronic database search using PsycINFO and Medline on November 30th, 2015, updated March 7th, 2019. We also conducted a supplemental snowball search on December 9th 2015 and a reverse citation search using Scopus and Web of Science. Studies were eligible to be included in this review if they clearly addressed the consistency between some type of hypothetical decision and a corresponding real decision or behaviour. Two reviewers extracted data using a standardized data collection form developed through an iterative consensus-based process. We extracted basic study information and data about each study's research area, design, and research question. Quotations from the articles were extracted and summarized into standardized factor statements. RESULTS: Of the 2444 articles that were screened, 68 articles were included in the review. The articles identified 27 factors that we grouped into 4 categories: decision maker factors, cognitive factors, task factors, and matching factors. CONCLUSIONS: We have summarized a large number of factors that may be relevant when considering whether hypothetical health services pilot work can be expected to yield results that are consistent with real-world behaviours. Our descriptive framework can serve as the basis for organizing future work exploring which factors are most relevant when seeking to develop complex health services interventions.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.257 | 0.187 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.009 | 0.001 |
| Bibliometrics | 0.002 | 0.004 |
| Science and technology studies | 0.002 | 0.001 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.004 | 0.003 |
| Research integrity | 0.001 | 0.008 |
| Insufficient payload (model declined to judge) | 0.007 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; both teacher heads agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".