Can clinical vignettes administered by mobile phone call be used to measure quality of contraceptive care practices in Malawi?
Bibliographic record
Abstract
Background Accurate and timely measures of quality of contraceptive care are required for program improvement but can be expensive and time-consuming to implement in low- and middle-income countries (LMICs). Clinical vignettes (CLVs) measure quality knowledge and have been shown to be a valid proxy of provider quality practice in the United States, and may be administered via mobile-phone, making them an attractive cost-savings alternative to field assessments. This cross-sectional study aims to validate mobile-phone CLVs for measuring quality of contraceptive service provision using simulated, “mystery” clients as the gold standard. Methods Researchers trained to simulate clients (SC) seeking contraceptive care visited 112 government-administered clinics in six districts of Malawi. The providers who saw the simulated clients were called via mobile phone approximately three weeks later for CLV administration. We calculated sensitivity, specificity, and area under the ROC receiver operating characteristic curve (AUC) and compared the proportion of quality behaviors measured from both methods. Results Overall, the quality measured by CLVs differed from quality measured with SCs. A higher proportion of providers reported clinical assessments and counseling on more topics during the CLV compared to the SCs, including counseling on what to do if the pill was not taken on time (59% vs. 26%; P <0.01). During the SC, providers asked more questions when taking client history and recommended the preferred method of the case scenario more often (81% vs. 49%; P <0.01). Conclusions The quality of care ascertained through mobile-phone CLV was not similar enough to quality ascertained through SCs to replace them. Mobile phone-based CLV also failed to capture all areas of quality performance. Continued efforts are needed to identify and test innovative methods to accurately and inexpensively measure quality of care to inform improvement of provider performance.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.003 | 0.002 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".