Can clinical vignettes administered by mobile phone call be used to measure quality of contraceptive care practices in Malawi?
Bibliographic record
Abstract
Background Accurate and timely measures of quality of contraceptive care are required for program improvement but can be expensive and time-consuming to implement in low- and middle-income countries (LMICs). Clinical vignettes (CLVs) measure quality knowledge and have been shown to be a valid proxy of provider quality practice in the United States, and may be administered via mobile-phone, making them an attractive cost-savings alternative to field assessments. This cross-sectional study aims to validate mobile-phone CLVs for measuring quality of contraceptive service provision using simulated, “mystery” clients as the gold standard. Methods Researchers trained to simulate clients (SC) seeking contraceptive care visited 112 government-administered clinics in six districts of Malawi. The providers who saw the simulated clients were called via mobile phone approximately three weeks later for CLV administration. We calculated sensitivity, specificity, and area under the ROC receiver operating characteristic curve (AUC) and compared the proportion of quality behaviors measured from both methods. Results Overall, the quality measured by CLVs differed from quality measured with SCs. A higher proportion of providers reported clinical assessments and counseling on more topics during the CLV compared to the SCs, including counseling on what to do if the pill was not taken on time (59% vs. 26%; P<0.01). During the SC, providers asked more questions when taking client history and recommended the preferred method of the case scenario more often (81% vs. 49%; P<0.01). Conclusions The quality of care ascertained through mobile-phone CLV was not similar enough to quality ascertained through SCs to replace them. Mobile phone-based CLV also failed to capture all areas of quality performance. Continued efforts are needed to identify and test innovative methods to accurately and inexpensively measure quality of care to inform improvement of provider performance.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.011 | 0.082 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.002 | 0.002 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.002 | 0.002 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".