Validity of Using Mobile Phone Surveys to Evaluate Community Health Worker Program in Mali
Bibliographic record
Abstract
Abstract BackgroundAssessing implementation strength through face-to-face interviews in hard-to-reach and unstable regions presents many challenges. Mobile phone-based interviews have become an alternative, but the validity of these data from this approach for Implementation Strength Assessment evaluation has not been sufficiently studied yet. The objective of this study was to assess the validity of mobile phone-based health provider interviews to measure the implementation strength of an integrated community case management (iCCM) and family planning program in Mali.MethodsFrom July to August 2018, interviewers administered a structured questionnaire to community health workers (ASCs) implementing the iCCM and family planning program in six districts in Mali. Interviews were conducted, first by phone, then verified through in-person visits. Survey questions addressed background information, training, supervision, demand generation activities and supplies of essential drugs and contraceptive methods. Sensitivity and specificity of the phone responses were calculated using the in-person response as the gold standard. A threshold of 80% for sensitivity and specificity respectively was considered acceptable.ResultsOf 157 ASCs interviewed by phone, 115 (73.2%) were reached in-person. Most indicators (9/10 iCCM indicators, 6/6 family planning indicators) for training, supervision, and availability of supplies on the day of interview, and those related to patient utilization reached the 80% threshold for sensitivity, but few (2/10 iCCM indicators, 1/6 family planning indicators) reached 80% for specificity. In contrast, most indicators of supply stock-outs in the last 3 months reached the threshold for specificity (5/6 iCCM indicators, 3/3 family planning indicators) but few reached the threshold for sensitivity (1/6 iCCM indicators, 1/3 family planning indicators).ConclusionsThe validity of data collected by phone were adequate for indicators of training, supervision, and day-of-interview commodity availability. Phone-based surveys are useful as a low-cost option for data collection in the assessment of implementation strength on general activities in inaccessible and resource-limited regions with mobile network connectivity.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.045 | 0.001 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.001 | 0.002 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.001 | 0.000 |
| Open science | 0.005 | 0.019 |
| Research integrity | 0.000 | 0.005 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".