The Good, Bad, and Ugly of Online Recruitment of Parents for Health-Related Focus Groups: Lessons Learned
Bibliographic record
Abstract
BACKGROUND: We describe our experiences with identifying and recruiting Ontario parents through the Internet, primarily, as well as other modes, for participation in focus groups about adding the influenza vaccine to school-based immunization programs. OBJECTIVE: Our objectives were to assess participation rates with and without incentives and software restrictions. We also plan to examine study response patterns of unique and multiple submissions and assess efficiency of each online advertising mode. METHODS: We used social media, deal forum websites, online classified ads, conventional mass media, and email lists to invite parents of school-aged children from Ontario, Canada to complete an online questionnaire to determine eligibility for focus groups. We compared responses and paradata when an incentive was provided and there were no software restrictions to the questionnaire (Period 1) to a period when only a single submission per Internet protocol (IP) address (ie, software restrictions invoked) was permitted and no incentive was provided (Period 2). We also compared the median time to complete a questionnaire, response patterns, and percentage of missing data between questionnaires classified as multiple submissions from the same Internet protocol (IP) address or email versus unique submissions. Efficiency was calculated as the total number of hours study personnel devoted to an advertising mode divided by the resultant number of unique eligible completed questionnaires . RESULTS: Of 1346 submitted questionnaires, 223 (16.6%) were incomplete and 34 (2.52%) did not meet the initial eligibility criteria. Of the remaining 1089 questionnaires, 246 (22.6%) were not from Ontario based on IP address and postal code, and 469 (43.1%) were submitted from the same IP address or email address (multiple submissions). In Period 2 vs Period 1, a larger proportion of questionnaires were submitted from Ontario (92.8%, 141/152 vs 75.1%, 702/937, P<.001), and a smaller proportion of same IP addresses (7.9%, 12/152 vs 47.1%, 441/937, P<.001) were received. Compared to those who made unique submissions, those who made multiple submissions spent less time per questionnaire (166 vs 215 seconds, P<.001), and had a higher percentage of missing data among their responses (15.0% vs 7.6%, P=.004). Advertisements posted on RedFlagDeals were the most efficient for recruitment (0.03 hours of staff time per questionnaire), whereas those placed on Twitter were the least efficient (3.64 hours of staff time per questionnaire). CONCLUSIONS: Using multiple online advertising strategies was effective for recruiting a large sample of participants in a relatively short period time with minimal resources. However, risks such as multiple submissions and potentially fraudulent information need to be considered. In our study, these problems were associated with providing an incentive for responding, and could have been partially avoided by activating restrictive software features for online questionnaires.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.184 | 0.248 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.002 | 0.001 |
| Science and technology studies | 0.005 | 0.007 |
| Scholarly communication | 0.007 | 0.011 |
| Open science | 0.007 | 0.006 |
| Research integrity | 0.004 | 0.005 |
| Insufficient payload (model declined to judge) | 0.006 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".