Best Practices and Recommendations for Research Using Virtual Real-Time Data Collection: Protocol for Virtual Data Collection Studies
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
BACKGROUND: The COVID-19 pandemic and the subsequent need for social distancing required the immediate pivoting of research modalities. Research that had previously been conducted in person had to pivot to remote data collection. Researchers had to develop data collection protocols that could be conducted remotely with limited or no evidence to guide the process. Therefore, the use of web-based platforms to conduct real-time research visits surged despite the lack of evidence backing these novel approaches. OBJECTIVE: This paper aims to review the remote or virtual research protocols that have been used in the past 10 years, gather existing best practices, and propose recommendations for continuing to use virtual real-time methods when appropriate. METHODS: Articles (n=22) published from 2013 to June 2023 were reviewed and analyzed to understand how researchers conducted virtual research that implemented real-time protocols. "Real-time" was defined as data collection with a participant through a live medium where a participant and research staff could talk to each other back and forth in the moment. We excluded studies for the following reasons: (1) studies that collected participant or patient measures for the sole purpose of engaging in a clinical encounter; (2) studies that solely conducted qualitative interview data collection; (3) studies that conducted virtual data collection such as surveys or self-report measures that had no interaction with research staff; (4) studies that described research interventions but did not involve the collection of data through a web-based platform; (5) studies that were reviews or not original research; (6) studies that described research protocols and did not include actual data collection; and (7) studies that did not collect data in real time, focused on telehealth or telemedicine, and were exclusively intended for medical and not research purposes. RESULTS: Findings from studies conducted both before and during the COVID-19 pandemic suggest that many types of data can be collected virtually in real time. Results and best practice recommendations from the current protocol review will be used in the design and implementation of a substudy to provide more evidence for virtual real-time data collection over the next year. CONCLUSIONS: Our findings suggest that virtual real-time visits are doable across a range of participant populations and can answer a range of research questions. Recommended best practices for virtual real-time data collection include (1) providing adequate equipment for real-time data collection, (2) creating protocols and materials for research staff to facilitate or guide participants through data collection, (3) piloting data collection, (4) iteratively accepting feedback, and (5) providing instructions in multiple forms. The implementation of these best practices and recommendations for future research are further discussed in the paper. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/53790.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.069 | 0.028 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.001 | 0.003 |
| Science and technology studies | 0.007 | 0.002 |
| Scholarly communication | 0.002 | 0.003 |
| Open science | 0.002 | 0.002 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it