The Dutch COVID-19 Contact Tracing App (the CoronaMelder): Usability Study
Bibliographic record
Abstract
BACKGROUND: Adoption and evaluation of contact tracing tools based on information and communications technology may expand the reach and efficacy of traditional contact tracing methods in fighting COVID-19. The Dutch Ministry of Health, Welfare and Sports initiated and developed CoronaMelder, a COVID-19 contact tracing app. This app is based on a Google/Apple Exposure Notification approach and aims to combat the spread of the coronavirus among individuals by notifying those who are at increased risk of infection due to proximity to someone who later tests positive for COVID-19. The app should support traditional contact tracing by faster tracing and greater reach compared to regular contact tracing procedures. OBJECTIVE: The main goal of this study is to investigate whether the CoronaMelder is able to support traditional contact tracing employed by public health authorities. To achieve this, usability tests were conducted to answer the following question: is the CoronaMelder user-friendly, understandable, reliable and credible, and inclusive? METHODS: Participants (N=44) of different backgrounds were recruited: youth with varying educational levels, youth with an intellectual disability, migrants, adults (aged 40-64 years), and older adults (aged >65 years) via convenience sampling in the region of Twente in the Netherlands. The app was evaluated with scenario-based, think-aloud usability tests and additional interviews. Findings were recorded via voice recordings, observation notes, and the Dutch User Experience Questionnaire, and some participants wore eye trackers to measure gaze behavior. RESULTS: Our results showed that the app is easy to use, although problems occurred with understandability and accessibility. Older adults and youth with a lower education level did not understand why or under what circumstances they would receive notifications, why they must share their key (ie, their assigned identifier), and what happens after sharing. In particular, youth in the lower-education category did not trust or understand Bluetooth signals, or comprehend timing and follow-up activities after a risk exposure notification. Older adults had difficulties multitasking (speaking with a public health worker and simultaneously sharing the key in the app). Public health authorities appeared to be unprepared to receive support from the app during traditional contact tracing because their telephone conversation protocol lacks guidance, explanation, and empathy. CONCLUSIONS: The study indicated that the CoronaMelder app is easy to use, but participants experienced misunderstandings about its functioning. The perceived lack of clarity led to misconceptions about the app, mostly regarding its usefulness and privacy-preserving mechanisms. Tailored and targeted communication through, for example, public campaigns or social media, is necessary to provide correct information about the app to residents in the Netherlands. Additionally, the app should be presented as part of the national coronavirus measures instead of as a stand-alone app offered to the public. Public health workers should be trained to effectively and empathetically instruct users on how to use the CoronaMelder app.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.008 | 0.020 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.001 | 0.000 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.000 | 0.001 |
| Research integrity | 0.001 | 0.000 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".