Performance of a Retinal Imaging Camera With On-Device Intelligence for Primary Care: Retrospective Study
Bibliographic record
Abstract
Background: Access to screening continues to be a barrier for the early detection of diabetic retinopathy (DR). Primary care-based diabetic retinopathy screening could improve access, but operational challenges, such as cost and workflow management, hamper the widespread adoption of retinal camera systems in primary care clinics in the United States. Objective: This study aimed to develop and evaluate a retinal screening system suitable for integration into a primary care workflow. Methods: We developed a nonmydriatic, 45° field imaging retinal camera system, the Verily Numetric Retinal Camera (VNRC; Verily Life Sciences LLC), able to generate high-fidelity retinal images enabled by on-device intelligent features. The VNRC output flows into cloud-based software that accepts and routes digitized images for grading. We evaluated the performance and usability of the VNRC in 2 studies. A retrospective performance study compared the performance of VNRC against a reference camera (Crystalvue NFC-700 [Crystalvue Medical]) as well as the correlation between VNRC capture status and gradability (as determined by ophthalmologist graders). The usability study simulated a primary care setting for a combined cohort of trained and untrained users (corresponding to patients in the simulation) and operators (corresponding to health care personnel in the simulation), where respondents completed a questionnaire about their user experience after attempting to capture images with the VNRC. Results: In the comparative performance study (N=108, K=206 images), a total of 98.5% (203/206) of images captured by the VNRC were graded as sufficient for clinical interpretation compared to 97.1% (200/206) of Crystalvue NFC-700 images (difference in proportion was 0.015, 95% CI -0.007 to 0.033). In the quality control algorithm evaluation (N=172, K=343 images), we found a positive association (φ=0.58, 95% CI 0.45-0.69) between gradability status (gradable or nongradable) as determined by ophthalmologists and the capture status (recapture not-needed or needed) as determined by the VNRC quality control algorithm. In the usability study (n=15 users and n=30 operators), all participating users (15/15) indicated that they were able to have both eyes screened easily. Most users and operators indicated agreement (from somewhat agree to strongly agree) with statements describing the imaging process as intuitive (15/15, 100% and 29/30, 97%), comfortable (15/15, 100% and 30/30, 100%), and allowing for a positive experience (15/15, 100% and 30/30, 100%), of users and operators, respectively. Conclusions: Our findings about the performance and usability of this retinal camera system support its deployment as an integrated end-to-end retinal service for primary care. These results warrant additional studies to fully characterize real-world usability across a wider and diverse set of primary care clinics.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.004 | 0.018 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.001 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.000 |
| Insufficient payload (model declined to judge) | 0.002 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".