Revisiting the Problem of Optic Nerve Detection in a Retinal Image Using Automated Machine Learning

To the Editor: We read with great interest the recent review article entitled “Big Data in Ophthalmology” by Cheng et al.1 The authors discuss characteristics of big data and their potential applications in ophthalmology including their use in artificial intelligence (AI) and deep learning design. However, big data are not always available and small sample sizes are inevitable, particularly in the study of rare diseases. Various misconceptions have caused clinicians to believe that AI cannot be applied to their smaller health care and imaging data (small data). In this report, we hope to demonstrate to your readership that ophthalmologists can apply deep learning models on small data without the need for large datasets or coding. Over the past 2 decades, computer-aided diagnosis (CAD) of retinal diseases has emerged as a powerful tool for disease detection. The initial step toward automatic identification of retinal pathology has traditionally been to locate the optic nerve, macula, and vascular arcades through handcrafted object segmentation.2 Feature-extraction systems were then used to detect pathologic features like exudates and haemorrhage by examining their geometric and color properties. The emergence of deep learning has revolutionized machine learning (ML) approaches for retinal diseases and has obviated the need for hand-engineering domain-specific features. Today, deep learning has been broadly applied to imaging data in ophthalmology and has also gained significant clinical traction. Until recently, the elaboration of ML models had been reserved to AI experts with coding skills. However, major advances in the democratization of AI were made possible by the release of automated ML (AutoML) platforms by leading technology companies. Using transfer learning and neural architecture search, AutoML allows users with small data to achieve state-of-the-art results by building on pretrained models that have been trained on big data. AutoML model design is carried out through a graphical interface that does not require coding.3 The feasibility of AutoML design for imaging and electronic health record data in ophthalmology has already been established.3,4 In this brief report, we demonstrate the design of an AutoML model to automatically locate the optic nerve in a retinal image without writing a single line of code. This proof-of-concept experiment that revisits a classic image analysis problem is a testament to the incredible advances that have been made in the field of AI since the early CAD systems. An ophthalmologist without coding experience (FA) carried out AutoML model design using images from the public database of the STARE Project (https://cecas.clemson.edu/∼ahoover/stare/). We identified 313 good-quality images depicting normal and pathologic fundi and uploaded them to the Google Cloud AutoML Vision Object platform for training (202 images), validation (30 images), and testing (81 images). The ophthalmologist annotated each image by generating bounding boxes using his computer mouse to surround the optic nerve (Supplemenatry Figure 1, https://links.lww.com/APJO/A90). We tested the AutoML model on the 81 images used by a landmark report from the STARE project that studied optic nerve detection using a traditional CAD algorithm called fuzzy convergence.5 The AutoML model had excellent diagnostic properties. The area under the precision-recall curve (AUPRC) was 0.983 with high precision (96.20%) and recall (93.83%). Those metrics are calculated by comparing the amount of overlap between a predicted bounding box (by AutoML) compared to the bounding box set by the ophthalmologist (ground truth) (Supplemenatry Figure 2, https://links.lww.com/APJO/A91). Accurate bounding boxes were more difficult to predict in cases of significant peripapillary atrophy and edema obscuring the margins of the optic disc. When we used the outcome described in the Hoover and Goldbaum study,5 all optic nerves (100%) were considered to have been successfully detected (vs 89% in their study). By revisiting this historic experiment and demonstrating improved performance without requiring skilled AI experts or a computationally expensive solution, we hope to have demonstrated to your readership the promise of AutoML. AutoML has the potential to be an innovative tool for the deployment of AI solutions on small data. By democratizing access to those solutions, valuable insights into rare retinal diseases can be gained. More work is needed to explain the fundamental mechanisms guiding these models’ predictions before the deployment of AutoML models.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.