Applications of Computer Vision in Underwater Ecology: A Case Study from the Northeast Pacific
Bibliographic record
Abstract
The world's oceans are undergoing rapid changes due to climate change and other anthropogenic impacts, affecting marine species' distribution, abundance, and behavior. Traditional ecological monitoring methods struggle to keep pace with these transformations, especially in underwater habitats. Recently, computer vision techniques have emerged to enhance the efficiency of video and image-based underwater monitoring. These methods facilitate the detection and classification of objects in visual data, potentially streamlining the process of counting and classifying marine organisms. However, the adoption of computer vision in marine ecology has been slow, partly due to its inaccessibility to ecologists and the lack of easily adaptable tools for ecological monitoring.This study investigates the application and validation of computer vision techniques for monitoring underwater pelagic macrofaunal diversity, using a case study from coastal British Columbia. Over 9000 hours of underwater video were collected from four sites over 18 months, using mounted cameras programmed to record five minutes of video every hour. Due to the infrequency of organisms present in the videos and challenges with underwater visibility, we created a stepwise iterative screening process to sequentially refine video data and aid the image annotation process. This involved using unsupervised classifiers (e.g. ResNet-18) to assist in reducing the number of background (i.e. 'empty water') images shown to annotators. To address the scarcity of annotations for certain taxa, an ‘adjacency filter’ was employed to increase the number of annotated frames for rare species. A rigorous QAQC process ensured standardization and minimized inter-annotator bias. Finally, a supervised computer vision model (YOLOv8) was trained with approximately 240,000 annotations to assess the presence and abundance of marine species over 18 months in the area. This approach provided high-resolution temporal data on the diversity and abundance of pelagic fish and gelatinous zooplankton at these sites.Here, we detail the process of employing these computer vision techniques for long-term underwater ecological monitoring, emphasizing accessibility for ecologists. A stepwise method for adapting computer vision techniques to achieve biodiversity monitoring objectives is presented, highlighting the strengths and limitations of our approach. We address our work in the context of the key barriers facing computer vision in underwater ecology, and provide tools for researchers seeking to incorporate AI in image or video-based marine research. Finally, we propose future directions for integrating these technologies into new and existing monitoring programs, and suggest priority areas for future research to advance the use of computer vision in underwater ecological monitoring.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.001 | 0.004 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".