Deep Learning-based Detection and Tracking of Capsule Robots using Ultrasound Feedback in Gastrointestinal Tract

Ingestible robotic capsules with locomotion capabilities and on‑board sampling mechanism have great potential for non‑invasive diagnostic and interventional use in the gastrointestinal tract. Real‑time tracking of capsule location and operational state is necessary for clinical application yet remains a significant challenge. To this end, this thesis investigates ultrasound-based detection and 3D tracking of capsule robots in the gastrointestinal tract by leveraging deep learning-based methods. As a pivotal step, an attention-based hierarchical deep learning approach is introduced. This method demonstrates the ability to simultaneously determine the mechanism state (i.e., the capsule is closed, open or lost) and in‑plane 2D pose of millimeter a capsule robot in ex-vivo tissue environments using ultrasound imaging. To train the neural networks, a representative dataset of the robotic capsule within ex‑vivo porcine stomachs is generated. The trained models achieve an accuracy of 96.9% for state classification and the mean estimation errors of 3.5 degrees and 1.76 mm for orientation and centroid position in ex-vivo experiments in porcine stomachs. However, conventional ultrasound B-mode imaging suffers from limited field of view, inability to image objects out of the scanning plane, and issues with low device visibility in echogenic in-vivo GI tract environments filled with bowel gas. To address these limitations and facilitate 3D tracking of capsule robots in tissue environments with intraluminal gas, an automatic robotic ultrasound tracking system for long-distance 3D tracking of a capsule robot in the GI tract is developed, with the ability to actively search for the lost capsule due to out-of-plane or out of the field of view motions. A hybrid deep learning model is proposed by combining transformer and convolutional neural networks for detecting and localizing the capsule robot. The attention mechanism built into the transformer enables the efficient capture of long-range capsule motions within the US image. The proposed system demonstrates continuous capsule tracking over 90 cm with a mean state detection accuracy of 90% and centroid localization accuracy of 1.5 mm across varying imaging parameters and artefact patterns.

Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.