Adaptive Sensor-Image Fusion for Enhanced 3D Pose Estimation in Cardiac Intervention
Bibliographic record
Abstract
Precise tracking of surgical instruments and anatomical structures is vital for minimally invasive cardiac surgery to perform safe and effective interventions. However, conventional methods that rely on frames for pose estimation are vulnerable to lighting fluctuations, occlusions, and motion blur, the challenges of real surgical environments. To mitigate these challenges, we propose a multi-modal pose estimation architecture allowing us to fuse image data, which can be processed using models such as Swin Transformer, along with position and orientation sensor data via a gated fusion, as well as a layered multi-head attention module. By adaptively addressing visual and sensor-based spatial details, the framework not only contributes to improved robustness against noise and uncertainty but also pose estimation accuracy. We extensively evaluate the proposed method against image-only and sensoronly baselines and demonstrate its superior performance in estimating both position and orientation. The gated fusion mechanism and multi-head attention structure improve the robustness against missing data and sensor noise, thereby enhancing prediction reliability. Position estimation has very close to perfect accuracy in the R<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> > 0.9997 range along all spatial axes. We see that orientation prediction also shows strong linear correlations, where the R<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> values exceed 0.9975 for all components of the quaternion, which confirms the model’s capability of capturing the rotational behavior. The implications of these results are significant, as the proposed framework offers a robust and accurate approach to surgical pose estimation, enabling more reliable cardiac surgical navigation.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".