SpatialGPT: Zero-Shot Vision-and-Language Navigation via Spatial CoT over Structured Spatial Memory
Bibliographic record
Abstract
Vision-and-Language Navigation (VLN) is a challenging multimodal task in which an autonomous agent must navigate unknown environments by following natural language instructions. Recent zero-shot VLN approaches leverage Large Language Models (LLMs), such as GPT, to interpret instructions and visual inputs for navigation inference without environment-specific training. However, these methods rely solely on the inherent spatial reasoning abilities of LLMs, which often fail to align panoramic observations with language instructions in zero-shot settings. To address this limitation, we propose SpatialGPT, a novel GPT-based VLN agent that incorporates spatial domain knowledge and the Chain-of-Thought (CoT) paradigm to enhance spatial reasoning. SpatialGPT integrates a Directional Connected Landmark List and a Spatial Knowledge Graph to jointly model local and global visual context as structured spatial memory. Built on this memory, we introduce a Synchronize-Align-Backtrack reasoning chain that synchronizes with instruction progress, aligns panoramic views to determine the next action, retrieves alternative paths or infers new frontiers during backtracking. Extensive experiments on the Room-to-Room (R2R) benchmark demonstrate that SpatialGPT achieves state-of-the-art zero-shot performance across all evaluation metrics, showcasing its enhanced spatial reasoning capabilities and strong generalization as an LLM-based VLN agent. The source code is available at SpatialGPT (GitHub)1.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.002 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.001 | 0.003 |
| Open science | 0.003 | 0.003 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.007 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".