Reinforcement Learning-Based Controller With NMPC-Assisted Training for Autonomous Surface Vessels
Bibliographic record
Abstract
Abstract For autonomous surface vessels, conventional model-based control strategies face challenges due to the inherent uncertainties and inaccuracies associated with the model. This study investigates the application of reinforcement learning (RL) controllers, specifically using the Deep Deterministic Policy Gradient (DDPG) algorithm, to reduce dependency on model accuracy by enabling controllers to learn through experience. To further enhance RL controller performance, this work integrates Nonlinear Model Predictive Control (NMPC) for assisted training. The methodology involves developing a DDPG-based RL controller for the approximated test vessel’s dynamics and training the RL controller for point tracking. Observations reveal that, while the RL controller successfully learns the control actions required for point tracking, the generated actions lack the smoothness necessary for real-world applications. To address this, NMPC is introduced to assist the RL controller in optimizing control actions during training, yielding results that are more suitable for practical implementation. The findings demonstrate that NMPC assisted DDPG controller can effectively learn vessel dynamics through iterative training, reducing the reliance on precise vessel models. Additionally, the integration of NMPC enhances control action quality, and it has been validated with performance indicators via iterative simulations data. The control variance measuring indicators showed a reduction of an average 60% compared to conventional DDPG controller and the tracking accuracy increase compared to NMPC by 58% on average. This work paves the way for a potential combination of established controllers with RL controller in order to enhance accuracy and performance.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.002 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".