Soft Actor-Critic with Inhibitory Networks for Retraining UAV Controllers Faster
Bibliographic record
Abstract
Active research in autonomous unmanned aerial vehicles (UAVs) seeks to combine the agility of Proportional-Integral-Derivative (PID) systems for low-level control with the adaptability of Deep Reinforcement Learning (DRL) to navigate through challenging, non-stationary environments. In the real world, there is often a need to quickly adapt trained DRL agents to more difficult tasks with conflicting rewards. For efficient retraining, the ability to leverage previously learned skills becomes critical. Unfortunately, using traditional DRL algorithms like soft actor critic (SAC) for retraining a policy can lead to catastrophic forgetting of the policy’s known skills. In this work, inspired by neuroscience research, we propose a novel approach using SAC with inhibitory networks to allow separate and adaptive state value evaluations, as well as distinct automatic entropy tuning. We validate our method through experiments using a quadcopter in a realistic simulation environment and demonstrate the advantage of retraining. Moreover, we present the superiority of our approach compared to baseline methods with respect to both sample efficiency and cumulative success.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.002 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".