Cascade: Enhancing Reinforcement Learning with Curriculum Federated Learning and Interference Avoidance — A Case Study in Adaptive Bitrate Selection
Bibliographic record
Abstract
Current reinforcement learning (RL) algorithms, particularly RL-based networking algorithms, demonstrate significant potential for overcoming limitations of manually-tuned heuristics. However, RL-based algorithms are known to be sample inefficient and may not perform well in a wide range of environments. Federated reinforcement learning (FRL) aims to enhance sample efficiency and improve model performance across a wide variety of environments. Nevertheless, many existing approaches neglect the challenges posed by the dynamic nature of training sample distributions in RL and the heterogeneity of data across clients in FRL. This may restrain the broader applicability of FRL algorithms. Addressing these gaps, we propose Cascade, the first curriculum federated reinforcement learning framework with interference avoidance, and we study Cascade in the context of RL-based adaptive bitrate (ABR) selection algorithms. To eliminate interference between two or more interfering tasks from different clients in FRL, we propose an interference avoidance technique that penalizes changes to model parameters important for other clients. We extensively evaluate Cascade in a wide range of network environments. Our experiments show that Cascade outperforms the state-of-the-art federated learning settings by a minimum of 21% in average reward and 15% in model skewness. These findings highlight the efficacy of Cascade and underscore the potential of enhancing FRL.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".