MaxExp-UCB: Enhanced Regret Bounds for Normalized Exploration Stochastic Multi-Armed Bandit Problems With High Action Spaces
Bibliographic record
Abstract
Comprehensive exploration is necessary for collectively exploring the arms, while still exploiting the optimal arm, in K−armed stochastic bandits, specifically for higher arm spaces like the rescue and surveillance operation where relatively higher exploration of the agent is expected in the search space. Under the case of Multi-armed Bandits (MAB), where K, is relatively higher action dimensions, many exploration algorithms like the epsilon-greedy for example use random and direct exploration, where sub-optimal actions may be chosen frequently, thus increasing regret linearly. In this paper, we study on the theoretical aspects of MaxExp-UCB algorithm, which promotes comprehensive exploration while still having a sub-linear regret growth. We introduce normalized exploration across bandit arms as 2 ln(t) N i (t) • δ(K−1) i̸ =i ⋆ N i (t) , and show that, in case of higher K values, the δ(K−1) i̸ =i ⋆ N i (t) term, becomes smaller, promoting further exploration to ensure a comprehensive search across all available arms in our MAB setting. We also conduct a theoretical study on the on the worst-case upper bound of the term δ(K−1) i̸ =i ⋆ N i (t) and prove that the upper bound is ≤ √ t 2δ(K−1) • (t) ∆•T • ln(t) N i (t). Finally, using the previous-worst case-bound, we derive and analyses the pseudo-regret bound in adapting comprehensive exploration and show that our regret has sub-linear properties. Through this we conclude that, our regret analysis is near to UCB regret bound, indicating the effectiveness of MaxExp-UCB in making near-optimal decisions while promoting comprehensive exploration across K-possible arms in MAB setting.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.009 | 0.031 |
| Meta-epidemiology (narrow) | 0.003 | 0.001 |
| Meta-epidemiology (broad) | 0.003 | 0.002 |
| Bibliometrics | 0.001 | 0.002 |
| Science and technology studies | 0.001 | 0.003 |
| Scholarly communication | 0.004 | 0.006 |
| Open science | 0.004 | 0.005 |
| Research integrity | 0.003 | 0.007 |
| Insufficient payload (model declined to judge) | 0.008 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".