Learning of Deception in Adversarial Games with Hierarchical Multiobjective Reinforcement Learning
Bibliographic record
Abstract
This thesis focuses on modelling deception in competitive differential games.By definition, differential games are games where two agents (or two groups of agents) have conflicting objectives.In other words, maximizing an agent's (or a group of agents') objective will minimize the other agent's (or a group of agents') pay-off.To increase the pay-off of one group, we implement deception in a differential game.On the other hand, the other group learns counter-deception to cope with the adversarials' strategy.To address deception and counter-deception, we implement the game of guarding territory and its derivatives.Our primary attempt consists of modelling deception using a hierarchical structure for pursuit-evasion games.The deception structure is made of two levels: a lower-level policy (LLP) that contains the optimal state-action values to reach a goal and a higher-level policy (HLP) that can change the lower-level policy's goal.While the players try to deceive their opponents using deception, the opponents try to detect deception and overcome the deceitful players.Reinforcement learning is the primary learning strategy in our study.We address the assignment problem in pursuit-evasion games, where multiple agents exist on each side.The assignment problem is solved by proposing a two-level policy system, where the pursuer's policy is calculated via a fuzzy actor-critic learning (FACL) algorithm.Our study is the first study on pursuit-evasion games where the assignment can be changed in the middle of the simulation.In addition, we proposed a hierarchical reinforcement learning scheme to provide broader control over the reward function hyperparameters.In other words, an HLP learns reward functions' hyperparameters.We combined all the proposed two-level structures.As a result, for the first time, we ii iii created a four-level policy system to solve all the mentioned problems simultaneously.The first level deals with learning the reward function in a four-level policy structure.The first level learns to generate the proper reward function at each time step.The second level deals with the primitive action, which is a steering angle of a robot.The third level deals with using deception and counter-deception.In an n-invader m-defender game, the assignment problem plays a pivotal role in the modelling.The fourth level assigns each invader to a defender to follow and assigns each defender to an invader to deceive.In this thesis, we also proposed a novel algorithm to solve multi-objective reinforcement learning problems in the continuous domain.Our proposed algorithm creates a Pareto front of non-dominant solutions, each representing a series of output parameters for a fuzzy logic controller.The algorithm can train a controller to address multiple conflicting reward functions.To the best knowledge, our proposed algorithm is the very first algorithm that addresses problems in the continuous domain.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.002 | 0.006 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.002 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".