LADDER: Level Analysis Dataset for Difficulty Evaluation and Ranking
Bibliographic record
Abstract
Standardized datasets are fundamental to scientific research. While fields like natural language processing and computer vision have widely accepted datasets that drive progress, video game research still lacks such resources, particularly for studying level difficulty. Existing studies rely on isolated, custom datasets, limiting cross-study comparisons and hindering the development of generalizable models. To bridge this gap, we introduce LADDER, a novel dataset specifically designed to analyze and evaluate level difficulty in video games. Unlike previous datasets that primarily focus on physiological and behavioral player data, LADDER integrates objective performance metrics (e.g., health lost, number of attempts before success), level characteristics (e.g., number of danger zones, object placement), and perceived difficulty ratings across multiple platformer games. This dataset enables researchers to establish benchmarks, enhance collaboration across disciplines, and improve study reproducibility. LADDER provides a standardized foundation for investigating the relationship between game design elements and player experience. By facilitating difficulty assessment and level balancing, it supports advancements in game design, player modeling, and adaptive gameplay systems. We present an overview of existing datasets, describe the methodology behind LADDER’s construction, and showcase its potential through preliminary analyses. The dataset is freely available online, offering a valuable resource for the scientific community to develop more engaging and accessible gaming experiences.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.002 | 0.012 |
| Meta-epidemiology (narrow) | 0.002 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.005 | 0.004 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.002 | 0.002 |
| Open science | 0.003 | 0.003 |
| Research integrity | 0.002 | 0.002 |
| Insufficient payload (model declined to judge) | 0.012 | 0.019 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".