Peeking Through the Windows: Hyperparameters, Administrative Data, and Selective Windowing
Bibliographic record
Abstract
With the growth of administrative data due to increased storage and digitization, there is a need for effective ways to process and analyze this information. This study examines the time-based nature of such data and suggests that choosing the right time window size is crucial and should be adjusted like other parameters in machine learning models. Additionally, an algorithm called the Time series Analysis to Investigate Binning (TAIB) algorithm is introduced. This algorithm determines which parts of the data might benefit from being examined in different time chunks, aiming to optimize the use of time-based data in machine learning. Two main datasets were utilized: one from The Calgary Drop-In and Rehab Centre (DI), covering 1991-2020, to categorize shelter users by their usage patterns; and the MIMIC III (Medical Information Mart for Intensive Care III) database, which provides details on ICU stays from 2001-2012, to predict the duration of a patient's stay. Both datasets were transformed into temporal matrices. Using the TAIB algorithm, the optimal features for time-based analysis were identified, leading to the creation of groups of feature sets for various time lengths. For testing, primary reliance was on basic implementations of several machine learning models. Preference was given to deep learning models due to their superior capability in managing vast data. Three experiments were conducted, modifying the model’s complexity, the features employed, and the type of model. The findings suggest that treating windowing as a hyperparameter improves the performance of machine learning models. Moreover, employing feature matrices offers an efficient alternative to using timesteps in environments constrained by system resources, simplifying the process of handling time-based data.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.010 | 0.038 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.002 | 0.004 |
| Open science | 0.002 | 0.002 |
| Research integrity | 0.001 | 0.004 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".