Active Learning for Online Nonlinear Neyman-Pearson Classification
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
Neyman-Pearson (NP) classification framework is suitable for solving binary classification problems with asymmetric error costs such as network intrusion detection and medical diagnosis. In these kind of applications, type I (detecting non-target as target, false positive) and type II (detecting target as non-target, false negative) errors have different consequences. In this paper, we propose an active learning method for online context tree based ensemble NP classifiers. Proposed method prioritizes training samples that have high uncertainty (greater than a constant threshold) among different classifiers of the ensemble model. We report the performance of the proposed active learning method by measuring the moving true positive rates (TPR) and NP scores with respect to the number of samples used in learning. Experiments are carried out on 4 different datasets and proposed model was compared with random sampling method, where new samples are selected randomly from the training set. In addition, we also show that in order to satisfy target false alarm rate of the NP problem, we need to sample training set with and exploration probability, independent from uncertainty measurement.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.004 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.002 | 0.001 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it