FAST-ODT: A Lightweight Outlier Detection Scheme for Categorical Data Sets
Bibliographic record
Abstract
Outlier detection is a key data analysis technique that aims to find unusual data objects in a data set. It has been widely used in varied areas, including communication networks, finance, medicine, environmental studies, etc. Many applications in these areas involve categorical data. For example, the data set used in the application of intrusion detection normally includes a group of captured packets, which tend to have categorical attributes such as “protocol”. Although there are many outlier detection algorithms for applications involving numerical data, only a few existing schemes can handle categorical data. And the schemes designed for categorical data seriously suffer from two problems: low detection precision and high time complexity. In this paper, we present two novel outlier detection algorithms for categorical data sets. First of all, we describe a simple scheme based on entropy, Outlier Detection Tree (ODT). With ODT, a classification tree is constructed to classify the data set into two classes: a normal class and an abnormal class. Thereafter, each data object is identified as an outlier or a normal one using the if-then rules in the tree. Furthermore, we propose an advanced outlier detection algorithm, FAST-ODT, which achieves both high detection accuracy and low time complexity. Our experimental results indicate that FAST-ODT outperforms the existing algorithms in terms of outlier detection precision and computational complexity.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.004 | 0.017 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.002 | 0.001 |
| Bibliometrics | 0.004 | 0.005 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.002 | 0.004 |
| Open science | 0.003 | 0.003 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.001 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".