Do We Really Need Hundreds of Machine Learning Models in Industry?
Bibliographic record
Abstract
Abstract Recent practice in tabular data-driven applications in the petroleum industry often follows the classic machine learning workflow, which selects the so-called "best" model based solely on accuracy. This accuracy-focused workflow involves exhaustive model comparison across hundreds of algorithms and tends to favor complex, hybrid or specific-tailored models. As a result, models that are well suited for field deployment, such as decision trees, are frequently overlooked despite offering lightweight structures and essential interpretability that support operator trust and practical use. In this regard, this study challenges the traditional paradigm by proposing a competitive oblique decision tree that strikes an optimal balance between accuracy, interpretability, and lightweight model structure. The proposed model reformulates decision tree training problem, which involves learning optimal nested IF–THEN prediction rules, as an unconstrained optimization task. A soft approximation is applied to enable differentiability. This reformulation enables effective optimization of all prediction rules through our gradient-based entire tree optimization approach. Extensive experiments on 26 tabular datasets show that our model achieves competitive predictive accuracy compared to 26 commonly used machine learning algorithms. Our tree with constant prediction, on regression datasets achieves an average improvement of 6.25% over traditional decision tree (CART), while our tree with linear prediction yields 7.22% improvement over CART and slightly outperforms random forest with a 0.1% gain. Beyond accuracy, our tree with linear prediction features a lightweight structure with only 4,650 parameters, several orders of magnitude smaller than random forest with 0.96 million, resulting in a 71 times speedup in prediction time. Interpretability is preserved through concise IF–THEN rules, with about 24.37≈ 21 prediction rules far fewer than the approximate 212.47 ≈ 5763 rules typically generated by CART. These performance advantages, combined with open-source availability, make our tree a strong alternative to hundreds of machine learning models and eliminates the need for tedious model selection.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.007 | 0.026 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.001 |
| Bibliometrics | 0.001 | 0.002 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.005 | 0.012 |
| Open science | 0.003 | 0.002 |
| Research integrity | 0.002 | 0.005 |
| Insufficient payload (model declined to judge) | 0.010 | 0.005 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".