Do We Really Need Hundreds of Machine Learning Models in Industry?
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
Abstract Recent practice in tabular data-driven applications in the petroleum industry often follows the classic machine learning workflow, which selects the so-called "best" model based solely on accuracy. This accuracy-focused workflow involves exhaustive model comparison across hundreds of algorithms and tends to favor complex, hybrid or specific-tailored models. As a result, models that are well suited for field deployment, such as decision trees, are frequently overlooked despite offering lightweight structures and essential interpretability that support operator trust and practical use. In this regard, this study challenges the traditional paradigm by proposing a competitive oblique decision tree that strikes an optimal balance between accuracy, interpretability, and lightweight model structure. The proposed model reformulates decision tree training problem, which involves learning optimal nested IF–THEN prediction rules, as an unconstrained optimization task. A soft approximation is applied to enable differentiability. This reformulation enables effective optimization of all prediction rules through our gradient-based entire tree optimization approach. Extensive experiments on 26 tabular datasets show that our model achieves competitive predictive accuracy compared to 26 commonly used machine learning algorithms. Our tree with constant prediction, on regression datasets achieves an average improvement of 6.25% over traditional decision tree (CART), while our tree with linear prediction yields 7.22% improvement over CART and slightly outperforms random forest with a 0.1% gain. Beyond accuracy, our tree with linear prediction features a lightweight structure with only 4,650 parameters, several orders of magnitude smaller than random forest with 0.96 million, resulting in a 71 times speedup in prediction time. Interpretability is preserved through concise IF–THEN rules, with about 24.37≈ 21 prediction rules far fewer than the approximate 212.47 ≈ 5763 rules typically generated by CART. These performance advantages, combined with open-source availability, make our tree a strong alternative to hundreds of machine learning models and eliminates the need for tedious model selection.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it