Hierarchical text classification of large-scale topics
Bibliographic record
Abstract
Topic modelling and classification of documents is a well studied problem in Natural Language understanding.Deep neural networks have displayed superior performance over the traditional supervised classifiers in text classification.They learn to extract useful features automatically when sufficient amount of data is presented.However, along with the growth in the number of documents comes the increase in the number of categories, which often results in poor performance of the multiclass classifiers.In this work, we use external knowledge of topic category taxonomy to aide the classification by introducing a deep hierarchical neural attention-based classifier.Our model performs better than or comparable to state-of-the-art hierarchical models with significantly fewer computational resources while maintaining high interpretability.iii Rsum La modlisation de sujet et la classification de documents est un problme bien tudi dans la comprhension du langage naturel.Les rseaux de neurones profonds affichent des performances suprieures celles des classificateurs superviss traditionnels dans la classification de textes.Ils apprennent extraire automatiquement des attributs utils lorsqu'une quantit suffisante de donnes est prsente.Cependant, avec la croissance du nombre de documents vient l'augmentation du nombre de catgories, ce qui entrane souvent une mauvaise performance des classificateurs nombreuses catgories.Dans ce travail, nous utilisons l'information donn par la taxonomie de catgorie de sujet pour aider la classification en introduisant un classificateur hirarchique bas sur l'apprentissage profond et les systmes d'attention.Notre modle est comparable et peut mieux performer que des modles hirarchiques de pointe tout en utilisant moins de ressources de calcul tout en conservant une haute interprtabilit.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.001 | 0.000 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".