XLNet4Rec: Recommendations Based on Users' Long-Term and Short-Term Interests Using Transformer
Bibliographic record
Abstract
The importance of understanding temporal dynamics in accurate recommendation systems is widely acknowledged. Sequential recommendation systems can efficiently model the dynamics of users and items over the period of time. Therefore, considering long-term and short-term interests of the user is essential for accurate recommendation. However, existing models considered users' long-term and short-term behavioural patterns, but ignore the side information, which plays an important role in improving the performance of the recommendations. As user behaviors contain a lot of information, such as consumption habits and dynamic preferences. In order to better locate user interests, our model called XLNet4Rec, considers the side information (additional features) along with IDs as inputs. Apart from this, unidirectional architectures such as GRU, LSTM restrict the power of hidden representation of users' behavior sequences and they follow the rigid ordered sequence, which is not always true in real world applications. Therefore, we used Transformers4rec (end-to-end RecSys framework), which consists XLNet, a transformer based architecture, employs the deep bidirectional approach to model user behavior sequences and also efficient in processing multiple features. In this paper, we proposed a recommendation system based on users' long-term and short-term interest using XLNet to predict the next user item interaction. Our model improves the quality and personalization of item recommendations for users. In this paper, we conducted experiments on two real-world datasets: Movielens and REES46. Empirical results show that our proposed model is more effective in recommending relevant items to the user compared to previous approaches.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.001 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.001 | 0.002 |
| Open science | 0.002 | 0.001 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.003 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".