Context-Aware in an Emerging Area in Conversational Agents: A Comparative Study of Recurrent Neural Networks and Transformer Models for Intent Detection
Bibliographic record
Abstract
The idea of a Cyber-Physical-Social System, or CPSS for short, is a relatively new concept that has emerged as a response to the requirement to comprehend the influence that Cyber-Physical Systems (CPS) have on people and vice versa. Conversational assistants (CAs), also called bots, are dedicated to oral or written communication. Over time, the CAs have gradually diversified to today touch various fields such as e-commerce, healthcare, tourism, fashion, travel, and many others sectors. Natural-language understanding (NLU) is fundamental in the Natural Language Processing (NLP) field. Identifying user intents from natural language utterances is a crucial step in conversational systems, and the diversity in user utterances makes intent detection even a challenging problem. Recently, with the emergence of Deep Neural Networks. New State of the Art (SOA) results have been achieved for different NLP tasks. Recurrent Neural networks (RNNs) and recent Transformer architectures are two major players in those improvements. In addition, RNNs have been playing an increasingly important role in sequence modeling in different application areas. On the other hand, Transformer models are new architectures that benefit from the attention mechanism, extensive training datasets, and compute power. First, this review paper presents a comprehensive overview of RNN and Transformer models. Then, a comparative study of the performance of different RNNs and Transformer architectures for the specific task of intent recognition for CAs which is a fundamental task of NLU.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".