Internalizing ASR with Implicit Chain of Thought for Efficient Speech-to-Speech Conversational LLM
Pourquoi ce travail est-il dans la base ?
Une base qui oublie comment elle a trouvé un travail ne peut pas être vérifiée. Voici les voies qui ont admis celui-ci.
Aucune affiliation canadienne. Une base fondée sur la seule affiliation (le devis habituel) n'aurait jamais vu ce travail. C'est l'un des travaux qui justifient l'inversion de la base.
Le tri à trois modèles
les 1 000 travaux triés →Les trois modèles l'ont jugé hors champ.
Machine learning method internalizing ASR chain of thought in a speech-to-speech conversational LLM; a model architecture contribution.
The preprint develops a speech-to-speech language-model method, not a study of research practice.
Speech LLM engineering for ASR chain-of-thought; AI systems research, not metaresearch.
Résumé
Current speech-based LLMs are predominantly trained on extensive ASR and TTS datasets, excelling in tasks related to these domains. However, their ability to handle direct speech-to-speech conversations remains notably constrained. These models often rely on an ASR-to-TTS chain-of-thought pipeline, converting speech into text for processing before generating audio responses, which introduces latency and loses audio features. We propose a method that implicitly internalizes ASR chain of thought into a speech LLM, enhancing its native speech understanding capabilities. Our approach reduces latency and improves the model's native understanding of speech, paving the way for more efficient and natural real-time audio interactions. We also release a large-scale synthetic conversational dataset to facilitate further research.
Conservé avec la notice de tri, où il sert de preuve aux étiquettes ci-dessus.
La notice
- Revue
- arXiv (Cornell University)
- Thématique
- Speech and dialogue systems
- Domaine
- Computer Science
- Établissements canadiens
- —
- Organismes subventionnaires
- Alliance de recherche numérique du CanadaNatural Sciences and Engineering Research Council of CanadaUniversity of British Columbia
- Mots-clés
- Speech recognitionPsychologyIndirect speechComputer scienceLinguisticsCognitive psychologyNatural language processingPhilosophy
- Résumé présent dans OpenAlex
- oui