Music Emotion Recognition and Modeling Based on Multimodal Signal Fusion
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
With the rapid development of the digital music industry, a vast amount of music resources has emerged, and the demand for accurate music emotion matching is becoming increasingly urgent.The transmission of music emotion involves multimodal information, such as audio and text.Single-modal recognition, due to its inability to fully capture the emotional nuances, has limitations, and multimodal signal fusion has become the key to achieving a more comprehensive recognition of music emotion.In current research on music emotion recognition, some methods rely on single modalities, such as audio feature-based recognition, which cannot interpret the deeper emotions in lyrics, resulting in low recognition accuracy for lyrical music.Some multimodal fusion studies use early feature concatenation or simple weighted strategies, failing to establish dynamic relationships between modalities.As a result, recognition errors are significant in cross-modal conflict scenarios, and robustness to cross-modal noise is insufficient.Against this backdrop, researching music emotion recognition and modeling based on multimodal signal fusion is of great significance.This study proposes a multimodal signal fusion-based music emotion recognition model, which makes breakthroughs through four core modules: in the feature extraction phase, improved Convolutional Neural Network (CNN) is used to extract emotional features from the audio time-frequency domain, and Bidirectional Long Short-Term Memory (BiLSTM) combined with the attention mechanism captures the semantic emotional tendencies of the text; the cross-modal interaction learning module designs a dynamic attention weight matrix, quantifying the contribution of different modalities in different emotional dimensions based on mutual information entropy; the feature fusion module introduces a cross-modal Transformer, which maps audio temporal features and text semantic features to a unified emotional vector space to address modality heterogeneity; the emotion classification layer uses a multi-output loss function to optimize both discrete emotional categories and continuous emotional dimension predictions.This research aims to improve the accuracy and robustness of music emotion recognition, providing a scalable model architecture and technical standards for multimodal emotion computation.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it