Deep Learning Approaches on Multimodal Sentiment Analysis
Bibliographic record
Abstract
Sentiment analysis aims to uncover people's sentiment based on some information about them, often using machine learning or deep learning algorithm to determine. The importance of such a technique heavily grows because it can help companies better understand users' attitudes toward things and decide future plans. Considering the maturity of sentiment analysis on a single modal such as text and image and the advancement of social media, which allow users to post with text and image simultaneously, novel methods are designed for multimodal sentiment analysis. Because of the infancy of this field, new methods are sometimes inefficient, unstable and lack robustness and still need further study. One big challenge on the multimodal sentiment analysis is to find a proper way that can overcome heterogeneous gap between different modalities, which implies that researchers should develop a method of combining various modalities properly to get higher efficiency and accuracy. Although much research has been performed and many new models are implemented, related summary is still lacking. This paper mainly introduces four interesting and relatively novel methods of solving the links among diverse modalities. First it briefly introduces some basic definitions and terminologies in sentiment analysis, followed by several popular and common datasets on multimodal sentiment analysis. After that it gives some evaluation index to assess one model's competence. Then it shows four models and compares pros and cons among them, showing the potential trend of future study in multimodal sentiment analysis. Finally, it proposes future challenges and opportunities.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.002 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.000 | 0.001 |
| Scholarly communication | 0.001 | 0.002 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.003 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".