Convolutional Features-Based Broad Learning With LSTM for Multidimensional Facial Emotion Recognition in Human–Robot Interaction
Bibliographic record
Abstract
Convolutional feature-based broad learning with long short-term memory (CBLSTM) is proposed to recognize multidimensional facial emotions in human–robot interaction. The CBLSTM model consists of convolution and pooling layers, broad learning (BL), and long- and short-term memory network. It aims to obtain the depth, width, and time scale information of facial emotion through three parts of the model, so as to realize multidimensional facial emotion recognition. CBLSTM adopts the structure of BL after processing was done at the convolution and pooling layer to replace the original random mapping method and extract features with more representation ability, which significantly reduces the computational time of the facial emotion recognition network. Moreover, we adopted incremental learning, which can quickly reconstruct the model without a complete retraining process. Experiments on three databases are developed, including CK+, MMI, and SFEW2.0 databases. The experimental results show that the proposed CBLSTM model using multidimensional information produces higher recognition accuracy than that without time scale information. It is 1.30% higher on the CK+ database and 1.06% higher on the MMI database. The computation time is 9.065 s, which is significantly shorter than the time reported for the convolutional neural network (CNN). In addition, the proposed method obtains improvement compared to the state-of-the-art methods. It improves the recognition rate by 3.97%, 1.77%, and 0.17% compared to that of CNN-SIPS, HOG-TOP, and CMACNN in the CK+ database, 5.17%, 5.14%, and 3.56% compared to TLMOS, ALAW, and DAUGN in the MMI database, and 7.08% and 2.98% compared to CNNVA and QCNN in the SFEW2.0 database.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.001 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.002 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".