Hybrid CNN-Transformer for Dynamic Indian Sign Language Recognition with Non-Manual Gesture Analysis
Bibliographic record
Abstract
Sign language is a component element of communication between the mute and hearingimpaired communities that are indispensable to them, but it is mostly closed-off to the general population.To step into that gap, the current paper outlines the design of a hybrid Vision Transformer-Convolutional Neural Network system, officially focused on Indian Sign Language (ISL) gesture recognition, strong dynamic gestures, and face muscles.The edited database is 1 100 video samples in 22 different classes, which were recorded in the heterogeneous environmental conditions, to provide the robustness.The empirical findings indicate that the hybrid model has an exemplary training accuracy of 100, validation accuracy of 88.6, and a test accuracy of 82.14 and thus outperforms the state-of-the-art that provides accuracy of 88.7 to 92% of training accuracy.Proposed system thus achieves enhanced accuracy by 7-11% in case of continuous sign gestures.Through this, inclusivity and accessibility to the deaf community are thereby enhanced and future possibilities involve data enhancement as well as the integration of NLP-based text-to-speech synthesis.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.000 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.003 | 0.004 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.002 | 0.007 |
| Open science | 0.001 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".