On the application of deep learning and multifractal techniques to classify emotions and instruments using Indian Classical Music
Bibliographic record
Abstract
Music is often considered as the language of emotions. The way it stimulates the emotional appraisal across people from different communities, culture and demographics has long been known and hence categorizing on the basis of emotions is indeed an intriguing basic research area. Indian Classical Music (ICM) is famous for its ambiguous nature, i.e. its ability to evoke a number of mixed emotions through only a single musical narration, and hence classifying evoked emotions from ICM becomes a more challenging task. With the rapid advancements in the field of Deep Learning , this Music Emotion Recognition (MER) task is becoming more and more relevant and robust, hence can be applied to one of the most challenging test case i.e. classifying emotions elicited from ICM. In this paper we present a new dataset called JUMusEmoDB which presently has 1600 audio clips (approximately 30 s each) where 400 clips each correspond to happy, sad, calm and anxiety emotional scales. The initial annotations and emotional classification of the database was done based on an emotional rating test (5-point Likert scale) performed by 100 participants. The clips have been taken from different conventional ‘ raga’ renditions played in two Indian stringed instruments – sitar and sarod by eminent maestros of ICM and digitized in 44.1 kHz frequency. The ragas , which are unique to ICM, are described as musical structures capable of inducing different moods or emotions. For supervised classification purposes, we have used Convolutional Neural Network (CNN) based architectures (resnet50, mobilenet v2.0, squeezenet v1.0 and a proposed ODE-Net) on corresponding music spectrograms of the 6400 sub-clips (where every clip was segmented into 4 sub-clips) which contain both time as well as frequency domain information. Along with emotion classification, instrument classification based response was also attempted on the same dataset using the CNN based architectures. In this context, a nonlinear technique, Multifractal Detrended Fluctuation Analysis (MFDFA) was also applied on the musical clips to classify them on the basis of complexity values extracted from the method. The initial classification accuracy obtained from the applied methods are quite inspiring and have been corroborated with ANOVA results to determine the statistical significance. This type of CNN based classification algorithm using a rich corpus of Indian Classical Music is unique even in the global perspective and can be replicated in other modalities of music also. The link to this newly developed dataset has been provided in the dataset description section of the paper. This dataset is still under development and we plan to include more data containing other emotional as well as instrumental entities into consideration.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.002 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.001 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.001 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".