Cross-modal relationships between music, emotion, and visual imagery: A comparative study of Iran, Canada, and Japan [Stage 1 Registered Report Snapshot]
Bibliographic record
Abstract
Many people experience emotions and visual imagery while listening to music. Previous research has identified cross-modal associations between musical features such as pitch, tempo, and visual features including height, and colour saturation/brightness. But while researchers have explored cross-cultural links between music and emotion as well as music and visual imagery, few studies have simultaneously investigated cross-cultural links between music, visual imagery, and emotion in order to distinguish the role of cultural experiences in contrast to innate perceptual capabilities. We hypothesize that cross-cultural emotion and cross-modal associations encompass both universality and diversity determined by innate perceptual capabilities and cultural experiences respectively and represented by: 1) cross-culturally consistent correlations between high level auditory features and low level visual features and dimensional emotions, 2) cross-cultural diversity in categorical emotional appraisal. In this study, we investigate the relationship between emotion and visual imagery induced by 6 musical excerpts in participants in Japan, Iran, and Canada through forced choice options such as matching excerpts with visual textures that vary in density, as well as dimensional and categorical emotion ratings for each excerpt manipulated in pitch and tempo. In our full manuscript, we will provide our power analysis and provide the details of our planned sample size.We will test the following four hypotheses: 1) tempo correlates with arousal levels cross-culturally, 2) tempo correlates with visual density cross-culturally, 3) emotion category ratings tend to vary between cultures and often even within cultures in response to slow tempo musical excerpts, and 4) emotion ratings show more consistency cross-culturally in response to faster tempo pieces.Each of the hypotheses helps us conclude which cross-modal and emotion associations are 1) cross-culturally consistent, or 2) culturally-dependent. If we find cross-cultural consistency across all three groups, it would suggest physiological link/embodied cognition represented in cross-modal associations. If we find consistency in two cultures only or none at all, it would suggest that cross-modal associations are culturally dependent. We also plan to conduct exploratory analyses of data on colour associations using the Berkeley Colour project stimuli.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.002 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.001 | 0.002 |
| Science and technology studies | 0.005 | 0.001 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.004 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".