Novel interfaces for audio manipulation
Bibliographic record
Abstract
In this work we investigate two novel methods for multi-parametric control of audio in the context of sound design and music creation.Both systems aim to improve user experience by leveraging mapping strategies from control input to synthesis parameters, extracting additional value from conventional and ubiquitous tools.A first engine uses landmark mapping of facial features and manipulation of basic, built-in plugins to effectively emulate spatial audio mapping using only hardware built into a standard laptop.A pilot experiment (N=4) shows that the system improves source location identification in users by 15% over baseline.We also propose an assistive engine for sound design, which leverages a simple statistical model based on expert domain presets to guide users in controlling a subtractive synthesizer.In a user study (N=11) the engine is shown to significantly improve user performance in a sound matching task (t(10) = 3.38, p = 0.01), boosting user performance by 0.89 standard deviations within-participant.We explain its effectiveness in terms of cognitive apprenticeship theory, where the engine, in offering suggestions in reaction to user movement in a tight feedback loop, acts as a mentor that exposes sound Abstract ii design thinking, and keeps the user in the zone of proximal development by making the task not too difficult nor too obvious.Taken together, these experiments prove that mapping alone can meaningfully augment the affordances of existing control and synthesis paradigms.iii Abrégé Dans ce mémoire, nous étudions deux nouvelles méthodes de contrôle multi-paramétrique de l'audio dans le contexte du design sonore et de la création musicale.Les deux systèmes visent à améliorer l'expérience utilisateur en exploitant des stratégies de mappage entre les entrées de contrôle et les paramètres de synthèse, extrayant ainsi une valeur ajoutée d'outils conventionnels et omniprésents.Un premier moteur utilise le mappage de points de repère des caractéristiques faciales et la manipulation de plugins intégrés de base pour émuler efficacement le mappage audio spatial en utilisant uniquement le matériel intégré dans un ordinateur portable standard.Une expérience pilote (N = 4) montre que le système améliore l'identification de la localisation des sources chez les utilisateurs de 15% par rapport à la référence.Nous proposons également un moteur d'assistance pour le design sonore, qui exploite un modèle statistique simple basé sur des presets d'experts du domaine pour guider les utilisateurs dans le contrôle d'un synthétiseur soustractif.Dans une étude utilisateur (N = 11), il est démontré que le moteur améliore significativement les performances des utilisateurs dans une tâche de correspondance sonore (t(10) = 3.38, Abrégé iv p = 0.01), augmentant les performances des utilisateurs de 0.89 écarts-types au sein des participants.Nous expliquons son efficacité en termes de théorie de l'apprentissage cognitif, où le moteur, en offrant des suggestions en réaction au mouvement de l'utilisateur dans une boucle de rétroaction serrée, agit comme un mentor qui expose la réflexion de design sonore et maintient l'utilisateur dans la zone de développement proximal en rendant la tâche ni trop difficile ni trop évidente.Ensemble, ces expériences prouvent que le mappage seul peut augmenter de manière significative les affordances des paradigmes de contrôle et de synthèse.v former system maps a head orientation control signal to stereo panning and filter cutoff parameters; the latter system maps eight synthesis parameter dials to each other, to visualize "suggested" dial adjustments to the user. Thesis StructureThis thesis consists of five chapters: an introduction covering the thesis topic, structure and summary of contributions, followed by a chapter of literature review; two chapters covering two systems for novel audio control developed by the author, including experiments (user studies) on these systems to validate their effectiveness; and a brief conclusion. Summary of ContributionsThis work presents two main contributions:1.A novel method to control a simple spatial audio engine by using facial landmarks from a pose-estimation algorithm running on a webcam feed.2. An assistant engine for a subtractive synthesizer that determines and displays parameter value suggestions to the user in realtime, that is statistically proven to improve user performance in sound design.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.005 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.001 |
| Bibliometrics | 0.001 | 0.000 |
| Science and technology studies | 0.000 | 0.001 |
| Scholarly communication | 0.002 | 0.004 |
| Open science | 0.002 | 0.003 |
| Research integrity | 0.002 | 0.001 |
| Insufficient payload (model declined to judge) | 0.020 | 0.004 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".