The AnySpeech Project —Open-vocabulary keyword spotting and phonetic transcription in any language
Bibliographic record
Abstract
Performing multilingual phonetic analysis is challenging, not least due to the lack of performant tools for performing basic analyses on (non-English) speech data. Without the capacity to perform basic operations such as transcription or search through speech data for understudied languages, critical questions in cross-linguistic phonetic analysis remain elusive [Blasi etal. Trends in Cog. Sci. 26(12), 1153–1170 (2022)]. To this end, we will demonstrate our recent advances towards multilingual representation learning and automatic phonetic transcription, as part of our AnySpeech initiative. Specifically, we will demonstrate two tools. First, the CLAP-IPA model—a phoneme-to-speech model, capable of performing open-vocabulary keyword spotting in any language without any parameter updates, including languages that were not in the training dataset [Zhu et al., NAACL, 750-772 (2024)]. Second, the IPAPack transcription model, a lightweight transcription model, capable of annotating 473 different phones, including those that were lacking in prior models (e.g., click consonants in Bantu languages). We will demonstrate that our models can be easily downloaded and set up on consumer-grade laptops with little effort. We anticipate that our tools will enable researchers to analyze speech recording repositories at scale, unlocking answers to critical questions in cross-linguistic phonetic analysis.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.008 | 0.018 |
| Meta-epidemiology (narrow) | 0.004 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.002 |
| Bibliometrics | 0.002 | 0.001 |
| Science and technology studies | 0.002 | 0.002 |
| Scholarly communication | 0.004 | 0.009 |
| Open science | 0.005 | 0.009 |
| Research integrity | 0.003 | 0.004 |
| Insufficient payload (model declined to judge) | 0.022 | 0.042 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".