GAN-Based Fine-Grained Feature Modeling For Zero-Shot Voice Cloning
Bibliographic record
Abstract
With the continuous development of deep learning and speech signal processing, speech synthesis technology has greatly improved in naturalness and comprehensibility, and many application technologies such as artificial intelligence voice assistant and personalized navigation have been widely used in real life, and the demand for personalized speech synthesis is increasing.Personalized speech synthesis requires models that can achieve speech timbre migration, also known as speech reproduction, with only a small number of target speaker speech samples.However, since human speech is highly expressive and contains rich information, including speaker identity information, prosody, rhythm, emotion and other factors, the limited speech data will lead to poor similarity and rhythmic performance of the model-generated speech, and the model needs to be fine-tuned to improve the quality of the synthesized speech.Therefore, personalized speech synthesis with few samples is a very challenging task.To achieve the goal of speech cloning, this paper proposes a personalized speech synthesis method based on FastSpeech2.By using fine-grained feature modeling module containing prosody extractor and prosody predictor, and a training strategy based on Generative adversarial network (GAN) and meta-learning, it is realized that personalized speech with high similarity and naturalness can be generated with a very short reference audio.The subjective and objective experiments also demonstrate that the model proposed in this paper can achieve high quality speech replication without fine-tuning the model under a few or even a single reference audio of the target speaker.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.002 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.001 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".