PMVC: Data Augmentation-Based Prosody Modeling for Expressive Voice Conversion

Framework of PMVC

Abstract

Voice conversion as the style transfer task applied to speech, refers to converting one person’s speech into a new speech that sounds like another person’s. Up to now, there has been a lot of research devoted to better implementation of VC tasks. However, a good voice conversion model should not only match the timbre information of the target speaker, but also expressive information such as prosody, pace, pause, etc. In this context, prosody modeling is crucial for achieving expressive voice conversion that sounds natural and convincing. Unfortunately, prosody modeling is important but challenging, especially without text transcriptions. In this paper, we firstly propose a novel voice conversion framework named ‘PMVC’, which effectively separates and models the content, timbre, and prosodic information from the speech without text transcriptions. Specially, we introduce a new speech augmentation algorithm for robust prosody extraction. And building upon this, mask and predict mechanism is applied in the disentanglement of prosody and content information. The experimental results on the AIShell-3 corpus supports our improvement of naturalness and similarity of converted speech.

Type
Publication
In 31st ACM International Conference on Multimedia
Click the Cite button above to demo the feature to enable visitors to import publication metadata into their reference management software.
Yimin Deng
Yimin Deng
University of Science and Technology of China
Huaizhen Tang
Huaizhen Tang
University of Science and Technology of China
Ning Cheng
Ning Cheng
Director