Accent modification for speech recognition of non-native speakers using neural style transfer
Authors:
- Kacper Radzikowski,
- Le Wang,
- Osamu Yoshie,
- Robert Marek Nowak
Abstract
Nowadays automatic speech recognition (ASR) systems can achieve higher and higher accuracy rates depending on the methodology applied and datasets used. The rate decreases significantly when the ASR system is being used with a non-native speaker of the language to be recognized. The main reason for this is specific pronunciation and accent features related to the mother tongue of that speaker, which influence the pronunciation. At the same time, an extremely limited volume of labeled non-native speech datasets makes it difficult to train, from the ground up, sufficiently accurate ASR systems for non-native speakers.In this research, we address the problem and its influence on the accuracy of ASR systems, using the style transfer methodology. We designed a pipeline for modifying the speech of a non-native speaker so that it more closely resembles the native speech. This paper covers experiments for accent modification using different setups and different approaches, including neural style transfer and autoencoder. The experiments were conducted on English language pronounced by Japanese speakers (UME-ERJ dataset). The results show that there is a significant relative improvement in terms of the speech recognition accuracy. Our methodology reduces the necessity of training new algorithms for non-native speech (thus overcoming the obstacle related to the data scarcity) and can be used as a wrapper for any existing ASR system. The modification can be performed in real time, before a sample is passed into the speech recognition system itself.
- Record ID
- WUTf1878862519e42d0b251b62bd1c7a15c
- Author
- Journal series
- EURASIP Journal on Audio Speech and Music Processing, ISSN 1687-4722, [1687-4714]
- Issue year
- 2021
- Vol
- 11
- Pages
- 1-10
- Publication size in sheets
- 0.50
- ASJC Classification
- ;
- DOI
- DOI:10.1186/s13636-021-00199-3 Opening in a new tab
- URL
- https://asmp-eurasipjournals.springeropen.com/articles/10.1186/s13636-021-00199-3 Opening in a new tab
- Language
- (en) English
- License
- File
-
- File: 1
- Accent modification for speech recognition of non-native speakers using neural style transfer, File WUTf1878862519e42d0b251b62bd1c7a15c.pdf / 917 KB
- WUTf1878862519e42d0b251b62bd1c7a15c.pdf
- publication date: 08-06-2021
- Accent modification for speech recognition of non-native speakers using neural style transfer, File WUTf1878862519e42d0b251b62bd1c7a15c.pdf / 917 KB
-
- Score (nominal)
- 100
- Score source
- journalList
- Score
- = 100.0, 26-05-2022, ArticleFromJournal
- Publication indicators
- = 0; = 2; : 2018 = 0.954; : 2020 (2 years) = 1.558 - 2020 (5 years) =2.008
- Uniform Resource Identifier
- https://repo.pw.edu.pl/info/article/WUTf1878862519e42d0b251b62bd1c7a15c/
- URN
urn:pw-repo:WUTf1878862519e42d0b251b62bd1c7a15c
* presented citation count is obtained through Internet information analysis and it is close to the number calculated by the Publish or PerishOpening in a new tab system.