Fine-tuning OpenAI’s Whisper model for Kazakh speech recognition

Abstract

Speech recognition remains a challenging task, especially for low-resource languages such as Kazakh. This study explores the effectiveness of fine-tuning OpenAI’s Whisper model on Kazakh speech data for improving speech recognition accuracy. The Kazakh speech corpus (KSC) is used as a benchmark dataset and to conduct experiments to evaluate how the adapted Whisper framework measures up against current leading speech recognition models, including deep neural network (DNN)-hidden Markov model (HMM), end-to-end long short-term memory (E2E-LSTM), and end-to-end transformer (E2E-transformer). Research outcomes indicate that the Whisper model outperforms all baselines after fine-tuning, showing a result with a word error rate (WER) of 12.5%, a character error rate (CER) of 3.15%, a character-level F1-score (chrF) of 93.72, and a bilingual evaluation understudy (BLEU) score of 76.05. These findings suggest that the Whisper model that is fine-tuned on Kazakh audio data can be a promising approach for improving Kazakh speech recognition accuracy. This work advances the development of speech recognition for the Kazakh language by demonstrating the effectiveness of transfer learning in low-resource languages. Subsequent research must prioritize collecting larger and more diverse speech data for Kazakh, including noisy and accented speech, to further improve speech recognition accuracy.