w2v-bert
Filtering by topic w2v-bert(1)Clear all filters
- PaperarXiv — Language & NLP (cs.CL)23 Jul 2026
DONDO: Open w2v-BERT Speech-Recognition Base Models for African Languages
Paul Azunre
DONDO, a family of open w2v-BERT 2.0-based automatic speech recognition models, is introduced for 27 African language varieties across six countries. The models, fine-tuned primarily on read speech from religious texts, achieve average word error rates of 10-13% across multilingual families. A lightweight language-conditioning mechanism allows a single multilingual checkpoint to be steered to target languages at inference, and all models are released under Apache-2.0 license on Hugging Face.
Original abstract
We present DONDO, a family of open, permissively licensed automatic speech recognition (ASR) base models for African languages, built on the w2v-BERT 2.0 self-supervised speech encoder. DONDO comprises twenty-one monolingual models and five multilingual models spanning twenty-seven language varieties across Ghana, Sierra Leone, Nigeria, Senegal, Kenya and Zimbabwe. Models are fine-tuned primarily on read speech drawn from religious texts, which offer broad, license-clear and orthographically consistent coverage for languages that otherwise lack transcribed audio. We describe a two-step (and, for one family, three-step) learning-rate-annealed fine-tuning procedure that first adapts a shared multilingual model at a high learning rate and then anneals it to recover, and in several cases surpass, strong monolingual baselines. We further describe a lightweight language-conditioning mechanism that injects a one-hot language identity as a sequence of prefix frames prepended to the acoustic features, allowing a single multilingual checkpoint to be steered to a target language at inference. Across the five multilingual families the annealed models reach average word error rates (WER) of 10-13%, closing most of the gap to monolingual models while covering many languages in a single checkpoint. All models are released on the Hugging Face KhayaAI organisation under the Apache-2.0 license (attribution only) so that others may fine-tune them freely, including for commercial use. We provide a conservative estimate that the languages covered are spoken by on the order of one hundred million first-language speakers, and by substantially more when second-language use is included.