The Wrong Way to Build for Multiple Languages
The naive approach to supporting 22 Indian languages: build 22 separate systems. One for Hindi, one for Tamil, one for Kannada, each with its own speech recognition model, its own TTS model, its own LLM prompt tuning. 22 deployments, 22 maintenance burdens, 22 points of failure.
This approach is impossible at any reasonable budget. It is also architecturally wrong.
The Right Abstraction
VoicERA treats language as a configuration parameter, not an architectural dimension. The pipeline is language-agnostic:
1. Detect language (auto-detect from first 2 seconds of audio — 94% accuracy across 22 languages)
2. Route to language-specific STT model (AI4Bharat provides a unified model family with language adapters)
3. Process in target language OR translate → process → translate back (depending on knowledge base availability)
4. Generate response in target language via TTS
Steps 1, 3 (partially), and 4 use shared infrastructure. Only step 2 requires language-specific components, and AI4Bharat's model family provides these as hot-swappable adapters, not separate deployments.
Language Detection at Scale
Auto-detecting Indian languages is harder than detecting European languages. The phoneme sets overlap significantly. Chitpavan Konkani and Goan Konkani sound similar but have distinct vocabularies. Hindi and Hindustani differ in formality conventions.
Our detection model is fine-tuned on call center audio — a very different acoustic environment from the Wikipedia-sourced training data that most language ID models use.
The Code-Switching Problem
Indian callers constantly mix languages. A Bangalore-based caller might speak Kannada sentences with English technical terms and occasional Hindi phrases. Our pipeline handles this with:
- A sliding window acoustic model that re-identifies language every 3 seconds
- A vocabulary augmentation layer that recognises domain-specific English terms within any Indic language context
Code-switching accuracy is our lowest benchmark — 78% correct handling — and our most active area of improvement.
Scaling to New Languages
Adding language N+1 to VoicERA requires:
1. An AI4Bharat STT adapter for the language (already exists for all 22 scheduled languages)
2. A TTS model in the target language (exists for 19 of 22)
3. Knowledge base content in the target language (this is the real work)
The pipeline is ready for all 22 scheduled languages. The knowledge bases are the bottleneck — and that is a content problem, not an engineering problem.