Choosing the speech engines behind your voice agent
No single vendor is best at transcription, synthesis and every language at once. Dialog365 treats the speech layer as pluggable, so each is chosen on merit.
A voice agent needs two speech capabilities: turning the customer's audio into text, and turning the reply into audio. These are separate problems with separate leaders, and the right answer changes by language, by latency budget and by how much the voice matters to the brand.
Supported engines
| Engine | Used for | Where it fits |
|---|---|---|
| Deepgram | Speech to text | Low-latency streaming transcription with strong real-time performance |
| ElevenLabs | Text to speech | The most natural-sounding voices, where brand voice matters most |
| Cartesia | Text to speech | Very low time-to-first-audio, suited to tight latency budgets |
| Sarvam | Speech to text and TTS | Indian languages and accents, where general models underperform |
| Microsoft Azure Cognitive Services | Speech to text and TTS | Broad language coverage and enterprise compliance posture |
What to optimise for
Transcription
Streaming latency and accuracy on your actual audio, in that order. A model that is two points more accurate on a clean benchmark but only returns final results at the end of the utterance will feel worse in conversation than a slightly less accurate model that streams partials. Test on recordings from your own lines, not on benchmark corpora.
Synthesis
Time to first audible byte first, naturalness second. A beautiful voice that takes 400ms to start speaking spends its advantage on dead air. Where both matter, the practical answer is often different engines for different contact types.
Why language coverage is not a single number
Vendors quote language counts, but coverage varies enormously in depth. A model may support Hindi in the sense that it produces Hindi text, while performing poorly on code-switched speech where English words appear mid-sentence — which is how a great many people actually speak. Region-specialised engines like Sarvam exist precisely because general models underperform there.
Why pluggable matters more than the current pick
This part of the stack moves faster than any other. An engine that is clearly best today may not be in six months, and being able to change one without re-architecting the agent is worth more than the specific choice made at launch.
Engines are selected per deployment, and can differ per language or contact type Swapping an engine does not change the agent's logic, prompts or routing New engines can be added without waiting on a platform release Audio quality gains are preserved end to end — see the piece on WebRTC
Frequently asked questions
Which STT engine does Dialog365 use?
Whichever suits the deployment. Deepgram, Sarvam and Azure Cognitive Services are all supported, and the choice is usually driven by language and latency requirements.
Can we use ElevenLabs voices?
Yes. ElevenLabs, Cartesia and Azure are supported for synthesis, and different voices can be used for different contact types.
What about Indian languages?
Sarvam is supported specifically for this, because general-purpose models tend to underperform on Indian languages and on code-switched speech.
Can we change engines later?
Yes. The speech layer is pluggable, so changing an engine does not change the agent's logic, prompts or routing.
See it on your own calls
Tell us how your contact centre runs today and we will map it to your setup.
Related
WebRTC and why call audio quality decides accuracy
Packet loss, codecs and what they do to transcription.
Why latency decides whether people talk to your voice agent
Sub-second turn-taking and what breaks below it.
Bring your own LLM
Model choice, routing and avoiding lock-in.
How Dialog365 stops the AI inventing answers
Grounding, closed-domain retrieval and refusal behaviour.