Save voice agent from its own echo
An early problem with voice agents: the agent’s voice comes back through the microphone, and the agent transcribes itself as if the user spoke. Headphones or a mute button work, but they take away the hands-free experience. The fix is acoustic echo cancelation, which uses the agent’s own speech as a reference and removes the echo and noise before speech recognition.
Takeaways
- Echo affects local, WebSocket and phone agents alike.
- Getting AEC to work is not straightforward. I got it running locally with a recent Python AEC library, Whisper for speech-to-text and Kokoro for text-to-speech.
- Echo is one of several blockers to production voice agents, alongside barge-in failures, short-audio transcription errors and background speakers. mulive, my voice library, is where I'm working through them.
Have a system that needs a second opinion?