A voice AI connected to an IP-PBX
what we learned

The chain, from call to response

A call arrives on a number, enters an Asterisk IP-PBX, which routes it to a SIP gateway. The gateway opens an audio stream to an application service: speech recognition, language model, voice synthesis. The response goes back the same way, in the same stream, never hanging up.

Each link is simple. It is their stacking that causes problems, because latencies add up and voice does not tolerate approximation.

The three constraints that decide everything

• Cumulative latency : the sum of speech, transcription, generation and synthesis times. Beyond a few hundred milliseconds, the caller thinks the line dropped and talks over.
• Jitter and packet loss : a voice stream does not tolerate rhythm variations well. A loaded network degrades comprehension long before dropping the call.
• Interruption : knowing when to stop talking when the caller resumes speaking. It is an audio buffer and voice activity detection problem, not a model problem.

What we learned connecting it for real

On a first deployment, the demo worked in the lab and the first real call was catastrophic: the perceived response time was twice as long as announced. The measurement done component by component was correct, it was the chaining that cost.

Consequence: we no longer measure an isolated component, but the time between the end of the caller's sentence and the first sound of the response. That is the only measurement that matches what the person on the line hears.

Guardrails, before performance

• Human transfer at any time a keyword, an explicit request, a model doubt: the call goes to a real extension.
• Closed scope the assistant only answers on a documented domain. Outside the scope, it does not guess: it transfers.
• Logging each call leaves an exploitable trace: timestamp, detected intent, transfer decision.

What we do not announce

Locally hosted video conferencing is still in development: we do not sell it. The voice PBX, IP phones, telephone conferences and the SIP trunk are in service. The rule is simple: an article, like a brochure, only describes what is already running at a client.