My work on Telvana involves connecting voice conversations to scheduling and other tools. That connection is where a lot of the interesting engineering happens. The caller is waiting for an answer while the model, audio pipeline, and external service are each doing their own work.
A booking is a useful example. The agent needs to understand the requested time, check availability, submit the request, and explain the result. If the caller changes their mind halfway through, the system needs to know how far the operation has progressed.
I approach this by making the steps explicit. What did the model finish deciding? What has the application started? What did the scheduling service confirm? Those answers determine what the agent can safely say next.
Wait for the complete request
Streaming tool arguments arrive in pieces. A piece can look like valid JSON before the model has finished the request. I want execution to depend on the API’s completion signal, followed by schema validation.
A tool with no arguments makes this distinction easier to see. An empty buffer could mean the tool needs no input, or it could mean the stream ended early. The completion event tells us which case we’re handling. Guessing here can turn an interrupted response into an action.
Give the application ownership of execution
The model proposes a tool call. Application code validates the inputs, checks the permissions, sets a deadline, and records the result. Keeping those responsibilities together makes failures easier to diagnose.
I also want separate outcomes for a successful booking, an unavailable appointment, and a request whose result is unknown. Each needs a different response. If a service times out after receiving a request, checking the booking state may be safer than immediately trying again. A stable request identifier helps when the downstream system supports deduplication.
Keep speech aligned with state
“Let me check that” acknowledges the request. “You’re booked” confirms a completed action. The latter should follow a successful tool result.
Interruptions need the same care. Stopping speech may leave a valid operation running. Changing an appointment time may invalidate a request that has yet to start. I want the conversation layer to consult the operation’s state before deciding what to cancel or repeat.
Test the awkward sequences
For this kind of work, I find a repeatable sequence more useful than a polished demo. Good cases include an incomplete argument stream, a slow dependency, an interruption during execution, and a reconnect after the tool has finished.
A trace helps establish the order of events. Listening to the recorded audio answers a different question: did the timing and wording make sense to the caller? Both matter. My goal is a system whose spoken answer agrees with what actually happened.