Every call you miss, answered — and the rule that makes it safe to let an agent act

About this series. Four pieces on what AI agents can and cannot automate in retail. Part 1 listed nine jobs a retailer can hand to an agent. This covers the second — answering the phone — and through it, what changes once an agent can do something that cannot be taken back.

What you would be buying

Reception is busy, or it is twenty to midnight. The call forwards, and a voice answers the question that would otherwise have become a missed call — from the same content your website agent uses. If it cannot help, it takes a callback number and puts the transcript in the team's inbox. If the caller agrees out loud, it sends them one text with a link.

What it recovers is the one channel where failure leaves no trace. An abandoned basket shows up in analytics. A bounced visitor shows up in analytics. A call that rings out shows up nowhere, and nobody files a ticket to say they could not reach you.

It also sends a text message, which means it acts — and an agent that acts can be wrong in a way you cannot take back. That is the rest of this article, because it is what you should interrogate before buying anything that touches a customer.

Start by writing down what it may never do

Before any of it was built we wrote one sentence: the bot is an overflow, never a gatekeeper. It never stands between a caller and a person who was available to answer. That single constraint settled most of the design arguments that followed, and it is the kind of sentence worth agreeing at board level rather than in a sprint.

Deliberately out of scope: taking bookings, taking payment, outbound calling, recording the call. Those are not missing features. They are the reason the rest is defensible.

The three things the voicebot may do: answer from the knowledge base, hand over to a person, send one text message with consent.
Three permitted actions. The list is short on purpose.

The rule that makes acting safe

The model chooses whether. A fixed template chooses what.

Our agent may send exactly one text message per call. The content is pre-written. The destination comes from configuration, not from the conversation. The number is one the caller confirms aloud, not the caller ID — because withheld numbers and wrong assumptions fail the same way. The model's only decision is whether to fire the tool.

This generalises directly to retail. An agent that issues a refund should choose which order from a list your systems supplied, never compose an amount. An agent that books a callback should pick from slots your rota exposes. An agent that applies a discount should select from codes that already exist. In every case the space of possible actions is enumerated by your systems, and the model picks from it.

The alternative — letting the model generate the parameters and trusting instructions to keep it sensible — is the architecture behind most of the agent failures that make the news.

Instructions are advice. Filters are enforcement.

Our guardrails run in two layers, and the distinction matters more than it sounds.

LayerWhat it isCan it be talked around?
Prompt instructionsWhat the model is told to do and avoidYes. Treat as advisory
Deterministic post-filtersCode that inspects the generated answer before it leaves — link allow-lists, content substitutionsNo. They run outside the model

Anything that must not happen is enforced in the second layer. A link the model invents is stripped by a rule, not discouraged by a sentence. And when retrieval finds nothing relevant, the system returns a written refusal without calling the model at all — you cannot hallucinate from a code path that never reaches the model.

Handing back to a human is a feature, not a failure

When the agent cannot answer, or the caller asks for a person, the call ends with a callback number captured and the transcript waiting in the team's inbox. A phone call cannot be put on hold and picked up tomorrow the way a chat can, so the handoff has to carry the context rather than the connection.

The failure mode of an ambitious phone agent is not a wrong answer. It is a customer who wanted a human, could not reach one, and does not call back — and that customer appears in no dashboard.

Acting in real time has a clock

Breakdown of one turn of a phone call: endpointing, rate-limit check, redaction, retrieval, generation and speech.
One turn of a real call, measured in production: 2.26 seconds from the caller falling silent to the first word back.

Worth noting what dominates it. The single largest remaining item is a rate-limit check — a database transaction protecting against abuse — not the language model. Retrieval is effectively free because it runs inside that check. The engineering that makes a real-time agent feel alive is mostly ordinary infrastructure.

One lesson on measurement, since it nearly misled us: our early calls reported about seven seconds, because the service had restarted between each one. We were measuring cold starts dressed as steady state. Warm, the same code answered in 3.5 seconds, and in region 2.26. If a vendor shows you a latency figure, ask what state the system was in.

What we would check before letting an agent act in a retail business

  • Is the action enumerated or generated? If the model composes the parameters, the design is wrong.
  • Is every action reversible, or approved above a threshold? Sending a message is cheap to undo. Refunding is not.
  • Is consent captured and logged where it is needed? Ours records the turn in which the caller agreed to be texted, against the send.
  • Can a human be reached, always? And does the handoff carry the context?
  • Does a deterministic test gate every deploy? Ours runs a fixed set of cases, including ones that must produce a refusal, on every release.

Next in the series: what we would not let an agent do in a retail business yet — and what would have to change first. Related: the agent that answers. All articles.

Start Your Data Transformation Journey

Discover practical, scalable solutions tailored to your business priorities.