Practice chats must test an AI agent’s fact checks

Denise Baptiste checked the sugar amount on two packages after a customer called about receiving a different brand. Baptiste, a customer-service representative at Elm Cart, initially thought the complaint was about price. The customer was concerned about the substitute for a mother with diabetes. Baptiste said the amounts did not match, so she told the customer not to use the substitute and requested a correction.

Nubank says it uses synthetic customers and simulated tool responses to screen changes to its support agents before live tests. An arXiv paper by researchers affiliated with Nubank and its simulation provider reports that simulated and production evaluation scores tracked closely across four deployed versions. In a separate live test, Nubank says a model selected after more than 16,000 simulated conversations increased self-service by 8.82 percentage points without a statistically significant change in customer satisfaction. Those are Nubank’s results, not a release rule for every support team. Three people who handle customers told Bonzai that a simulation should test whether an agent checks the facts before answering, while a live test must show what the practice conversation could not.

Baptiste would require an agent to ask for the package’s sugar amount, compare it with the ordered item and help correct a mismatch rather than confidently declare the substitute safe. Kenji Moreau, owner and repair technician at FixPoint Repair, would require an agent responding to a black-screen complaint to ask whether the phone rings, vibrates or shows anything when plugged in, then recommend testing power before naming a replacement screen. Moreau said a screen replacement can cost $180 to $320. Priya Desai, owner at Ember & Bean, would require an agent to explain the shop’s two-hour weekday seating limit and the lunchtime seating problem accurately, acknowledge a frustrated regular without blame and bring Desai in rather than argue.

Each also sees a gap between a scripted exchange and a customer in front of them. Baptiste said practice might miss the moment a complaint about a different brand becomes a concern about diabetes. Moreau said a customer’s description of a dead screen could mean several things; at the counter, Moreau can plug in the phone and check it. Desai said a regular might go quiet or leave earlier than usual without objecting to the seating rule. “A practice conversation can get the words right, but it doesn’t always catch that pause,” Desai said.

In a live test, Baptiste would check whether the order correction went through and whether the customer called again about the substitution. Moreau would watch for an agent recommending a screen too soon or giving unsafe battery advice, and would stop it for either. Desai would listen in with longtime regulars, watch whether the agent noticed someone becoming upset or quiet and stepped aside when needed, and pay attention to whether customers still felt comfortable returning.

For all three, a simulated conversation could clear an agent to try a live test only after it showed the right checks and response. The live test would still have to show whether that answer held up when a customer’s concern changed, a device needed checking or a regular went quiet.

Baptiste would start with the order number and look for another call about the substitution. “If a customer has to call back and explain the sugar amount all over again, the first answer didn’t do the job, no matter how polite it sounded,” Baptiste said.