An AI rollout changes what counts as a correct answer. A demonstration may be judged on a chosen response. A service has to select the right record, use its latest status and take an appropriate action repeatedly, including when information is missing.
That is one useful way to examine the gap between an impressive AI demonstration and a dependable service. The question is which decisions require language judgement, and which should come from the system running the transaction.
The Breakdown
In a May 29, 2026 engineering account, Swiggy describes an early cancellation-support design that asked a language model to identify the stage of a delivery delay. It produced hallucinations. The team moved that decision to its order-timeline API, while retaining the model for recognising the broader customer intent.
The team rejected historical human-agent disposition codes as a direct training signal because people had context unavailable to the bot. It describes examining over 100,000 chats and maintaining about 500 representative test queries. This is a first-party engineering account, not an independent assessment of service quality.
Swiggy also says its bot kept trying to retain customers who repeatedly requested cancellation. A repetition threshold moved the conversation towards cancellation handling. Following a system’s goal can still frustrate its user.
Seeing a run is different from testing its result
LangChain’s State of Agent Engineering survey, published on June 12, 2026, illustrates a measurement gap. Among respondents, 89% reported some form of observability, while 52.4% ran offline evaluations and 37.3% ran evaluations in production. Observability records what a system did; evaluation tests whether the result met a defined requirement.
The public survey collected 1,340 responses between November 18 and December 2, 2025; 63% came from technology. It is a vendor-run, self-selected sample, with no India-specific result or representative geographic breakdown supplied. It cannot tell us what proportion of Indian AI projects fail.
For a deployment team, the practical implication is to test the information available at the moment a decision is made. A historical answer may look correct because a reviewer knows what happened later. The live system must decide without that hindsight.
A business action needs its own checks
Razorpay’s March 2026 description of Agent Studio offers a separate implementation example. It says offers come from merchant-configured discounts and that a platform validation layer checks actions before execution. These are the supplier’s stated controls; the page does not independently demonstrate their effectiveness or publish an error rate.
For an Indian merchant or enterprise buying an assistant, that distinction creates concrete acceptance tests. Can it recognise a request when the customer changes their mind? Does it fetch the current transaction state? What happens when the underlying service is unavailable? Can the workflow stop or hand over without improvising an answer?
These tests concern behaviour across a working process. They sit alongside access controls, which MM covered in its CERT-In advisory explainer, rather than replacing them.
The Crunch
A production test should examine the conversation, the live facts and the action that follows. Fluency helps the interaction. Reliability depends on keeping those parts connected without asking the model to invent missing state.