Oil India is seeking a generative-AI system for geological and geophysical analysis. Published in August, the tender closes on October 22, 2026.
The acceptance question
September clarifications identify factual accuracy above 95% and hallucinations below 5% as intended acceptance thresholds. Component test sets, ground truth and measurement methods are to be agreed before Phase 2.
Several component targets remain indicative. Evaluation will consider documented limitations in OIL’s input data. A phase acceptance certificate is required before invoicing.
The specification requires support for human review and override of agent outputs.
These are requirements for a proposed system. Delivered performance remains unverified.
Why a percentage needs context
The US National Institute of Standards and Technology’s July 2024 Generative AI Profile offers useful context. Its voluntary guidance recommends evaluating systems in conditions similar to deployment. It cautions against projecting broad capabilities from narrow or anecdotal tests.
The testing sample therefore matters. A benchmark can establish how a system answered particular questions; buyers still need to establish whether those questions represent the work their staff will give it. NIST also recommends comparison with known reference answers and verification of citations, including during ongoing monitoring.
For a buyer, this suggests three practical questions: what records define a correct answer, who checks the output, and what happens when the evidence is incomplete? A score without those definitions offers limited guidance about operational reliability.
The Crunch
AI acceptance needs a repeatable test that reflects the work the customer wants done.