Skip to content
ODOOVATIONShome

Acceptance guide

What an AI agent pilot must prove

6 min read·Published by ODOOVATIONS, October 2026

A pilot is worth paying for only if it ends with evidence. Before the build starts, agree one bounded task, the tools and data the agent may touch, a set of test cases with expected outcomes, which actions need human approval, what happens when something fails, and the cost per task you can afford. The pilot passes when the test cases pass, failures hand over cleanly and the cost stays inside budget.

Six things to agree before building

ItemExample (synthetic)Why it matters
Bounded taskAnswer order-status questions and draft a delay apologyAn agent with no boundary cannot be tested
Authorised toolsRead orders; draft emails; no refundsLimits the damage of a wrong decision
Evaluation set40 real questions with expected answers, including 8 it should refuseTurns "seems good" into a pass rate
Approval rulesAny money movement waits for a managerKeeps consequential actions human
Failure handlingTool down → say so, hand over, do not pretendThe most common real-world failure
Cost budgetUnder an agreed cost per completed taskA pilot that is too expensive to run is not a success

What the pilot report should contain

  • Pass rate on the evaluation set, with the failures listed.
  • How many cases were handed to a person, and why.
  • Every tool call, so behaviour can be audited.
  • Measured cost per task.
  • A recommendation: run, change or stop.

No agent is right every time. A pilot should show how often it is wrong, how errors are caught, and whether that is acceptable for the task.

Red flags in a pilot proposal

  • "Zero hallucinations" or guaranteed accuracy.
  • No test set, only a live demo.
  • Agents allowed to act on money or customer data without approval.
  • Model and API costs left out of the estimate.

Questions

For one bounded workflow, typically two to four weeks after scope and data are ready.

Ready to talk through your project?

Send a voice note, spreadsheet or a plain description of where your business is struggling. We’ll help you find the right next step.