Acceptance guide
What an AI agent pilot must prove
A pilot is worth paying for only if it ends with evidence. Before the build starts, agree one bounded task, the tools and data the agent may touch, a set of test cases with expected outcomes, which actions need human approval, what happens when something fails, and the cost per task you can afford. The pilot passes when the test cases pass, failures hand over cleanly and the cost stays inside budget.
Six things to agree before building
| Item | Example (synthetic) | Why it matters |
|---|---|---|
| Bounded task | Answer order-status questions and draft a delay apology | An agent with no boundary cannot be tested |
| Authorised tools | Read orders; draft emails; no refunds | Limits the damage of a wrong decision |
| Evaluation set | 40 real questions with expected answers, including 8 it should refuse | Turns "seems good" into a pass rate |
| Approval rules | Any money movement waits for a manager | Keeps consequential actions human |
| Failure handling | Tool down → say so, hand over, do not pretend | The most common real-world failure |
| Cost budget | Under an agreed cost per completed task | A pilot that is too expensive to run is not a success |
What the pilot report should contain
- Pass rate on the evaluation set, with the failures listed.
- How many cases were handed to a person, and why.
- Every tool call, so behaviour can be audited.
- Measured cost per task.
- A recommendation: run, change or stop.
No agent is right every time. A pilot should show how often it is wrong, how errors are caught, and whether that is acceptable for the task.
Red flags in a pilot proposal
- "Zero hallucinations" or guaranteed accuracy.
- No test set, only a live demo.
- Agents allowed to act on money or customer data without approval.
- Model and API costs left out of the estimate.
Related services
Questions
For one bounded workflow, typically two to four weeks after scope and data are ready.
Ready to talk through your project?
Send a voice note, spreadsheet or a plain description of where your business is struggling. We’ll help you find the right next step.

