Brief
Most AI pilots die the same way: no data path, no evaluation set, no owner after the workshop. Here is the failure pattern and the engineering sequence that breaks it.
A team picks a flashy use case. A vendor or internal lab builds a demo on clean sample data. Leadership applauds. Then someone asks for production access to the real system of record — messy permissions, partial history, conflicting IDs — and the project stalls.
Or it ships into a corner of the product with no evaluation set, no regression gate, and no named owner for model quality after the launch party. Six weeks later accuracy drifts, operators stop trusting it, and the ‘AI program’ becomes a cautionary slide.
This is not a model problem first. It is an engineering and operating-model problem.
Data path: who owns access, retention, and quality for the inputs the model needs?
Acceptance cases: which examples must pass, including the awkward ones operators already know?
Failure policy: what happens when the system is unsure — escalate, abstain, or block?
Ownership: which team watches quality and latency after go-live, and who can pause it?
Integration: does the capability sit inside the workflow people already use, or in a side chat nobody opens twice?
Understand the live estate and the constraint that decides what is safe to change.
Assess feasibility with representative data — not sanitized demos.
Engineer narrow interfaces and human-reviewed actions before wide write access.
Evaluate on every material change. Ship with a rollback and a pause switch.
Hand over runbooks, tests, and decision records your team can own without the partner in the room.
If your AI initiative cannot survive a question about evaluation, data, and ownership, it is not ready for production — and expanding the pilot budget will not fix that.
AI engineering starts where the demo ends.