How do you scope an AI proof of concept that ships in weeks?

01Why most AI pilots take too long and prove too little

AI pilots usually fail in one of two ways. Some try to prove everything at once: every file type, every integration, every user role. They take months, and by the time they are ready the business question has moved. Others prove the wrong thing: a model works in a notebook on clean sample data, and nobody learns whether the workflow works for the people who would use it. This is part of our larger guide, What is forward deployed engineering?.

The useful middle is a pilot that is small in scope but real in setting. It runs inside the existing product, uses the existing login and payment flows, and puts a new workflow in front of real users, even if it only handles the easiest version of the input.

This is not a new idea in machine learning. Google's long standing Rules of Machine Learning advise keeping the first model simple and getting the infrastructure right. Their reasoning is that the hard part is getting data into the system, defining what good and bad mean and integrating the result into the application. A pilot built on that principle proves the plumbing and the workflow first. Accuracy on difficult inputs can be improved once you know the workflow is worth it.

02How we scope a pilot

We start by naming the one question the pilot has to answer. Usually it is a version of: will users actually use this new workflow, and does it fit how they work? Everything else is secondary. Then we work through a fixed sequence:

1. 01

   Write the question and the success criteria.

   Specific and measurable, for example: agencies can publish metadata for a file in under a few minutes, and attorneys can find and request it. Vague goals such as "good accuracy" produce pilots that never end.

2. 02

   Pick the simplest input that still tests the workflow.

   If the workflow can be proven with plain text, start with plain text. Scanned documents, complex formats and OCR come later.

3. 03

   Reuse what the product already has.

   Payment, delivery, login, notifications and admin screens. A pilot that rebuilds these is not a pilot.

4. 04

   Put one ID on the whole lifecycle.

   One request, one identifier, from first action to final delivery. It makes the pilot measurable and auditable from day one.

5. 05

   List what is deferred, in writing.

   Formats, integrations, roles and automation that are out of scope. This list protects the timeline more than any plan.

6. 06

   Agree how the second phase will be decided.

   What pilot feedback would justify investing in the harder inputs, and what would stop the work.

The table below shows how we sort scope into a first phase and a later phase.

03What we see in production

On a national marketplace for legal transcripts, connecting court reporting agencies and attorneys, the client wanted to move to a metadata only workflow. Attorneys would find transcripts through metadata, with nothing visible before the agency approved and the attorney paid. The platform already existed, with upload, search, request, payment and download flows.

We scoped Phase 1 as a small pilot. It handled plain text transcripts and front matter extraction only. PDF, DOCX, scanned documents, OCR, archives and integrations were deferred. Phase 1 reused the existing payment and delivery flows and tied the whole lifecycle to one Request ID, so every step from search to delivery could be tracked. It was built and delivered within a few weeks in June 2026, and the client could test the new workflow on the existing product before investing in the harder formats.

Once the workflow was proven, Phase 2 added the harder parts. Metadata extraction moved into the agency's browser, so the agency selects a file rather than uploading it, and only approved metadata reaches the platform. Extraction covered plain text, DOCX, PDF and scanned documents, with OCR that reports confidence per field. We also added end to end encrypted delivery, so the platform cannot read transcripts at any point.

By late July 2026 every workflow item on the client's own Phase 2 checklist was checked as done, and the platform moved into a controlled pilot with agencies and attorneys. The client wrote that metadata extraction accuracy had improved significantly. Pilot feedback then raised privacy concerns about open search, and we proposed replacing it with counsel of record matching and private requests routed to agencies. That is exactly the kind of learning a pilot should produce: it changed the plan for the next phase.

Two earlier projects, both delivered through a partner agency for its clients, show the same principle from other angles. On a building sustainability simulation platform, the client's simulation engine was not ready to deploy. Rather than wait, we designed the first phase around uploaded simulation result files, so the platform could ship and be tested with real data while the engine matured. That first phase reached production in about ten weeks from kickoff.

On a new password manager product, the riskiest question was key management, so the technical lead ran a series of security proofs of concept on encryption and key exchange before any feature code was written. Work on that product stopped after the first feature sprint, but the risky decisions had been tested first, which is the right order.

That first phase was also deliberately narrow: one 3D model format, one baseline code and results uploaded by an admin. Self registration and the other model formats moved to a later phase, which kept a dense domain shippable.

04The trade-offs of a narrow pilot

A narrow scope is not free. Plan for these trade-offs openly:

• Users will ask for the deferred items. That is good news. Track the requests: they become the evidence for the next phase.
• The simplest input may not represent the hardest cases. Be honest in the pilot report about what was not tested.
• Reusing existing flows can expose old limits. A payment or delivery flow designed for one model may need small changes for the new one. Budget a little time for it.
• Pilots can drift into production by accident. Decide in advance what has to be true before more users are added, such as security review, audit coverage or a cleanup of the demo environment.

For a view of what production AI costs once a pilot succeeds, see how much production AI costs in SaaS. Your team may be tempted to skip the pilot and generate the whole feature with AI coding tools. Our article on why AI generated code is not enough for brownfield SaaS explains where that approach breaks.

05A short checklist for your next AI proof of concept

Before the build starts, check that you can answer each of these in one sentence:

• What single question will this pilot answer?
• Who are the real users, and how many will try it?
• What is the simplest input that still tests the workflow?
• Which existing flows will we reuse instead of rebuild?
• What is explicitly out of scope?
• How will we track each request from start to finish?
• What result would justify a second phase, and what result would stop it?

If the answers fit on one page, the pilot can usually ship in weeks. If they do not, the scope is still too wide.

Frequently asked questions