Scoping Generative AI Development Services Honestly
Author : James Smith | Published On : 14 Oct 2026
We have commissioned Generative AI Development Services four times. One is in production and genuinely useful. One works and nobody uses it. Two never reached a customer.
The difference was visible in the scoping conversation, before anybody wrote any code. Here is what we now insist on hearing.
Ask what happens when it is wrong
The first question, and the one that eliminates most proposals.
Every generative system is sometimes wrong. The question is what that costs, who notices, and how quickly it can be corrected.
Our successful build drafts responses for a person to review. Being wrong costs that person thirty seconds. One of our failures sent output to customers directly, where being wrong cost an apology and a correction.
Firms offering Generative AI Development Services that cannot answer this in the first meeting are selling a demonstration rather than a system.
Insist on grounding, not recall
The design decision that separates useful from impressive.
A system answering from what a model remembers will sometimes invent a confident, plausible, wrong answer. A system answering from your own documents, with the source shown beside the answer, can be checked in seconds.
Our production system shows the paragraph it drew from on every response. People trust it because they can verify it, and that verification takes less time than writing the answer themselves.
Measure the current process first
Nobody can tell whether a build succeeded without a baseline.
Before any work starts we measure how long the task takes today, how often it happens, what accuracy looks like, and what errors currently cost.
Our unused system saved perhaps eleven minutes a month per person, which nobody had calculated beforehand. It worked exactly as specified and the specification was not worth building.
Start where somebody already checks the work
The lowest risk place to begin.
Our successful build went into a process that already had a reviewer. Putting a draft in front of that person changed nothing about the control and simply made their job faster.
That meant the first deployment carried almost no risk. We learned how the system behaved on real cases before removing any supervision, and we removed it gradually.
Expect most of the effort to be data work
The scoping surprise in every engagement.
Our production build spent about seven weeks on retrieval. Which documents, how they are chunked, how permissions are respected, how stale content is identified, and what happens when nothing relevant exists.
The model work was perhaps a fortnight. Anybody quoting a generative build as mostly model work has not done one.
Connect it to the systems that hold the answers
Grounding depends on reach.
Our build draws on documents in SharePoint, records in our line of business systems, and reference data from our warehouse. Getting those connected, with permissions inherited rather than bypassed, was most of the integration effort.
Azure Data Engineering work ran alongside the build for that reason. The pipelines feeding the retrieval layer were built properly rather than assembled from exports, because an assistant answering from stale data is worse than no assistant at all.
Expect the integration to outlast the model
A point worth making to anybody budgeting for this.
Models change. Ours has been swapped twice since launch, each time in about a week, because the system was built so the model is a replaceable component.
What has not changed is the retrieval layer, the permission model, the logging, and the pipelines underneath. That work was most of the cost and all of it is still in service.
That data layer is maintained like any other production path rather than as part of an AI project that finished.
Write the evaluation before the build
Not afterwards, when the criteria will flex to fit the result.
We define what a good answer contains, what makes one unacceptable, and we collect two hundred real cases including ones with no valid answer. The system is scored against those, and correct refusal counts as success.
That document is written with the people who do the work today. Engineers writing their own success criteria will produce a system that passes them.
Fit it into an actual strategy
A collection of pilots is not an AI automation strategy, and most organisations have the former.
Ours now ranks candidate processes by volume, variation, reversibility and measured cost. High volume, low variation, fully reversible work goes first. Interesting problems wait.
That ranking is why we stopped two of the four builds. Both were interesting and neither would have paid back.
What we require before approving a build
- A clear answer on the cost of being wrong.
- Grounded retrieval with sources shown.
- A measured baseline from the existing process.
- A first deployment where somebody already reviews.
- Written evaluation criteria and real test cases.
- Permissions inherited from the source systems.
- A named owner and a switch that stops it.
Common questions
Who should own the pipelines afterwards?
The data team, as production paths. Our Azure Data Engineering group maintains ours rather than leaving them with a project that has finished.
How long does a first build take?
Ours took four months, with most of it on retrieval and integration rather than on the model.
Should we build or buy?
Buy where a product fits a general need. Generative AI Development Services are worth commissioning where the value depends on your own documents and workflow.
What does the data work involve?
Pipelines, permissions, freshness and chunking. Ours were built properly rather than from exports, because an assistant answering from stale records is worse than none.
Who should own the evaluation?
Somebody outside the delivery team. Criteria owned by the builders drift towards what the system already does.
Where does the data work sit?
With the data team. Generative AI Development Services can build the retrieval layer, and the pipelines underneath belong to whoever maintains your production data.
What is the most common failure?
Building something that works and saves nobody meaningful time. Measure the current process before committing.
What the four taught us
The two that failed were not technically worse than the one that worked.
They solved problems where being wrong was expensive, or where the time saved was too small to notice. Generative AI Development Services can build almost anything now, which makes deciding what is worth building the part that actually determines the outcome.
