The AI Deployment Last Mile Costs More Than the Model

Model choice is the smallest decision you will make about an AI feature, and it is the one that will eat the most meetings. The AI deployment last mile — everything between a model that scores well on your test data and a model your staff actually use while a customer is waiting — is where the money goes and where projects quietly stop. Swapping one model for a better one changes one adapter and one eval run. Building the path that puts its output in front of a person who can act on it or override it, with a record of which they did, is the actual build.

A benchmark score says nothing about that gap. It is measured on data you chose, in an environment with no users and no audit trail.

What the AI deployment last mile actually includes

Somewhere to run inference at a known cost per call. Every output carries a version stamp, so when a decision is questioned you can say which model produced it. The eval set runs in CI and fails the build when quality drops. A platform engineer can roll the whole thing back alone, without waking a data scientist. There is a screen showing the inputs the model used, because someone at your end will have to explain one specific output to one specific customer. Anything below a set confidence threshold routes into a human queue. And a boundary, written down and enforced in code, where personally identifiable data stops.

That list is ordinary application and infrastructure engineering. Machine learning does not appear in it. It belongs in the same estimate as the model itself, priced line by line so you can see what the operating half costs.

Why building your own model stopped being the expensive option

Pre-training absorbed the part that used to be unaffordable. Someone else paid for the enormous corpus and the compute run behind it. What is left for you is adaptation to your own material — your claim codes, your patient population, your defect photographs, your support history — and that adaptation runs against records you already hold.

A vendor’s one-size-fits-all tool used to be the only affordable route to a working model. Now it is one option among several, and it competes against a version tuned to how your operation actually behaves.

Whether you adapt weights or retrieve your own documents at query time is a separate decision, and RAG vs fine-tuning sets out where each earns its keep. The last mile is identical either way. Retrieval needs an index that stays current with the source systems; fine-tuning needs a retraining path someone will still remember how to run. Both need the eval set.

Does an AI platform lock you in?

Buy the environment you build in and the lock-in does not disappear. It moves up a layer. You stop being stuck with a vendor’s product roadmap and start being stuck with their runtime, their deployment model and their per-inference pricing.

The architecture that survives a platform change is dull and specific. Model calls sit behind an interface your code owns. Prompts and evaluation sets live in your repository, under version control, where a reviewer can see what changed between two releases. Custom AI vs SaaS AI works through the same trade at the product level. Get that boundary right and swapping provider is a rewrite of one adapter, not of the product.

Who owns the model after launch?

A purchased tool comes with somebody else’s on-call rotation. A model tuned on your data comes with yours.

Data drifts. A new scanner in the imaging suite, or a new set of codes from the regulator, and accuracy decays without anything visibly breaking. A model that has degraded throws no stack trace and returns no error code. It starts being wrong at a rate your team absorbs quietly, until someone senior notices the numbers have stopped matching reality.

What makes that survivable is a schedule. The eval set runs on every deploy and again on a cron. The human queue is where your next training set comes from — every decision a reviewer makes there is a labeled example you would otherwise pay for. Budget those reviewer hours into the running cost from the first month, because that line does not go to zero after launch.

What the deployment layer costs

Treat it as a line item rather than overhead. A Tier 2 production set — everything in Terraform, a staging environment matching production, deploys with rollback, metrics and alerting, and a restore drill you watch succeed — sits at ₹4L – ₹6L in our cost of DevOps and cloud setup guide. That is before the model-specific pieces: the evaluation harness, the trace store, the review interface. Indicative for a scoping conversation; the real figure comes out of paid discovery against a written scope, and is fixed before any code is written.

Set that against a notebook that scores well on held-out data. It tells you almost nothing about whether the work can run in production. The pilot answers whether a model can do the job at all; the last mile answers whether your organization can operate it. Ask for both in one estimate, split into two figures, so nobody can approve the first half and discover the second.

How to scope the last mile before you commission a model

Put these in writing before anyone picks a model:

• Which decision does this change, and who makes that decision today?
• What does the system do when it is not sure?
• Which single number tells you it is working, and which screen shows that number?
• What would make you switch it off?

An off switch with no named owner is not an off switch. If nobody can say who stops the model and on what evidence, it stays live by default, and the answer to the first question is nobody.

If the data does not exist yet and there is no realistic path to collecting it, the honest recommendation is to instrument the workflow first and commission the model once the data exists. That is why “there is no data yet and no way to collect it for now” appears on our AI-integrated software page under the reasons not to hire us.

We would take a narrower model with a complete deployment path around it over a stronger model with none. You give up accuracy at launch and you will hear about it in the first review meeting. What you get back is a system your own team can measure and correct without the people who built it in the room. That is the difference between a feature you own and one you rent.