Hosted vs Self-Hosted AI Agents: What You Have to Own

Owning the model weights is the least useful thing you can own in a self-hosted AI agent. The hosted vs self-hosted AI agents argument gets held at that layer anyway — whose weights, whose GPUs — because it is the layer with a logo on it. OpenClaw crossed 300,000 GitHub stars by April running on a low-power Mac mini, and Gemini Spark does a near-identical job on virtual machines its users never see. Then the compliance review arrives and asks whether you can export a complete run trace, and which of a customer’s documents a given run touched. Neither answer turns on where inference happened.

What “self-hosted” actually means for an AI agent

Take the agent apart and you get six pieces, each of which can sit on either side of your network boundary.

The weights and the inference that runs them. The orchestration loop that plans steps and retries failures. The tool layer, which decides what the agent may call and holds the credential it calls with. The context store, holding your documents and their embeddings. The traces, meaning a record of every step the agent took and what came back. And the secrets.

A product can call itself self-hosted while pieces two through six run on your infrastructure and the model stays a commercial API call. That is still a real boundary. Prompt content crosses it, per call, under the vendor’s terms. Everything else stays put. Running the weights yourself is a separate and much larger decision, and treating the two as one question is how a hosting debate stalls without ever reaching something anyone can sign.

Before you compare products, write those six lines down and mark who holds each one.

Hosted vs self-hosted AI agents: what you give up either way

Hosted buys you the vendor’s integrations on day one and the vendor’s ops team forever. In exchange you inherit their retention policy, their subprocessor list, their model choice and their upgrade timing. Your agent’s behavior can change because someone else shipped on a Tuesday.

Self-hosted orchestration takes the opposite trade. Credentials stay in your vault and the agent gets its own scoped token instead of borrowing a vendor connector. Context and traces stay inside your network, so when a customer asks which of their documents a particular run touched, you answer from your own logs. You pin the model version and move when you are ready. You also own the upgrades and the on-call rotation for all of it.

The line that matters most to a buyer sits at the tool layer. An agent that acts needs a credential, and whoever holds that credential holds your blast radius. Most of that is API and integration work — internal endpoints behind scoped tokens with their own rate limits — whichever way the hosting question goes.

When a hosted agent fails a compliance review

Encryption is rarely the sticking point, because it is the one thing every vendor has already answered. The narrower questions are where a deal stalls:

• Can data be held in a named jurisdiction, and is that in the contract?
• Can you approve the subprocessor list before it changes?
• Can retention be set to zero, and can you verify it independently?
• Is the training opt-out contractual, or a settings toggle someone can flip?
• Can you pull a complete run trace on demand when a customer disputes an action?

Several of those are contract questions, not architecture questions. If you are building into fintech or healthcare, assume all five get asked in writing before anything goes live. What a contract cannot settle is the architectural half: data that may not leave a network at all, or traces at a level of detail the vendor never captured in the first place.

Send that list to the vendor before the pilot and get the answers into the agreement rather than a sales deck.

What running your own model weights actually costs

Reserved GPU capacity, kept warm enough to hold your latency budget, plus quantization work when the capacity you can afford won’t. An evaluation harness that is now entirely yours, with no provider benchmark to lean on. Upgrades you schedule, test and roll back yourself. Nobody to escalate to at 3am when throughput collapses.

One thing comes back that is hard to buy anywhere else: frozen behavior. The model does not shift under a workflow you already validated. For an agent whose output passed a clinical or regulatory sign-off, that is worth real money.

Our scoping guide for agent builds puts self-hosted and private-cloud models in the top tier, from ₹40L. That tier prices several agents sharing tools and policy, role-based access and retained audit trails, in an environment where data cannot leave. The self-hosted model is one line inside it, two tiers above a staff-facing agent running on a commercial API.

Run your own weights when residency is a legal hard stop, or when volume is high enough and the task narrow enough that a small open model wins on arithmetic. Otherwise rent.

How to set up an agent when the data is regulated

Own the boundary and rent the intelligence.

The model sits behind an adapter with a narrow interface — generate and embed — so changing provider is a config edit plus a re-run of the evaluation set. Retrieval and conversation history live in a store you operate. Secrets live in your vault, and the agent gets its own identity with an allowlist naming the routes it may touch. Redaction runs before anything leaves the network, and the trace is written to storage you control before the vendor sees a token.

Built that way, the hosted model is a replaceable dependency and your compliance posture rests on the layer you run. It also keeps the custom versus SaaS decision reversible: start on a vendor’s API, move to open weights on your own hardware when the volume justifies it, and the only things that change are the adapter and the bill.

Should you self-host your AI agent?

Rent the model, own everything around it.

That gives up the connectors a hosted platform ships with, so you build the integrations yourself and the first agent takes longer to do anything useful. It also accepts that the provider can change model behavior without asking you, which is why an evaluation set of real cases with known-good outcomes belongs in the first build and gets re-run on every provider change.

The credential is the part we would not trade. An agent acting inside your systems on a token you do not hold is a boundary nobody can describe to a risk reviewer, and the scoping week of an AI agent engagement exists to settle which systems the agent touches and what it must never do alone. Fix that as the constraint and the hosting question answers itself: rented model, your infrastructure, your logs, your token, and a slower start bought in exchange.