TransIT AI
Sign up

Blog · July 22, 2026 · Knox Hutchinson

Why We Made the AI Run Local — and What Never Leaves Your Network

In June we explained why the AI in Transit is investigation-only — four abilities, a per-vendor permit list, your click on every command, all enforced by architecture rather than policy. That post answered the first question a network engineer asks about AI on production gear: can it change anything? No.

This post answers the second question: where do my bytes go?

As of v3.0, there’s a new answer available: nowhere. The agent can run entirely against a model you host — Ollama on the machine in front of you, or an OpenAI-compatible endpoint on your own infrastructure. No API calls to us. No API calls to Anthropic or OpenAI. Device output, prompts, and the model’s answers stay on hardware you control.

The Other Half of the Trust Problem

For a lot of teams, “your AI requests go to Anthropic or OpenAI through our proxy, and our metering stores no prompt or completion content” is a satisfying answer. We built that pipeline carefully and documented it in our security model.

For another set of teams, it’s disqualifying — and no amount of careful engineering on our side changes that. Defense and government networks. Carrier NOCs. Regulated environments with data-residency language in their contracts. Labs that are air-gapped on purpose. Security teams whose AI policy is one sentence long: nothing leaves.

Those shops don’t need a better cloud story. They need the cloud out of the loop entirely.

Bring Your Own Model

BYOM ships in two shapes:

  • Ollama on the machine running Transit. Zero config: Transit detects the local server and lists your models in the chat picker under Local — your hardware.
  • An OpenAI-compatible endpoint you control — vLLM, LM Studio, llama.cpp’s server, LiteLLM — running wherever your infrastructure lives.

One requirement: the model has to support tool calling. The agent works by calling tools — list sessions, read scrollback, propose a command — so a model that can’t do function-calling can’t drive it. Transit checks for the capability and hides models that don’t report it, so you can’t pick one that will fail confusingly halfway through an investigation. As a practical floor, recent tool-tuned families at 8B parameters and up work well; the setup guide walks the whole flow, including picking a model that fits your hardware.

What Leaves Your Network (and What Doesn’t)

In local mode:

  • Your prompts — processed on your hardware. Never sent to Transit.
  • The device output the agent reads — processed on your hardware. Never sent to Transit.
  • The model’s answers — generated on your hardware. Never sent to Transit.
  • Anthropic and OpenAI — not contacted. There is no provider API key anywhere in the local path, ours or yours. Nothing to send; nothing to leak.

What does reach us: sign-in, entitlement, and billing — the account plumbing that turns the feature on. That’s the whole list, and it carries no conversation content.

Local Means Local: The :cloud Trap

Here’s a detail we care about more than we probably had to.

Ollama can also serve models tagged :cloud (or -cloud). Select one and your inference quietly proxies through ollama.com’s hosted service. That’s a legitimate Ollama feature — but it is not local, and a model picker that mixes “runs on this machine” and “leaves through ollama.com” under one Local label would break the exact guarantee this feature exists to make.

So Transit hides :cloud models from the picker entirely. Not a warning icon. Not a tooltip. Hidden. If a model appears in Transit’s Local group, inference happens on hardware you control.

Local means local.

Nothing Relaxes in Local Mode

Everything from the investigation-only post applies to local models unchanged:

  • The AI still has exactly four abilities — list, read, propose, ask.
  • Every proposed command still passes the per-vendor, default-deny permit list.
  • Every command still requires your explicit click.
  • Device output still passes redaction before the model sees it.

That last one surprises people: why redact for a model on your own machine? Because the enforcement lives outside the model, and it doesn’t branch on where the model runs. A local model is still a model — it can still be wrong, and device output can still try to prompt-inject it. The permit list and the approval click don’t care how the model was persuaded, and that’s as true on localhost as it is in the cloud.

Remote Endpoints Without the Foot-Guns

If your model server lives off-box, two rules apply:

  • HTTPS is required, and the certificate verifies against your OS trust store — so the internal CA your machines already trust works without any Transit-specific ceremony. There is no “skip verification” toggle.
  • Bearer tokens are write-only. If your endpoint needs a token, Transit stores it in the OS keychain, sends it only to that endpoint, and has no UI that reads it back out.

Offline, Honestly

Local models are a paid-tier feature, and your subscription is normally confirmed when you sign in. For disconnected work, that check is designed to travel: signing in mints a signed offline pass that keeps local models working for up to 7 days without any connectivity. The conversations themselves never need a network path to anything but your model.

If your environment stays disconnected for longer than a week at a stretch, the current design needs a periodic check-in. We’d rather tell you that plainly than imply air-gapped-forever and let you discover the boundary in the field.

What This Looks Like in Practice

You pull a tool-capable model, confirm it reports the tools capability, and flip Settings → Local models on. Your models appear in the chat picker under Local. Full walkthrough here.

Then it’s the same product. You SSH into the lab switch and ask why the BGP neighbor is down. The agent proposes show bgp neighbor — the permit list checks it, you click, the output is redacted and handed to your model, and the diagnosis comes back. The first message pays a one-time delay while the model loads into memory; after that it’s conversational.

Two honesty notes about running small models:

  • If responses crawl, lower the context window in Settings — bigger windows cost memory and prompt-processing time.
  • Transit refuses a conversation that would overflow the model’s context window rather than silently truncating it. With a small local model, silent truncation is the failure you don’t see — the model quietly forgets its instructions and gets confidently worse. We made it a visible error instead.

The Security Review Argument

If you need sign-off from a security team, local mode gives them a short list to verify:

  • Inference traffic goes to the local Ollama server, or to an HTTPS endpoint you designate, validated against the OS trust store.
  • No provider API key exists anywhere in the local path.
  • Prompts, device output, and completions are never transmitted to Transit or to any AI provider.
  • Ollama’s cloud-proxied :cloud models are structurally excluded from the picker.
  • The permit list, the approval click, and output redaction are identical to cloud mode.
  • The only Transit-bound traffic is sign-in, entitlement, and billing — no conversation content in any mode.

These aren’t configuration choices someone can drift away from. They’re how the local path is built.

Not Enterprise-Gated, on Purpose

The industry default is to put on-prem AI behind a “contact sales” wall. We went the other way: local models start on Transit’s first paid tier — Hobbyist, $9/month — and are included on every tier above it.

The people who most need local inference are frequently the least “enterprise”: the homelabber with a gaming GPU, the two-person NOC inside a company whose IT policy vetoes cloud AI, the engineer who wants to prove the concept on a Tuesday without a procurement cycle. Gating the privacy story behind enterprise pricing would have been exactly backwards.

The terminal is a free download; pricing for the AI is public.

Set up local models in about ten minutes →

Frequently Asked Questions

Does Transit see my prompts or device output when I use a local model? No. In local mode, prompts, device output, and the model’s answers are processed entirely on your hardware or an endpoint you control. Only sign-in, entitlement, and billing traffic reaches Transit — no conversation content.

Can the agent run in an air-gapped environment? Conversations with a local model need no connectivity beyond your own model server. Subscription entitlement is confirmed by a signed offline pass that lasts up to 7 days without connectivity, so a fully disconnected site needs a periodic check-in — that’s the honest boundary of the current design.

Which local models work with Transit? Any Ollama model — or OpenAI-compatible endpoint model — that supports tool calling. Transit hides models that don’t report the tools capability. Recent tool-tuned families at 8B parameters and up are a practical floor; the setup guide covers picking one that fits your hardware.

What are Ollama’s :cloud models, and why does Transit hide them? Models tagged :cloud or -cloud proxy inference through ollama.com’s hosted service — they run in Ollama’s cloud, not on your machine. Because that would silently break the “nothing leaves” guarantee, Transit excludes them from the model picker entirely. Local means local.

Do the safety guardrails change when I use a local model? No. The four-ability limit, the per-vendor default-deny permit list, the mandatory approval click, and output redaction all run identically in local mode. The enforcement lives outside the model, so it doesn’t matter where the model runs.

Can I point Transit at a model server on another machine? Yes — any OpenAI-compatible endpoint (vLLM, LM Studio, llama.cpp, LiteLLM). Off-box endpoints require HTTPS, verified against your OS trust store (an internal CA works), and bearer tokens are stored write-only in your OS keychain and sent only to that endpoint.

Which plan includes local models? Every paid tier, starting with Hobbyist at $9/month — which also includes BYOK, running Transit’s standard cloud models on your own Anthropic or OpenAI key. Bundled cloud usage starts on Operator.