TransIT AI
Sign up

AI agent

Use local models (Ollama & self-hosted)

Bring Your Own Model (BYOM) runs the Transit agent on models you host — Ollama on your own machine, or a self-hosted / OpenAI-compatible endpoint you control. The conversation never touches Transit’s cloud; the policy gate, per-command approval dialog, and scrollback redaction apply exactly as they do with cloud models.

Local models are included on the Hobbyist tier ($9/mo — local models plus BYOK cloud models on your own key) and on every higher tier. Like any AI feature they require a signed-in account with an active subscription — a signed offline pass keeps them working for up to 7 days without connectivity.

This page walks the recommended setup: pick a model that fits your hardware, run it in Ollama, then turn it on in Transit.

What you’ll need

  • A Transit account on a paid tier (Hobbyist or above), signed in.
  • Ollama installed on the machine running Transit.
  • A tool-capable model. The agent works by calling tools (list_open_sessions, read_scrollback, propose_command), so a model that can’t do function-calling won’t appear in Transit’s picker. 8B parameters and up from a recent tool-tuned family (Qwen 3.x, Gemma 4, gpt-oss) is a good floor.

Step 1 — Pick a model for your hardware (recommended)

whichllm detects your hardware and ranks local models by real benchmarks and fit — a better starting point than guessing by parameter count. It’s a small Python tool, so install uv (Astral’s Python manager) first:

# 1. Install uv
curl -LsSf https://astral.sh/uv/install.sh | sh

# 2. Run whichllm (one-off), or `uv tool install whichllm` to keep it
uvx whichllm@latest

It auto-detects Apple Silicon, NVIDIA, AMD, Intel, or CPU-only and prints a ranked list with quality score and tokens/sec, e.g.:

#1  Qwen/Qwen3.6-27B      27.8B   Q5_K_M   score 92.8   27 t/s

Two things to keep in mind when reading its output:

  • whichllm ranks for general quality and speed, not tool-calling. Pick a result from a tool-capable family; you’ll confirm capability in Step 3.
  • whichllm prints Hugging Face model IDs, not Ollama tags — the two don’t always match, so you map to an Ollama tag in the next step.

Step 2 — Pull the model in Ollama

Install Ollama and make sure it’s running (the desktop app starts a local server on 127.0.0.1:11434 automatically). Then map the recommendation to its Ollama tag and pull it. For example, the Hugging Face model google/gemma-4-26B-A4B-it is gemma4:26b-a4b-it-qat on Ollama:

ollama pull gemma4:26b-a4b-it-qat

The model downloads to disk now and loads into memory on its first use, so your first message in Transit has a one-time load delay.

Step 3 — Confirm it supports tools

Transit only lists models that report the tools capability. Check before you rely on one:

ollama show gemma4:26b-a4b-it-qat

Look for tools in the Capabilities section. If it’s absent, the model can’t drive the agent and Transit will hide it — pick a different model.

Step 4 — Turn it on in Transit

  1. Open Settings → Local models and toggle it On. Transit detects your running Ollama and lists your tool-capable models.
  2. In the chat model picker, choose your model under the Local — your hardware group (shown as e.g. gemma4:26b-a4b-it-qat · local).
  3. Start chatting. Proposed commands still require your explicit approval in the modal, exactly as with cloud models.

If responses are very slow, lower the context window under Settings → Local models — larger windows use more memory. Transit refuses a conversation that would overflow the window rather than letting the model silently forget its instructions.

Custom and remote endpoints

On any paid tier you can also point Transit at a remote Ollama server or an OpenAI-compatible endpoint on your own infrastructure — vLLM, LM Studio, llama.cpp’s server, or LiteLLM. Add one under Settings → Local models → custom endpoints. Off the local machine, HTTPS is required (verified against your OS trust store, so an internal CA works), and you can attach a bearer token that’s stored write-only in your OS keychain and sent only to that endpoint.

What stays private

Conversations with a local model — your prompts and the model’s output — are processed entirely on hardware or an endpoint you control and are not sent to Transit or to any cloud AI provider. Only the account, sign-in, entitlement, and billing information needed to enable the feature reaches Transit.

One caveat: Ollama can also run cloud-hosted models tagged :cloud (and -cloud), which proxy inference through ollama.com. Those aren’t local, so Transit deliberately hides them from the picker — local means local.