Local AI guide

Warpdrive normally drives a coding agent through somebody's API — your Claude or OpenAI account, or a subscription. It does not have to. Open Settings, pick a model from a list that already knows what your computer can handle, and Warpdrive installs it and points its agents at it. The work happens on your hardware and costs nothing per question.

What this is#

An option, offered the same way the local gateway is — not something Warpdrive turns on behind your back. Choosing it does the fiddly parts of running a model locally: it finds an AI runtime you already have or installs one, downloads the model you picked, checks it answers, and writes the configuration that makes Warpdrive's sandbox agents use it. Nothing is installed system-wide, nothing needs an administrator password, and deleting one folder undoes all of it.

The models on offer are small — 1 to 14 billion parameters, roughly 1 to 9 GB on disk — and the one preselected for you is the largest that comfortably fits your machine. That is deliberate: a small model that answers in a few seconds on a laptop is more useful than a large one that swaps. It is not as strong as Claude or GPT-5; see known limits for an honest picture of the trade.

What your computer needs#

Warpdrive reads the machine before it offers anything — memory, cores, and whether there is a GPU worth using — and marks one model as the recommendation:

A graphics card helps a lot but is not required — these models run on the CPU. The runtime itself is a separate download: about 150 MB on macOS, larger on Windows and Linux because those builds bundle GPU libraries. Setup tells you the exact size and asks before downloading anything.

Set it up#

In the desktop app it is a settings screen with a dropdown — the same way you choose to run a local gateway. From a terminal it is one command. Both do exactly the same thing.

1. In the app

   Open Settings → Local AI. It reads your machine, offers the models that suit it — the right one is already selected — and Set up on this machine does the rest. Models too large for your computer stay listed but cannot be picked, so the reason is never a mystery.

2. Linux gateway app

   ./warpdrive-local-ai

   The Linux package has no desktop window, so setup runs from the folder you extracted the tarball into, beside warpdrive-gateway.

3. Alpha CLI — any platform

   warpdrive local-ai setup

   Add --yes to accept the download prompts, which is also what you want over SSH or from a script. warpdrive local-ai catalog --text prints the same model list the app's dropdown shows.

It says what it is about to download and how big it is, asks once, then runs unattended — a few minutes on a normal connection. When it finishes it names the model and the address it is answering on.

Using it#

Nothing else to switch on. New instances start with the local model selected, so the agent inside them works with no API key. Instances you already created keep whatever they were configured with.

Check what is running at any time:

warpdrive local-ai status

The runtime keeps running in the background after setup, and the gateway app starts nothing extra for it. If you reboot and want it back, run warpdrive local-ai start — or just run setup again, which is safe to repeat.

What setup actually does#

• Looks for an OpenAI-compatible server already listening on this machine: Ollama on port 11434, LM Studio on 1234, llama.cpp on 8080. If one answers, it is used as-is.
• Otherwise downloads the official Ollama archive, verifies it against the checksum published with that release, and unpacks it into ~/.warpdrive/local-ai/. No installer runs and nothing is placed outside that folder.
• Starts the runtime bound to 0.0.0.0:11434, which is what lets the sandbox containers reach it. Model weights go to ~/.ollama/models, shared with any Ollama you already had.
• Downloads the model, then asks it one question and waits for a real answer, so a broken setup fails here rather than in your first prompt.
• Registers the model as an OpenCode provider in ~/.warpdrive/opencode/global-config.json, which Warpdrive stages into every sandbox at boot. An existing default model is left alone unless you pass --set-default.
• Writes what it chose to ~/.warpdrive/local-ai/config.json.

Already have LM Studio or Ollama#

Then setup installs nothing. It detects the running server, uses a model you have already loaded (preferring a coding model when there is a choice), and does only the wiring step. Pass --no-install if you want it to fail rather than ever download a runtime.

Command reference#

Choosing a model#

The app's dropdown lists a curated set — small coding models and small general-purpose ones — each marked with its download size and whether your machine can hold it, with one marked as the recommendation. Anything you have pulled yourself appears there too. The default pick is a Qwen 2.5 Coder build: small, permissively licensed, and good at the kind of editing an agent does.

From the terminal, any model your runtime can serve works:

warpdrive local-ai setup --model=llama3.2:3b        # a general-purpose small model
warpdrive local-ai setup --model=qwen2.5-coder:14b  # bigger, needs ~10 GB free RAM

Bigger is not automatically better here: a model that does not fit in memory falls back to swapping and gets slow enough to be unusable. Start with the recommendation, and step up only if answers feel thin and your machine has room.

What leaves your machine#

During setup: the download requests for the runtime and the model. After that, if the agent is using the local model, the prompts and your code stay on the machine — there is no API call to anyone.

This is about the AI, not about everything Warpdrive does. Signing in, the marketplace, and a hosted gateway still talk to the network as they always did, and an agent you have separately configured with a cloud provider still uses that provider.

Troubleshooting#

Known limits#

• A 1.5–7B model is meaningfully weaker than a frontier model. It is good at small, well-specified edits and explanations, and it will struggle with long multi-file reasoning. Treat it as the free tier of your own hardware, not a drop-in replacement.
• The wiring targets OpenCode, the agent that accepts arbitrary providers. Claude Code and Codex authenticate against their own vendors and are unaffected.
• Speed depends on your hardware. Apple Silicon and machines with a discrete GPU are several times faster than an older CPU-only laptop.
• The Windows and Linux runtime archives are large (over 1 GB) because they carry GPU libraries. macOS is about 150 MB.
• Setup downloads from Ollama's official release and verifies the published checksum, but the runtime is third-party software with its own release cadence.