Local AI coding agent: Claude Code, Codex, OpenCode setup

A local AI coding agent runs the model on hardware you control, so your code never goes to a model provider and there is no per-token bill. Serve an open-weight model with Ollama on your own computer or vLLM on a GPU server, then point Claude Code, Codex CLI or OpenCode at it. The cost is quality and speed: a model that fits on one consumer GPU is usually weaker at long, multi-file work than the strongest hosted models.

Use a local model with your agent

Ollama or vLLM?

Start with Ollama unless you already run GPU servers.

ollama pull qwen3

On a shared GPU server, with a Qwen3-Coder model:

vllm serve Qwen/Qwen3-Coder-30B-A3B-Instruct --served-model-name qwen3-coder --enable-auto-tool-choice --tool-call-parser qwen3_xml

The vLLM flags matter: without them, automatic tool calls are off and the agent can’t act. Full steps: vLLM.

Claude Code

With Ollama, run ollama launch claude. Or set the variables yourself, in PowerShell on Windows:

$env:ANTHROPIC_AUTH_TOKEN = "ollama"
$env:ANTHROPIC_API_KEY = ""
$env:ANTHROPIC_BASE_URL = "http://localhost:11434"
claude --model qwen3

On macOS or Linux:

export ANTHROPIC_AUTH_TOKEN=ollama
export ANTHROPIC_API_KEY=""
export ANTHROPIC_BASE_URL=http://localhost:11434
claude --model qwen3

For vLLM, set ANTHROPIC_BASE_URL to the server and map ANTHROPIC_DEFAULT_OPUS_MODEL, ANTHROPIC_DEFAULT_SONNET_MODEL and ANTHROPIC_DEFAULT_HAIKU_MODEL to your served model name; see vLLM.

Claude Code is built for Claude models, and Anthropic doesn’t support routing it to others. It works through these servers, but some features, such as built-in web search, don’t.

Codex CLI

With Ollama, run ollama launch codex, or:

codex --oss -m qwen3

For vLLM, add a custom provider to ~/.codex/config.toml with base_url = "http://gpu-host:8000/v1"; see vLLM. Codex only speaks the Responses API, which both Ollama and vLLM serve.

OpenCode

With Ollama, run ollama launch opencode, or add a provider to opencode.json:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Ollama (local)",
      "options": { "baseURL": "http://localhost:11434/v1" },
      "models": { "qwen3": { "name": "Qwen3" } }
    }
  }
}

For vLLM, use the same shape with baseURL set to http://gpu-host:8000/v1. LM Studio (http://127.0.0.1:1234/v1) and llama.cpp’s llama-server (http://127.0.0.1:9931/v1) work this way too.

The port gotcha: current llama-server listens on port 9931 by default. OpenCode’s docs and older guides still show 8080, the old default. Start it with --port 8080 or use 9931 in baseURL. More in OpenCode with local models.

Contenox

1. Open Backends → Providers.
2. Choose Ollama / Ollama Cloud under Local, or vLLM under Cloud.
3. Keep the default address for Ollama, or enter the vLLM server’s host and port only, without /v1.
4. Select Connect and wait for Connected.

The models appear in the Model chip. Connect a local server for routine work and a cloud provider for hard tasks, then switch per session. Download Contenox.

What hardware do I need?

Memory decides what you can run. The model’s weights plus its context must fit in GPU memory, or in unified memory on Apple silicon. If they don’t, part of the model moves to the CPU and responses slow down a lot.

• The context costs memory too. An agent sends instructions, tool definitions, files and command output every turn. Ollama’s docs recommend at least 64000 tokens of context for agents and coding tools, and a longer context needs more memory. Ollama’s default is often lower; see the context fix.
• Bigger models are more reliable. Developers report that models in the 7B to 9B range often fail at multi-step agent tasks. They work for questions and focused edits.
• CPU-only works for trying things out, but expect slow answers.

With Ollama, run ollama ps while a model is loaded. PROCESSOR shows whether it runs fully on the GPU, and CONTEXT shows the context length it got.

Which local models work for a coding agent?

The model must support tool calls; without them, the agent can’t read files or run commands.

• Ollama: filter the library to models with tool support.
• vLLM: the tool calling guide lists the model families it can parse tool calls for, and the parser flag each needs.

New open-weight models appear every few weeks. Try a candidate on a small, real task, such as fixing one failing test, before giving it larger work. See How to choose a model for coding.

Related

• Best Ollama models for coding: which models hold up in agent work
• Best Ollama harness: which agent to pair with a local model
• How to run local LLMs for coding: runtimes, model sizes and setup in depth
• Ollama: connect a model on your computer
• vLLM: connect a team GPU server
• How to choose a model for coding