vLLM coding agent: Claude Code, Codex, OpenCode setup

A vLLM coding agent uses a model you serve yourself with vLLM, usually on a GPU server your team shares. vLLM speaks the OpenAI APIs and the Anthropic Messages API, so Claude Code, Codex CLI and OpenCode can all use it with just the server address. The step people miss: vLLM does not turn on automatic tool calling by default. Start it with --enable-auto-tool-choice and the --tool-call-parser that matches your model.

Use vLLM with your agent

Start the server

vllm serve Qwen/Qwen3-Coder-30B-A3B-Instruct \
  --served-model-name qwen3-coder \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --api-key YOUR_KEY

--served-model-name gives the model a short name. Claude Code needs it: names with a slash, like the Hugging Face path, don’t work there.

The server listens on port 8000. Check it from the machine where the agent runs. In PowerShell on Windows:

curl.exe http://gpu-host:8000/v1/models -H "Authorization: Bearer YOUR_KEY"

On macOS or Linux:

curl http://gpu-host:8000/v1/models -H "Authorization: Bearer YOUR_KEY"

If that returns your model, the agent can reach it.

Which tool-call parser does my model need?

The parser must match the format the model writes its tool calls in. From the vLLM tool calling guide:

The guide lists more families, and some need a --chat-template file from vLLM’s examples. A wrong parser often fails quietly: the model describes the tool call in plain text instead of making it.

What else should I set when starting vLLM?

• --api-key (or the VLLM_API_KEY environment variable): requires a key on /v1 requests. Anyone who can reach an open server can use it.
• --max-model-len: the context length. Agents send long prompts, so give it as much as your GPU memory allows. If you leave it out, vLLM takes the value from the model’s config.
• --host and --port: where the server listens. The port defaults to 8000.

Claude Code

Point Claude Code at the server and map all three model tiers to your served name. In PowerShell on Windows:

$env:ANTHROPIC_BASE_URL = "http://gpu-host:8000"
$env:ANTHROPIC_API_KEY = "YOUR_KEY"
$env:ANTHROPIC_AUTH_TOKEN = "YOUR_KEY"
$env:ANTHROPIC_DEFAULT_OPUS_MODEL = "qwen3-coder"
$env:ANTHROPIC_DEFAULT_SONNET_MODEL = "qwen3-coder"
$env:ANTHROPIC_DEFAULT_HAIKU_MODEL = "qwen3-coder"
claude

On macOS or Linux:

ANTHROPIC_BASE_URL=http://gpu-host:8000 \
ANTHROPIC_API_KEY=YOUR_KEY \
ANTHROPIC_AUTH_TOKEN=YOUR_KEY \
ANTHROPIC_DEFAULT_OPUS_MODEL=qwen3-coder \
ANTHROPIC_DEFAULT_SONNET_MODEL=qwen3-coder \
ANTHROPIC_DEFAULT_HAIKU_MODEL=qwen3-coder \
claude

vLLM’s own example uses dummy for both keys on a server started without --api-key.

The slow-every-turn trap: newer Claude Code builds send a per-request attribution header that can defeat vLLM’s prefix caching. vLLM versions above 0.17.1 handle it. On older ones, set CLAUDE_CODE_ATTRIBUTION_HEADER to "0" in the env section of ~/.claude/settings.json. See prompt caching.

The web search gotcha: Claude Code’s built-in web search fails with tool type 'web_search_...' is not supported, because vLLM rejects that server-side tool. Add a web search MCP server instead. And since Anthropic doesn’t support Claude Code with non-Claude models, other features may break the same way.

Codex CLI

Add a provider to ~/.codex/config.toml. Codex ignores provider settings in a project’s .codex/config.toml, so it has to be the user-level file.

model_provider = "vllm"
model = "qwen3-coder"

[model_providers.vllm]
name = "vLLM"
base_url = "http://gpu-host:8000/v1"
env_key = "VLLM_API_KEY"

Set the key in PowerShell with setx VLLM_API_KEY "YOUR_KEY" and open a new terminal, or on macOS or Linux with export VLLM_API_KEY="YOUR_KEY". Then run codex.

Codex sends requests in the Responses API format only; it’s the only wire_api value Codex accepts. vLLM serves it at /v1/responses.

OpenCode

Add vLLM as a custom provider in opencode.json:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "vllm": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "vLLM",
      "options": {
        "baseURL": "http://gpu-host:8000/v1",
        "apiKey": "{env:VLLM_API_KEY}"
      },
      "models": {
        "qwen3-coder": {
          "name": "Qwen3 Coder",
          "limit": { "context": 65536, "output": 8192 }
        }
      }
    }
  }
}

Set limit.context to the --max-model-len you started vLLM with, so OpenCode knows how much room is left. Pick the model with /models.

Contenox

1. Open Backends → Providers and, under Cloud, choose vLLM.
2. In Base URL, enter the host and port only, for example http://gpu-host:8000. Contenox adds the /v1/... paths itself, so leave /v1 off.
3. If the server has a key, open Advanced and enter it in API Key (optional), or enter an Environment variable name such as VLLM_API_KEY, set the variable and restart Contenox.
4. Select Connect and wait for Connected.

Contenox reads the model and the context length the server reports, and the Token Limit chip defaults to it. Download Contenox.

Common errors

• Connection refused or Unreachable: the server isn’t running, the port is blocked from your computer, or (in Contenox) the address has a path after the port. Run the curl check above from the same machine.
• 401 or Key rejected: the key doesn’t match the server’s --api-key.
• 400 error mentioning tool_choice: the server was started without --enable-auto-tool-choice and --tool-call-parser. Restart it with both. With containers, check that the flags reach the actual vllm serve command.
• Model not found: the name in the agent doesn’t match --served-model-name.
• The model replies in text instead of calling tools: the parser doesn’t match the model, or the model needs a chat template.
• Requests fail on long sessions: the conversation exceeds --max-model-len. Raise it if memory allows, or lower the context limit in your agent.

Related

• Run models locally: hardware and model choice for local agents
• Ollama: the single-computer option
• How to run local LLMs for coding
• Keep secrets away from AI agents