Only Codex, of the big three, has a native Windows sandbox, and none of the open-source agents that run any model has an official Terminal-Bench entry. Those two gaps decide most of this AI coding harness comparison: 16 coding agents on license, models, subscription sign-in, local models, Windows, permissions and sandbox, every cell checked against the tool’s own docs. A ”?” means the docs don’t say.
Models and sign-in
| Harness | Who makes it | License | Models | Sign in with a subscription | Local models |
|---|---|---|---|---|---|
| Claude Code | Anthropic | Proprietary | Claude only (Anthropic, Bedrock, Vertex, Foundry) | Claude plans, or an Anthropic Console account | No official support |
| Codex | OpenAI | Apache-2.0 | OpenAI, plus Bedrock, Vertex and custom providers in config.toml | ChatGPT plans | Yes: --oss with Ollama or LM Studio |
| OpenCode | Anomaly Innovations (the SST team) | MIT | Any provider | ChatGPT, GitHub Copilot, GitLab Duo; not Claude plans | Yes: Ollama, LM Studio, llama.cpp |
| GitHub Copilot CLI | GitHub (Microsoft) | Proprietary | Copilot’s models (Claude, GPT and others), plus your own OpenAI-compatible, Azure or Anthropic keys | Copilot plans | Yes: Ollama, vLLM, Foundry Local |
| Cursor CLI | Anysphere | Proprietary | Cursor-hosted models only | Cursor account | ? |
| Antigravity CLI | Google’s terms; no open-source license | Gemini plus a fixed list including Claude models | Google account (Free, Pro, Ultra), or an API key | Not in the CLI (the SDK only) | |
| Gemini CLI | Apache-2.0 | Gemini only | Gemini Code Assist Standard or Enterprise only, since June 2026 | ? | |
| DeepSeek Harness | DeepSeek | MIT | DeepSeek, plus Anthropic, OpenAI, Kimi, GLM and custom endpoints | DeepSeek account | Through a custom endpoint to a self-hosted server |
| Aider | Paul Gauthier | Apache-2.0 | Any provider | No (a Copilot token workaround is documented) | Yes: Ollama, LM Studio, any OpenAI-compatible server |
| goose | Agentic AI Foundation (Linux Foundation); started by Block | Apache-2.0 | Any provider | Through Claude Code or Codex as ACP agents | Yes: Ollama |
| Cline | Cline Bot Inc. | Apache-2.0 | Any provider | ChatGPT; Claude through Claude Code; Cline’s own plans | Yes: Ollama, LM Studio |
| Pi | Mario Zechner | MIT | Any provider | Yes, with /login (Copilot documented) | Yes: llama.cpp built in; Ollama, LM Studio, vLLM by config |
| Hermes Agent | Nous Research | MIT | Any provider | ChatGPT; Claude Max with extra usage only; Copilot through a plugin; Nous Portal | Yes: Ollama, LM Studio, llama.cpp, vLLM |
| Kilo Code | Anaconda, since July 2026 | MIT | Any provider, through Kilo Gateway or your own keys | ChatGPT plans; Kilo’s own plans; not Claude plans | Yes: Ollama, LM Studio |
| IBM Bob | IBM | Proprietary | Routes each task between Claude, Mistral and IBM’s Granite models | IBM account; Bob plans per instance | Yes: self-hosted, on-premises and air-gapped since October 2026 |
| Contenox | Alexander Ertli | Apache-2.0 | Most major providers by API key, plus OpenAI-compatible servers | ChatGPT; Claude through hosted Claude Code; Copilot as an unofficial sign-in | Yes: Ollama, vLLM |
The Claude subscription trap: a Claude Pro or Max plan works in Claude Code and almost nowhere else. Unless it has approved them, Anthropic doesn’t allow third-party tools to offer claude.ai sign-in; OpenCode removed its Claude plugin for that reason, and Kilo Code asks for an API key instead. The tools that “support” a Claude plan (Cline, goose, Contenox) do it by running Claude Code itself.
Windows, permissions and sandbox
| Harness | Windows native | Permission model | OS sandbox | Interface |
|---|---|---|---|---|
| Claude Code | Yes | Modes, with auto (a model approves routine actions) as the default; allow, ask and deny rules | macOS, Linux, WSL2; not native Windows | Terminal, VS Code, JetBrains, desktop app, web |
| Codex | Yes | Sandbox mode plus approval policy; --approve-for-me hands approvals to a reviewer agent | macOS, Linux, native Windows | Terminal, IDE extension, ChatGPT desktop app, cloud |
| OpenCode | Yes, but WSL recommended | Allow, ask or deny per tool; most tools allow by default | None; the docs recommend Docker or a VM | Terminal, desktop app (beta), IDE extension, web |
| GitHub Copilot CLI | Yes | Asks by default; allow and deny rules; Assisted mode lets a model approve | Preview, off by default: macOS, Linux, recent Windows 11 builds | Terminal |
| Cursor CLI | Yes | Allow and deny rules; --force skips approvals | macOS, Linux; on Windows only inside WSL2 | Terminal (same agent as the Cursor editor) |
| Antigravity CLI | Yes | Modes (auto, plan, default, accept-edits); allow, ask and deny rules | Linux, macOS, native Windows (Windows 11 24H2 with a 2026 update) | Terminal, desktop app, editors through ACP |
| Gemini CLI | Yes | Approval modes default, auto_edit, plan, yolo; trusted folders | macOS, Linux (containers), native Windows | Terminal, VS Code companion |
| DeepSeek Harness | ? | Presets: workspace-write (asks before risky actions) or danger-full-access | Linux, macOS; Windows partial | Web UI on localhost, CLI, ACP |
| Aider | Yes | Confirms actions; --yes-always approves all | None | Terminal, experimental browser UI |
| goose | Yes | Four modes; the default, Completely Autonomous, never asks | None (the experimental macOS sandbox was removed) | Desktop app, CLI |
| Cline | Yes | Asks by default; auto-approve per category; YOLO | None locally | VS Code, JetBrains, CLI, desktop app, cloud agents |
| Pi | Yes | None by design; extensions can add approval gates | None; the docs recommend a container or VM | Terminal, SDK |
| Hermes Agent | Yes | smart (a model judges risk, the default), manual or off | None on the host; Docker, SSH or cloud backends isolate commands | Terminal, desktop app, editors through ACP, messaging apps |
| Kilo Code | Yes | Allow, ask or deny per tool, ask by default | macOS, Linux; not Windows | VS Code, JetBrains, CLI, desktop app (beta), cloud |
| IBM Bob | Yes | ? | ? | Bob IDE, Bob-Shell in the terminal, IBM-managed or on-premises |
| Contenox | Yes, Windows first | Visible rules: Allow, Ask me first or Block for reads, edits and commands | Native Windows: writes limited to the project folder | Desktop app; contenox-runtime for scripts and CI |
Roo Code is not in the table: it shut down on 15 May 2026 and its repository is archived. Its README points users to Cline and a community fork.
Two status notes. DeepSeek Harness is a developer preview: DeepSeek warns of breaking changes and says it hasn’t been security audited. Aider still works, but its last release was in February 2026, and before that August 2025.
Is there a harness leaderboard?
Not one that ranks all of these. The closest:
- Terminal-Bench 2.0 lists agent and model separately, and its open submission files let you compare harnesses on the same model. For Claude Opus 4.6: about 63% in Terminus 2 (the benchmark’s own minimal agent), about 60% in GitHub Copilot CLI, about 74% in KRAFTON AI’s Terminus-KIRA. The submissions are public.
- Terminal-Bench 4.0, the current version, mostly lists each vendor’s model in its own harness. As of 4 November 2026, Claude Code with Opus 5.5 leads at 64.8%, followed by Claude Code with Sonnet 5.5 at 61.8% and Codex with GPT-6 Astra at 58.2%. That compares model-and-harness pairs, not harnesses.
- “How good is your harness?”, a paper at an ICML 2026 workshop, separated harness from model on Terminal-Bench 2.0 and found that picking the best harness gains about as much as picking the best model, and that some harnesses suit some models better than others.
None of the open-source, any-model harnesses (OpenCode, Kilo Code, Cline, Pi) has an official Terminal-Bench 4.0 entry, so “OpenCode vs Claude Code” on a benchmark has no current, like-for-like answer. Run both on one real task in your repo instead. What the benchmarks measure, and how much to trust them, is in AI coding benchmarks.
Which one for whom
- You pay for Claude Pro or Max: Claude Code. It’s the only harness that uses that plan directly, and it’s built alongside the models.
- You pay for ChatGPT: Codex. It’s open source, runs local models with
--oss, and has the only native Windows sandbox among the big three. - You work at a bank, an insurer or anywhere code can’t leave the building: IBM Bob. It went generally available in April 2026 and runs on-premises or air-gapped since October, with task routing between Claude, Mistral and Granite. It’s priced per instance, with paid add-ons for Java, IBM i, Z and DevSecOps.
- You pay for Copilot at work: Copilot CLI, or OpenCode with your Copilot sign-in if you want an open-source harness on the same plan.
- You want one harness for every model, open source: OpenCode in the terminal; Kilo Code or Cline inside VS Code or JetBrains. Kilo Code adds a sandbox on macOS and Linux; OpenCode has none.
- You want to run local models: Codex, OpenCode, Copilot CLI, Pi and Hermes Agent all document Ollama. Pi has llama.cpp built in.
- You mainly use DeepSeek: DeepSeek Harness, if a developer preview is acceptable; otherwise OpenCode or Kilo Code with a DeepSeek key. DeepSeek is cheap enough to change the math: one month of the Contenox team’s own use on the DeepSeek API was 23,815 requests and 7.86 billion tokens.
- You want a tiny harness you can read and change: Pi. It never asks before acting, so run it in a container.
- You want an agent in Slack, Telegram or email, not just code: Hermes Agent.
- You’re on Windows and want a sandbox: Codex natively, Antigravity CLI on a current Windows 11, or Copilot CLI’s preview sandbox. Claude Code and Cursor need WSL2 for theirs.
- “Best harness for vibe coding”: the one with the least setup between you and a running app: Claude Code or Codex with the subscription you have, or a desktop app if the terminal is in your way. Keep a sandbox or tight permissions on, because vibe coding means you aren’t reading every command.
Contenox
Contenox is a desktop app built for Windows first, open source under Apache-2.0. You open a project folder, pick a provider (a ChatGPT subscription, an API key or a local model through Ollama), and the agent works there while you watch and approve. It can also host Claude Code from its Agents page, so a Claude plan works too, under the same rules. What runs without asking is a set of rules you can read in Permissions, and on Windows a sandbox keeps writes inside the project folder. Download it here.
Related guides
- What is an AI coding harness?: harness vs agent vs model, and harness engineering
- Best AI coding agents: ranked by what they’re best at
- Open source AI coding agents: every license, checked
- AI coding agents on Windows: what works natively and what needs WSL
- YOLO mode: what each tool’s skip-all switch turns off