Applied Intelligenceindependent · by Arseny Gorokh
ai · 02
applied research note ·2026 · 05 · 14 ·ai-02 ·16 min read ·empirical study

Which models can run Claude Code through DIAL?Five upstreams, and the cost winner is open-source.

If you are paying per seat for a coding agent at team scale, the live question is whether a cheaper open model through your own gateway could do the same work. This note runs one Claude Code harness against five upstream models through EPAM's DIAL gateway and scores each on the two things that decide a rollout: did the task pass, and what did each success cost. An open 480B model passed every task at $0.101 each, the cheapest of the field; the result that goes in your budget is that capability tracked model scale, not vendor.

$0.101
cost per successful task for the cheapest model that passed everything, an open 480B coder, 38 percent under Haiku and well under Sonnet.
100%
pass rate for that open 480B model, 24 of 24 tasks, matching the two strongest Anthropic models in the matrix.
38%
pass rate for a 30B open model, which fails every multi-step task with the same one-call-then-stall signature, below the viability line.
120
trials, five models by eight tasks by three, run in 31 minutes wall-clock with zero retries and zero rate-limit errors.
itwo questions hiding in one experiment

Does the harness route, and does it survive the swap.

Before you can swap in a cheaper model, two things have to be true, and only one is plumbing: Claude Code has to reach the model through the gateway at all, and the harness has to keep working when you point it at an open-source upstream instead of an Anthropic one.

The first question is settled quickly; the second is the one that decides your bill. Its answer is that the harness is portable at Sonnet scale, not at Haiku scale, and the determining variable is model scale, not vendor.

The headline

Claude Code works through the gateway; an open 480B model is the cheapest thing that passes everything.

At verified Bedrock pricing, the open 480B coder is the cheapest 100-percent-pass model in the matrix, 38 percent cheaper than Haiku and roughly five times cheaper than Sonnet, while nothing tested can replace Claude Haiku 4.5 at the Haiku tier. The cost flip versus an earlier bake-off comes from replacing the SDK's fallback Opus rate for unrecognised models with actual console rates.

iithe narrow research question

Sustain a plan, not just emit a tool call.

"Does open-source work with Claude Code?" collapses on contact, because most modern coder models can emit a single tool-use block. The harness needs more: the model must plan, observe a tool result, update its plan, and continue. The gating question is agentic-loop sustainment, not structured-output emission. Two falsifiable hypotheses frame it.

  • H1, plumbing. Claude Code routes through the gateway via an Anthropic-to-OpenAI shim with no harness changes. Falsified by any partial-message rate above zero across 100 consecutive runs, or any tool-use round-trip divergence between the local-Anthropic and gateway-Anthropic baselines.
  • H2, portability. Harness viability with non-Anthropic upstreams is a function of model scale; sub-100B active-parameter models cannot sustain the multi-step loop. Falsified by a sub-100B open model passing the multi-step tasks at a rate comparable to Haiku 4.5.
iiithe bridge

One adapter, two routes, one wire surface.

The bridge is the only custom code between Claude Code and the upstream. It exposes Anthropic's messages shape on the wire, translates each request into the gateway's OpenAI-shaped chat-completions API, and streams the response back in the shape the harness expects. Two wrapper configurations differ only in which upstream the harness's three model slots point at; everything else, the agentic loop, the system prompt, the tool inventory, is untouched. One behaviour is load-bearing: the bridge strips cache-control markers when the upstream is not Anthropic, because the open routes otherwise return a no-route error.

Figure 1 · the request path. The harness is stock; only the bridge is custom, and model selection is environment-driven.
01
Harness
Claude Code, stock CLI, messages API.
02
The bridge
Anthropic shape to OpenAI shape; strips cache-control for open routes; emits cost and token telemetry.
03
DIAL gateway
project routing, auth.
04
Upstreams
Anthropic on Bedrock (Opus 4.7 / Sonnet 4.6 / Haiku 4.5); open Qwen3-Coder 480B; open Qwen3-Coder 30B.
ivthe task portfolio

Eight tasks, escalating so the shape of failure carries the diagnosis.

Each task is self-contained Python under a hundred lines, with a hidden validator that exits non-zero on any deviation, removing LLM-judge bias. The portfolio climbs from one-shot answers to the full agentic loop, where the hardest task requires the model to remember what it just did across a tool call and update an external surface from memory. No model lands between "passes all multi-step" and "passes none": failure is at the loop level, not the task level.

Table 1 · the eight-task gauntlet, by the capability it stresses.
tiertaskwhat it stresses
single-turnT1 read and explain a trap valuecomprehension, ignoring noise.
single-turnT4 generate from boilerplatespec-driven generation, pattern match.
multi-fileT2 rename across several filesmulti-file consistency, aliased imports.
multi-fileT3 bump a constant, leave tests aloneconditional edit, negative space.
multi-fileT5 "fix the test" where the bug is elsewhereinvestigation under misleading framing.
multi-fileT6 refactor without touching autha negative constraint held across edits.
full loopT7 plan, run the tests, reportthree independent tool roles, three or more turns.
full loopT8 refactor, test, update the README from memorythe hardest: retain state across a tool call, five or more turns.
vinstrumentation

Pass rate misleads without cost per success.

A 38 percent pass rate could mean "half competent" or "passes the easy tasks and bricks the hard ones in a structured way," so the metric portfolio is designed to disambiguate. Hidden validators provide outcome ground truth without an LLM judge, whose sharpest failure in the prior bake-off was over-rating models whose prose style resembled its own.

Table 2 · the metric portfolio, and why each metric is in the matrix.
metricwhat it measureswhy it is here
success ratevalidator exit-zero fraction across three trialsoutcome ground truth, no LLM-judge bias.
avg turnsmean tool-use turns per triala diagnostic: Anthropic clusters at 7 to 10, failing open models at 1 to 2.
frictionturns plus five times contract violationspenalises success via brute retry instead of clean planning.
violationsharness-contract breaks, untyped tool argsseparates "model is wrong" from "model does not speak the protocol."
wall p50 / p95wall-clock medians and tails on successful trialsp50 is throughput, p95 is the long-tail cost; they diverge.
cost per successstandardised cost per successful trialthe only honest economic metric across models.
vithe result

The cheapest model that passes everything is the big open one.

At Bedrock-verified pricing, the open 480B model is not just viable, it is the floor. The earlier bake-off had it looking expensive only because the SDK billed unrecognised models at the Opus fallback rate; the console rates tell the real story.

Table 3 · the matrix, by pass rate and cost per successful task (Bedrock-verified rates). 24 trials per model, 8 tasks by 3.
modelpasscost / successstanding
Qwen3-Coder 480B open24/24$0.101cheapest viable, the Pareto floor.
Claude Haiku 4.524/24$0.1621.6x the 480B; nothing open replaces it at this tier.
Claude Sonnet 4.622/24$0.4744.7x; the 480B is the Sonnet-tier substitute.
Claude Opus 4.724/24$0.6916.8x; the capability ceiling, at a price.
Qwen3-Coder 30B open9/24moota toy here: fails 15 of 24, cost-per-success is undefined in the failing cluster.

One honest uncertainty: the Anthropic 4.x Bedrock rates come from the SDK's internal table, which agreed with verified rows within about five percent but was not independently console-checked. And the gateway contract price is the only number that matters in production; the ranking survives any contract that does not inflate the 480B input rate above roughly a 6.7x markup.

viicache, latency, and the frontier

Capability tracks parameter scale, not vendor.

The open 480B model's input-token total exceeds Sonnet's on the same workload for a structural reason, not a model one: the gateway forwards cache-control intact for Anthropic, so Bedrock caches the 20K-token system prompt and reads it back at about a tenth of the cost, whereas the bridge must strip cache-control for the open routes, so they pay full rate every turn. That is a gateway gap, not a model gap; cache support for non-Anthropic routes would narrow the cost difference an estimated 30 to 50 percent. On latency, per-call the open 480B is actually slightly faster than Anthropic; the long tail comes from turn count and tool-result round-trips, so the harness makes the tail.

Figure 2 · the viability frontier: pass rate against cost per success, lower-left is the floor. The 30B model sits off this chart at 38 percent pass, below the roughly 90-percent viability line, where cost per success is undefined.

The closing thesis: harness viability is scale-driven, not vendor-driven. Three open lineages tested across this and the prior bake-off, a 27B, a 123B, and the 30B here, all fail with the same one-call-then-stall signature; only the 480B, roughly an order of magnitude larger in active parameters, closes the loop. Open models keep up at roughly 400B-plus active parameters on a coder-tuned base, and not below.

·what to do, and the limits

Put the 480B in the Sonnet slot. Never the 30B in the Haiku slot.

  • For operators. Adopt the open 480B as the Sonnet-tier default; never put a sub-100B open model in the Haiku slot. The economics and the loop both favour it.
  • For gateway operators. Ship cache-header support for non-Anthropic routes; it is the single change that most narrows the cost gap.
  • For open-model trainers. The target is multi-turn tool-use transcripts with sustained plans, not single-shot tool-call benchmarks.
  • For replication. The real cutoff lives somewhere in the 30B-to-480B gap; test 70B to 150B coder models to find it.
About this note

An independent applied-AI engineering note by Arseny Gorokh. Not an official EPAM publication. EPAM's DIAL and the named models are the public subject; the harness, tasks, and bridge are the author's own. The limits bound the claim: a single operator and network path, synthetic tasks, a print-mode artefact, placeholder pricing pending a contract rate, un-normalised tokenizers, no human-eval cross-check, one harness version. Re-run the matrix before treating any magnitude as portable.

References

  1. AWS Bedrock pricing console (US East, 2026-05-14), source of the verified open-model rates that anchor the corrected cost story.
  2. Claude Code SDK internal price table, source of the Anthropic 4.x Bedrock rates (within ~5% of verified rows) and of the Opus-rate fallback that caused the original overstatement.
  3. An independent council critique of the prior bake-off, which produced the loop-sustainment framing and the hidden-validator decision.
  4. The prior bake-off (May 2026): a 123B and a 27B open model whose failure signature this study reproduces, basis for the cross-vendor scale claim.
  5. The cost-per-success discipline: headline cost and self-reported speed are not the honest economic metric; cost per successful task is.
An Applied Intelligence publication · independent · 2026 ai-02 Set in Newsreader & Spline Sans Mono · the Applied Intelligence reading style