The coding model that stays on your servers (2026)

30.5 B total parameters. 3.3 B active per token. 256 K native context (Qwen model card). Roughly 16 GB of VRAM at AWQ 4-bit. Apache 2.0. That is the delta from twelve months ago, and it is the reason a small regulated-industry dev team can, in April 2026, run a credible in-house coding assistant on a single NVIDIA RTX PRO 5000 Blackwell 48 GB workstation without asking anyone's permission.

Who this is for: a fifteen-person Polish fintech whose clearing partner just sent over an outsourcing questionnaire; a Warsaw software house building an integration for a bank spółdzielczy under KNF Rekomendacja D; a kancelaria prototyping its own document-AI under tajemnica adwokacka. None of them are going to run GPT-5 over their client's code. All of them need the answer to one question: what can we actually host ourselves, today, that is good enough?

The thesis of this post is narrow: Qwen3-Coder-30B-A3B is the first 2026-class open-weights coding model where a 48 GB workstation plus Apache 2.0 plus on-prem inference gives a small regulated-industry team a credible in-house alternative to Cursor/Copilot. Everything below is evidence for that claim and the honest caveats that come with it.

In short. Qwen3-Coder-30B-A3B has 30.5 billion parameters, of which 3.3 billion are active per token, a 256 K context and an Apache 2.0 licence, and needs about 16 GB of VRAM at AWQ 4-bit. It fits one workstation with a 48 GB card and is a credible in-house alternative to Cursor or Copilot for a small regulated SME, with stated limits against frontier models.

As of April 2026; updated measurements are in preparation.

Why MoE changed the hardware math

For the last two years, a "30 B coding model" meant a 30 B dense transformer, and a dense 30 B at AWQ 4-bit meant roughly 18 GB of weights plus a KV cache that scaled with context length and batch size. On a 48 GB card, that left you maybe 8–12 K of usable working context before throughput collapsed. Fine for a tab-complete demo. Not fine for agentic edits across a real repository.

Mixture-of-Experts rewrites that arithmetic. Qwen3-Coder-30B-A3B is a 30.5 B-parameter MoE with 128 experts and 8 routed per token, so per-token compute is determined by the 3.3 B active parameters, not the 30.5 B total (Hugging Face model card). During inference, the model behaves - compute-wise - like a 3 B model. You still pay the full VRAM footprint to hold 128 experts in memory, but the per-token FLOPs and the KV-cache growth rate are those of a much smaller model. That is the whole point of MoE for local inference: keep the learned capacity of a 30 B model, pay the per-token cost of a 3 B model.

Concretely, on a single 48 GB Blackwell-class card running vLLM with AWQ-Marlin and chunked prefill, we observe 45–60 tok/s sustained at 64 K working context and batch 1, with roughly 16 GB for weights, 14–18 GB for KV cache at that context, and the rest for activations and CUDA overhead. That is not a projection. It is what the machine does.

The April 2026 shortlist

Three open-weights coding models are worth holding on the shortlist today. The numbers below are from primary sources; where a cell is cautious, it's because the leaderboard number is the vendor's self-reported score and not yet reproduced independently.

Two honest caveats on the table. First, Qwen's 69.6% SWE-bench Verified is a self-reported figure and the aggregate number in the Qwen3-Coder announcement family; an independent evaluation may give a different result. Second, the Qwen 2.5 Coder 32B line is there as the "what you would have deployed six months ago" reference point - it still works, but on SWE-bench Verified the MoE successor leaves it well behind while using less per-token compute. The generational jump is real.

Devstral Small 1.1 is the hedge. 24 B dense, Apache 2.0, 128 K context, 53.6% on SWE-bench Verified, and - this matters more than the raw score - it was built in collaboration between Mistral AI and All Hands AI, the team behind OpenHands (Hugging Face model card). If you care about agentic tool-use in a framework with a real track record, Devstral is not a consolation prize. It is a different bet with its own pedigree.

A vLLM launch that actually runs

The following is the command we run in production on a single 48 GB Blackwell for the SADP dev agent. Comments inline.

# /etc/systemd/system/sadp-coder.service (ExecStart)
vllm serve Qwen/Qwen3-Coder-30B-A3B-Instruct-AWQ \
  --host 0.0.0.0 --port 8000 \
  --served-model-name qwen3-coder-30b-a3b \
  \
  # AWQ-Marlin is the fused kernel path; plain awq is ~30% slower on Blackwell.
  --quantization awq_marlin \
  --dtype float16 \
  \
  # 0.90 leaves headroom for activations + CUDA workspaces on a 48 GB card.
  # Push to 0.93 only if your OS is headless and nothing else shares the GPU.
  --gpu-memory-utilization 0.90 \
  \
  # 64 K working context is the sweet spot: fits KV cache at batch 1-2,
  # covers a realistic repo-review prompt, and doesn't tank throughput.
  # The model supports 262144 native, but you do not have the VRAM for it.
  --max-model-len 65536 \
  \
  # Chunked prefill keeps long-prompt TTFT reasonable and lets decode
  # run concurrently with prefill of the next request. Default on in
  # recent vLLM, explicit here for auditability.
  --enable-chunked-prefill \
  --max-num-batched-tokens 8192 \
  \
  # Prefix caching is the single biggest win for agentic coding workloads
  # where the system prompt + repo map is repeated across turns.
  --enable-prefix-caching \
  \
  # Two concurrent requests is realistic for a 3-person team sharing one
  # workstation. Bump only if you've measured KV headroom.
  --max-num-seqs 2 \
  \
  # Speculative decoding with a 0.5 B Qwen draft model yields ~1.4x decode
  # throughput on this workload. Optional; remove if the draft model OOMs.
  --speculative-model Qwen/Qwen2.5-Coder-0.5B-Instruct \
  --num-speculative-tokens 4

Three flags that regularly cause grief and deserve a note. --gpu-memory-utilization is the single biggest footgun: vLLM preallocates KV cache against this budget at startup, so "it worked yesterday" and "it OOMs today" usually means somebody changed --max-model-len without rebalancing. --enable-prefix-caching is worth the memory it costs once your agents start reusing a repo-map prefix across turns. And --speculative-model only pays off if the draft model's tokenizer matches the target; Qwen-on-Qwen is fine, cross-family is not.

Where Qwen3-Coder-30B-A3B is genuinely worse than the frontier

The honest part. Three things a 30B-A3B model running on your workstation will not do as well as Claude Sonnet 4 or GPT-5-class frontier models.

Repo-scale multi-file patch coherence. On codebases above roughly 50 K LOC, where a plausible edit touches six files in four packages and needs to stay consistent with an architectural convention that is only implicit in the existing code, frontier proprietary models still win. They have been RL-trained on much longer agentic trajectories with much more compute. A local 30B-A3B will get the first three files right and then quietly lose the thread in the fourth. You will notice this in PR reviews, not in benchmarks.

Very-long-horizon tool use. "Run the failing test, read the trace, find the flaky dependency, patch it, rerun, confirm, commit" is a 20–40 tool-call chain. Frontier models have been post-trained on exactly this distribution with scale you cannot replicate locally. Open-weights coders are catching up fast but are not there yet, and the gap is widest on tasks that require recovery from a mid-chain mistake.

Novel-language reasoning outside the training mix. If your codebase is 80% Elixir, 80% OCaml, or a Cobol-to-Java migration, benchmarks on Python/TypeScript/Java/Go do not predict your experience. This is true for every model, but the training-mix asymmetry hurts open-weights models more because you cannot pay OpenAI to widen the mix for you.

None of these are fatal. All of them are real.

Why the on-prem 30 B still wins for the small regulated buyer

Given that honest list, the question writes itself: why not just use Claude or GPT-5 through an enterprise API with a DPA? Four reasons, and for a Polish small SMB the first two matter before the EU-level ones do (covered in Sovereign AI - why now).

KNF Rekomendacja D. If you sell code into any KNF-supervised entity - bank, insurer, SKOK, bank spółdzielczy, fintech płatniczy - your client's outsourcing questionnaire asks where the AI runs, who owns the weights, and what happens on exit. "Our vendor uses OpenAI under an EU DPA" is an answer that generates follow-up questions. "The model runs on our own hardware, on our own LAN, weights hash-pinned at deploy, logs retained under our schema" is an answer that ends the exchange. The second sentence is only available if the weights are on your GPU.

UODO / RODO, Article 28. Every piece of code that quotes a production log, a schema with personal data, a stack trace with email addresses - any of those sent to a cloud LLM creates an art. 28 processor relationship you have to paper. On-prem inference collapses the compliance surface to one controller and zero processors. For a twenty-person company, the cost of not needing to negotiate another umowa powierzenia with an American LLM vendor is measured in weeks of counsel time.

DORA Article 28 + AI Act Article 12. Above the Polish regulator, the EU layer applies. DORA-in-scope entities must demonstrate contractual exit rights, subcontractor transparency, and an ICT risk register that covers third-party inference; AI Act high-risk systems must log who/what/when/on-whose-behalf of every inference in a way that survives a conformity-assessment audit. A cloud endpoint gives you a usage dashboard; an on-prem vLLM behind your own gateway gives you the raw request/response pair plus a deterministic hash of the prompt template. Guess which one the auditor wants.

The trade you are actually making. You are giving up part of the capability of the frontier models in exchange for zero egress, deterministic cost, auditable inference, and a supply-chain surface that stops at your rack. For a dev shop building KSeF or e-Doręczenia integrations, a fintech writing payment-auth code, or a kancelaria under tajemnica adwokacka, that trade is a rounding error on the loss side and a door-opener on every procurement conversation. Your clients - or your clients' clients - are already asking the question, whether or not procurement has written it down.

The 2026 change is that the capability gap has narrowed enough to make the trade. When the gap was wide you could not make it in good conscience. Now you can, and your auditor will thank you.

The Devstral Small hedge

One last thing. Do not deploy a single-vendor stack even if the vendor ships Apache 2.0. Weights-licence risk is real - it is not litigation risk, it is "Qwen3 paused updates for eight weeks and we have a customer incident open" risk. Devstral Small 1.1 is the hedge worth naming: 24 B dense, Apache 2.0, 128 K context, built with the OpenHands tool-calling lineage, and different enough in training mix that its failure modes do not correlate with Qwen's. Keep it warm on a second machine, or in a second container on the same machine. A regulated-SME inference stack with exactly one weights provider is not a sovereign stack. It is a single point of failure with a permissive licence.

This is the stack I run for SADP. The data never leaves the building; the model does not either. If that is the 2026 you are planning for - whether you sell code into KNF-supervised clients, ship an integration under KSeF, or just want your developers' keystrokes to stop flowing to a US cloud - let's talk.