Dense or MoE: which open model to pick

Pick the wrong open-weight model in 2026 and you do not lose a benchmark point. You lose the hardware spend, months of integration, and the only audit-defensible window you had before the AI Act obligations for Annex III high-risk systems start to apply (from 2 December 2027, moved from 2 August 2026). The choice that matters is not "best benchmark score". It is dense vs MoE, mapped onto your hardware, your workload, and your jurisdiction.

Who this is for: a regulated SME in Poland (FinTech, insurance, legal, advisory, GovTech) with a technical decision-maker who is about to write a purchase order for a GPU node and wants to know whether the right answer is "one big dense model" or "one sparse MoE". The wrong call costs more than the GPU.

In short. Choosing between a dense and an MoE model starts with VRAM, not benchmarks. Dense wins at coding, long documents and fine-tuning; MoE wins with many parallel users and short prompts. For EU-based firms Llama 4 drops out on licence, so the decision rests on hardware budget, number of users, context length and jurisdiction.

As of May 2026; updated measurements are in preparation.

Start with VRAM, not benchmarks

Before you compare architectures, size the hardware. These numbers come from primary source measurements, not model-card marketing.

Three hard rules fall out of the table immediately.

Rule 1: an RTX 5070 Ti 16 GB does not fit Qwen3.6-27B at any useful context length. The Q4 file is ~16-18 GB, but runtime VRAM with overhead is ~20 GB even at 1 K context. Add KV cache and you need more headroom. For 27B dense, plan on 24 GB minimum (RTX 4090), 32 GB for comfort (RTX 5090), or 64 GB unified memory (MacBook M4 Max) if you want the full 262 K context window.

Rule 2: MoE does not save VRAM. All 35 B parameters of Qwen3.6-35B-A3B must sit in memory; only 3 B are active per token. The file is ~20-22 GB, runtime ~24-26 GB before KV cache. You do not run this on a 24 GB card.

Rule 3: KV cache for Qwen3.6-27B (hybrid DeltaNet + GQA attention) costs 64 KB per token in BF16. At 8 K context that is 512 MB. At 32 K it is 2.0 GB. At 128 K it is 8.2 GB. At 262 K it is 16.8 GB. That is on top of the ~20 GB base. This is why a 24 GB card works for coding (short context) but chokes on long-document analysis.

The 100-word architecture cheat sheet

A dense model sends every token through every weight. Compute scales with total parameters. Predictable, easy to quantise, easy to fine-tune. Stable KV cache layout.

A Mixture-of-Experts (MoE) model sends every token through a router that picks N experts from M (for example 8 of 256). Only the selected experts do work, so per-token compute matches a much smaller model, but the entire bank of experts must sit in VRAM.

Consequence: MoE buys you cheaper compute per token (faster inference, better throughput) at a larger VRAM footprint than a dense model of equal quality. Q4 quantisation on MoE is more fragile because the router is sensitive to weight noise. Dense wins on coding, long context, consistency, and fine-tuning. MoE wins on throughput for short prompts with many concurrent users.

Where dense wins

Coding and agentic tool use

The case: your team writes code, runs tests, and queries an on-prem assistant. The model must call tools reliably, maintain context across a 32-64 K repo, and quantise predictably so that a 4-bit weight does not break the router.

The pick: Qwen3.6-27B dense at AWQ 4-bit or Q4_K_M.

The hardware: RTX 4090 24 GB (tight, up to ~32 K context), RTX 5090 32 GB (comfortable), or MacBook M4 Max 64 GB (best choice, fits full 262 K).

Why: Qwen3.6-27B scores 77.2% on SWE-bench Verified, which beats Qwen3.5-397B-A17B (76.4%) at roughly one fifteenth of the total parameters (Qwen model card). A dense model quantises predictably and gives stable tool-use behaviour because every weight is active on every token. No expert drift, no routing instability. Boring is the feature.

vllm serve Qwen/Qwen3.6-27B-AWQ \
  --quantization awq \
  --max-model-len 65536 \
  --gpu-memory-utilization 0.90 \
  --enable-prefix-caching \
  --max-num-seqs 2

When not to pick it: if you need to serve 10+ developers in parallel from a single GPU, batch throughput becomes the bottleneck and an MoE wins.

Long-context document analysis

The case: you analyse contracts, audit files, regulatory drafts, or patient records that run to hundreds of pages. The model must hold 64-128 K of context without losing track of references on page 1 when answering about page 400.

The pick: Qwen3.6-27B dense.

The hardware: MacBook M4 Max 64 GB or RTX 5090 32 GB. RTX 4090 24 GB works up to ~32-48 K context; beyond that KV cache eats the card.

Why: dense models have a single, stable KV cache. No expert-switching noise, no context-dependent routing changes. A 262 K native context fits the full Polish civil code plus a client file in one prompt. The hybrid DeltaNet layers in Qwen3.6-27B keep KV cache growth linear, not quadratic, so 128 K is practical on 32 GB and 262 K is practical on 64 GB.

When not to pick it: if your documents are short (under 8 K) and you have many concurrent users, MoE gives better throughput.

Fine-tuning on your own data

The case: you need to adapt a model to your domain (clinical terminology, legal citations, internal taxonomies) with a single GPU and a small labelled set.

The pick: Qwen3.6-27B dense.

The hardware: one A6000 48 GB or one H100 80 GB.

Why: QLoRA on a dense model is a solved problem. Unsloth, Axolotl, and standard PEFT all work out of the box. Iteration time: days. QLoRA on an MoE requires expert-balancing loss, the router drifts under low-rank adapters, and open-source tooling support is partial. Iteration time: weeks, often with multi-GPU FSDP.

When not to pick it: if you already have a multi-GPU cluster and an in-house ML engineer who has shipped MoE fine-tuning before. Most SMEs do not have that engineer.

Where MoE wins

Multi-tenant inference for many concurrent users (short prompts)

The case: an internal AI portal, 10 to 50 employees querying in parallel: brief drafting, ticket triage, short Q&A, code snippets. Typical prompt under 8 K tokens.

The pick: Qwen3.6-35B-A3B (MoE, 256 experts, 8 routed + 1 shared active).

The hardware: one NVIDIA RTX PRO 6000 96 GB, or two NVIDIA RTX PRO 5000 48 GB, or one RTX 5090 32 GB with a very tight batch (not recommended for production).

Why: 3 B active parameters per token is what lets vLLM with PagedAttention batch 8 to 16 concurrent requests at 200 to 400 ms TTFT. Continuous batching exploits the fact that different tokens activate different experts, which keeps the GPU busy.

When not to pick it: if your users submit long documents (>32 K) or you only have 24 GB of VRAM. The 35 B model at Q4 needs ~24-26 GB just for weights before any KV cache.

Datacenter throughput for a SaaS with ~100 paying users

The case: a Polish SaaS product, around one hundred paying users, AI features as a first-class part of the offering, single-tenant deployment in an EU colo.

The pick: Mistral Small 4 (119B-2603), MoE with 4 of 128 experts active per token.

The hardware: one H200 141 GB, or two H100 80 GB at Q4 / FP8.

Why: Mistral reports roughly 40% lower end-to-end latency vs Mistral Small 3 in latency-optimised configuration, and up to 3x more requests per second in throughput-optimised configuration (Mistral model card). 6.5 B active parameters are not a problem for an H200. Apache 2.0 means no commercial restrictions in the EU, in contrast to Llama 4.

When not to pick it: if you do not have at least one H200 or an air-gapped pair of H100s, the entry cost (80 to 150 k EUR) is not justified for a hundred-user SaaS. Stay on Qwen3.6-35B-A3B until your unit economics force the upgrade.

Three industry examples

Interactive agency or e-commerce (5-10 people)

You build WordPress and WooCommerce sites, write custom integrations, and handle short coding tasks. Your team is small, your repos are not huge, and your budget for hardware is one workstation.

Hardware: RTX 5070 Ti 16 GB. Model: Qwen3.5-9B or Qwen3-14B dense (both fit comfortably in 16 GB at Q4). Qwen3-14B offers a 128 K context window - enough for most repos and coding tasks, though shorter than the 262 K in Qwen3.6-27B. Use case: code completion, plugin debugging, SQL query generation, short content drafts. You do not need 27B for this workload; 8B dense at Q4 on a 16 GB card is a practical starting point that leaves VRAM for the OS and IDE. The RTX 5070 Ti is a consumer card without ECC; this is a minimum-viable setup, not a production configuration for sensitive data.

Financial advisory (20 people)

You analyse client documentation, prepare grant applications, and handle confidential financial data. Documents are long (100-300 pages), and your team of 10-20 people queries the model in parallel.

Hardware: NVIDIA RTX PRO 5000 48 GB or two RTX 4090 24 GB. Model: Qwen3.6-35B-A3B MoE at Q4.

Use case splits into two:

(a) Internal AI portal with short prompts. Q&A, grant drafting, brief consistency checks - prompts under 8 K tokens, 10-20 concurrent users. MoE handles this well. 3 B active parameters per token keep latency low under load.

(b) Full-document analysis of 100-300 page files. At ~500 tokens per page, a 300-page document is ~150 K tokens. That exceeds the ~32-48 K context you can fit on a 48 GB card with Qwen3.6-35B-A3B. With this MoE you must chunk the document and accept that cross-page references may break. If you need the full document in one prompt, switch to Qwen3.6-27B dense and plan for 48-64 GB of VRAM. The trade-off is clear: MoE buys throughput for many users, but it does not fit the whole file.

Solo law practice (1 person)

You analyse case files of 500+ pages, draft procedural submissions, and research case law and statutes. Attorney-client privilege means zero cloud. You need the model to remember facts from page 12 when drafting conclusions on page 480.

Hardware: MacBook M4 Max 64 GB or RTX 4090 24 GB (with context capped to ~32-48 K). Model: Qwen3.6-27B dense at Q4. Use case: case-file ingestion, draft argumentation, statute cross-checking. The 262 K context window fits a full case file plus the relevant civil code articles. Dense architecture matters here because consistency of legal argumentation across a 40-page submission is non-negotiable; you do not want routing noise changing the tone between paragraphs.

The Llama 4 elephant in the room

Llama 4 Scout and Maverick are technically interesting. Scout has a 10 M token context window. Maverick has 128 experts and a 402 B parameter bank. On any benchmark page, they look like the obvious 2026 choice.

They are not. The Llama 4 Community Licence and Meta's Acceptable Use Policy explicitly do not grant the rights for multimodal Llama 4 models to individuals domiciled in, or companies with a principal place of business in, the European Union. Because Llama 4 Scout and Maverick are natively multimodal, a Polish JDG, sp. z o.o., or any EU-registered legal entity is excluded from building a product on these weights, even on its own hardware. For an AI Act conformity assessment, a model your jurisdiction is licence-excluded from is not "high risk handled correctly". It is "non-compliant from day one".

For a regulated SME in the EU, the open-weights shortlist in May 2026 is Qwen and Mistral. That is it.

A decision tree

1. Are you EU-domiciled? If yes, Llama 4 is out. Continue.
2. Is your main use case coding or agentic work with tool use? If yes, pick dense (Qwen3.6-27B).
3. Do you analyse documents longer than 32 K tokens, or draft submissions that require consistency across sections? If yes, pick dense (Qwen3.6-27B).
4. Do you have 10+ concurrent users and short prompts (under 8 K)? If yes, pick MoE (Qwen3.6-35B-A3B).
5. Do you fine-tune on your own data? If yes, pick dense (Qwen3.6-27B).
6. Everything else (chatbot, Q&A, short tasks, high traffic) → MoE (Qwen3.6-35B-A3B).

Mistral Small 4 enters the picture when you have outgrown a 48 GB single-node setup and have at least one H200 or a 2x H100 air-gapped pair.

What to verify before you commit

1. Licence vs jurisdiction. Read the actual licence text on the model card, not a summary. Pay attention to revenue thresholds, geographic carve-outs, and downstream-use clauses. Llama 4 is the obvious trap.
2. Benchmark on your data, not theirs. SWE-bench Verified and MMLU are useful for ranking, not for predicting your TTFT on your repo. Spin up vLLM with the candidate model, replay one week of your real prompts, measure.
3. KV cache real-world test. "262 K native context" on the model card does not mean 262 K on your GPU. Measure the KV cache footprint at your target batch size and context length before you size the hardware.
4. Plan the six-month upgrade. Qwen ships a new generation roughly every quarter, Mistral every six months. The model you deploy in May 2026 will not be the model you run in May 2027. Make sure your serving stack (vLLM, your gateway, your audit log schema) is model-agnostic.
5. AI Act conformity logging from day one. The obligations for Annex III high-risk systems are to apply from 2 December 2027 (earlier: 2 August 2026), and the automatically generated logs of such a system are to be kept for at least six months (Article 19(1) for providers, Article 26(6) for deployers). Build the log schema before you build the product.

Closing

The 2026 question is not "dense or MoE in the abstract". It is: given your hardware budget, your concurrent users, your context length, and your EU jurisdiction, which open-weight model survives both the GPU bill and the auditor?

For most regulated Polish SMEs in May 2026, the answer is: Qwen3.6-27B dense for coding, long documents, and solo or small-team workloads. Qwen3.6-35B-A3B MoE for multi-tenant portals with short prompts and many users. Mistral Small 4 when you grow into datacenter-class hardware. Llama 4 is off the table for EU buyers.

The broader argument for keeping the weights on your own rack lives in Sovereign AI - why now. If you are about to write a PO for a GPU node and want a second opinion before you do, let's talk.