DeepSeek V4: 1.6T MoE, CSA/HCA, mHC, Muon and 1M context
Technical DeepSeek V4 Pro and Flash analysis: 1.6T/49B and 285B/13B, compressed attention, mHC, Muon, 1M context, MIT licence and deployment.
- AUTHOR
- Karol Rapacz / CEO of Breachroad · OSCP · PNPT
- PUBLISHED
- 24 April 2026
- READING TIME
- 17 min read
- TOPIC
- AI Security
DeepSeek released DeepSeek V4 Pro and Flash on 24 April 2026. The official model card describes text-only Mixture-of-Experts models with three reasoning modes and one-million-token context. Pro has 1.6 trillion total and 49 billion active parameters; Flash has 285 billion total and 13 billion active. Weights and code are distributed under MIT alongside an API.
The main architectural changes combine Compressed Sparse Attention (CSA), Heavily Compressed Attention (HCA), Manifold-Constrained Hyper-Connections (mHC) and the Muon optimiser while retaining DeepSeekMoE and Multi-Token Prediction from V3.
CSA and HCA
Classical attention grows quadratically and requires a large KV cache. CSA compresses cache along sequence and applies sparse attention. HCA uses heavier compression followed by dense attention over the reduced representation. The hybrid seeks global view and local detail at lower cost.
Compression is not lossless. Test documents with dispersed exceptions, code with distant definitions, multi-stage logs and conflicting instructions. A 1M limit does not guarantee uniform attention over every token.
mHC and signal stability
Manifold-Constrained Hyper-Connections constrain residual mappings to doubly stochastic matrices on the Birkhoff polytope. Intuitively, flows among paths maintain stable mass rather than degenerating in a deep complex network. This is architectural rather than an API switch; assess it through outcome stability and cost.
Muon, MTP and reasoning modes
The model card says Muon improves convergence and training stability. MTP predicts multiple tokens and can support faster decode. Non-think, Think High and Think Max let applications route cost.
Use Non-think for extraction, High for complex analysis and Max only where internal evaluations justify latency and spend. Visible reasoning is not proof; require sources and tool evidence.
Pro versus Flash
Pro activates 49B of 1.6T; Flash activates 13B of 285B. Flash may fit high-volume routing, Pro the hardest tasks. Total weights still dominate self-hosting memory and expert communication.
Compare on identical prompts, runtime and quantisation. Measure task pass rate, factuality, context retrieval, function calls, refusals, first-token latency, throughput, VRAM and energy. Do not project an API benchmark onto a local quant.
Open licence, complete supply chain
MIT applies to released assets, while tokenisers, kernels, servers, containers, adapters and data have their own provenance. Pin official revision and hash, use safe formats, review loader code and register quantisation provenance.
The model card’s statements about public/licensed training data and privacy filtering are vendor disclosures. Organisations still need legal and privacy assessment for their own prompt and fine-tuning data.
Secure deployment
API and self-hosted access need an identity-aware gateway, quotas, data classification and DLP. Long context expands prompt-injection surface. Validate tool schemas, allowlist destinations and approve mutations.
For self-hosting, isolate inference, deny default egress, patch runtimes and test tenant boundaries. Never expose a raw OpenAI-compatible endpoint without authentication merely because it is local.
Profiling hybrid attention
Evaluate CSA, HCA and MHC as parts of the complete runtime. Profile prefill and decode at 4K, 32K, 128K and the target maximum, together with KV-cache growth, inter-GPU traffic and batching behaviour. Model-card results may assume optimised kernels unavailable in your server version. Record driver, CUDA or accelerator library, runtime commit and scheduler settings in the deployment manifest.
Test long context with many similar documents, conflicting versions and evidence at different positions. A needle test does not show whether the model resolves source freshness and reliability. Require fragment identifiers in answers and verify quotations programmatically against documents.
Choosing Pro or Flash
A router can send classification and simple extraction to Flash, escalating complex or low-confidence tasks to Pro. Calibrate the threshold using cost per correct result. Record what escalated, how often and with what outcome. If Flash frequently causes an expensive retry, the apparent saving disappears. High-impact actions need independent validation whichever variant handled them.
Register V4 in model governance, run LLM evaluations and protect access through an LLM gateway. For Pro versus Flash testing on your workloads, contact BreachRoad.
Primary sources: DeepSeek Transparency Center, DeepSeek V4 Model Card.
