NVIDIA Nemotron 3 Super: 120B-A12B, Mamba, LatentMoE and NVFP4
Technical Nemotron 3 Super analysis: hybrid Mamba-Transformer, 120B/12B MoE, LatentMoE, MTP, native NVFP4, 1M context and self-hosting.
- AUTHOR
- Karol Rapacz / CEO of Breachroad · OSCP · PNPT
- PUBLISHED
- 10 March 2026
- READING TIME
- 16 min read
- TOPIC
- AI Security
NVIDIA released Nemotron 3 Super on 10 March 2026, an open agentic model with 120 billion total and 12 billion active parameters per token. It combines Mamba and Transformer layers, sparse MoE, LatentMoE, Multi-Token Prediction and NVFP4 pretraining. NVIDIA published BF16, FP8 and NVFP4 variants, a base model, post-training data and a technical report.
The release matters because model, numeric format, speculative decoding and hardware were engineered as one inference stack.
120B-A12B and Mamba–Transformer
MoE stores many experts while a router activates a subset. Twelve billion active parameters reduce token FLOPs, but 120B total still drives weight memory and distribution. Inter-GPU communication, state/cache, routing and expert imbalance also matter.
Mamba state-space layers carry information efficiently through long sequences without full quadratic attention at every step. Transformer layers retain precise attention relationships. The hybrid seeks long context and throughput without losing exact reference behaviour.
LatentMoE, MTP and NVFP4
LatentMoE targets routing and expert-representation quality. MTP predicts several future tokens, enabling native speculative decoding: verified predictions increase decode throughput. Acceleration depends on acceptance rate and sequence shape rather than a fixed multiplier.
NVFP4 is a 4-bit format optimised for NVIDIA platforms. Low-precision-aware pretraining differs from naïve after-the-fact quantisation. Quality still depends on GPU, runtime, kernels and calibration.
NVIDIA reports up to 2.2× GPT-OSS-120B throughput and 7.5× Qwen3.5-122B in an 8K-input/64K-output setup. That is a vendor result on defined hardware; short chats, low batch or other accelerators may rank differently.
One-million-token context
NVIDIA reports 1M context and strong RULER performance. Capacity is not a recommendation to send everything. Prefill, memory, latency and data risk grow. Manage context with retrieval and compaction, then test needle retrieval, tool histories, passage citation and instruction stability.
Open artefacts and reproducibility
Published base/post-trained checkpoints and datasets improve inspectability but do not eliminate supply-chain risk. Pin revision and hash, track every licence, review loader code and quarantine third-party adapters.
Record model card, tokenizer, chat template, runtime, quantisation and driver as one registry entry. A model name alone is not a reproducible deployment.
Secure agent deployment
Open weights enable private-network inference, but agents still need sandboxing, a tool gateway, schema validation and least-privilege credentials. Long context can carry injected instructions from documents or repositories. Separate instructions from data and approve mutations.
Evaluate agentic reasoning, coding, target languages, 1M retrieval, function calling, refusals and adversarial inputs. Measure energy, VRAM, throughput, first-token latency and whole-task correctness.
Infrastructure benchmark plan
Use the same prompt set, input/output lengths, batch and cache policy across systems. Run cold start, steady load and a traffic spike separately. Report median and p95/p99 TTFT, inter-token latency, requests per second, OOM errors, energy and cost per correct task. A figure measured at 8K input and 64K output does not predict a chat workload producing 500-token responses.
For NVFP4, inspect quality by task class rather than benchmark average. Quantisation may affect code, less common languages, strict JSON and long reasoning unevenly. BF16/FP8 and NVFP4 artifacts need separate registry records, evaluations and rollback paths.
Cluster resilience
Test GPU loss, worker restart, queue saturation and a mismatched kernel version. Scheduling should shed load by priority rather than degrade every tenant. Health must mean more than a live process: canary prompts should check tokenizer, generation and tool calling. Protect weight copies and caches because private adapters may encode organisational knowledge.
Combine deployment with a model registry, safe weight formats and agent sandboxing. For independent benchmarking and threat modelling, contact BreachRoad.
Primary sources: NVIDIA Research — Nemotron 3 Super, technical report.
