Nemotron 3 Ultra will be available from Nvidia in few days.
Rohan Paul Twitter · Rohan Paul (@rohanpaul_ai) · 2026-06-01
Nvidia is releasing Nemotron 3 Ultra within days, featuring a hybrid state-space model and mixture-of-experts architecture that enables extended reasoning and tool use on long sequences without standard attention-mechanism limitations.
Appears in
Extraction
Topics: nvidiallm-architecturestate-space-modelsmixture-of-experts
Claims
- Nvidia's Nemotron 3 Ultra will be released within days.
- The model uses a hybrid architecture combining state-space models (SSM) and mixture-of-experts (MoE).
- The SSM component enables effective processing of long sequences, allowing extended reasoning and tool use beyond what standard attention mechanisms support.
- Hybrid SSM+MoE architectures avoid the memory and compute bottlenecks associated with attention scaling on long contexts.
Key quotes
Nemotron 3 Ultra will be available from Nvidia in few days.
The SSM part is built for long sequences, so the model can keep reasoning or using tools for longer without getting crushed by the usual attention