Nvidia Drops Nemotron 3 Super Amid $26 Billion Open-Model AI Bet—America's Answer to Qwen?

Nvidia has released Nemotron 3 Super, a new 120-billion-parameter open-weight AI model designed to optimize autonomous agents and handle ultra-long context tasks efficiently. This model aims to address the escalating costs and decreasing accuracy associated with multi-agent systems by employing a hybrid architecture that balances computational efficiency with deep reasoning capabilities.

Nemotron 3 Super Architecture and Capabilities

The Nemotron 3 Super model features a hybrid Mamba-Transformer Mixture-of-Experts (MoE) architecture, with 12 billion active parameters out of a total of 120 billion. This design allows for cost-effective inference while maintaining the complexity required for advanced workflows. A key feature is its native 1-million-token context window, enabling agents to retain extensive information, such as entire codebases, in memory.

The architecture uniquely combines Mamba-2 state-space layers for efficient long-token stream processing, Transformer attention layers for precise recall, and a novel "Latent MoE" design. This approach compresses token embeddings before routing them to experts, allowing for greater activation of specialists at a comparable compute cost. Furthermore, the model was pretrained natively in Nvidia's 4-bit floating-point format (NVFP4), ensuring accuracy at lower precision from the outset, unlike models compressed after high-precision training.

Performance and Benchmarks

Nemotron 3 Super demonstrates significant performance improvements over its predecessor, delivering over five times the throughput. Compared to external models, it achieves 2.2x faster inference throughput than OpenAI's GPT-OSS 120B and 7.5x faster than Alibaba's Qwen3.5-122B. Initial testing indicates robust reasoning capabilities, even with ambiguous or erroneous prompts, with the model accurately identifying errors and handling complex logic and math problems.

Nvidia's Strategic Investment in Open-Source AI

The launch of Nemotron 3 Super is part of Nvidia's broader strategy, backed by a $26 billion investment over five years dedicated to building open-weight AI models. This initiative is driven by the increasing dominance of Chinese open-source AI models, which have rapidly gained market share globally. Research indicates that Chinese models now represent approximately 30% of global open-model usage, with Alibaba's Qwen surpassing Meta's Llama as a leading self-hosted model.

This strategic move aims to counter the growing influence of Chinese AI labs and secure Nvidia's position in the AI ecosystem. While U.S. companies have largely focused on proprietary models, Chinese firms have actively contributed to the open-source community. Nvidia's investment in open-weight models, coupled with its hardware dominance, seeks to prevent the formation of an AI ecosystem reliant on Chinese chips and models that bypass Nvidia's infrastructure.