NVIDIA Nemotron Labs 3 Puzzle 75B A9B: Compressed Hybrid MoE LLM Delivering 2.03x Server Throughput

NVIDIA's Nemotron Labs has released Puzzle 75B A9B, a compressed hybrid Mixture-of-Experts language model with 75 billion total parameters but only 9 billion active per token, achieving a reported 2.03x improvement in server throughput over its uncompressed counterpart. The hybrid MoE architecture is the key technical innovation here — combining dense and sparse routing to maximize GPU utilization while preserving model quality at scale. For infrastructure engineers and teams running self-hosted inference, a 2x throughput gain at this parameter scale is significant and could substantially reduce per-token compute costs on NVIDIA hardware. The model is likely optimized for NVIDIA's own GPU stack (H100/H200/B200), so teams not on NVIDIA infrastructure should verify compatibility before planning deployments. This release reinforces NVIDIA's strategy of moving up the stack from silicon into model architecture, making its hardware more competitive by co-designing models that exploit its memory bandwidth and interconnect strengths.