Ask the Experts: Dory Looped Transformer Model | Nemotron Labs

Can one 4B model trade compute for accuracy at inference time, with no change to its weights? Nemotron Labs Dory 1.0, an open research model from NVIDIA, explores that question with a looped transformer that reuses a shared block of layers, taking SWE-bench Verified from 50.3 at one loop to 61.8 at four loops. In this Ask the Experts session, you’ll get direct access to the NVIDIA researchers behind it, and we’ll cover how looping works, why last-loop KV caching keeps memory flat, and what the early results suggest about trading compute for accuracy.

What you’ll learn:

• How looped transformer models trade inference compute for accuracy, and where extra loops stop helping
• How last-loop KV caching cuts memory overhead for long-context workloads
• How native NVFP4 pre-training and the BF16, MXFP8, and NVFP4 checkpoints run on DGX Spark and Hopper and Blackwell GPUs
• How to evaluate loop-count tradeoffs on your own tasks

Have questions about looped transformer models, loop-count tradeoffs, or what’s next for this research? Drop them live, and our guests will answer them in real time.