Nvidia's open speech model untangles eight voices

NVIDIA has released Nemotron 3 Diarization, an open-weights speech model built to answer a question that meeting software, call centres and captioning tools all wrestle with: who spoke when. The model card carries a release date of 23 September 2026 and describes a single checkpoint that handles up to eight speakers, in streaming or offline mode.

Diarization is the unglamorous half of speech AI. Transcription tells you what was said, and diarization tells you who said it, which is the part that turns a wall of text into something a human can follow in a six-person stand-up.

The latency ladder

The numbers NVIDIA publishes are the interesting part. Frames are measured in 80 millisecond steps, and the card lists four supported streaming configurations: an offline-style setup with a 30.4 second input buffer, a low-latency profile at 1.04 seconds, a very low latency profile at 0.64 seconds, and an ultra-low profile at 0.32 seconds. The card notes the hardware can run as low as 80 milliseconds of buffering for latency-critical work, though 0.32 seconds is the lowest configuration NVIDIA recommends.

Latency here means input buffer latency, so processing time sits on top of it, and the model card says so plainly. Output frame resolution is configurable in multiples of 10 milliseconds, and because inference is chunked there is no cap on how long a recording can be.

How it keeps speakers apart

Nemotron 3 Diarization follows Sortformer, NVIDIA's earlier work on the problem, and resolves the speaker-permutation headache by ordering its output channels according to when each voice first arrives. For streaming it adds the Arrival-Order Speaker Cache and a first-in, first-out queue from Streaming Sortformer, so speaker identities survive across chunks instead of being relabelled every few seconds.

That is the failure mode practitioners complain about most: a system that is accurate in a lab but renames everyone halfway through a conference call.

Licence and how to run it

The weights ship under the openmdw-1.1 licence and NVIDIA says the model is ready for commercial as well as non-commercial use. It runs through NVIDIA NeMo Speech or the Transformers stack, with Python 3.12 or later required, and there is a hosted demo space that pairs diarization with streaming speech recognition.

Our opinion

Open diarization matters less for transcribing a podcast than for the things nobody has built yet, like a meeting recorder that knows when the quiet person actually spoke, or call analytics small firms can run on their own hardware instead of renting per-minute. The licence choice is the real signal: NVIDIA is putting a capability that usually arrives as a cloud API into a downloadable file, and that will annoy exactly the vendors whose margins depend on it staying there. The caveat worth holding on to is the one the card itself admits, that 0.32 seconds is a buffer target and not a promise about how fast your box will be.