NVIDIA Nemotron 3.5 ASR: 600M-Parameter Multilingual Streaming Speech-to-Text

What Is Nemotron 3.5 ASR?

NVIDIA Nemotron 3.5 ASR is a 600M-parameter streaming Automatic Speech Recognition (ASR) model released on June 4, 2026. It's part of NVIDIA's Nemotron Speech ecosystem within the NeMo framework.

The key achievement: a single checkpoint covers 40 language-locales — no per-language models, no model-swapping. The model transcribes in real time with native punctuation and capitalization built in, producing production-ready text without a separate punctuation-restoration step.

Open weights are available on Hugging Face under the OpenMDW-1.1 license.

Architecture: Cache-Aware FastConformer-RNNT

Nemotron 3.5 ASR uses a Cache-Aware FastConformer-RNNT architecture — an evolution of NVIDIA's original English streaming model nvidia/nemotron-speech-streaming-en-0.6b.

Key architectural innovations:

• FastConformer backbone — Efficient attention mechanism optimized for streaming audio
• Cache-aware design — Maintains a sliding window cache of previous audio frames, eliminating the latency drift found in traditional sliding-window models
• RNNT loss — RNN Transducer loss for accurate sequence-to-sequence speech recognition
• Prompt-based language-ID conditioning — A single model handles all 40 language-locales through conditional prompting

This architecture directly addresses a fundamental limitation of Whisper and similar models: sliding-window ASR systems suffer from latency drift as the window shifts through long audio streams. Nemotron 3.5's cache-aware approach keeps latency stable throughout the entire audio duration.

Target Workloads

Nemotron 3.5 ASR targets two primary workloads:

Language Support: 40 Language-Locales

A single 600M-parameter model covers 40 language-locales, including:

• English (US, UK, AU, IN), Spanish (ES, MX), French (FR, CA), German
• Italian, Portuguese (BR, PT), Dutch, Russian, Japanese, Korean
• Chinese (Mandarin), Hindi, Arabic, Turkish, Polish, Swedish
• Danish, Finnish, Norwegian, Thai, Vietnamese, Indonesian, and more

This multilingual capability in a single checkpoint is a significant advantage over models like Whisper, which require separate models for different language modes.

Benchmark Performance

While NVIDIA hasn't published a comprehensive benchmark table, the model is positioned as competitive with Whisper on standard English benchmarks and superior on streaming-specific metrics:

• Streaming latency — Lower and more stable than Whisper's sliding-window approach
• Clinical ASR — NVIDIA's own clinical workflow evaluation shows competitive results on medical terminology recognition
• Multilingual WER — Competitive word error rates across supported languages

NVIDIA has published a clinical ASR evaluation workflow (June 9, 2026) using Nemotron Speech for evaluating pronunciation accuracy on rare medical terminology — drug names, procedure names, and specialty-specific diagnoses.

Deployment & Ecosystem

Nemotron 3.5 ASR integrates into NVIDIA's broader ecosystem:

• NVIDIA NeMo Data Designer — For data preparation and synthetic data generation
• Magpie TTS Multilingual — For creating pronunciation-aware synthetic training data
• NVIDIA Agent Skills — For automated clinical ASR evaluation workflows
• DGX Spark / arm64 — Edge deployment support for on-device transcription

Clinical ASR Applications

NVIDIA has demonstrated Nemotron 3.5 ASR's capability in the clinical domain, where accuracy on rare medical terminology is critical. Drug names like Acetaminophen, Amlodipine, Cefazolin, and Biktarvy are not part of everyday vocabulary — off-the-shelf speech systems can sound fluent and still miss the words that matter most.

The clinical workflow uses synthetic data generation (SDG) to create pronunciation-aware evaluation datasets without real patient data, enabling rapid benchmark creation without HIPAA compliance overhead.

Related Articles

Qwen-AgentWorld 35B-A3B — Language World Models for General Agents Best AI Models for Hermes Agent — Local and cloud deployment options