NVIDIA Nemotron 3.5 Transcribe Unifies Speech AI Tasks

NVIDIA just released Nemotron 3.5 Transcribe, a 34-billion-parameter Mixture-of-Experts speech model that folds what used to be a scattered set of audio tools into one open-weight checkpoint. It brings together transcription across 40 languages, speaker attribution, translation, and audio question answering, which changes how enterprises can handle large stores of recorded audio and run speech inference on their GPUs.

i10X Quick Take

Quick Take

The breakthrough: Nemotron 3.5 Transcribe acts as a unified speech LLM, replacing multi-tool pipelines with a single model capable of processing one-hour recordings and accurately attributing overlapping speech.
The architecture: Built on a hybrid Mamba-2 and Transformer MoE backbone, it holds 34 billion total parameters but activates only 4.3 billion per step, allowing it to run efficiently on a single H100 80GB GPU.
The limitation: This is strictly an offline model designed for recorded audio; developers building real-time, streaming voice agents will need to rely on different architectures.
The data engine: NVIDIA also open-sourced Granary v2, a massive 463,000-hour curated multilingual dataset that reveals the exact data pipeline used to train highly adaptable, promptable speech systems.

What happened

NVIDIA launched Nemotron 3.5 Transcribe under the commercially viable OpenMDW-1.1 license, delivering a specialized “speech-augmented language model” (SALM). Unlike standard Automatic Speech Recognition (ASR) systems that only output plain text, Nemotron 3.5 can be prompted to translate across 31 languages, identify up to eight speakers even during rapid overlapping conversations, format text with exact word timestamps, and redact Personally Identifiable Information (PII) directly during inference.

To achieve this, NVIDIA engineered a complex architecture combining a 0.6B Transformer ASR encoder and a frozen diarization module with a 52-layer MoE backbone. Crucially, the model integrates 23 Mamba-2 layers alongside its MoE layers. This state-space model (SSM) integration allows the system to handle a massive 65,536-token context window – enough to ingest a full hour of audio – without the quadratic memory explosion typical of pure Transformer architectures.

Alongside the model, NVIDIA released Granary v2, a comprehensive data curation pipeline spanning 48 languages. By open-sourcing the dataset and the training methodology simultaneously, NVIDIA is providing the AI developer community with both the inference engine and the data curation methodology required to build highly adapted, domain-specific speech applications.

Why it matters now

The release of Nemotron 3.5 Transcribe signals a structural shift in how AI infrastructure handles speech. Historically, processing a corporate meeting or a call center log required a brittle, multi-stage pipeline: an ASR model like Whisper for the text, a separate diarization model to figure out who was speaking, and a third LLM call to translate or extract decisions. This approach multiplied GPU memory requirements, increased failure points, and compounded latency. By combining these tasks into one promptable checkpoint, NVIDIA is driving compute consolidation at the inference layer.

The architecture itself serves as a blueprint for the next generation of multimodal LLMs. By utilizing a Mixture-of-Experts approach, the model only activates 4.3 billion parameters per token. This keeps inference costs low enough that API providers – like DeepInfra, which is already pricing the model at a highly competitive $0.002 per audio minute – can undercut traditional transcription services. Furthermore, integrating Mamba-2 layers proves that hybrid SSM-Transformer architectures are moving from research papers into production-grade enterprise tools, specifically to solve the context-length bottleneck in audio processing.

Strategically, NVIDIA is deepening its moat by commoditizing the software layer to sell more hardware. By releasing highly capable open-weight models natively optimized for BF16, FP8, and NVFP4 precision formats, NVIDIA ensures that the most efficient way to process enterprise audio lakes is by deploying their latest Blackwell and Hopper GPUs.

Who is most affected

Stakeholder	Impact	Why
Enterprise AI Architects	Pipeline Consolidation	They can replace complex, multi-model transcription pipelines with a single promptable checkpoint, radically reducing inference overhead and maintenance.
Commercial ASR APIs	Margin Pressure	Open-weight models that natively handle diarization, translation, and PII redaction directly challenge the premium pricing of proprietary, legacy speech APIs.
Hardware Buyers & Self-Hosters	Optimized Utilization	The MoE design (4.3B active parameters) means this heavy-duty model can run on a single H100, lowering the barrier to self-hosting massive audio workloads securely.

About the analysis

Researched and written with AI assistance under the editorial responsibility of the i10X News team. The title image is an AI-generated illustration. Read our AI Use Policy. Spotted an error? Request a correction.

This independent i10x analysis maps the architectural advancements and market impacts of NVIDIA’s new speech LLM, synthesizing technical model cards, benchmark data, and developer documentation to inform AI founders, CTOs, and infrastructure teams.

i10x Perspective

While the consumer AI narrative fixates on real-time, low-latency streaming voice assistants, NVIDIA is quietly claiming the enterprise backend. Nemotron 3.5 Transcribe proves that the future of corporate knowledge extraction isn’t just about faster transcription – it is about deep, asynchronous, prompt-driven audio understanding. We’ve seen this pattern before when open models reshaped text workflows, and the companies that control the offline “audio lake” will own the next massive wave of enterprise intelligence.