Conformer-2

What is Conformer-2?

Conformer-2 represents the latest advancement in automatic speech recognition (ASR) from AssemblyAI, a leading provider of speech AI solutions. This state-of-the-art model is designed to transcribe spoken English audio with exceptional accuracy, even in challenging real-world conditions. Trained on an impressive 1.1 million hours of diverse English audio data, Conformer-2 builds directly on the foundation of its predecessor, Conformer-1, while delivering targeted enhancements in key areas like proper noun recognition, alphanumeric transcription, and overall noise robustness. For developers and businesses building AI applications that rely on voice data—such as call center analytics, podcast summarization, or virtual meeting transcription—Conformer-2 serves as a critical component in creating reliable, scalable speech-to-text pipelines.

Unlike generic ASR tools, Conformer-2 is optimized for practical, industry-specific use cases where precision matters most. It addresses common pain points in speech recognition, such as misinterpreting names, numbers, or handling background noise, making it invaluable for applications in customer service, media monitoring, and content creation. By leveraging cutting-edge research inspired by large language model scaling laws, AssemblyAI has crafted a model that not only matches but exceeds benchmarks in user-centric metrics, ensuring transcripts that are more readable and actionable.

How Does Conformer-2 Work?

At its core, Conformer-2 employs a sophisticated architecture rooted in the Conformer model family, which combines convolutional and recurrent neural networks for superior sequence modeling in audio processing. The training process draws from the noisy student-teacher (NST) methodology introduced in Conformer-1, but takes it further with model ensembling. This technique involves multiple "teacher" models generating pseudo-labels on vast unlabeled datasets, which then train the "student" model—Conformer-2 itself. Ensembling reduces variance and boosts robustness by exposing the model to a broader range of predictions, mitigating individual model failures and enhancing performance on unseen data.

Data scaling plays a pivotal role in Conformer-2's capabilities. Following insights from DeepMind's Chinchilla paper on optimal training compute for large models, AssemblyAI scaled the dataset to 1.1 million hours—170% more than Conformer-1—while expanding the model to 450 million parameters. This balanced approach adheres to speech-specific scaling laws, where audio hours are equated to text tokens (using a heuristic of 1 hour ≈ 7,200 words or 9,576 tokens). The result? A model that generalizes better across diverse audio sources, from clean podcasts to noisy phone calls.

Inference speed is another hallmark of Conformer-2. Despite its larger size, optimizations in AssemblyAI's serving infrastructure, including a custom GPU cluster with 80GB A100s and a fault-tolerant Slurm scheduler, reduce latency by up to 53.7%. For instance, transcribing a one-hour audio file now takes just 1.85 minutes, down from 4.01 minutes with Conformer-1. This efficiency is achieved without sacrificing accuracy, making it feasible for real-time or high-volume applications.

To integrate Conformer-2, users access it via AssemblyAI's API, which is generally available and set as the default model. No code changes are needed for existing users—they'll automatically benefit from the upgrades. The API supports features like the new speech_threshold parameter, allowing rejection of low-speech audio files (e.g., music or silence) to control costs and focus processing on relevant content. Getting started is straightforward: sign up for a free API token, explore the documentation, or test via the web-based Playground by uploading files or YouTube links.

Key Improvements and Performance Results

Conformer-2 maintains word error rate (WER) parity with Conformer-1 but shines in practical metrics that align with real-world needs. Here's a breakdown of its advancements:

• Proper Noun Error Rate (PPNER) Improvement (6.8%): Traditional WER overlooks the impact of errors in entities like names or addresses. AssemblyAI's custom PPNER metric, based on Jaro-Winkler similarity, evaluates character-level accuracy for proper nouns. Across 60+ hours of labeled data from domains like call centers and webinars, Conformer-2 reduces PPNER, leading to more consistent and readable transcripts. For example, in customer interactions, correctly capturing a client's name can prevent downstream miscommunications.

• Alphanumeric Transcription Accuracy (31.7% Improvement): Numbers and codes are crucial in finance, e-commerce, or verification scenarios. Conformer-2 was tested on 100 synthesized sequences (5-25 digits, voiced by 10 speakers), achieving a 30.7% relative reduction in character error rate (CER). It shows lower variance too, meaning fewer catastrophic mistakes—ideal for applications like transcribing credit card details or order confirmations.

• Noise Robustness (12.0% Improvement): Real audio often includes background noise, unlike sterile benchmarks. Using the LibriSpeech-clean dataset augmented with Gaussian noise at varying signal-to-noise ratios (SNR), Conformer-2 outperforms Conformer-1, especially at 0 dB SNR (equal signal and noise). This 43% edge over competitors in noisy conditions makes it robust for podcasts, broadcasts, or remote meetings.

These gains stem from enhanced pseudo-labeling with multiple teachers and diverse training data, ensuring the model handles variability in accents, speeds, and environments.

Use Cases and Practical Value

Conformer-2 empowers a wide array of AI-driven applications. In media and content creation, it excels at transcribing podcasts or videos, enabling auto-summarization, chapter detection, or sentiment analysis. For customer service and call centers, its noise handling and entity recognition improve analytics on support calls, identifying action items or customer pain points. Businesses in finance and e-commerce benefit from accurate numeric transcription for transaction logs or IVR systems.

The model's value lies in its scalability and ease of integration. Developers can build generative AI apps—like voice-enabled chatbots or automated report generation—without wrestling with custom training. AssemblyAI's enterprise-grade security, benchmarks, and support further enhance its appeal. Early adopters report faster processing and higher-quality outputs, directly impacting productivity and user experience.

Who is Conformer-2 For?

This model targets product teams, developers, and enterprises working with spoken data. If you're in AI research, needing robust ASR for experiments; a startup building no-code speech tools; or a large organization scaling media monitoring—Conformer-2 fits. It's particularly suited for those frustrated by off-the-shelf ASR's limitations in noisy or entity-heavy audio. Non-technical users can leverage the Playground for quick tests, while API users integrate it into workflows via Python, JavaScript, or other languages.

Why Choose Conformer-2?

In a crowded ASR landscape, Conformer-2 stands out for its research-backed innovations and customer-focused metrics. It avoids the pitfalls of overtrained or under-scaled models, delivering speed without compromise. Backed by AssemblyAI's in-house hardware and ongoing R&D into multimodality and self-supervised learning, it's future-proof. Plus, with free trials and transparent pricing, it's accessible for experimentation.

For the best results with speech recognition, start with Conformer-2 in your next project. Whether optimizing for accuracy in proper nouns, ensuring numeric precision, or braving noisy environments, this model sets a new standard. Explore AssemblyAI's docs for code samples, or contact sales for custom integrations—unlocking the full potential of voice AI has never been easier.