Zoom AI Speech-to-Text

Real-time STT with ZoomAISTTService over the Zoom AI Services Scribe API in Live mode

​

Overview

ZoomAISTTService provides real-time speech recognition with the Scribe API from Zoom AI Services, streaming audio over a WebSocket in Live mode. Scribe detects when each user turn ends, so the service drives turn detection for live streaming without a local VAD. It supports Scribe’s additional features, including English Pro model, multilingual models, language detection, speaker diarization, custom vocabulary, and text polishing, etc.

Zoom AI STT API Reference

Pipecat’s API methods for Zoom AI STT integration

Example Implementation

Complete voice agent with Scribe API

Scribe Documentation

Official Zoom Scribe API documentation

Zoom AI Services

Get credentials and manage your Zoom AI Services plan

​

Installation

To use Zoom AI services, install the required dependency:

uv add "pipecat-ai[zoom-ai]"

​

Prerequisites

​

Zoom AI Account Setup

Before using Zoom AI STT, you need:

1. Zoom AI Services access: Sign up through Zoom AI Services
2. API key: Create a Zoom AI Services API key. See Get credentials.

​

Required Environment Variables

• ZOOM_AI_API_KEY: Your Zoom AI Services API key

​

Configuration

​

ZoomAISTTService

api_key · str · required

Zoom AI Services API key. It is sent on every connection and reconnect, so it must stay valid for as long as the service runs.

url · str · default: "wss://api.zoom.us/v2/aiservices/scribe/live"

Scribe Live WebSocket URL. A plaintext ws:// URL to a host other than localhost logs a warning, since the API key and audio are sent unencrypted.

sample_rate · int | None · default: "None"

Input audio sample rate in Hz. If None, uses the pipeline’s audio sample rate. Audio is resampled to 16 kHz for Scribe.

should_interrupt · bool · default: "True"

Whether a user turn started by Scribe interrupts the bot.

settings · ZoomAISTTService.Settings · default: "None"

Runtime-configurable settings for the STT service. See Settings below.

​

Settings

Runtime-configurable settings passed via the settings constructor argument using ZoomAISTTService.Settings(...). Settings map to the Scribe API fields of the same name; see the Scribe documentation for what each feature does and its availability. See Service Settings for how settings work in Pipecat.

Parameter	Type	Default	Description
model	str | None	None	Recognition model, such as zoom-scribe-en-pro or zoom-scribe-multilingual. None lets Scribe choose a compatible model. See Select a model.
language	Language | str	Language.EN_US	Language of the audio; with language detection, the preferred initial language.
silence_duration_ms	int | None	350	Milliseconds of silence that end a turn. Pipecat’s default is 350ms optimized for voice agent use case. None uses Scribe’s default.
diarization	bool | None	False	Whether Scribe labels each transcript with its speaker. See Configuring Scribe Features.
language_detection	str | None	None	Language detection mode; Live mode supports "continuous". None turns it off. See With the Multilingual Model.
vocabulary	list[str] | None	None	Custom vocabulary: names, acronyms, and terms to recognize. See Configuring Scribe Features.
text_polishing	bool | None	False	Whether to return Text Polishing output as the transcript. It supports only en-US.
extra	dict[str, Any]	{}	Other Scribe fields that have no setting, shaped like the session.update event and merged into it, so new Scribe features can be used without an immediate Pipecat update. For an example, see With the Multilingual Model.

​

Usage

​

Basic Setup

import os

from pipecat.services.zoom_ai.stt import ZoomAISTTService

stt = ZoomAISTTService(
    api_key=os.getenv("ZOOM_AI_API_KEY"),
)

​

Configuring Scribe Features

Scribe features are set through Settings. This example uses the English Pro model, adds a custom vocabulary, and turns on speaker diarization:

import os

from pipecat.services.zoom_ai.stt import ZoomAISTTService
from pipecat.transcriptions.language import Language

stt = ZoomAISTTService(
    api_key=os.getenv("ZOOM_AI_API_KEY"),
    settings=ZoomAISTTService.Settings(
        model="zoom-scribe-en-pro",
        language=Language.EN_US,
        vocabulary=["Pipecat", "Zoom Scribe", "Kubernetes"],
        diarization=True,
    ),
)

With diarization on, each TranscriptionFrame carries Scribe’s event in result, with the speaker label in result["speaker"] (for example "Speaker 1"), and result["speaker_segments"] when several people spoke in one turn.

See Select a model, Custom vocabulary, and the Scribe documentation for what each feature does.

​

With the Multilingual Model

For callers who may switch languages, use zoom-scribe-multilingual with continuous language detection. language is the language Scribe expects first. Each TranscriptionFrame has the language Scribe detected for that transcript in language.

import os

from pipecat.services.zoom_ai.stt import ZoomAISTTService
from pipecat.transcriptions.language import Language

stt = ZoomAISTTService(
    api_key=os.getenv("ZOOM_AI_API_KEY"),
    settings=ZoomAISTTService.Settings(
        model="zoom-scribe-multilingual",
        language=Language.EN_US,
        language_detection="continuous",
        # Optional: use candidate_languages to restrict detection.
        extra={"config": {"language_detection": {"candidate_languages": ["en-US", "fr-FR"]}}},
    ),
)

See Detect languages and Multilingual model languages for the supported languages.

​

Updating Settings Mid-Stream

Send an STTUpdateSettingsFrame to change settings during a conversation. The service reconnects with a new Scribe session once the current turn ends:

from pipecat.frames.frames import STTUpdateSettingsFrame

await worker.queue_frame(
    STTUpdateSettingsFrame(
        delta=ZoomAISTTService.Settings(vocabulary=["Pipecat", "Order 1001"])
    )
)

​

Notes

• Final transcripts only: The service pushes a final transcript per turn, with no interim results.
• Idle sessions: Scribe ends a session after 30 seconds without audio, so the service sends keepalive silence when input stops (see Session limits). Keepalive silence and the silence sent to end a turn count toward usage.
• Errors: If Zoom refuses the connection (for example, an invalid API key), the error includes Zoom’s reason. If Scribe rejects the session’s settings, the error has the INVALID_REQUEST category, since a new session would be rejected the same way. Errors include the session’s tracking ID (x-zm-trackingid), which Zoom support uses to find a session. See Error Handling.
• Time to final segment (TTFS): Scribe defines the turn boundary, so the service reports no TTFS metric.

​

Event Handlers

Zoom AI STT supports the standard service connection events:

Event	Description
on_connected	Connected to Scribe Live WebSocket
on_disconnected	Disconnected from Scribe Live WebSocket
on_connection_error	Connection to Scribe Live failed

@stt.event_handler("on_connected")
async def on_connected(service):
    print("Connected to Zoom Scribe")