> ## Documentation Index
> Fetch the complete documentation index at: https://docs.pipecat.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Zoom AI Speech-to-Text

> Real-time STT with ZoomAISTTService over the Zoom AI Services Scribe API in Live mode

## Overview

`ZoomAISTTService` provides real-time speech recognition with the [Scribe API](https://developers.zoom.us/docs/ai-services/scribe/) from Zoom AI Services, streaming audio over a WebSocket in [Live mode](https://developers.zoom.us/docs/ai-services/scribe/live-mode/). Scribe detects when each user turn ends, so the service drives turn detection for live streaming without a local VAD. It supports Scribe's additional features, including English Pro model, multilingual models, language detection, speaker diarization, custom vocabulary, and text polishing, etc.

> [card] [Zoom AI STT API Reference](https://reference-server.pipecat.ai/en/latest/api/pipecat.services.zoom_ai.stt.html)
>
> Pipecat's API methods for Zoom AI STT integration

> [card] [Example Implementation](https://github.com/pipecat-ai/pipecat/blob/main/examples/voice/voice-zoom-ai.py)
>
> Complete voice agent with Scribe API

> [card] [Scribe Documentation](https://developers.zoom.us/docs/ai-services/scribe/)
>
> Official Zoom Scribe API documentation

> [card] [Zoom AI Services](https://www.zoom.ai/)
>
> Get credentials and manage your Zoom AI Services plan

## Installation

To use Zoom AI services, install the required dependency:

```bash theme={null}
uv add "pipecat-ai[zoom-ai]"
```

## Prerequisites

### Zoom AI Account Setup

Before using Zoom AI STT, you need:

1. **Zoom AI Services access**: Sign up through [Zoom AI Services](https://www.zoom.ai/)
2. **API key**: Create a Zoom AI Services API key. See [Get credentials](https://developers.zoom.us/docs/ai-services/auth/).

### Required Environment Variables

* `ZOOM_AI_API_KEY`: Your Zoom AI Services API key

## Configuration

### ZoomAISTTService

**api\_key** `str` *required*

Zoom AI Services API key. It is sent on every connection and reconnect, so it
must stay valid for as long as the service runs.

**url** `str` (default: wss\://api.zoom.us/v2/aiservices/scribe/live)

Scribe Live WebSocket URL. A plaintext `ws://` URL to a host other than
localhost logs a warning, since the API key and audio are sent unencrypted.

**sample\_rate** `int | None` (default: None)

Input audio sample rate in Hz. If `None`, uses the pipeline's audio sample
rate. Audio is resampled to 16 kHz for Scribe.

**should\_interrupt** `bool` (default: True)

Whether a user turn started by Scribe interrupts the bot.

**settings** `ZoomAISTTService.Settings` (default: None)

Runtime-configurable settings for the STT service. See [Settings](#settings)
below.

### Settings

Runtime-configurable settings passed via the `settings` constructor argument using `ZoomAISTTService.Settings(...)`. Settings map to the Scribe API fields of the same name; see the [Scribe documentation](https://developers.zoom.us/docs/ai-services/scribe/) for what each feature does and its availability. See [Service Settings](https://docs.pipecat.ai/pipecat/fundamentals/service-settings.md) for how settings work in Pipecat.

| Parameter | Type | Default | Description |
| - | - | - | - |
| `model` | `str \| None` | `None` | Recognition model, such as `zoom-scribe-en-pro` or `zoom-scribe-multilingual`. `None` lets Scribe choose a compatible model. See [Select a model](https://developers.zoom.us/docs/ai-services/scribe/speech-understanding/#select-a-model). |
| `language` | `Language \| str` | `Language.EN_US` | Language of the audio; with language detection, the preferred initial language. |
| `silence_duration_ms` | `int \| None` | `350` | Milliseconds of silence that end a turn. Pipecat's default is 350ms optimized for voice agent use case. `None` uses Scribe's default. |
| `diarization` | `bool \| None` | `False` | Whether Scribe labels each transcript with its speaker. See [Configuring Scribe Features](#configuring-scribe-features). |
| `language_detection` | `str \| None` | `None` | Language detection mode; Live mode supports `"continuous"`. `None` turns it off. See [With the Multilingual Model](#with-the-multilingual-model). |
| `vocabulary` | `list[str] \| None` | `None` | [Custom vocabulary](https://developers.zoom.us/docs/ai-services/scribe/custom-vocabulary/): names, acronyms, and terms to recognize. See [Configuring Scribe Features](#configuring-scribe-features). |
| `text_polishing` | `bool \| None` | `False` | Whether to return [Text Polishing](https://developers.zoom.us/docs/ai-services/scribe/text-polishing/) output as the transcript. It supports only en-US. |
| `extra` | `dict[str, Any]` | `{}` | Other Scribe fields that have no setting, shaped like the [`session.update` event](https://developers.zoom.us/docs/ai-services/scribe/live-mode/#configure-the-session) and merged into it, so new Scribe features can be used without an immediate Pipecat update. For an example, see [With the Multilingual Model](#with-the-multilingual-model). |

## Usage

### Basic Setup

```python theme={null}
import os

from pipecat.services.zoom_ai.stt import ZoomAISTTService

stt = ZoomAISTTService(
    api_key=os.getenv("ZOOM_AI_API_KEY"),
)
```

### Configuring Scribe Features

Scribe features are set through `Settings`. This example uses the English Pro model, adds a custom vocabulary, and turns on speaker diarization:

```python theme={null}
import os

from pipecat.services.zoom_ai.stt import ZoomAISTTService
from pipecat.transcriptions.language import Language

stt = ZoomAISTTService(
    api_key=os.getenv("ZOOM_AI_API_KEY"),
    settings=ZoomAISTTService.Settings(
        model="zoom-scribe-en-pro",
        language=Language.EN_US,
        vocabulary=["Pipecat", "Zoom Scribe", "Kubernetes"],
        diarization=True,
    ),
)
```

With diarization on, each `TranscriptionFrame` carries Scribe's event in `result`, with the speaker label in `result["speaker"]` (for example `"Speaker 1"`), and `result["speaker_segments"]` when several people spoke in one turn.

See [Select a model](https://developers.zoom.us/docs/ai-services/scribe/speech-understanding/#select-a-model), [Custom vocabulary](https://developers.zoom.us/docs/ai-services/scribe/custom-vocabulary/), and the [Scribe documentation](https://developers.zoom.us/docs/ai-services/scribe/) for what each feature does.

### With the Multilingual Model

For callers who may switch languages, use `zoom-scribe-multilingual` with continuous language detection. `language` is the language Scribe expects first. Each `TranscriptionFrame` has the language Scribe detected for that transcript in `language`.

```python theme={null}
import os

from pipecat.services.zoom_ai.stt import ZoomAISTTService
from pipecat.transcriptions.language import Language

stt = ZoomAISTTService(
    api_key=os.getenv("ZOOM_AI_API_KEY"),
    settings=ZoomAISTTService.Settings(
        model="zoom-scribe-multilingual",
        language=Language.EN_US,
        language_detection="continuous",
        # Optional: use candidate_languages to restrict detection.
        extra={"config": {"language_detection": {"candidate_languages": ["en-US", "fr-FR"]}}},
    ),
)
```

See [Detect languages](https://developers.zoom.us/docs/ai-services/scribe/speech-understanding/#detect-languages) and [Multilingual model languages](https://developers.zoom.us/docs/ai-services/scribe/speech-understanding/#multilingual-model-languages) for the supported languages.

### Updating Settings Mid-Stream

Send an `STTUpdateSettingsFrame` to change settings during a conversation. The service reconnects with a new Scribe session once the current turn ends:

```python theme={null}
from pipecat.frames.frames import STTUpdateSettingsFrame

await worker.queue_frame(
    STTUpdateSettingsFrame(
        delta=ZoomAISTTService.Settings(vocabulary=["Pipecat", "Order 1001"])
    )
)
```

## Notes

* **Final transcripts only**: The service pushes a final transcript per turn, with no interim results.
* **Idle sessions**: Scribe ends a session after 30 seconds without audio, so the service sends keepalive silence when input stops (see [Session limits](https://developers.zoom.us/docs/ai-services/scribe/live-mode/#session-limits)). Keepalive silence and the silence sent to end a turn count toward usage.
* **Errors**: If Zoom refuses the connection (for example, an invalid API key), the error includes Zoom's reason. If Scribe rejects the session's settings, the error has the `INVALID_REQUEST` category, since a new session would be rejected the same way. Errors include the session's tracking ID (`x-zm-trackingid`), which Zoom support uses to find a session. See [Error Handling](https://docs.pipecat.ai/pipecat/fundamentals/error-handling.md).
* **Time to final segment (TTFS)**: Scribe defines the turn boundary, so the service reports no TTFS metric.

## Event Handlers

Zoom AI STT supports the standard [service connection events](https://docs.pipecat.ai/api-reference/server/events/service-events.md):

| Event | Description |
| - | - |
| `on_connected` | Connected to Scribe Live WebSocket |
| `on_disconnected` | Disconnected from Scribe Live WebSocket |
| `on_connection_error` | Connection to Scribe Live failed |

```python theme={null}
@stt.event_handler("on_connected")
async def on_connected(service):
    print("Connected to Zoom Scribe")
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.