LLMTextProcessor

A processor for aggregating LLMTextFrames into logical units before passing them to downstream services

​

Overview

LLMTextProcessor is a processor designed to aggregate LLMTextFrames into coherent text units before passing them to downstream services, such as TTS. By utilizing text aggregators, it ensures that text is properly segmented and structured, enhancing the quality of subsequent processing. This processor expects LLMTextFrames as input and outputs AggregatedTextFrames containing the aggregated text.

When an LLMTextProcessor is in use, text aggregation is handled upstream and the TTS service’s built-in aggregator is bypassed.

The benefit of pre-aggregating LLM text frames is that it allows for more controlled and meaningful text synthesis. Downstream services can operate on complete sentences or logical text blocks. For TTS services, this means being able to customize how certain types of text are spoken (e.g., spelling out phone numbers, stripping out url protocols, or inserting other tts-specific annotations) or even skipping over certain text segments entirely (e.g., code snippets or markup). For other services, such as RTVI, it allows for sending these logical text units as separate bot-output messages, supporting custom client-side handling and rendering (e.g. collapsible code blocks, clickable links, etc.).

LLMTextProcessor API Reference

Pipecat’s API methods for LLMTextProcessor

Example Implementation

Complete example with LLMTextProcessor and custom TTS and RTVI handling

​

Constructor Parameters

text_aggregator · BaseTextAggregator · default: "None"

An instance of a text aggregator (e.g., PatternPairAggregator or a custom aggregator type) used to aggregate incoming text from LLMTextFrames. If None, a SimpleTextAggregator will be used by default, aggregating text based on sentence boundaries.

text_transforms · list[tuple[TextType | str, TextTransform]] · default: "None"

Callables that rewrite each aggregation, as (text_type | "*", transform) pairs. A transform takes the text and its type and returns the rewritten text, and "*" matches every type. The rewrite reaches the TTS service, the context, and the client. A transform that leaves only whitespace drops the aggregation. A transform registered on the TTS service rewrites only what is spoken. TextTransform is in pipecat.utils.text.transforms.

​

Methods

​

add_text_transformer

processor.add_text_transformer(transform_function, text_type="*")

Registers a transform for aggregations of text_type, or for every type with "*".

​

remove_text_transformer

processor.remove_text_transformer(transform_function, text_type="*")

Removes the transform registered with the same function and text_type.

​

Usage

The LLMTextProcessor should be integrated into your pipeline after the LLM service and before any services that consume text, such as TTS. It processes incoming LLMTextFrames, aggregates their text content, and outputs AggregatedTextFrames.

For more usage examples, check out the docs for the PatternPairAggregator.

from pipecat.pipeline.pipeline import Pipeline
from pipecat.processors.aggregators.llm_text_processor import LLMTextProcessor
from pipecat.utils.text.pattern_pair_aggregator import MatchAction, PatternPairAggregator

...

llm_text_aggregator = PatternPairAggregator()
llm_text_aggregator.add_pattern(
    type="code",
    start_pattern="<code>",
    end_pattern="</code>",
    action=MatchAction.AGGREGATE,
)
llm_text_processor = LLMTextProcessor(text_aggregator=llm_text_aggregator)

...

# Pipeline - The following pipeline is typical for a STT->LLM->TTS bot + RTVI
#            with the addition of the LLMTextProcessor to handle special text segments.
pipeline = Pipeline(
    [
        transport.input(),
        rtvi,
        stt,
        context_aggregator.user(),
        llm,
        llm_text_processor,
        tts,
        transport.output(),
        context_aggregator.assistant(),
    ]
)