Skip to main content

LLM and tools

LLM
Add function calling to your Pipecat bot. Examples exist for each LLM provider supported in Pipecat.View Recipe β†’
LLM
Skip the schema boilerplate: pass Python functions straight into the context and let Pipecat derive the tool schema from the signature and docstring.View Recipe β†’
LLM
Run long tool calls in the background so the bot keeps talking while the work finishes. Covers @tool_options, on_function_calls_started, and cancellation on interruption.View Recipe β†’
LLM
Use a model’s thinking/reasoning mode in a voice pipeline, including reasoning alongside function calls. Examples exist for Anthropic, Google, and OpenAI Responses.View Recipe β†’
LLM
Give the agent live web search with KeenableWebSearch, so it can answer questions about current events beyond the model’s training data.View Recipe β†’
Integration
Use MCP tools to interact with external services.View Recipe β†’
Integration
Use MCP tools over Streamable HTTP transport for remote MCP servers.View Recipe β†’
Integration
Combine tools from multiple MCP servers (stdio and HTTP) in a single bot.View Recipe β†’

Context and memory

Context
Automatically compress conversation history as it approaches the token limit, preserving in-flight function calls. See context summarization for configuration, including using a cheaper dedicated LLM for the summary.View Recipe β†’
Context
Save and restore conversation context so a returning user picks up where they left off. Examples exist for standard LLMs and speech-to-speech models.View Recipe β†’
Context
Answer questions from a document the model was never trained on, by routing them through a second, cheaper model holding the source in its context window.View Recipe β†’
Context
Store and recall memories across conversations with Mem0, so the bot personalizes its greeting and responses based on past interactions.View Recipe β†’

Speech and audio

Recording & Logging
Collect audio frames from the user and bot for later processing or storage.View Recipe β†’
Audio
Play a background sound in your Pipecat bot. The audio is mixed with the transport audio to create a single integrated audio stream.View Recipe β†’
Audio
Play sound effects in your Pipecat bot.View Recipe β†’
TTS
Use VoiceFormatter so currency, phone numbers, dates, and acronyms are spoken the way a person would say them β€” β€œ$42.50” becomes β€œforty-two dollars and fifty cents” rather than β€œdollar sign four two point five zero”.View Recipe β†’
TTS
Compose individual text transforms instead of the VoiceFormatter bundle, for fine-grained control over which run and in what order.View Recipe β†’
TTS
Use PatternPairAggregator to switch TTS voices inside a single streamed response β€” a narrator and two characters in one story.View Recipe β†’
STT/TTS/LLM
Change STT, TTS, and LLM settings mid-session β€” voice, model, language, temperature. See service settings for the API.View Recipes β†’

Turn management

User Interaction
Specify a strategy for mute to mute user input, allowing the bot to continue without interruption.View Recipe β†’
User Interaction
Use a wake phrase to wake up your Pipecat bot.View Recipe β†’
User Interaction
Detect when a user is idle and automatically respond.View Recipe β†’
User Interaction
Detect when a user has finished speaking and automatically respond. Learn more about smart-turn model.View Recipe β†’
User Interaction
Learn how to configure interruption strategies for your Pipecat bot.View Recipe β†’
User Interaction
Use the LLM to detect when the user was cut off mid-thought, suppress the response, and re-engage after a timeout instead of answering half a sentence.View Recipe β†’
User Interaction
Use a classifier to keep the turn open when the user was cut off mid-thought or asked for time, so the LLM only answers complete turns.View Recipe β†’
User Interaction
Extend a turn-detecting STT’s timing with your own stop strategy β€” hold the turn open for a beat after Deepgram Flux proposes the end, so afterthoughts land in the same user message.View Recipe β†’
Events
Handle user and bot end of turn events to add custom logic after a turn.View Recipe β†’

Multilingual

Multilingual
A ParallelPipeline example showing how to dynamically switch languages.View Recipe β†’
Multilingual
Translate the user’s speech into another language and speak it back in real time.View Recipe β†’

Telephony

Telephony
Build a keypad-driven phone menu with no STT in the pipeline: DTMFAggregator turns each key sequence into a transcription the LLM reacts to. See IVR navigation for the other direction.View Recipe β†’

Vision and video

Vision
Pass a video frame from a live video stream to a model and get a description.View Recipe β†’
Video
Output more than one video track at once β€” the default camera track plus a second processed track sent to a custom destination.View Recipe β†’

Pipeline and reliability

Pipeline
Write your own FrameProcessor to inspect or transform frames as they move through the pipeline. See custom frame processors for the walkthrough.View Recipe β†’
Pipeline
Put two providers behind a ServiceSwitcher and move work between them mid-session, manually or on failure.View Recipe β†’
Pipeline
React to processor health with is_usable and on_usable_changed: fall back to a second TTS provider when the first stops working, and end the bot only when the last one is gone.View Recipe β†’
Pipeline
Pass database handles, HTTP clients, or per-user state through PipelineWorker(app_resources=...) and read them from tool handlers and custom processors.View Recipe β†’

Debugging and observability

Recording & Logging
Capture user and bot transcripts for later processing or storage.View Recipe β†’
Recording & Logging
Parse user email from the LLM response.View Recipe β†’
Debugging
Learn how to debug your Pipecat bot with an Observer by observing frames flowing through the pipeline.View Recipe β†’
Debugging
Enable heartbeat frames to detect a stalled or blocked pipeline in production.View Recipe β†’
Debugging
A live graphical debugger for the Pipecat voice and multimodal conversational AI framework. It lets you visualize pipelines and debug frames in real time β€” so you can see exactly what your bot is thinking and doing.View Recipe β†’

πŸ’‘ Need Help Getting Started?

More Recipes

Browse all recipes on GitHub

Examples

Browse complete applications

Discord Community

Get help from the community