> ## Documentation Index
> Fetch the complete documentation index at: https://docs.pipecat.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Pipecat Classifiers

> Answer small yes/no, choice, and score questions with classifiers, and get probabilities without a conversation LLM turn.

Besides holding the conversation, a voice agent makes lots of small decisions. Did the call reach a person or a voicemail? Which button does "the blue one" mean? Is this click worth a comment? Is the customer about to cancel?

You could ask your conversation LLM each time, but that costs a full LLM turn, adds text to the context, and leaves you parsing free-form answers. A **classifier** is built for these questions. You ask it typed questions about some state and get typed answers with probabilities, in one call, without touching the conversation.

## Pipeline Integration

A classifier is not a pipeline component. It's a plain object: whoever needs answers creates one, keeps it, and calls it. Nothing is added to the pipeline and no frames flow into it.

Most of the time you don't call it yourself. Pipecat components that make small decisions take one as a `classifier` argument and ask it for you:

```python theme={null}
import os

from pipecat.classifiers.typesafe.jev.classifier import JevClassifier
from pipecat.extensions.voicemail.voicemail_detector import VoicemailDetector

classifier = JevClassifier(api_key=os.getenv("TYPESAFE_API_KEY"))

voicemail = VoicemailDetector(classifier=classifier)
```

The rest of this page shows how to ask a classifier directly, which is also how those components use it.

## Asking a Question

Create a classifier and ask it a question about a **state**, the thing the question is about:

```python theme={null}
import os

from pipecat.classifiers.base_classifier import YesNoQuestion
from pipecat.classifiers.typesafe.jev.classifier import JevClassifier

classifier = JevClassifier(api_key=os.getenv("TYPESAFE_API_KEY"))

results = await classifier.yes_no(
    "Hi, you've reached Dana. Leave a message.",
    {"voicemail": YesNoQuestion(instructions="is this a voicemail greeting?")},
)

results["voicemail"].is_yes  # True
results["voicemail"].probability  # 0.97
```

Questions are passed by name, and the results come back under the same names. That's what lets you ask several questions at once, as shown below.

## Three Kinds of Question

Every question is one of three kinds, and each kind has its own result type.

> [tab] Yes or no
>
> A `YesNoQuestion` asks whether the state meets a condition. Add `yes` and `no` when the question alone leaves the boundary open.
>
> ```python theme={null}
> YesNoQuestion(
>     instructions="Does the customer want money back?",
>     yes="asks for a refund, a credit, or a charge to be reversed",
>     no="anything else",
> )
> ```
>
> The `YesNoResult` has the `probability` that the answer is yes and `is_yes`, which is true when yes is the likelier answer.

> [tab] Choice
>
> A `ChoiceQuestion` asks which of several options fits. Each option maps to a description of when it applies, or `None` when its name says enough.
>
> ```python theme={null}
> ChoiceQuestion(
>     instructions="Which team should take this call?",
>     options={
>         "billing": "invoices, payments, refunds",
>         "technical": "bugs, outages, errors",
>         "sales": "pricing, new contracts",
>         "other": None,
>     },
> )
> ```
>
> The `ChoiceResult` has the `choice`, a `probabilities` dict with one entry per option, and the `confidence` in the choice.

> [tab] Score
>
> A `ScoreQuestion` asks where the state falls on an ordered scale. List at least two `levels`, lowest first.
>
> ```python theme={null}
> ScoreQuestion(
>     instructions="How upset is the customer?",
>     levels=["calm", "frustrated", "angry"],
> )
> ```
>
> The `ScoreResult` has a `score` from 0 (the first level) to one less than the number of levels. It's the probability-weighted position, so a score of 1.4 sits between "frustrated" and "angry". It also has the probability of each level in `levels`, and `probability(level)` to look one up.

## Asking Several Questions at Once

Several questions about the same state go in one call rather than one call each. `ask()` takes any mix of kinds and returns a result of the right type for each:

```python theme={null}
from pipecat.classifiers.base_classifier import (
    ChoiceQuestion,
    ScoreQuestion,
    YesNoQuestion,
)

transcript = [
    {"role": "assistant", "content": "Thanks for calling Acme. How can I help you today?"},
    {"role": "user", "content": "I was charged twice for March. This is the third time I'm calling."},
    {"role": "assistant", "content": "I'm sorry about that. Let me look at the account."},
    {"role": "user", "content": "Honestly, if this isn't fixed today I'm cancelling."},
]

results = await classifier.ask(
    transcript,
    {
        "refund": YesNoQuestion(instructions="Does the customer want money back?"),
        "department": ChoiceQuestion(
            instructions="Which team should take this call?",
            options={"billing": None, "technical": None, "sales": None, "other": None},
        ),
        "mood": ScoreQuestion(
            instructions="How upset is the customer?",
            levels=["calm", "frustrated", "angry"],
        ),
    },
)
```

With `ask()`, each result's type depends on its question. When every question is the same kind, `yes_no()`, `choice()` and `score()` return results that are already typed.

Notice the state here. It can be plain text, or structured data such as a transcript with speaker roles, a dict of fields, or a trimmed screen snapshot. Structured data tells the classifier more: with roles, it knows which lines the customer said.

## Asking About Images

Some classifiers can look at images as well as the state. Build a `ClassifierImage` from an image frame, such as a frame from the user's camera, and pass it as `images=`:

```python theme={null}
from pipecat.classifiers.base_classifier import ClassifierImage, YesNoQuestion

image = await ClassifierImage.from_frame(frame)
results = await classifier.yes_no(
    "A frame from the user's camera.",
    {"person": YesNoQuestion(instructions="Is there a person in the frame?")},
    images=[image],
)
```

`ClefClassifier`, `JevClassifier`, `OpenAIDecisionsClassifier` and `LLMClassifier` take images (the LLM behind an `LLMClassifier` must be able to see them). A classifier's `supports_images` property says whether it can, and one that cannot raises `ClassifierError` when given images.

## Choosing a Classifier

Every classifier Pipecat includes answers the same questions and returns the same results, so you can swap one for another.

| | `JevClassifier` | `ClefClassifier` | `LLMClassifier` |
| - | - | - | - |
| **Backed by** | Jev, TypeSafe's hosted classification model | Clef, Cloudflare's decision model on Workers AI | Any Pipecat LLM service with `run_inference()` |
| **Speed** | About a tenth of a second | A few hundred milliseconds | A full LLM request |
| **Probabilities** | Calibrated | The model's own | Whatever the LLM writes |
| **Setup** | `uv add "pipecat-ai[typesafe]"` and a `TYPESAFE_API_KEY` | `uv add "pipecat-ai[cloudflare]"` and a Cloudflare account ID and token | None beyond the LLM you already use |
| **Metrics** | Time and tokens | Time and tokens | Time only |

`LLMClassifier` wraps an LLM service you already have. It calls the service directly, outside the pipeline, so the service doesn't need to be in one:

```python theme={null}
from pipecat.classifiers.llm.classifier import LLMClassifier
from pipecat.services.openai.llm import OpenAILLMService

llm = OpenAILLMService(
    api_key=os.getenv("OPENAI_API_KEY"),
    settings=OpenAILLMService.Settings(model="gpt-4o-mini"),
)
classifier = LLMClassifier(llm=llm)
```

Start with `LLMClassifier` if you'd rather not add a dependency, and switch to `JevClassifier` when decisions sit on the path to the user hearing a reply, such as voicemail detection, where every extra fraction of a second is noticeable.

## Acting on the Answer

`is_yes` is true when yes is the likelier answer, a probability of 0.5 or more. When acting on a wrong answer is costly, compare the probability with a threshold of your own:

```python theme={null}
result = (await classifier.yes_no(state, {"cancel": question}))["cancel"]
if result.probability > 0.8:
    await offer_retention_discount()
```

Thresholds are only meaningful when the probabilities are calibrated, which is what Jev provides. `JevClient` pins the Jev model version by default, so thresholds you tune keep holding until you choose to upgrade.

## Using a Classifier in Your Own Code

When you ask a classifier from your own processor or agent, keep three things in mind:

* **Lifecycle.** Call `setup()` with your task manager before the first question and `cleanup()` when you are done. `JevClassifier` opens its connection in `setup()`, so the first question does not pay for it.
* **Errors.** A classifier that cannot answer, or doesn't answer in time, raises `ClassifierError`. Every classifier has a timeout, so a call never hangs. Decide what your code does without an answer.
* **Metrics.** After every call, the classifier fires `on_metrics` with the time it took and, when it knows, the tokens it used. A classifier can't push frames, so a processor that owns one pushes the data as a `MetricsFrame`.

```python theme={null}
from pipecat.classifiers.base_classifier import ClassifierError, YesNoQuestion
from pipecat.frames.frames import MetricsFrame
from pipecat.processors.frame_processor import FrameProcessor


class ChurnDetector(FrameProcessor):
    def __init__(self, classifier):
        super().__init__()
        self._classifier = classifier
        self._classifier.add_event_handler("on_metrics", self._on_metrics)

    async def setup(self, setup):
        await super().setup(setup)
        await self._classifier.setup(self.task_manager)

    async def cleanup(self):
        await super().cleanup()
        await self._classifier.cleanup()

    async def _on_metrics(self, classifier, data):
        await self.push_frame(MetricsFrame(data=data))

    async def _is_churning(self, transcript) -> bool:
        question = YesNoQuestion(instructions="Might the customer leave?")
        try:
            results = await self._classifier.yes_no(transcript, {"churn": question})
        except ClassifierError:
            return False
        return results["churn"].probability > 0.8
```

## Where Pipecat Uses Classifiers

Pipecat components that make small decisions take a `classifier` argument, so you choose what answers them. A few examples:

* **[Voicemail detection](https://docs.pipecat.ai/pipecat/fundamentals/voicemail.md).** `VoicemailDetector` asks a choice question, a person or a voicemail, about the transcript so far after each transcription. It acts on the latest answer once the caller goes quiet.
* **[Turn completion](https://docs.pipecat.ai/api-reference/server/utilities/turn-management/user-turn-strategies.md#classifieruserturnstrategies).** `ClassifierUserTurnStrategies` asks a choice question, whether the user's turn is complete, cut off mid-sentence, or waiting for time to think, at each pause with new words. The turn stays open until the classifier finds it complete or a timeout passes, so the LLM only answers finished turns.
* **[Controlling the UI](https://docs.pipecat.ai/pipecat/learn/ui-worker.md).** `UIWorker` asks which element on screen the user means (a choice among the named elements), whether something is true of the screen (yes or no), and which elements match a description (one yes/no question per element, all in one call). That's how it finds, checks and acts on the page without an LLM turn.
* **[Evals](https://docs.pipecat.ai/pipecat/evals/scenario-configuration.md#judging-with-a-classifier).** The eval judge asks a classifier whether each reply the agent gives meets a scenario's criterion: yes, no, or not answered yet. By default that is an `LLMClassifier` over a local model, and a scenario's `judge:` block can swap in a `JevClassifier`, which answers each check in a few hundred milliseconds.
* **Your own code.** Anything that owns a classifier can ask it questions of its own. For example, a custom `UIWorker` job can ask which checkbox each item the user named refers to, one choice question per item in a single call.

## Key Takeaways

* **A classifier is a plain object** that answers typed questions about some state. It doesn't sit in the pipeline.
* **Three kinds of question**: yes or no, a choice among options, and a score on a scale, each with probabilities.
* **Ask several at once**: questions go by name, and all the questions about one state share one call.
* **One interface for every classifier**: `JevClassifier` for fast, calibrated answers, `ClefClassifier` over Cloudflare's Workers AI, and `LLMClassifier` over any LLM service you already use.

## What's Next

With the single-agent basics and classifiers covered, let's see how Pipecat coordinates multiple agents, starting with giving an agent its own LLM and tools.

> [card] [Multiple LLM Agents](https://docs.pipecat.ai/pipecat/learn/multiple-llm-agents.md)
>
> Give an agent its own LLM and register tools with the @tool decorator

> [card] [Try the Classifiers Example](https://github.com/pipecat-ai/pipecat/blob/main/examples/features/features-classifiers.py)
>
> Ask a yes/no, a choice and a score question about a call transcript, with
> Jev, Clef, or an OpenAI model.

> [card] [Classifiers API Reference](https://docs.pipecat.ai/api-reference/server/classifiers/overview.md)
>
> Question and result types, and the classifiers Pipecat includes.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.