> ## Documentation Index
> Fetch the complete documentation index at: https://braintrust.dev/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Configure sustained score alerts with Time window alerts

> Alert when LLM score degradations persist over a time window using Time window alerts or webhook-based external aggregation.

export const plans_0 = "Any"

export const deployments_0 = "Any"

export const data_plane_version_0 = undefined

export const use_case_0 = "Use case - Alert when LLM judge score breaches persist over a time window, such as sustained low factuality for 5 minutes"

<Note>
  **Applies to:**

  * Plan - {plans_0}
  * Deployment - {deployments_0}
  * {data_plane_version_0}
  * {use_case_0}
</Note>

## Summary

Use a [Time window alert](/docs/observe/alerts#create-a-time-window-alert) to notify when an aggregate score, error rate, or other scalar SQL calculation crosses a threshold over a time window. Time window alerts are the built-in path for sustained score alerts. Use webhook-based external aggregation only when **Time window** alerts are unavailable in your deployment or when you need custom paging logic that the built-in alert does not express.

## What is happening

Log alerts match individual events in evaluation batches. They work well for single-row conditions, but a single outlier can trigger a noisy alert when you care about an aggregate or sustained trend. A **Time window** alert evaluates one SQL calculation per window and compares the numeric result to a threshold, so it can alert on conditions like average score, p90 score, count, or percentage below a threshold.

## Fix or suggestion

### Option 1: create a Time window alert

Action: [create a Time window alert](/docs/observe/alerts#create-a-time-window-alert) that computes the aggregate inside Braintrust and notifies you when the result crosses your threshold.

For sustained score alerts, configure the calculation to return one numeric value for each evaluated window. Then set **Window length (minutes)** to the aggregation window, set **Alert trigger** to your threshold comparison, and use **Advanced settings** for persistence, recovery, no-data, late-data, schedule, and repeat notification behavior.

Example SQL calculation for average factuality:

```sql theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
SELECT AVG(scores.factuality) AS avg_factuality
FROM project_logs('<PROJECT_ID>')
WHERE scores.factuality IS NOT NULL
  AND metadata.environment = 'prod'
```

<Note>
  For self-hosted deployments, **Time window** alerts require [data plane v2.10.0 or later](/docs/data-plane-changelog).
</Note>

### Option 2: use webhook-based external aggregation

Action: use a log alert webhook as a trigger, then query or aggregate data in your own service before paging.

Use this fallback when **Time window** alerts are not available in your deployment, or when your paging policy needs custom state, enrichment, or routing outside the built-in alert configuration.

1. Create a log alert whose SQL filter captures candidate breaches, for example `scores.factuality IS NOT NULL AND scores.factuality < 0.8 AND metadata.environment = 'prod'`. Set the action to **Webhook** and point it at your aggregator endpoint.
2. Use the webhook payload as a trigger. The alert webhook payload is fixed. It includes the alert metadata, count, time window, and related logs URL, but not every matching log row or score value.
3. In your aggregator, use the alert time window and the same SQL filter to query Braintrust, or consume the same trace data from your own pipeline.
4. Compute the windowed condition you care about, such as percentage below threshold, average score, p90, or consecutive count, and only page if that aggregate crosses your threshold.

Minimal Python pseudocode (conceptual):

```python theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
THRESHOLD = 0.8
PCT_REQUIRED = 0.5  # 50 percent of events

def on_webhook_event(payload):
    time_start = payload["details"]["time_start"]
    time_end = payload["details"]["time_end"]

    rows = query_braintrust_logs(
        start=time_start,
        end=time_end,
        filter="scores.factuality IS NOT NULL AND metadata.environment = 'prod'",
    )

    scores = [
        row["scores"]["factuality"]
        for row in rows
        if row.get("scores", {}).get("factuality") is not None
    ]

    if not scores:
        return

    pct_below_threshold = sum(score < THRESHOLD for score in scores) / len(scores)

    if pct_below_threshold >= PCT_REQUIRED:
        page_ops.send_page({
            "time_start": time_start,
            "time_end": time_end,
            "count": len(scores),
            "pct_below_threshold": pct_below_threshold,
        })
```

Keep the aggregator simple and observable.

## How to confirm it worked

* For a **Time window** alert, use **Recent evaluation preview** to confirm the calculation returns the expected value, then create test logs that cross and recover from the threshold.
* For a **Webhook** action on a log alert, click **Test** next to the **Webhook URL** field. Confirm the webhook receives the expected payload.
* For a **Slack** action, testing from the UI is not available. Log an event that matches the filter, then confirm the channel receives the notification.
* For external aggregation, send test events that simulate sustained and transient breaches. Confirm your aggregator only pages for sustained cases and remains silent for single outliers. Check aggregator logs and metrics for window counts and the page history.

## Notes

* See [Alerts](/docs/observe/alerts) for full alert configuration reference.
* Time window alerts evaluate scalar SQL calculations. Keep the external aggregation pattern when you need behavior outside that model, such as joining Braintrust data with another incident system before deciding whether to page.