Alerts

Alerts notify the right people the moment something worth attention happens in a conversation. An alert is a workspace-level entity that you assign to one or more agents — exactly like tools or prompts. When a bound agent finishes processing a message (or fails), ToothFairyAI evaluates that agent's alerts against the conversation's analytics, system metrics and error context, and dispatches notifications through every channel you enable.

Where a Trigger runs an agent in reaction to an event, an Alert notifies people about what happened inside an agent's run. The two are complementary: triggers act, alerts inform.

Menu location

Alerts can be created and managed from the following menu: Settings > Alerts > Create alert

Alerts are then bound to agents from the agent editor, under the Analytics & Alerts section (see Assigning alerts to an agent).

How alerts work

1. You create an alert (conditions + recipients + notification channels).
2. You assign one or more alerts to an agent from that agent's Analytics & Alerts section.
3. After every agent run, ToothFairyAI fetches all the alerts assigned to that agent, builds a single evaluation context from the run, and checks each alert's conditions.
4. Every alert whose conditions match (and is not in cooldown) fires its notifications.

One agent, many alerts

A single agent can be bound to multiple alerts — they are evaluated independently. You do not need to cram every check into one alert. See Multiple alerts per agent.

Create an alert

Overview

1. Click on the Create alert button.
2. Name — Assign a unique name to identify the alert.
3. Description (optional) — Describe what this alert watches for. This text is also used as the notification body.
4. Trigger type — Categorises the alert's intent:
   • Analytics — watch conversation analytics (toxicity, satisfaction, quality, custom fields, …).
   • System — watch run-level metrics (duration, units of intelligence consumed, errors).
   • Custom — a user-defined check driven by your own custom analytics fields.
5. Severity — info, warning or critical. Surfaced in notifications and webhooks so recipients can triage.

Conditions

6. Conditions — One or more field / operator / value rules. A field path can be dotted (e.g. customAnalytics.churnRisk or qualityScore). All conditions must match for the alert to fire (they are AND-ed).

   Operator	Meaning
   eq	Field equals the value
   neq	Field does not equal the value
   gt	Field is numerically greater than the value
   gte	Field is numerically greater than or equal to the value
   lt	Field is numerically less than the value
   lte	Field is numerically less than or equal to the value
   contains	Field contains the value
   not_contains	Field does not contain the value

   Add as many condition rows as you need; empty field rows are ignored on save. See Evaluation context for the fields you can target.

Recipients

7. Recipients — Select workspace users to notify. Each selected user receives in-app, push and email notifications (per the channels enabled below) based on their account.
8. Additional emails (optional) — Comma-separated external email addresses (people who are not workspace users). External recipients can only receive email — in-app and push are not possible for them.
9. Cooldown (minutes) — The minimum time between repeated notifications for the same alert. Once an alert fires, it will not fire again until the cooldown elapses, even if conditions still match. Set to 0 to disable cooldown.

Notifications

10. Active — Arm or pause the alert. Inactive alerts are never evaluated.

11. In-app — Creates a notification that appears in the web/Electron top-bar bell and the mobile app inbox.

12. Push — Sends an Expo push notification to each recipient's registered mobile devices.

13. Email — Sends an email via SES to each recipient (and any additional emails).

14. Webhook — Emits an alert_triggered event to the workspace's configured webhook and as a native ToothFairyAI trigger event, so external systems (and other agents' triggers) can react. See Webhooks.

15. Click on the Create button to save the alert.

Evaluation context

Every condition targets a field in a single merged context built from the agent run. The fields you can target are:

Conversation analytics

Produced by ToothFairyAI's conversation analyser after each run.

Field	Type	Meaning
piiDetected	boolean	Whether personally identifiable information was detected
piiTypes	string[]	PII types found (e.g. email, phone, address)
piiOccurrences	number	Total count of PII occurrences
toxicityDetected	boolean	Whether toxicity was detected
toxicityScore	number	Toxicity score
toxicityCategories	string[]	Toxicity categories detected
customerSatisfactionScore	number	Estimated customer satisfaction (0–1)
customerSatisfactionFeedback	string	Short feedback summary
requestCompleted	boolean	Whether the user's request was completed
requestCompletionScore	number	Request completion score
escalationRequired	boolean	Whether escalation to a human was flagged
escalationReason	string	Why escalation was flagged
qualityScore	string	Quality rating (e.g. low, medium, high)
qualityNotes	string	Short quality notes

Custom analytics fields

Any field you define under the agent's Analytics & Alerts section is extracted and made available under customAnalytics.<yourFieldName>. See Custom analytics fields.

System metrics

Field	Type	Meaning
duration	number	Run duration (seconds)
consumedUoI	number	Units of Intelligence consumed by the run
error	boolean	Whether the run ended in an error
errorMessage	string	Error message when the run failed

Field absence

If a condition targets a field that is absent from the context (for example an error field on a successful run), that condition is treated as not satisfied. This means an error-only alert should target error eq true.

Condition examples

Goal	Field	Operator	Value
Toxic conversation	toxicityDetected	eq	true
Low satisfaction	customerSatisfactionScore	lt	0.4
Escalation needed	escalationRequired	eq	true
High spend per run	consumedUoI	gt	50
Run failed	error	eq	true
Custom churn risk	customAnalytics.churnRisk	eq	high
Custom sentiment	customAnalytics.sentiment	eq	negative

Assigning alerts to an agent

Alerts are assigned to agents — not the other way around.

1. Open the agent you want to monitor in the agent editor.
2. Scroll to the Analytics & Alerts section.
3. Use Alerts to trigger for this agent to select one or more alerts.
4. Save the agent.

From that point, every run of that agent is checked against all selected alerts.

Multiple alerts per agent

An agent can be bound to multiple alerts, and each is evaluated independently on every run. There is no limit and no ordering dependency between them.

• Every assigned, active alert is fetched and checked after each run.
• Each alert's conditions are evaluated against the same run context.
• Cooldown is per-alert. If alert A is in cooldown it is skipped, but alert B still fires if its conditions match.
• Each alert dispatches to its own recipients and channels — they do not share a recipient list.

Design pattern

Prefer several focused alerts over one giant alert. For example, on a support agent:

• one alert for toxicityDetected eq true → notify the safety team,
• one alert for escalationRequired eq true → notify the support lead,
• one alert for consumedUoI gt 50 → notify the billing admin.

Each can have its own severity, cooldown and recipients.

Custom analytics fields

Custom analytics fields let you extract your own metrics from every conversation and then build alert conditions against them. They are defined per-agent, in the Analytics & Alerts section, in the Custom analytics fields JSON editor.

How they work

1. You define a list of fields (name, type, description, optional enum).
2. After each run, the conversation analyser is instructed to extract those fields from the conversation.
3. The extracted values are stored on the chat as customAnalytics and added to the alert evaluation context under customAnalytics.<name>.
4. Your alert conditions can then reference customAnalytics.<name> with any operator.

How to populate them for the best results

Custom analytics fields are entered in the Custom analytics fields JSON editor. Two shapes are accepted — use whichever you prefer:

• Field-definition array (simplest) — a JSON array of {name, type, description, enum?} objects.
• JSON Schema object (Functions-parameters style) — an object with a properties map whose keys are the field names. Useful if you copy a schema from a Function's parameters.

Both are normalised to the same internal list, so they are equivalent. The array form is shown below.

Each field is an object with:

Property	Required	Values	Notes
name	yes	any string	The key the value is stored and referenced under. Use a stable, descriptive camelCase name (e.g. churnRisk). Alert conditions reference it as customAnalytics.<name>.
type	no	string | number | boolean	Defaults to string. Determines how the value is coerced and which operators make sense (use gt/lt with number). In the JSON Schema form, integer is also accepted and treated as number.
description	no	any string	The most important field for accuracy. This is what the analyser reads to decide what to extract. Be specific about what to look for and how to score it.
enum	no	array of strings	Only valid for string fields. Constrains the value to a fixed set — strongly recommended for categorical fields so conditions are reliable.

Example — a support / customer-success agent:

[
  {
    "name": "churnRisk",
    "type": "string",
    "description": "Estimated risk that this customer will churn based on tone, complaints and cancellation language. Score as low, medium or high.",
    "enum": ["low", "medium", "high"]
  },
  {
    "name": "sentiment",
    "type": "string",
    "description": "Overall customer sentiment at the end of the conversation.",
    "enum": ["positive", "neutral", "negative"]
  },
  {
    "name": "outcomeResolved",
    "type": "boolean",
    "description": "Whether the customer's issue was fully resolved by the end of the conversation."
  },
  {
    "name": "upsellOpportunity",
    "type": "boolean",
    "description": "Whether the conversation surfaced a genuine upsell or cross-sell opportunity."
  }
]

The same fields as a JSON Schema object (Functions-parameters style) — equivalent:

{
  "type": "object",
  "properties": {
    "churnRisk": {
      "type": "string",
      "description": "Estimated risk that this customer will churn based on tone, complaints and cancellation language. Score as low, medium or high.",
      "enum": ["low", "medium", "high"]
    },
    "sentiment": {
      "type": "string",
      "description": "Overall customer sentiment at the end of the conversation.",
      "enum": ["positive", "neutral", "negative"]
    },
    "outcomeResolved": {
      "type": "boolean",
      "description": "Whether the customer's issue was fully resolved by the end of the conversation."
    },
    "upsellOpportunity": {
      "type": "boolean",
      "description": "Whether the conversation surfaced a genuine upsell or cross-sell opportunity."
    }
  },
  "required": ["churnRisk"]
}

With those fields defined, you can create alerts such as:

Alert	Condition
At-risk customer	customAnalytics.churnRisk eq high
Angry customer	customAnalytics.sentiment eq negative
Unresolved issue	customAnalytics.outcomeResolved eq false
Sales lead	customAnalytics.upsellOpportunity eq true

Best practices for custom fields

1. Write precise descriptions. The analyser is an LLM — the clearer the instruction, the more consistent the extraction. Describe what to look for and how to classify it.
2. Use enum for categorical fields. A constrained set (low/medium/high) makes eq conditions reliable. Free-text strings invite variation that breaks equality checks.
3. Use number for anything you will threshold. Only number fields work correctly with gt / gte / lt / lte.
4. Keep names stable. If you rename a field, existing alerts that reference the old customAnalytics.<oldName> path will stop matching.
5. Keep the list focused. Every field is extracted on every run, which adds cost and latency. Define only the fields you will actually alert on.
6. Pair with analytics instructions. The Analytics instructions field above the custom fields editor lets you give the analyser extra context (e.g. "this agent handles inbound billing disputes") that improves extraction quality across all fields.

Values must exist to match

A condition targeting customAnalytics.churnRisk only fires when the analyser actually returned a value for churnRisk on that run. If extraction is skipped or returns nothing, the condition is treated as not satisfied. Use enum and clear descriptions to maximise extraction consistency.

Notification channels

Channel	Who receives it	Notes
In-app	Selected workspace users	Appears in the web/Electron top-bar bell and the mobile app notification inbox.
Push	Selected workspace users	Delivered to each user's registered mobile devices via Expo push.
Email	Selected workspace users and additional emails	Sent via SES. External (non-user) recipients only receive email.
Webhook	The workspace's webhook endpoint	Emits an alert_triggered event to the configured webhook and as a native trigger event.

Webhook / trigger event payload

When Webhook is enabled, an alert_triggered event is emitted. External systems subscribed to the workspace webhook receive it, and it is also available as a native ToothFairyAI event — so another agent's Trigger can react to an alert firing.

Managing alerts

Once created, alerts appear in the alerts list with their trigger type, severity, status and last-triggered time.

Edit an alert

1. Click an existing alert from the list.
2. Modify any settings as needed.
3. Click Save to update the alert.

Delete an alert

1. Open the alert you want to delete.
2. Click the Delete button.
3. Confirm the deletion when prompted.

Deleting an alert that is assigned to agents

Deleting an alert does not automatically remove its id from agents' assigned alert lists, but a deleted alert is never evaluated or dispatched. You can safely leave stale references; they are ignored.

Best practices

1. One concern per alert. Focused alerts are easier to triage and let you route each to the right people and severity.
2. Set a cooldown. For alerts that can match on many consecutive runs (e.g. low satisfaction), a cooldown prevents notification fatigue.
3. Use the right operator. Use eq/neq for booleans and enums, gt/lt for numbers, and contains for substrings.
4. Prefer enums. Categorical custom fields with a fixed enum produce reliable equality conditions.
5. Test the agent first. Run the agent manually, confirm the analytics (including custom fields) are being produced, then build alert conditions against fields you know are populated.
6. Route by severity. Use critical for things needing immediate action and info for awareness, so recipients can prioritise.

Availability

Alerts are evaluated after every run of an agent that has at least one alert assigned. Evaluation is best-effort: if alert evaluation itself fails, the agent's response to the user is not affected.