Skip to main content

Escalate

When the active model cannot solve a task — repeated failed attempts, a hard domain problem, quality clearly blocked by model capability — the agent can temporarily switch to a stronger deployed model for a limited number of turns, then revert automatically.

The upgrade target is not a guess: candidates are ranked with benchmark data served by the models endpoint — an overall intelligence index, per-domain capability indexes (finance, legal, engineering, economics, ...) and per-evaluation agentic scores. Models that score at least gain_threshold points better than the current model on the relevant domain are eligible; the best scorer wins (cheapest on ties).

Areas​

Ranking is area-aware. The agent picks the area matching the task (or the tool infers it from the free-text reason):

AreaScores models on
generalOverall intelligence index (default)
agenticSRE/ops agents, analyst agents, Terminal-Bench
codingSciCode, Terminal-Bench 2.1/4.0
mathCritPt, Humanity's Last Exam, GPQA
scienceGPQA, Humanity's Last Exam, SciCode evals
longcontextLong-context reasoning evals
legalLegal capability index, HarveyLab
financeFinance & accounting index, Tau Banking
economicsEconomics index, GDPval
engineeringEngineering index, SRE bench
strategyStrategy & ops index, GDP.pdf pass rate
medicalHealthcare & medical index, Omniscience

Examples: a failing TypeScript refactor escalates with area: "coding", a contract review with area: "legal", an integral proof with area: "math".

Regional edge case​

A session pinned to a regional model (model id ending in __regional, served from the AU/EU/US in-region pools) can only escalate to models available in that same region. The tool resolves candidates to their <id>__regional variant and silently skips models without one — a regional session can never route work to a global-pool model. Global sessions prefer the global pool.

Enable​

Escalate is available in two ways:

  1. Perk toggle (recommended) — turn on the escalate perk for the project, the same way you enable computer-use or cloud hand-off. All defaults apply (see below).
  2. Explicit config — add an escalate section to tfcode.json to control the knobs:
{
  "escalate": {
    "enabled_for": ["build", "tf_engineer", "tf_orchestrate"],
    "endpoint": "https://ai.toothfairyai.com/models_list",
    "area": "general",
    "gain_threshold": 5,
    "turns": 3,
    "api_key": "optional-if-endpoint-needs-auth"
  }
}
  • enabled_for — Agents allowed to escalate (default: build, tf_engineer, tf_orchestrate). Also accepts perk names.
  • endpoint — Models endpoint exposing deployed model benchmark data (e.g. https://chat.host/v1/models or a models_list URL). Defaults to env TF_MODELS_URL. Two shapes are accepted: an OpenAI-style {data: [...]} array where each entry nests benchmark data under toothfairy, or a models_list {templates: {...}} object.
  • area — Default ranking area when the tool call omits one (default: general).
  • gain_threshold — Minimum score improvement over the current model required to escalate (default: 5).
  • turns — Escalation budget in assistant turns before auto-revert (default: 3).
  • api_key — Bearer token sent to the endpoint if it requires auth.

The catalog is cached for 6 hours (override the cache file location with env TF_ESCALATE_CACHE); a stale cache is still used if the endpoint is unreachable.

Agent experience​

The tool exposes two actions:

  • action: "escalate" (default) — with reason describing what the current model cannot do and optionally area. Responds with the chosen model, its tier and area scores.
  • action: "revert" — switch back to the original model immediately.

Escalation state is per-session and in-memory: a restart drops any active escalation. Each new user prompt consumes one budget turn; when the budget is exhausted the session reverts to the original model automatically. Escalating requires the escalate permission (auto-allowed by default).

Last updated on