Skip to main content

Serving & Availability

By default, a fine-tuned model sleeps when nobody uses it and wakes automatically on the first request (the wake-up takes roughly 5–10 minutes and is not billed). That suits occasional use perfectly — you pay nothing while the model is idle and only per token when it answers.

If your team relies on a model during business hours, waiting for a wake-up every morning isn't ideal. This page explains how to keep a model warm on your own terms — and what it costs.

The three modes​

ModeWhat happensBest for
On-demand (default)Wakes on first request (~5–10 min, free), sleeps after ~15 min idleOccasional / unpredictable use
ScheduledKept warm inside recurring windows you define (e.g. weekdays 07:00–17:00, your timezone); on-demand behaviour outside the windowBusiness-hours workflows
AlwaysKept warm continuously; cancel any time by switching back to on-demandCritical, round-the-clock workloads

All configuration is done through the Fine-Tuning API and is attached to the model's Training Log — so it travels with the model and is fully scriptable. Changes take effect within ~2 minutes.

Reading the current configuration​

curl "https://api.toothfairyai.com/finetuning/serving/{trainingLogId}" \
  -H "x-api-key: YOUR_API_KEY"
{
  "modelId": "ToothFairyAI/gemma-3-1b-abcd1234",
  "mode": "scheduled",
  "schedule": {"days": ["mon","tue","wed","thu","fri"], "start": "07:00", "end": "17:00", "timezone": "Australia/Sydney"},
  "lane": {"status": "ready", "port": 8100, "pinned_until": 1760000000},
  "warm_seconds_accrued": 7200
}

The lane object is the live state: stopped (asleep), starting/creating (waking up), or ready (warm and answering instantly).

Option A: a recurring schedule (no cron needed)​

Set once and forget — the platform turns the model on at the start of each window and lets it sleep after the end:

curl -X POST "https://api.toothfairyai.com/finetuning/serving/{trainingLogId}" \
  -H "x-api-key: YOUR_API_KEY" -H "Content-Type: application/json" \
  -d '{
    "mode": "scheduled",
    "schedule": {
      "days": ["mon", "tue", "wed", "thu", "fri"],
      "start": "07:00",
      "end": "17:00",
      "timezone": "Australia/Sydney"
    }
  }'

Rules: days is any list of mon…sun or ["everyday"]; start/end are HH:MM in your chosen timezone; end must be later than start (overnight windows are not supported). "mode": "always" keeps the model warm around the clock; "mode": "on-demand" switches back to the default behaviour.

Option B: on/off from your own scheduler​

Prefer to drive it yourself? Turn the model on and off exactly when you want:

# 07:00 — your cron: warm it up for the next 10 hours
curl -X POST "https://api.toothfairyai.com/finetuning/serving/{trainingLogId}/power" \
  -H "x-api-key: YOUR_API_KEY" -H "Content-Type: application/json" \
  -d '{"state": "on", "hours": 10}'

# 17:00 — your cron: release it for the evening
curl -X POST "https://api.toothfairyai.com/finetuning/serving/{trainingLogId}/power" \
  -H "x-api-key: YOUR_API_KEY" -H "Content-Type: application/json" \
  -d '{"state": "off"}'

state: "on" triggers the wake-up and holds the model warm for hours (minimum 1). The call returns immediately — poll GET /serving/{trainingLogId} until lane.status is "ready". Holds extend each other, never shorten: booking 3 hours and then 1 more keeps it warm for 4 hours total. state: "off" releases the hold right away; the model then sleeps after the normal short idle grace.

What warm time costs​

Warm hours are billed in 1-hour units (minimum 1 unit) at your model's size-tier hourly rate, on top of normal per-token usage:

Model sizeWarm hour rate
Small$2.20 / hour (220 UoI)
Medium$3.60 / hour (360 UoI)
Large$5.20 / hour (520 UoI)
Extra large / Largest$24.40 / hour (2,440 UoI)

Worked example: a Medium model scheduled weekdays 07:00–17:00 runs 10 warm hours/day → $36/day (~3,600 UoI). Always-on on a Medium model is roughly $2,600/month — that tier makes most sense for enterprise workloads. Accrued warm time appears in warm_seconds_accrued on the serving status call and is billed once per day; the wake-up itself is never billed, and per-token charges during warm windows are unchanged.

Deleting a training log​

If the Training Log is deleted, its schedule is automatically disabled and the model's lane switches off within ~2 minutes — it will not keep waking up and billing warm hours for a deleted model. Warm time that already ran is still billed (it was real compute). The serving endpoints return 404 Training log not found for deleted logs, so no new schedules or holds can be placed on them.

Last updated on