Predictions

toothfairyai@latest…

Manager for OpenAI-compatible streaming predictions.

The workspace is resolved from the x-api-key header, so workspace_id is optional (deprecated) and is only sent when explicitly provided.

Example:

>>> stream = client.predictions.create(
...     model="sorcerer",
...     messages=[{"role": "user", "content": "Hello!"}],
...     max_tokens=100,
... )
>>> for event in stream:
...     if event.text:
...         print(event.text, end="", flush=True)

Accessed via client.predictions.

Methods

Method	HTTP	Endpoint
`create`	POST	`POST /predictions`
`collect`	—	derived

create

Create a streaming prediction.

def create(
    model: str,
    messages: List[Dict[str, Any]],
    workspace_id: Optional[str] = None,
    max_tokens: Optional[int] = None,
    temperature: Optional[float] = None,
    top_p: Optional[float] = None,
    top_k: Optional[int] = None,
    repetition_penalty: Optional[float] = None,
    reasoning_mode: Optional[str] = None,
    reasoning_effort: Optional[str] = None,
    reasoning_budget: Optional[int] = None,
    thinking_budget: Optional[int] = None,
    seed: Optional[int] = None,
    frequency_penalty: Optional[float] = None,
    presence_penalty: Optional[float] = None,
    stop: Optional[Any] = None,
    tools: Optional[Any] = None,
    tool_choice: Optional[Any] = None,
    response_format: Optional[Any] = None,
    verbosity: Optional[str] = None,
    user: Optional[str] = None,
    region: Optional[str] = None
) -> PredictionStream

Endpoint: POST /predictions · AI streaming service (SSE)

Request fields

Python kwarg	Wire field	Type	Required	Description
`model`	`model`	`string`	yes	Model alias to use. The backend accepts any key in its chat-templates registry (~150+ aliases, e.g. sorcerer, mystica, glm-5, kimi-k2p5, deepseek-ai/DeepSeek-V3.2, minimax-m2p5, and many gpt-/claude-/gemini-* aliases). The values below are recommended ToothFairyAI aliases. Fine-tuned models are also accepted by name (e.g. `ToothFairyAI/Meta-Llama-3.1-8B-abcd1234`) — list your workspace's models via `GET /models_list?workspaceid=<id>`. The first request to an idle fine-tuned model waits ~5–10 minutes for automatic wake-up (not billed).
`messages`	`messages`	`array<object>`	yes	Array of conversation messages following OpenAI format
`workspace_id`	`workspaceid`	`string`	no	DEPRECATED — no longer required. The workspace is resolved from the `x-api-key` header. This field is accepted for backward compatibility but is ignored.
`region`	`region`	`string`	no	Optional. Selects the geographic provider pool used to serve the request. `null` or `"global"` (default) uses the global pool and picks the model as-is. `"au"`, `"us"`, or `"eu"` restrict routing to providers deployed in that region (enforces data sovereignty; the global pool is NOT used as fallback). This controls the upstream provider pool independently of which API server you call.

Allowed: always, never | | reasoning_budget | reasoning_budget | integer | no | Maximum tokens the model may spend on its internal thinking process (reasoning models only). Falls back to the model's configured thinking_budget when omitted. Some models cap this (e.g. 1024 minimum). | | seed | seed | integer | no | Deterministic seed for reproducible sampling where the provider supports it. When supported, identical seed + inputs produce similar (not always byte-identical) outputs. | | frequency_penalty | frequency_penalty | number | no | OpenAI-style frequency penalty. Positive values reduce the likelihood of tokens that have already appeared, proportional to their frequency. Range typically -2.0 to 2.0. | | presence_penalty | presence_penalty | number | no | OpenAI-style presence penalty. Positive values increase the likelihood of new tokens that have not yet appeared. Range typically -2.0 to 2.0. | | response_format | response_format | object | no | OpenAI-compatible response format spec, e.g. {"type": "json_schema", "json_schema": {...}} or {"type": "json_object"}. Enforces structured output where the provider supports it. | | verbosity | verbosity | string | no | Output verbosity hint where supported (e.g. OpenAI 'o' series). Typical values: low, medium, high.

Always pass snake_case keyword arguments — the Python kwarg column shows the exact name to use for each wire field (as a named parameter where it appears in the method signature, otherwise via **kwargs). The SDK converts it deterministically to the camelCase wire key the API expects. Passing camelCase directly is deprecated: it emits a warning and converts to the same wire key.

Response

This is a streaming endpoint. The response is delivered as Server-Sent Events (SSE) rather than a single JSON object. The method yields SSE events (or resolves with the accumulated text for collect-style helpers).

Example

client.predictions.create(model="…", messages="…")

collect

Convenience: create a prediction and return the full accumulated text.

def collect(model: str, messages: List[Dict[str, Any]]) -> str

Derived method — delegates to another SDK call and performs no direct HTTP request.

Example

client.predictions.collect(model="…", messages="…")

Predictions​

Methods​

create​

Request fields​

Response​

Example​

collect​

Example​

Predictions

Methods

create

Request fields

Response

Example

collect

Example