Predictions

Predictions

toothfairyai@latest…

Manager for OpenAI-compatible streaming predictions.

The workspace is resolved from the x-api-key header, so workspace_id is optional (deprecated) and is only sent when explicitly provided.

Example:

>>> stream = client.predictions.create(
...     model="sorcerer",
...     messages=[{"role": "user", "content": "Hello!"}],
...     max_tokens=100,
... )
>>> for event in stream:
...     if event.text:
...         print(event.text, end="", flush=True)

Accessed via client.predictions.

Methods

Method	HTTP	Endpoint
create	POST	POST /predictions
collect	—	derived

create

Create a streaming prediction.

def create(
    model: str,
    messages: List[Dict[str, Any]],
    workspace_id: Optional[str] = None,
    max_tokens: Optional[int] = None,
    temperature: Optional[float] = None,
    top_p: Optional[float] = None,
    top_k: Optional[int] = None,
    repetition_penalty: Optional[float] = None,
    reasoning_mode: Optional[str] = None,
    reasoning_effort: Optional[str] = None,
    reasoning_budget: Optional[int] = None,
    thinking_budget: Optional[int] = None,
    seed: Optional[int] = None,
    frequency_penalty: Optional[float] = None,
    presence_penalty: Optional[float] = None,
    stop: Optional[Any] = None,
    tools: Optional[Any] = None,
    tool_choice: Optional[Any] = None,
    response_format: Optional[Any] = None,
    verbosity: Optional[str] = None,
    user: Optional[str] = None,
    region: Optional[str] = None
) -> PredictionStream

Endpoint: POST /predictions · AI streaming service (SSE)

Request fields

Field	Type	Required	Description
model	string	yes	Model alias to use. The backend accepts any key in its chat-templates registry (~150+ aliases, e.g. sorcerer, mystica, glm-5, kimi-k2p5, deepseek-ai/DeepSeek-V3.2, minimax-m2p5, and many gpt-/claude-/gemini-* aliases). The values below are recommended ToothFairyAI aliases.
messages	array<object>	yes	Array of conversation messages following OpenAI format
workspaceid	string	no	DEPRECATED — no longer required. The workspace is resolved from the x-api-key header. This field is accepted for backward compatibility but is ignored.
region	string	no	Optional. Selects the geographic provider pool used to serve the request. null or "global" (default) uses the global pool and picks the model as-is. "au", "us", or "eu" restrict routing to providers deployed in that region (enforces data sovereignty; the global pool is NOT used as fallback). This controls the upstream provider pool independently of which API server you call.

Allowed: global, au, us, eu | | max_tokens | integer | no | Maximum tokens to generate in the response. Automatically clamped to model's maximum. | | temperature | number | no | Sampling temperature for randomness. Higher = more creative, lower = more deterministic. | | top_p | number | no | Nucleus sampling parameter for diversity. | | tools | array<object> | no | List of tools/functions the model may call. Follows OpenAI function calling format. | | tool_choice | string | object | no | Controls tool calling behavior: 'auto' (model decides), 'none' (no tools), 'required' (must call tool) | | reasoning_effort | string | no | Reasoning effort level for thinking models (sorcerer_15, mystica_15, etc.)

Allowed: low, medium, high | | stop | string | array<string> | no | Stop sequences where generation will halt | | top_k | integer | no | Top-K sampling. Limits the next-token selection to the K most probable tokens. Supported by most providers (some ignore it for OpenAI-compatible backends). Lower = more focused, higher = more diverse. Omit to use the provider default. | | repetition_penalty | number | no | Repetition / frequency penalty on already-used tokens. 1.0 = no penalty (default). Values >1 discourage repetition, values <1 encourage it. Applied where supported (Fireworks, Together, DeepInfra). | | reasoning_mode | string | no | When the model supports thinking, controls whether reasoning is emitted. always (default) emits thinking on every call, never disables it. Ignored by non-reasoning models.

Allowed: always, never | | reasoning_budget | integer | no | Maximum tokens the model may spend on its internal thinking process (reasoning models only). Falls back to the model's configured thinking_budget when omitted. Some models cap this (e.g. 1024 minimum). | | seed | integer | no | Deterministic seed for reproducible sampling where the provider supports it. When supported, identical seed + inputs produce similar (not always byte-identical) outputs. | | frequency_penalty | number | no | OpenAI-style frequency penalty. Positive values reduce the likelihood of tokens that have already appeared, proportional to their frequency. Range typically -2.0 to 2.0. | | presence_penalty | number | no | OpenAI-style presence penalty. Positive values increase the likelihood of new tokens that have not yet appeared. Range typically -2.0 to 2.0. | | response_format | object | no | OpenAI-compatible response format spec, e.g. {"type": "json_schema", "json_schema": {...}} or {"type": "json_object"}. Enforces structured output where the provider supports it. | | verbosity | string | no | Output verbosity hint where supported (e.g. OpenAI 'o' series). Typical values: low, medium, high.

Allowed: low, medium, high | | user | string | no | OpenAI-style end-user identifier, passed through for tracking/abuse monitoring. Does not influence generation. | | thinking_budget | integer | no | Token budget allocated for reasoning/thinking (passed through to supporting models) |

The Python SDK accepts snake_case keyword arguments and converts them to the camelCase wire fields shown above.

Response

This is a streaming endpoint. The response is delivered as Server-Sent Events (SSE) rather than a single JSON object. The method yields SSE events (or resolves with the accumulated text for collect-style helpers).

Example

client.predictions.create(model="…", messages="…")

collect

Convenience: create a prediction and return the full accumulated text.

def collect(model: str, messages: List[Dict[str, Any]]) -> str

Derived method — delegates to another SDK call and performs no direct HTTP request.

Example

client.predictions.collect(model="…", messages="…")