Vultr Inference API (1.1.3)

The model identifier to use for generation.

Total completion token allowance, including thinking and visible output. The gateway forwards zero if supplied; whether zero is accepted depends on the upstream engine.

Input messages. Models operate on alternating user and assistant conversational turns. Consecutive user or assistant turns are combined into a single turn.

If the final message uses the assistant role, the response content continues immediately from that message's content (assistant prefill).

A text-only system message may follow a user turn, as the final message or before an assistant turn. Use the top-level system parameter for initial instructions. Turn-scoped clear_at and per-message output_config are not supported.

System prompt providing context or instructions to the model. May be a string or an array of text blocks.

Whether to stream the response using server-sent events. Streaming events use typed event names: message_start, content_block_start, content_block_delta, content_block_stop, message_delta, message_stop.

Controls randomness in the model responses. Valid range: 0 to 1.

Enables nucleus sampling with a cumulative probability cutoff.

Limits sampling to the top K options.

Custom sequences that, when encountered in the generated text, cause the model to stop.

Additional metadata about the request.

Output controls. Effort is translated through the selected model's reasoning configuration. Unsupported configured levels return 422.

Thinking configuration. The adapter signs its own thinking blocks for continuation; it does not emit redacted thinking blocks.

Definitions for tools available to the model.

The model to use. If omitted, the gateway resolves its default model. A -normalize suffix is accepted for compatibility and ignored; every response is normalized. A :<variant> suffix is parsed separately from the model ID; :express derives a thinking budget when the model supports enforced budgets and no explicit budget is supplied.

The message context to use for the chat completion request, separated by system, user, assistant, tool, and developer roles. A final assistant message is treated as a prefill and continued (see continue_final_message).

Continue the final message instead of starting a new assistant turn. Implied whenever the final message has the assistant role, so it only makes the intent explicit. The final message must carry non-empty text and no tool_calls; trailing whitespace is trimmed, thinking is disabled for the continuation, and only the newly generated text is returned. Use it to resume a response that ended with finish_reason length.

Indicates whether the response should be streamed.

Deprecated name for max_completion_tokens. If both are sent, max_completion_tokens takes precedence.

Maximum completion tokens, including reasoning and visible output. The effective limit may shrink to fit the model context window.

Shorthand for reasoning.effort. When both are supplied, reasoning.effort takes precedence. Configured models map this level to their supported reasoning control; unsupported levels return 422.

Reasoning controls. Configured models map effort through their database profile. Models advertising reasoning.supports_max_tokens: true in /v1/models enforce the budget; other models accept it without enforcing it.

The number of chat completion choices to generate for each input message.

If you would like a different response from the same message, changing the seed will change the response. A null value generates a random seed.

A value between 0.0 and 2.0 that controls the randomness of the model's output. When set closer to 1, such as 0.8, the outcome is more unpredictable and creative. Values nearing 0, like 0.2, produce more predictable and less creative results. A temperature of zero requests greedy decoding; exact reproducibility depends on the model engine.

A value between 0.0 and 1.0 that controls the probability of the model generating a particular token. A higher value will result in more diverse outputs, while a lower value will result in more repetitive outputs.

A value between -2.0 and 2.0 that controls how much the model penalizes generating repetitive responses.

A value between -2.0 and 2.0 that controls how much the model penalizes generating responses that contain certain words or phrases.

A list of strings that the model will stop generating text if it encounters any of them.

Whether to return log probabilities of the output tokens or not. If true, returns the log probabilities of each output token returned in the content of message.

An integer between 0 and 20 specifying the number of most likely tokens to return at each token position, each with an associated log probability. logprobs must be set to true if this parameter is used.

The vector store collection to search for relevant context.

The model that will be inferred for chat completion. A -normalize suffix is accepted for compatibility and ignored; every response is normalized.

The message context to use for the chat completion request, separated by system, user, assistant, tool, and developer roles.

Deprecated name for max_completion_tokens. If both are sent, max_completion_tokens takes precedence.

Maximum completion tokens, including reasoning and visible output.

Shorthand for reasoning.effort. Configured models map this level to their supported reasoning control.

Reasoning controls. Configured models map effort through their database profile. Models advertising reasoning.supports_max_tokens: true in /v1/models enforce the budget; other models accept it without enforcing it.

The number of chat completion choices to generate for each input message.

If you would like a different response from the same message, changing the seed will change the response. A null value generates a random seed.

A value between 0.0 and 2.0 that controls the randomness of the model's output. When set closer to 1, such as 0.8, the outcome is more unpredictable and creative. Values nearing 0, like 0.2, produce more predictable and less creative results. A temperature of zero requests greedy decoding; exact reproducibility depends on the model engine.

A value between 0.0 and 1.0 that controls the probability of the model generating a particular token. A higher value will result in more diverse outputs, while a lower value will result in more repetitive outputs.

A list of strings that the model will stop generating text if it encounters any of them.

A value between -2.0 and 2.0 that controls how much the model penalizes generating repetitive responses.

A value between -2.0 and 2.0 that controls how much the model penalizes generating responses that contain certain words or phrases.

Indicates whether the response should be streamed.

Whether to return log probabilities of the output tokens or not. If true, returns the log probabilities of each output token returned in the content of message.

An integer between 0 and 20 specifying the number of most likely tokens to return at each token position, each with an associated log probability. logprobs must be set to true if this parameter is used.

If omitted, the gateway resolves its default model. A :<variant> suffix is parsed separately from the model ID; :express derives a thinking budget when the model supports enforced budgets and no explicit budget is supplied.

Input text or structured input items.

Maximum output tokens, including reasoning and visible output.

Reasoning controls. Configured models map effort through their database profile. Models advertising reasoning.supports_max_tokens: true in /v1/models enforce the budget; other models accept it without enforcing it.

The embeddings model to use.

A string, a list of strings, a list of token ids, or a list of token id lists. Up to 2048 inputs. Multimodal inputs are rejected.

base64 returns each embedding as base64-encoded little-endian float32 values.

Truncate embeddings to this many dimensions, for models trained to support it (e.g. Qwen3-Embedding).

The decision model to use.

What to decide about. A string, or structured data (an object or array), which the model reads as JSON.

The rerank model to use.

The search query to rank the documents against.

The documents to rank, as plain strings or objects with a text field. Image documents are rejected.

Return only the highest scoring top_n results. All results are returned when omitted.

The model that will be used to generate text-to-speech audio.

The text to generate audio for, up to a maximum of 2,000 characters.

The text to be converted into embeddings and stored in the vector store collection.

A description of the contents in this collection item record. If omitted, this value will default to a shortened version of the text stored in the collection.

A text description of the desired image(s). The maximum length is 2000 characters, counted as Unicode characters rather than bytes.

The model to use for image generation. Defaults to z-image-turbo. GET /v1/models lists the image models currently served as entries with an image output modality. An unknown model returns 422; a known model that is not currently served returns 404.

The number of images to generate. Must be between 1 and 10. The images in one request are generated together, so changing n can change every image, including the first, for the same seed.

The format in which generated images are returned. url (the default) returns a link to the PNG that expires after one hour; b64_json returns the PNG inline as base64.

The size of the generated images. Must be one of 256x256, 512x512, 1024x1024 (default value), 1792x1024, and 1024x1792. Other dimensions are not supported, and width and height are ignored.

Random seed. Must be an integer; fractional values are rejected rather than rounded. When omitted, each request uses a random seed.

Repeating a request with the same seed and the same value for every other field, including n and size, returns the same images from the same model version. A model or engine update can change the images a seed produces.

The number of denoising steps. Generation time grows roughly linearly with steps. When omitted, the model's default is used: 8 for z-image-turbo.

For z-image-turbo the last step of the schedule lands at zero noise and does not change the image, so N steps apply N - 1 denoising updates. The default of 8 applies 7; 9 matches the 8 updates in the upstream Z-Image-Turbo reference.

Classifier-free guidance strength. When omitted, the model's default is used: 0 for z-image-turbo, which is also the upstream recommendation for Turbo models.

0 turns guidance off. Values above 0 turn it on, which enables negative_prompt and roughly doubles the model work per step. Higher values are not a quality setting for z-image-turbo.