Vultr Inference API (1.1.3)

The model identifier to use for generation.

Total completion token allowance, including thinking and visible output. The gateway forwards zero if supplied; whether zero is accepted depends on the upstream engine.

Input messages. Models operate on alternating user and assistant conversational turns. Consecutive user or assistant turns are combined into a single turn.

If the final message uses the assistant role, the response content continues immediately from that message's content (assistant prefill).

A text-only system message may follow a user turn, as the final message or before an assistant turn. Use the top-level system parameter for initial instructions. Turn-scoped clear_at and per-message output_config are not supported.

System prompt providing context or instructions to the model. May be a string or an array of text blocks.

Whether to stream the response using server-sent events. Streaming events use typed event names: message_start, content_block_start, content_block_delta, content_block_stop, message_delta, message_stop.

Controls randomness in the model responses. Valid range: 0 to 1.

Enables nucleus sampling with a cumulative probability cutoff.

Limits sampling to the top K options.

Custom sequences that, when encountered in the generated text, cause the model to stop.

Additional metadata about the request.

Output controls. Effort is translated through the selected model's reasoning configuration. Unsupported configured levels return 422.

Thinking configuration. The adapter signs its own thinking blocks for continuation; it does not emit redacted thinking blocks.

Definitions for tools available to the model.

The model to use. If omitted, the gateway resolves its default model. Append -normalize to route through the response normalizer. A :<variant> suffix is parsed separately from the model ID; :express derives a thinking budget when the model supports enforced budgets and no explicit budget is supplied.

The message context to use for the chat completion request, separated by system, user, assistant, tool, and developer roles. A final assistant message is treated as a prefill and continued (see continue_final_message).

Continue the final message instead of starting a new assistant turn. Implied whenever the final message has the assistant role, so it only makes the intent explicit. The final message must carry non-empty text and no tool_calls; trailing whitespace is trimmed, thinking is disabled for the continuation, and only the newly generated text is returned. Use it to resume a response that ended with finish_reason length.

Indicates whether the response should be streamed.

Deprecated name for max_completion_tokens. If both are sent, max_completion_tokens takes precedence.

Maximum completion tokens, including reasoning and visible output. The effective limit may shrink to fit the model context window.

Shorthand for reasoning.effort. When both are supplied, reasoning.effort takes precedence. Configured models map this level to their supported reasoning control; unsupported levels return 422.

Reasoning controls. Configured models map effort through their database profile. Models advertising reasoning.supports_max_tokens: true in /v1/models enforce the budget; other models accept it without enforcing it.

The number of chat completion choices to generate for each input message.

If you would like a different response from the same message, changing the seed will change the response. A null value generates a random seed.

A value between 0.0 and 2.0 that controls the randomness of the model's output. When set closer to 1, such as 0.8, the outcome is more unpredictable and creative. Values nearing 0, like 0.2, produce more predictable and less creative results. A temperature of zero requests greedy decoding; exact reproducibility depends on the model engine.

A value between 0.0 and 1.0 that controls the probability of the model generating a particular token. A higher value will result in more diverse outputs, while a lower value will result in more repetitive outputs.

A value between -2.0 and 2.0 that controls how much the model penalizes generating repetitive responses.

A value between -2.0 and 2.0 that controls how much the model penalizes generating responses that contain certain words or phrases.

A list of strings that the model will stop generating text if it encounters any of them.

Whether to return log probabilities of the output tokens or not. If true, returns the log probabilities of each output token returned in the content of message.

An integer between 0 and 20 specifying the number of most likely tokens to return at each token position, each with an associated log probability. logprobs must be set to true if this parameter is used.

The vector store collection to search for relevant context.

The model that will be inferred for chat completion. Append -normalize to the model name (e.g. <model>-normalize) to normalize non-standard OpenAI responses such as reasoning_content → reasoning and fix tool call ID formats.

The message context to use for the chat completion request, separated by system, user, assistant, tool, and developer roles.

Deprecated name for max_completion_tokens. If both are sent, max_completion_tokens takes precedence.

Maximum completion tokens, including reasoning and visible output.

Shorthand for reasoning.effort. Configured models map this level to their supported reasoning control.

Reasoning controls. Configured models map effort through their database profile. Models advertising reasoning.supports_max_tokens: true in /v1/models enforce the budget; other models accept it without enforcing it.

The number of chat completion choices to generate for each input message.

If you would like a different response from the same message, changing the seed will change the response. A null value generates a random seed.

A value between 0.0 and 2.0 that controls the randomness of the model's output. When set closer to 1, such as 0.8, the outcome is more unpredictable and creative. Values nearing 0, like 0.2, produce more predictable and less creative results. A temperature of zero requests greedy decoding; exact reproducibility depends on the model engine.

A value between 0.0 and 1.0 that controls the probability of the model generating a particular token. A higher value will result in more diverse outputs, while a lower value will result in more repetitive outputs.

A list of strings that the model will stop generating text if it encounters any of them.

A value between -2.0 and 2.0 that controls how much the model penalizes generating repetitive responses.

A value between -2.0 and 2.0 that controls how much the model penalizes generating responses that contain certain words or phrases.

Indicates whether the response should be streamed.

Whether to return log probabilities of the output tokens or not. If true, returns the log probabilities of each output token returned in the content of message.

An integer between 0 and 20 specifying the number of most likely tokens to return at each token position, each with an associated log probability. logprobs must be set to true if this parameter is used.

If omitted, the gateway resolves its default model. A :<variant> suffix is parsed separately from the model ID; :express derives a thinking budget when the model supports enforced budgets and no explicit budget is supplied.

Input text or structured input items.

Maximum output tokens, including reasoning and visible output.

Reasoning controls. Configured models map effort through their database profile. Models advertising reasoning.supports_max_tokens: true in /v1/models enforce the budget; other models accept it without enforcing it.

The model that will be used to generate text-to-speech audio.

The text to generate audio for, up to a maximum of 2,000 characters.

The text to be converted into embeddings and stored in the vector store collection.

A description of the contents in this collection item record. If omitted, this value will default to a shortened version of the text stored in the collection.

A text description of the desired image(s). The maximum length is 2000 characters.

The model to use for image generation. Defaults to flux.1-dev.

The number of images to generate. Must be between 1 and 10.

The format in which generated images are returned. Must be one of url or b64_json.