RAG Chat Completion

POST /v1/chat/completions/RAG Models: Chat and reasoning View .md

Create a chat completion with context retrieved from items or files in a vector store collection. Reasoning effort, thinking budget, and the express variant follow the Chat Completions behavior.

Body

application/json

Chat Completion RAG Request Payload

collection string required

The vector store collection to search for relevant context.
model string required

The model that will be inferred for chat completion. A -normalize suffix is accepted for compatibility and ignored; every response is normalized.
messages array of objects required

The message context to use for the chat completion request, separated by system, user, assistant, tool, and developer roles.

at least 1 items
2 fields
- role string
  
  The role of the author of this message.
  
  systemassistantusertooldeveloper
- content string or array of objects
  
  The contents of the message: a string, or an array of content parts (text, images, audio, video and documents). Media parts are accepted on user messages. See chat-content-part for documents: they are converted to text, which becomes part of the prompt.
  One of 2
  - string
  - array of objects
max_tokens integer

Deprecated name for max_completion_tokens. If both are sent, max_completion_tokens takes precedence.
max_completion_tokens integer

Maximum completion tokens, including reasoning and visible output.

default 32768at least 1
reasoning_effort string

Shorthand for reasoning.effort. Configured models map this level to their supported reasoning control.

ultramaxxhighhighmediumlowminimalnone
reasoning object

Reasoning controls. Configured models map effort through their database profile. Models advertising reasoning.supports_max_tokens: true in /v1/models?format=openrouter (a max_tokens property under the reasoning parameter in the default list) enforce the budget; other models accept it without enforcing it.
4 fields
- effort string
  
  ultramaxxhighhighmediumlowminimalnone
- enabled boolean
  
  When false, sets effort to none.
- summary string
  
  Accepted but not supported by the current engines.
  
  autoconcisedetailed
- max_tokens integer
  
  Thinking-token budget. Must be below the effective completion limit so the model has room to answer. Ignored when thinking is disabled or a final assistant message is being continued.
  
  at least 1
n integer

The number of chat completion choices to generate for each input message.

default 1
seed integer

If you would like a different response from the same message, changing the seed will change the response. A null value generates a random seed.
temperature number (float)

A value between 0.0 and 2.0 that controls the randomness of the model's output. When set closer to 1, such as 0.8, the outcome is more unpredictable and creative. Values nearing 0, like 0.2, produce more predictable and less creative results. A temperature of zero requests greedy decoding; exact reproducibility depends on the model engine.

default 1range 0–2
top_p number (float)

A value between 0.0 and 1.0 that controls the probability of the model generating a particular token. A higher value will result in more diverse outputs, while a lower value will result in more repetitive outputs.

default 1range 0–1
stop string or array of strings

The strings the model stops generating at, up to 4. A single string is accepted as shorthand for a one-item array.
One of 2
- string
- array of strings
frequency_penalty number (float)

A value between -2.0 and 2.0 that controls how much the model penalizes generating repetitive responses.

default 0range -2–2
presence_penalty number (float)

A value between -2.0 and 2.0 that controls how much the model penalizes generating responses that contain certain words or phrases.

default 0range -2–2
stream boolean

Indicates whether the response should be streamed.
logprobs boolean

Whether to return log probabilities of the output tokens or not. If true, returns the log probabilities of each output token returned in the content of message.
top_logprobs integer

An integer between 0 and 20 specifying the number of most likely tokens to return at each token position, each with an associated log probability. Optional: logprobs: true alone returns the sampled token's log probability. Requires logprobs to be true.

range 0–20
tool_choice string or object

Controls which (if any) tool is called by the model. A tool choice other than none requires tools.
One of 3
- string
  
  none means the model will not call any tool and instead generates a message. auto means the model can pick between generating a message or calling one or more tools. required means the model must call one or more tools.
- Chat Completion Function Tool Choiceobject
  
  Specifies a tool the model should use. Use to force the model to call a specific function.
  - type string required
    
    For function calling, the type is always function.
    
    function
  - function object required
    1 field
    
    name string required
    
    The name of the function to call.
- Allowed toolsobject
  
  Constrains the tools available to the model to a pre-defined set. The gateway sends the engine only the allowed tools, with mode as the tool choice.
  - type string required
    
    Allowed tool configuration type. Always allowed_tools.
    
    allowed_tools
  - allowed_tools object required
    2 fields
    
    mode string required
    
    auto allows the model to pick from among the allowed tools and generate a message. required requires the model to call one or more of the allowed tools.
    
    autorequired
    
    tools array of objects required
    
    The functions the model may call, each named in the request's tools.
    
    at least 1 items
    
    2 fields
    
    type string required
    
    For function calling, the type is always function.
    
    function
    
    function object required
    
    1 field
    
    name string required
    
    The name of the function to call.
tools array of objects

A list of function tools the model may call.
2 fields
- type string
  
  The type of the tool. Currently, only function is supported.
- function object
  
  An object representing a function for use with chat completion tool calling.
  4 fields
  - name string
    
    The name of the function to be called. Must be a-z, A-Z, 0-9, or contain underscores and dashes, with a maximum length of 64
  - description string
    
    A description of what the function does, used by the model to choose when and how to call the function.
  - parameters object
    
    The parameters the functions accepts, described as a JSON Schema object.
  - strict boolean
    
    Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the parameters field. Only a subset of JSON Schema is supported when strict is true.

Responses

application/json, text/event-stream · chat-completion-response-payload

6 fields

id string

A unique identifier for the chat completion, prefixed chatcmpl-.
object string

Always chat.completion.
created integer

The Unix timestamp (in seconds) of when the chat completion was created.
model string

The model that was inferred for chat completion.
choices array of objects

The resulting assistant message(s) for the chat completion.
5 fields
- index integer
  
  The index of the choice in the list of choices.
- message object
  
  An object containing the role and content for a chat completion message.
  4 fields
  - role string
    
    The role of the author of this message.
    
    systemassistantusertooldeveloper
  - content string
    
    The contents of the message. Always a string, empty when the model only called tools.
  - reasoning string or null
    
    The reasoning (chain-of-thought) the model produced, or null. Every model reports it here; reasoning_content is not used.
  - tool_calls array of objects
    
    The tool calls generated by the model, such as function calls.
    
    3 fields
    
    id string
    
    The ID of the tool call, in the form call_<id>. Send it back unchanged as the tool_call_id of the tool result.
    
    type string
    
    The type of the tool. Currently, only function is supported.
    
    function object
    
    An object containing the the function that the model called for a chat completion choice.
    
    2 fields
    
    name string
    
    The name of the function to call.
    
    arguments string
    
    The arguments to call the function with, as generated by the model in JSON format.
- logprobs object or null
  One of 2
  - Chat Completion Logprobsobject
    
    An object containing the log probability information for a chat completion choice.
    
    content array of objects
    
    A list of message content tokens with log probability information.
    
    4 fields
    
    token string
    
    The text content of the token.
    
    logprob number (float)
    
    The log probability of the token.
    
    bytes array of integers
    
    A list of integers representing the UTF-8 bytes representation of the token.
    
    top_logprobs array of objects
    
    List of the most likely tokens and their log probability, at this token position.
    
    3 fields
    
    token string
    
    The text content of the token.
    
    logprob number (float)
    
    The log probability of the token.
    
    bytes array of integers
    
    A list of integers representing the UTF-8 bytes representation of the token.
  - null
- finish_reason string
  
  Why the model stopped, normalized across engines. tool_calls whenever the message carries tool calls.
  
  stoplengthtool_callscontent_filtererror
- native_finish_reason string or null
  
  The engine's own finish reason, unmodified.
usage object

An object containing usage information for this request.
5 fields
- completion_tokens integer
  
  Number of tokens in the generated completion, including reasoning tokens.
- prompt_tokens integer
  
  Number of tokens in the prompt. Includes vector search embeddings tokens for RAG requests.
- total_tokens integer
  
  Total number of tokens used in the request (prompt + completion).
- prompt_tokens_details object
  1 field
  - cached_tokens integer
    
    Prompt tokens served from the prefix cache. 0 when the engine does not report it.
- completion_tokens_details object
  1 field
  - reasoning_tokens integer
    
    The part of completion_tokens spent on reasoning. 0 when the engine does not report it.

curl https://api.vultrinference.com/v1/chat/completions/RAG \
  -H "Authorization: Bearer $VULTR_INFERENCE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-v4.1-flash",
    "collection": "handbook",
    "messages": [
      {
        "role": "user",
        "content": "How many vacation days do new hires get?"
      }
    ]
  }'

import os
import requests

response = requests.post(
    "https://api.vultrinference.com/v1/chat/completions/RAG",
    headers={"Authorization": f"Bearer {os.environ['VULTR_INFERENCE_API_KEY']}"},
    json={
        "model": "deepseek-v4.1-flash",
        "collection": "handbook",
        "messages": [
            {
                "role": "user",
                "content": "How many vacation days do new hires get?",
            },
        ],
    },
)
print(response.json())

const response = await fetch("https://api.vultrinference.com/v1/chat/completions/RAG", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.VULTR_INFERENCE_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    "model": "deepseek-v4.1-flash",
    "collection": "handbook",
    "messages": [
      {
        "role": "user",
        "content": "How many vacation days do new hires get?"
      }
    ]
  }),
});
console.log(await response.json());

Response · 200

{
    "id": "string",
    "object": "string",
    "created": 0,
    "model": "string",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "system",
                "content": null,
                "reasoning": null,
                "tool_calls": []
            },
            "logprobs": {
                "content": []
            },
            "finish_reason": "stop",
            "native_finish_reason": "string"
        }
    ],
    "usage": {
        "completion_tokens": 0,
        "prompt_tokens": 0,
        "total_tokens": 0,
        "prompt_tokens_details": {
            "cached_tokens": 0
        },
        "completion_tokens_details": {
            "reasoning_tokens": 0
        }
    }
}