Limits

At a glance#

Each model allows each subscription a number of requests a minute and a number at once. Most models have the defaults of their kind:

Some models have their own limits instead, higher or lower: each model's page gives its limits, and GET /v1/models?format=openrouter lists them as rate_limits, by output. Each limit counts one model at a time, so traffic to one never uses up another's, and a model that makes both video and images counts each apart.

Requests a minute#

Past the limit the answer is 429, with Retry-After set to the seconds until the minute turns over:

{
  "error": {
    "code": 429,
    "message": "Rate limit exceeded. Please retry later.",
    "type": "rate_limit_error",
    "metadata": {
      "error_type": "rate_limit_exceeded"
    }
  }
}

On /v1/messages the body is Anthropic's rate_limit_error, with the same header. A streaming request gets the 429 itself, not a stream.

Requests at once#

A request beyond the number at once waits its turn, first come first served, for up to two minutes. One that waits longer gets a 429 with Retry-After: 5, and a message that says it was the number at once rather than the rate. Nothing is sent while a request waits, so leave your client's timeout generous.

Videos and songs are jobs: the number at once is how many may be pending or rendering, and one more is refused straight away with a 429 and Retry-After: 30. A job counts until it completes, fails or is cancelled.

Retrying#

Retry a 429 after Retry-After, and a 502, 503 or 504 after a growing wait. Other errors fail the same way again until the request changes: Errors has the whole list.

import os
import time

import requests

URL = "https://api.vultrinference.com/v1/chat/completions"
HEADERS = {"Authorization": f"Bearer {os.environ['VULTR_INFERENCE_API_KEY']}"}


# Retry what a later attempt can fix: 429 after Retry-After, and 502, 503 and 504 with a growing wait
def post(body: dict, attempts: int = 5) -> dict:
    for attempt in range(attempts):
        response = requests.post(URL, headers=HEADERS, json=body, timeout=600)
        if response.status_code not in (429, 502, 503, 504) or attempt == attempts - 1:
            response.raise_for_status()
            return response.json()
        time.sleep(int(response.headers.get("Retry-After", 2**attempt)))


reply = post({"model": "deepseek-v4.1-flash-1017", "messages": [{"role": "user", "content": "Say hello."}]})
print(reply["choices"][0]["message"]["content"])

const URL = "https://api.vultrinference.com/v1/chat/completions";

// Retry what a later attempt can fix: 429 after Retry-After, and 502, 503 and 504 with a growing wait
async function post(body: object, attempts = 5): Promise<any> {
  for (let attempt = 0; ; attempt++) {
    const response = await fetch(URL, {
      method: "POST",
      headers: { Authorization: `Bearer ${process.env.VULTR_INFERENCE_API_KEY}`, "Content-Type": "application/json" },
      body: JSON.stringify(body),
    });
    if (![429, 502, 503, 504].includes(response.status) || attempt === attempts - 1) {
      if (!response.ok) throw new Error(`${response.status}: ${await response.text()}`);
      return response.json();
    }
    const wait = Number(response.headers.get("Retry-After") ?? 2 ** attempt);
    await new Promise((resolve) => setTimeout(resolve, wait * 1000));
  }
}

const reply = await post({ model: "deepseek-v4.1-flash-1017", messages: [{ role: "user", content: "Say hello." }] });
console.log(reply.choices[0].message.content);

Context window#

A model reads at most its context window, prompt and answer together. Each model's page gives its window.

• If the prompt and max_completion_tokens would not fit, the answer's budget is cut to what does and the request goes ahead. finish_reason: "length" is the sign it ran out.
• If the prompt alone fills the window, the answer is a 400, context_length_exceeded, with the count:

Your input is ~1203887 tokens, which exceeds the model context limit of 1048576. Please shorten your messages.

Documents and media count as the text they become (Documents and media).

Output length#

Without max_completion_tokens (or max_tokens) a chat completion writes up to 32,768 tokens; /v1/responses does the same without max_output_tokens. That is a default, not a ceiling: your own cap can be higher, up to what fits in the model's context window beside the prompt, or lower to bound what one request can cost. /v1/messages has no default: max_tokens is required.

Body and file sizes#

Over a size limit the answer is 413, and its message gives the limit.

Time#

A request may run for about nine minutes; one that has not answered by then gets a 504. Videos and songs are jobs instead: they render for up to half an hour without holding a connection open.