# Limits

How many requests a minute, how many at once, how much a model reads and writes, and how large a body can be.

## At a glance

| Limit | Default |
| --- | --- |
| Requests a minute | 120 per model, per subscription |
| Requests at once | 16 per model, per subscription, on chat, responses and messages |
| Output | 32,768 tokens unless you set a lower cap |
| Time | About nine minutes per request |

Some models have lower limits. Each limit counts one model at a time, so traffic to one never uses up another's.

## Requests a minute

Past the limit the answer is 429, with `Retry-After` set to the seconds until the minute turns over:

```json
{
  "error": {
    "code": 429,
    "message": "Rate limit exceeded. Please retry later.",
    "type": "rate_limit_error",
    "metadata": {
      "error_type": "rate_limit_exceeded"
    }
  }
}
```

On `/v1/messages` the body is Anthropic's `rate_limit_error`, with the same header. A streaming request gets the 429 itself, not a stream.

## Requests at once

A request beyond the number at once waits its turn, first come first served, for up to two minutes. One that waits longer gets a 429 with `Retry-After: 5`, and a message that says it was the number at once rather than the rate. Nothing is sent while a request waits, so leave your client's timeout generous.

## Retrying

Retry a 429 after `Retry-After`, and a 502, 503 or 504 after a growing wait. Other errors fail the same way again until the request changes: [Errors](/docs/concepts/errors#status-codes) has the whole list.

```python
import os
import time

import requests

URL = "https://api.vultrinference.com/v1/chat/completions"
HEADERS = {"Authorization": f"Bearer {os.environ['VULTR_INFERENCE_API_KEY']}"}


# Retry what a later attempt can fix: 429 after Retry-After, and 502, 503 and 504 with a growing wait
def post(body: dict, attempts: int = 5) -> dict:
    for attempt in range(attempts):
        response = requests.post(URL, headers=HEADERS, json=body, timeout=600)
        if response.status_code not in (429, 502, 503, 504) or attempt == attempts - 1:
            response.raise_for_status()
            return response.json()
        time.sleep(int(response.headers.get("Retry-After", 2**attempt)))


reply = post({"model": "deepseek-v4.1-flash", "messages": [{"role": "user", "content": "Say hello."}]})
print(reply["choices"][0]["message"]["content"])
```

```typescript
const URL = "https://api.vultrinference.com/v1/chat/completions";

// Retry what a later attempt can fix: 429 after Retry-After, and 502, 503 and 504 with a growing wait
async function post(body: object, attempts = 5): Promise<any> {
  for (let attempt = 0; ; attempt++) {
    const response = await fetch(URL, {
      method: "POST",
      headers: { Authorization: `Bearer ${process.env.VULTR_INFERENCE_API_KEY}`, "Content-Type": "application/json" },
      body: JSON.stringify(body),
    });
    if (![429, 502, 503, 504].includes(response.status) || attempt === attempts - 1) {
      if (!response.ok) throw new Error(`${response.status}: ${await response.text()}`);
      return response.json();
    }
    const wait = Number(response.headers.get("Retry-After") ?? 2 ** attempt);
    await new Promise((resolve) => setTimeout(resolve, wait * 1000));
  }
}

const reply = await post({ model: "deepseek-v4.1-flash", messages: [{ role: "user", content: "Say hello." }] });
console.log(reply.choices[0].message.content);
```

## Context window

A model reads at most its context window, prompt and answer together. Each [model's page](/docs/models) gives its window.

- If the prompt and `max_completion_tokens` would not fit, the answer's budget is cut to what does and the request goes ahead. `finish_reason: "length"` is the sign it ran out.
- If the prompt alone fills the window, the answer is a 400, `context_length_exceeded`, with the count:

```text
Your input is ~1203887 tokens, which exceeds the model context limit of 1048576. Please shorten your messages.
```

Documents and media count as the text they become ([Documents and media](/docs/concepts/documents)).

## Output length

A chat completion writes up to 32,768 tokens unless `max_completion_tokens` (or `max_tokens`) sets a lower cap; `/v1/responses` reads `max_output_tokens`. A lower cap bounds what one request can cost. `/v1/messages` has no default: `max_tokens` is required.

## Body and file sizes

| What | Limit |
| --- | --- |
| A request to `/v1/embeddings` or `/v1/rerank` | 16 MB |
| A multipart upload, on any route | 100 MB |
| A file for `/v1/files` | 100 MB, and 1,000 files a subscription |
| Audio for `/v1/audio/transcriptions` | 25 MB |
| A PDF, read as a document | 300 pages |
| An image for `/v1/images/edits` | 20 MB each, 64 MB together |
| Media for `/v1/videos` | Images 30 MB, videos 50 MB, audio 15 MB |
| Inputs to `/v1/embeddings` | 2,048 a request |
| Results from `/v1/rerank` | 100 |

Over a size limit the answer is 413, and its message gives the limit.

## Time

A request may run for about nine minutes; one that has not answered by then gets a 504. Videos and songs are [jobs](/docs/concepts/async-jobs) instead: they render for up to half an hour without holding a connection open.