# Time lyrics and speech to their audio

Send a recording and the words in it, and get a start and an end for every word and line, for karaoke, captions and subtitles.

`POST /v1/audio/alignments` takes a recording and the words it holds, and answers with when each word and each line is sung or spoken. Nothing is transcribed: every word that comes back is one you sent, so the text stays exactly yours. Use it to light up lyrics as a song plays, to cue captions, or to cut a recording at a line.

The model is `aligner-1`. It is synchronous: a three-minute song takes about 5 seconds, plus the upload. It costs $0.002 a minute of audio, billed by the second; [its model page](/docs/models/aligner-1) has the price and limits as they stand.

| Limit | |
| --- | --- |
| Audio | 25 MB and 15 minutes, any common format (mp3, wav, flac, m4a, ogg, webm, aac) |
| Text | 12,000 characters |
| Rate | 30 requests a minute and 5 at once per account |

## Align a recording

This makes a short clip with the speech endpoint, then aligns it. The Python version also writes the lines out as SRT captions.

```bash
# A short clip to align: the speech endpoint reads the text aloud
curl -s https://api.vultrinference.com/v1/audio/speech \
  -H "Authorization: Bearer $VULTR_INFERENCE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "qwen3-tts-12hz-1.7b", "voice": "vivian", "response_format": "mp3", "input": "The quick brown fox jumps over the lazy dog. It barked twice, then slept."}' \
  -o clip.mp3

# Align it with the words it holds, one line per line; speech mode skips separating a vocal from music
curl -s https://api.vultrinference.com/v1/audio/alignments \
  -H "Authorization: Bearer $VULTR_INFERENCE_API_KEY" \
  -F model=aligner-1 \
  -F file=@clip.mp3 \
  -F mode=speech \
  -F 'text=The quick brown fox jumps over the lazy dog.
It barked twice, then slept.' | jq '{lines: [.lines[] | {text, start, end, certain}], usage}'
```

```python
import os

import requests

API = "https://api.vultrinference.com/v1"
headers = {"Authorization": f"Bearer {os.environ['VULTR_INFERENCE_API_KEY']}"}
text = "The quick brown fox jumps over the lazy dog.\nIt barked twice, then slept."

# A short clip to align: the speech endpoint reads the text aloud
clip = requests.post(f"{API}/audio/speech", headers=headers, json={
    "model": "qwen3-tts-12hz-1.7b",
    "voice": "vivian",
    "response_format": "mp3",
    "input": text.replace("\n", " "),
})
clip.raise_for_status()

# Align it; speech mode skips separating a vocal from music
response = requests.post(
    f"{API}/audio/alignments",
    headers=headers,
    data={"model": "aligner-1", "text": text, "mode": "speech"},
    files={"file": ("clip.mp3", clip.content, "audio/mpeg")},
)
response.raise_for_status()
alignment = response.json()

for word in alignment["words"]:
    print(f"{word['start']:6.2f} {word['end']:6.2f}  {word['text']}")


# Each line as an SRT caption
def stamp(seconds):
    ms = round(seconds * 1000)
    return f"{ms // 3600000:02}:{ms // 60000 % 60:02}:{ms // 1000 % 60:02},{ms % 1000:03}"


with open("captions.srt", "w") as f:
    for n, line in enumerate(alignment["lines"], 1):
        f.write(f"{n}\n{stamp(line['start'])} --> {stamp(line['end'])}\n{line['text']}\n\n")

print("duration", alignment["usage"])
```

## The text you send

Send the words in the order they are heard, one line of the song or script per line. The line breaks are yours to choose: each becomes a line in the answer, with its own start and end.

- **Write out every repeat.** A chorus sung three times is written three times. The words are placed in order, so a chorus written once is placed once.
- **Label sections in square brackets.** A line that is only `[Chorus]` or `[Verse 2 - Singer B]` names the lines under it and is never aligned. Each line in the answer carries its section's number and label.
- **Write numbers as words.** The model reads letters, so `21` has nothing to place; write `twenty-one`.
- **Send only what is heard.** Every word you send is placed somewhere, so a line that is not in the recording still gets times, usually with a low score. Leave it out instead. A pause or an instrumental break with no words is left empty, not stretched over.

## Songs and speech

`mode` is `lyrics` by default, which separates the vocal from the music before aligning; that is what a song needs. Send `mode=speech` for a podcast, an interview, narration or anything else without a backing track: it skips the separation, and `separated` in the answer is `false`.

## The answer

```json
{
  "duration": 220.89,
  "separated": true,
  "lines": [{"index": 0, "section": 1, "tag": "Verse", "text": "The lamp was low, the door was barred with iron", "start": 21.22, "end": 25.66, "score": 0.981, "certain": true}],
  "words": [{"line": 0, "text": "lamp", "start": 21.48, "end": 21.79, "score": 0.94}],
  "unaligned": [],
  "usage": {"type": "duration", "seconds": 220.89, "cost": 0.00736593}
}
```

- `lines` has one entry per line of your text, in order, with times in seconds from the start of the audio. `section` counts the `[Label]` sections from 0 and `tag` is the label, or `null` before the first one.
- `words` has every word of every line, written as you wrote it, with the line it belongs to.
- `score` is how well the audio matched, from 0 to 1. `certain` is the model's own judgement that a line is placed right. It is usually right, but not always: on the benchmark below about 2% of lines were more than a second off and still marked certain. Check the lines that are not certain first.
- `unaligned` lists lines the model could not place at all, with the reason, instead of leaving them out silently.

## Timing a song you generate

If the song comes from `POST /v1/music`, you do not need to download it and send it back: add `"timestamps": true` to the
music request and the finished job's outputs carry the same `lines`, `words` and `unaligned` (as `outputs[].timestamps`),
made by `aligner-1` from the audio the model produced and the lyrics you sent. It works for models whose entry in
`GET /v1/music/models` says `timestamps: true` (minimax-music3 and ace-step-1.5), needs lyrics, and the song's language
must be one `aligner-1` lists. The timing is billed as above, by the second, and shown in the job's `usage.timestamps`;
the job's `usage.cost` includes it. If the timing fails the song still completes, without `timestamps`, with a
`timestamps_unavailable` warning, and nothing is billed for the timing. The request is in [Write songs for MiniMax Music 3](/docs/workflows/music-minimax-music3).

## How accurate it is

On JamendoLyrics, a public set of 79 songs with every word's start marked by hand, half the words are placed within 0.04 seconds of where they are sung, and 92 to 98% within 0.3 seconds, in English, French, German and Spanish. On 69 minutes of meeting recordings in speech mode, 98.8% of words land within 0.3 seconds of the recordings' published timings, which were made by machine rather than by hand.

## Languages

`aligner-1` reads the Latin alphabet: English, Spanish, French, German, Portuguese, Italian, Dutch, Turkish and the other languages its model page lists. Accented letters are read as their plain ones, so `niño` and `Mädchen` align, and come back as you wrote them.

Text in another script (Chinese, Japanese, Korean, Cyrillic, Arabic and others) has nothing the model can read, and the request is a 400 that says so. A `language` the model does not list is a 400 naming the ones it does. Neither is billed.

## Billing and errors

A call that returns lines bills the whole recording, rounded up to the second: 220.89 seconds bills 221. `usage.cost` is that charge in dollars. A refused request bills nothing.

| Status | When |
| --- | --- |
| 400 | A field is missing or not allowed, the audio cannot be decoded, or the text has nothing the model can read |
| 413 | The audio or the text is over the limits above |
| 429 | Over the rate limits; wait for `Retry-After` |

- [`POST /v1/audio/alignments`](/docs/reference/create-alignment.md): Create Alignment