Time lyrics and speech to their audio
Send a recording and the words in it, and get a start and an end for every word and line, for karaoke, captions and subtitles.
POST /v1/audio/alignments takes a recording and the words it holds, and answers with when each word and each line is sung or spoken. Nothing is transcribed: every word that comes back is one you sent, so the text stays exactly yours. Use it to light up lyrics as a song plays, to cue captions, or to cut a recording at a line.
The model is aligner-1. It is synchronous: a three-minute song takes about 5 seconds, plus the upload. It costs $0.002 a minute of audio, billed by the second; its model page has the price and limits as they stand.
| Limit | |
|---|---|
| Audio | 25 MB and 15 minutes, any common format (mp3, wav, flac, m4a, ogg, webm, aac) |
| Text | 12,000 characters |
| Rate | 30 requests a minute and 5 at once per account |
Align a recording
This makes a short clip with the speech endpoint, then aligns it. The Python version also writes the lines out as SRT captions.
# A short clip to align: the speech endpoint reads the text aloud
curl -s https://api.vultrinference.com/v1/audio/speech \
-H "Authorization: Bearer $VULTR_INFERENCE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model": "qwen3-tts-12hz-1.7b", "voice": "vivian", "response_format": "mp3", "input": "The quick brown fox jumps over the lazy dog. It barked twice, then slept."}' \
-o clip.mp3
# Align it with the words it holds, one line per line; speech mode skips separating a vocal from music
curl -s https://api.vultrinference.com/v1/audio/alignments \
-H "Authorization: Bearer $VULTR_INFERENCE_API_KEY" \
-F model=aligner-1 \
-F file=@clip.mp3 \
-F mode=speech \
-F 'text=The quick brown fox jumps over the lazy dog.
It barked twice, then slept.' | jq '{lines: [.lines[] | {text, start, end, certain}], usage}'import os
import requests
API = "https://api.vultrinference.com/v1"
headers = {"Authorization": f"Bearer {os.environ['VULTR_INFERENCE_API_KEY']}"}
text = "The quick brown fox jumps over the lazy dog.\nIt barked twice, then slept."
# A short clip to align: the speech endpoint reads the text aloud
clip = requests.post(f"{API}/audio/speech", headers=headers, json={
"model": "qwen3-tts-12hz-1.7b",
"voice": "vivian",
"response_format": "mp3",
"input": text.replace("\n", " "),
})
clip.raise_for_status()
# Align it; speech mode skips separating a vocal from music
response = requests.post(
f"{API}/audio/alignments",
headers=headers,
data={"model": "aligner-1", "text": text, "mode": "speech"},
files={"file": ("clip.mp3", clip.content, "audio/mpeg")},
)
response.raise_for_status()
alignment = response.json()
for word in alignment["words"]:
print(f"{word['start']:6.2f} {word['end']:6.2f} {word['text']}")
# Each line as an SRT caption
def stamp(seconds):
ms = round(seconds * 1000)
return f"{ms // 3600000:02}:{ms // 60000 % 60:02}:{ms // 1000 % 60:02},{ms % 1000:03}"
with open("captions.srt", "w") as f:
for n, line in enumerate(alignment["lines"], 1):
f.write(f"{n}\n{stamp(line['start'])} --> {stamp(line['end'])}\n{line['text']}\n\n")
print("duration", alignment["usage"])The text you send
Send the words in the order they are heard, one line of the song or script per line. The line breaks are yours to choose: each becomes a line in the answer, with its own start and end.
- Write out every repeat. A chorus sung three times is written three times. The words are placed in order, so a chorus written once is placed once.
- Label sections in square brackets. A line that is only
[Chorus]or[Verse 2 - Singer B]names the lines under it and is never aligned. Each line in the answer carries its section's number and label. - Write numbers as words. The model reads letters, so
21has nothing to place; writetwenty-one. - Send only what is heard. Every word you send is placed somewhere, so a line that is not in the recording still gets times, usually with a low score. Leave it out instead. A pause or an instrumental break with no words is left empty, not stretched over.
Songs and speech
mode is lyrics by default, which separates the vocal from the music before aligning; that is what a song needs. Send mode=speech for a podcast, an interview, narration or anything else without a backing track: it skips the separation, and separated in the answer is false.
The answer
{
"duration": 220.89,
"separated": true,
"lines": [
{
"index": 0,
"section": 1,
"tag": "Verse",
"text": "The lamp was low, the door was barred with iron",
"start": 21.22,
"end": 25.66,
"score": 0.981,
"certain": true
}
],
"words": [
{
"line": 0,
"text": "lamp",
"start": 21.48,
"end": 21.79,
"score": 0.94
}
],
"unaligned": [],
"usage": {
"type": "duration",
"seconds": 220.89,
"cost": 0.00736593
}
}lineshas one entry per line of your text, in order, with times in seconds from the start of the audio.sectioncounts the[Label]sections from 0 andtagis the label, ornullbefore the first one.wordshas every word of every line, written as you wrote it, with the line it belongs to.scoreis how well the audio matched, from 0 to 1.certainis the model's own judgement that a line is placed right. It is usually right, but not always: on the benchmark below about 2% of lines were more than a second off and still marked certain. Check the lines that are not certain first.unalignedlists lines the model could not place at all, with the reason, instead of leaving them out silently.
Timing a song you generate
If the song comes from POST /v1/music, you do not need to download it and send it back: add "timestamps": true to the
music request and the finished job's outputs carry the same lines, words and unaligned (as outputs[].timestamps),
made by aligner-1 from the audio the model produced and the lyrics you sent. It works for models whose entry in
GET /v1/music/models says timestamps: true (minimax-music3 and ace-step-1.5), needs lyrics, and the song's language
must be one aligner-1 lists. The timing is billed as above, by the second, and shown in the job's usage.timestamps;
the job's usage.cost includes it. If the timing fails the song still completes, without timestamps, with a
timestamps_ warning, and nothing is billed for the timing. The request is in Write songs for MiniMax Music 3.
How accurate it is
On JamendoLyrics, a public set of 79 songs with every word's start marked by hand, half the words are placed within 0.04 seconds of where they are sung, and 92 to 98% within 0.3 seconds, in English, French, German and Spanish. On 69 minutes of meeting recordings in speech mode, 98.8% of words land within 0.3 seconds of the recordings' published timings, which were made by machine rather than by hand.
Languages
aligner-1 reads the Latin alphabet, Chinese, Japanese and Cyrillic: English, Spanish, French, German, Portuguese, Italian, Dutch, Turkish, Chinese, Japanese, Russian, Ukrainian and the other languages its model page lists. Accented letters are read as their plain ones, so niño and Mädchen align, and come back as you wrote them. Chinese is timed a character at a time, Japanese a kana or a run of kanji at a time.
Korean, Arabic and other scripts are not read yet, and a text with nothing in a script the model reads is a 400 that says so. A language the model does not list is a 400 naming the ones it does. Neither is billed.
Billing and errors
A call that returns lines bills the whole recording, rounded up to the second: 220.89 seconds bills 221. usage.cost is that charge in dollars. A refused request bills nothing.
| Status | When |
|---|---|
| 400 | A field is missing or not allowed, the audio cannot be decoded, or the text has nothing the model can read |
| 413 | The audio or the text is over the limits above |
| 429 | Over the rate limits; wait for Retry-After |