Make songs with ACE-Step 1.5

ACE-Step 1.5 makes a song, with vocals or without, from a short style caption and optional lyrics. It renders fast, takes the tempo, key and time signature as settings of its own, and makes the length you ask for to the second.

Everything else is optional: bpm, key and time_signature set the music, seed makes a take repeatable, and response_format picks the file type. Its model page has every field and the price.

A song is an async job: you create it, then poll it or get a webhook. It is usually done in seconds, so Prefer: wait on the create request often answers with the finished song.

Make a song#

This 30-second song is done in a few seconds on turbo. The Python version asks for hifi.

# Create the song and wait up to 60 seconds for it (Prefer: wait); a 30-second turbo song is done in a few seconds, so it answers 200
curl -s https://api.vultrinference.com/v1/music \
  -H "Authorization: Bearer $VULTR_INFERENCE_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait=60" \
  -d '{
    "model": "ace-step-1.5",
    "prompt": "Indie folk, warm female alto, fingerpicked acoustic guitar, soft brushed drums, intimate and hopeful",
    "lyrics": "[Verse]\nMorning light on the kitchen floor\nCoffee cups and an open door\n\n[Chorus]\nStay a while, stay a while\nThe day can wait for us",
    "duration": 30,
    "bpm": 96,
    "key": "G major",
    "time_signature": "4/4",
    "seed": 7
  }' | tee song.json | jq '{id, status, usage}'

# Download it once its status is completed
curl -s -o song.mp3 "https://api.vultrinference.com/v1/music/$(jq -r .id song.json)/content" \
  -H "Authorization: Bearer $VULTR_INFERENCE_API_KEY"

import os
import time

import requests

API = "https://api.vultrinference.com/v1"
headers = {"Authorization": f"Bearer {os.environ['VULTR_INFERENCE_API_KEY']}"}

lyrics = "\n".join([
    "[Verse]",
    "Morning light on the kitchen floor",
    "Coffee cups and an open door",
    "",
    "[Chorus]",
    "Stay a while, stay a while",
    "The day can wait for us",
])

# Create the job; it answers 202 at once. quality "hifi" renders on the
# 50-step model and bills 7.5 times the seconds; leave it out for turbo.
job = requests.post(f"{API}/music", headers=headers, json={
    "model": "ace-step-1.5",
    "prompt": "Indie folk, warm female alto, fingerpicked acoustic guitar, soft brushed drums, intimate and hopeful",
    "lyrics": lyrics,
    "duration": 30,
    "bpm": 96,
    "key": "G major",
    "time_signature": "4/4",
    "quality": "hifi",
    "seed": 7,
}).json()

# Poll at the pace the API asks for (openai-poll-after-ms), until the job is final
wait_ms = 2000
while job["status"] in ("pending", "in_progress"):
    time.sleep(wait_ms / 1000)
    response = requests.get(job["polling_url"], headers=headers)
    job = response.json()
    wait_ms = int(response.headers.get("openai-poll-after-ms", wait_ms))

print(job["status"], job.get("usage"))
if job["status"] == "completed":
    with open("song.mp3", "wb") as f:
        f.write(requests.get(f"{API}/music/{job['id']}/content", headers=headers).content)

Turbo or hifi#

quality picks the model the song renders on:

Turbo is quick and cheap enough to try many captions and seeds. Hifi is a different model, so the same request and seed make a different song on it, not a finer render of the turbo take. Any other quality is a 400 that lists the tiers.

The caption#

ACE-Step reads the caption as a description of the sound. Name the genre and subgenre, the mood, the main instruments and the voice, in a sentence or a list of tags:

Indie folk, warm female alto, fingerpicked acoustic guitar, soft brushed drums, intimate and hopeful

Keep it to the music: the story belongs in the lyrics. The model reads at most about 200 tokens of caption, so the API holds prompt to 750 characters rather than let the end be cut off without a word.

Tempo, key and time signature#

bpm, key and time_signature are fields the model takes directly, not hints written into the caption:

Any other time signature is a 400 that lists these four. Leave a field out and the model chooses.

The lyrics#

The lyrics are sections in order, each a tag in square brackets followed by its lines:

[Verse]
Morning light on the kitchen floor
Coffee cups and an open door

[Chorus]
Stay a while, stay a while
The day can wait for us

[Bridge]
And if the rain comes in

[Chorus]
Stay a while, stay a while

The tags it knows are [Intro], [Verse], [Pre-Chorus], [Chorus], [Bridge], [Interlude], [Instrumental] and [Outro]. Spacing and case do not matter ([pre chorus] is [Pre-Chorus]), and [Inst] is read as [Instrumental]. Any other tag is passed on as you wrote it. Everything that is not a tag is sung, so keep stage directions and singer names in the caption.

The model reads about 2,000 tokens of lyrics, which is why lyrics is held to 4,096 characters.

The lyrics' language#

language tells the model which language the lyrics are in: en (the default), zh, ja, ko, es, fr, de and 44 more, listed under languages in GET /v1/music/models. yue is Cantonese. A regional tag such as zh-CN or pt-BR is read as its language. Name the language whenever the lyrics are not English. A song without lyrics is sent as unknown.

Instrumentals#

Leave lyrics out, or send an empty string, for a song with no vocals. Describe the lead instrument in the caption instead of a voice.

A reference recording#

reference_audio gives the model a recording whose voice and tone the song takes up, such as a singer or a mix you like. Send one of:

• an https URL: the API fetches it when the song renders
• a base64 data URI: data:audio/mpeg;base64,...
• a song from one of your own jobs: {"type": "job_output", "job_id": "music_...", "index": 0}, kept for 7 days

It must be 10 to 600 seconds long and at most 50 MB, in wav, mp3, flac, ogg, m4a or aac. A data URI or a job's song is checked when you send the request, and anything wrong is a 400. A URL is checked when the song renders, and if the file breaks these rules the job fails with invalid_request and is not billed.

input_references takes the same audio as a list, each item with "role": "reference"; send a reference once, either way.

Length#

duration is the song's exact length: 10 to 600 seconds, 120 by default. Unlike a model whose duration is only a ceiling, ACE-Step fills the time you ask for, so fit the lyrics to it: a few sections for 30 seconds, a full verse-chorus song for three minutes or more.

You pay per second of audio, rounded up, at the tier's price. A 30-second song costs 30 seconds on turbo and 225 on hifi.

Takes and seeds#

The same request with the same seed makes the same song on the same tier. Keep a seed you like and change one thing at a time, such as a line of lyrics or the tempo; changing the tier changes the model, and with it the song. ACE-Step makes one song per request (n is 1); for several candidates, send several requests, each billed on its own.

Getting the song#

ACE-Step sends the finished file, not a stream, so its jobs have no stream_url. Poll the job at the pace openai-poll-after-ms asks for, send Prefer: wait=60 on the create request, or set callback_url for a webhook.

GET /v1/music/{id}/content gives the song as mp3 (the default, 256 kbps), wav, flac, opus, aac or raw pcm, at 48 kHz in stereo, the model's native rate. sample_rate asks for another (8 to 48 kHz). It is kept for 7 days.

What it does not do yet#

ACE-Step can also edit audio: repaint part of a song, cover it in another style, continue it, separate a stem, or add a part. These work on a song you send as source_audio, which no task takes yet, so they are not offered: task takes only text2music.

When something sounds wrong#