Skip to main content
NexGPU

Music generation model

Run OpenAI Jukebox on a single Tesla V100 32GB

The 5b_lyrics top-level prior holds 11.5GB of weights, and upsampling — not the top level — eats the clock. The code was written against PyTorch 1.4 and CUDA 10.0, so the V100 it grew up on is still the right card, and it happens to be the cheapest one we rent.

Sign up freeView pricing

Jukebox is OpenAI's raw-audio music model from 2020, published as "Jukebox: A Generative Model for Music" by Dhariwal, Jun, Payne, Kim, Radford and Sutskever. It does not write MIDI and it does not predict spectrograms. A three-level VQ-VAE compresses 44.1kHz waveforms into discrete codes at hop lengths of 8, 32 and 128 with a 2048-entry codebook per level, and sparse Transformers model those code sequences autoregressively. It was trained on 1.2 million songs, 600k of them in English with lyrics scraped from LyricWiki, which is why it remains one of the very few open models you can condition on raw lyrics and have it actually sing them.

The first line of github.com/openai/jukebox reads Status: Archive — code provided as-is, no updates expected. That is not a dealbreaker, it is a prerequisite. What you get is a 2020 snapshot that wants Python 3.7.5, mpi4py 3.0.3, and PyTorch 1.4 with cudatoolkit 10.0. The HuggingFace port has been demoted along the way too: it moved from models/jukebox into models/deprecated/jukebox, the last release that shipped it was transformers v4.57.0, and v5.0.0 removed the module entirely. Take that route and you are pinning transformers to 4.x.

So who still runs it? Three groups. Researchers reproducing or extending raw-audio autoregressive modelling. MIR people using Jukebox as a feature extractor — jukemirlib pulls layer 36 activations for tagging, key and chord tasks and states a 13GB VRAM floor. And people who treat the VQ-VAE as a ready-made audio codec and train diffusion models in its latent space, which is exactly what jukebox-diffusion does. All three need to pull roughly 16GB of checkpoints, all three need a card with enough VRAM and a matching architecture, and none of them should be buying hardware for it.

01 —

Three top-level priors, one shared upsampler pair

You swap the top prior; the VQ-VAE and both upsamplers stay the same

VersionParametersVRAMContextNotes
5b_lyrics (prior_5b_lyrics)5B top prior + a 1280-wide, 18-layer lyrics encoder~11.5GB resident in fp16, plus ~1GB of KV cache per samplen_ctx 8192 ≈ 23.8 seconds of raw audioThe only 5B variant with lyrics conditioning; the checkpoint is 11.46GB. Uses the v2 label set: 4,111 artists and 120 genres. Upstream caps max_batch_size at 3 on a 16GB V100.
5b (prior_5b)5B top prior, no lyrics encoder~10.3GB resident in fp16n_ctx 8192 ≈ 23.8 seconds of raw audioTakes artist, genre and timing conditioning only — good for instrumentals. The 10.29GB checkpoint saves you 1.2GB of VRAM over 5b_lyrics.
1b_lyrics (prior_1b_lyrics)~1B, single_enc_dec architecture~3.8GB loaded in fp32, ~400MB of KV cache per samplen_ctx 6144 ≈ 17.84 seconds of raw audioGet this working before you touch the 5B models. Only a 1.96GB checkpoint, max_batch_size goes straight to 16, and it uses the v3 label set: 7,898 artists and 604 genres.
upsampler_level_1 / upsampler_level_0~1B each (width 1920, depth 72, single head)2.33GB of weights each, sampled at max_batch_size 16n_ctx 8192, covering roughly 5.9s and 1.5s of raw audioAll three models download the same pair from the 5b/ directory. The overwhelming majority of sampling time is spent here, not in the top-level prior.
VQ-VAE (5b/vqvae.pth.tar)~2M parametersA 7.73MB checkpoint; VRAM cost is negligible44.1kHz waveforms, three levels at 8x / 32x / 128x compressionThe smallest and most durable piece of the stack. jukemirlib and jukebox-diffusion both call it directly as an off-the-shelf audio codec.

02 —

Four ways to run it, four cards

Check the architecture first, then the VRAM

  • Reproduce the original stack and get demos out of 1b_lyrics

    Tesla V100 32GB$0.188/GPU-hour

    CUDA 10.0 only targets up to sm_75; the V100's sm_70 is comfortably inside that, the published hyperparameters were tuned on this exact card, and it is the cheapest GPU we rent.

  • High-throughput 5b_lyrics sampling with max_batch_size pushed above 3

    Tesla V100 32GB (scale out with mpiexec)$0.188/GPU-hour

    11.5GB of weights plus ~1GB of KV cache per sample; 32GB gives you a full 20GB more headroom than the 16GB reference card, so the top level runs a dozen-plus samples in one pass instead of five groups of three.

  • Rebuild on modern PyTorch, or run the transformers 4.57 port

    A100 PCIE 80GB$0.824/GPU-hour

    sample.py shuttles each prior on and off the GPU between levels via prior.cuda() / prior.cpu(); with 80GB all four models stay resident and that churn disappears.

  • Extract layer-36 Jukebox representations with jukemirlib for MIR tasks

    RTX 4090 24GB$0.540/GPU-hour

    The documented floor is 13GB of VRAM, so 24GB is ample; feature extraction is a short throughput-bound job, and under per-second billing the faster card ends up cheaper overall.

03 —

From bare instance to your first wav

Four steps — on the first run most of the wall clock is checkpoint download

  1. 01

    Boot the box and build a Python 3.7.5 environment

    Pick a Tesla V100 32GB in the NexGPU console with the Ubuntu CLI or PyTorch image, SSH in, install conda. PyTorch must be 1.4 with cudatoolkit 10.0 — a newer build will trip over renamed ops the moment you start sampling.

    conda create -y --name jukebox python=3.7.5 && conda activate jukebox && conda install -y mpi4py=3.0.3 && conda install -y pytorch=1.4 torchvision=0.5 cudatoolkit=10.0 -c pytorch
  2. 02

    Install jukebox itself

    requirements.txt lists only six packages, but numba is pinned to 0.48.0 and librosa to 0.7.2 — do not casually upgrade either. Confirm nvidia-smi sees the card before you start sampling.

    git clone https://github.com/openai/jukebox.git && cd jukebox && pip install -r requirements.txt && pip install -e .
  3. 03

    Prove the pipeline with 1b_lyrics first

    The first run pulls weights from openaipublic.azureedge.net into ~/.cache/jukebox/models/. For this path that is a 7.73MB VQ-VAE, two 2.33GB upsamplers and a 1.96GB top prior — about 6.6GB. Output lands in sample_1b/level_0/, and index.html animates the lyric alignment as the audio plays.

    python jukebox/sample.py --model=1b_lyrics --name=sample_1b --levels=3 --sample_length_in_seconds=20 --total_sample_length_in_seconds=180 --sr=44100 --n_samples=16 --hop_fraction=0.5,0.5,0.125
  4. 04

    Switch to 5b_lyrics and size the batch for 32GB

    The top prior becomes the 11.46GB checkpoint and total download rises to about 16.1GB. The upstream max_batch_size of 3 was chosen for a 16GB card; on 32GB, at 11.5GB plus ~1GB per sample, 12 to 15 still leaves headroom and the top level finishes in a single pass. Watch nvidia-smi once before committing to a long run. To seed from your own audio, add --mode=primed --audio_file=yours.wav --prompt_length_in_seconds=12.

    python jukebox/sample.py --model=5b_lyrics --name=sample_5b --levels=3 --sample_length_in_seconds=20 --total_sample_length_in_seconds=180 --sr=44100 --n_samples=15 --hop_fraction=0.5,0.5,0.125

What three one-minute tracks overnight actually costs

Do the arithmetic with the published speeds. Checkpoints first: a 7.73MB VQ-VAE, two 2.33GB upsamplers and the 11.46GB 5b_lyrics top prior come to about 16.1GB, and inbound transfer is not billed. Upstream states that fully sampling 20 seconds through all three levels takes roughly 3 hours on a V100, and that run produces a batch of 3: 3 x $0.188 = $0.564, or about $0.19 per 20-second demo. A full minute is roughly 9 hours: 9 x $0.188 = $1.692, still 3 samples, so about $0.56 each. Pulling down 3GB of 44.1kHz wav costs 3 x $0.0081 = $0.024. Total for the night: $1.692 + $0.024 = about $1.72, less than a coffee. Want to keep the weights for next time? 16.1GB x $0.414/GB-month = $6.66/month; destroy the volume instead and re-downloading is a fifteen-minute job — compute billing stops the second the instance stops, while storage keeps billing until it is destroyed. For comparison, the same 9 hours on an A100 SXM4 80GB at $1.088/GPU-hour is $9.79, and since Jukebox is bottlenecked on serial autoregressive decoding rather than raw FLOPs, that extra spend buys you almost nothing.

04 —

FAQ

Jukebox is archived. Is a 2020 model still worth self-hosting?

If all you want is a good-sounding song, newer systems have left it behind — sampling is slow, output is mono, and fidelity is not its strong suit. But it is still one of the very few raw-audio autoregressive models with a full 5B-parameter top prior released publicly and lyric conditioning that actually sings. For reproduction work, music representations and latent-space diffusion, the substitutes are thin. The cheapest way to decide is to spin up a Tesla V100 32GB on NexGPU at $0.188/GPU-hour, billed per second, run it overnight, and stop the instance the moment you have your answer.

How much VRAM does Jukebox need? Is a 24GB RTX 4090 enough?

Capacity is not the problem. Measured on a 16GB V100, the three top priors occupy 3.8GB, 10.3GB and 11.5GB, and KV cache adds roughly 400MB per sample for 1b_lyrics and 1GB for 5b_lyrics — 24GB swallows all of that. The obstacle is architecture: the original environment is pinned to CUDA 10.0, whose compile targets stop at sm_75, and the 4090 is sm_89. Either port the whole dependency stack to modern PyTorch, or use a card the code was built for. On NexGPU that means Tesla V100 32GB (sm_70) at $0.188/GPU-hour or Tesla T4 16GB (sm_75) at $0.298/GPU-hour.

How long does it take to generate a minute of music, and why is it so slow?

The README is blunt about it: roughly 3 hours to fully sample 20 seconds on a V100. The paper breaks it down further — about an hour to generate a minute of top-level tokens, and about 8 hours to upsample that minute back to 44.1kHz audio. The bottleneck is the two upsampling stages, not the top prior. So the right move is to raise n_samples and generate many candidates in parallel rather than queueing them one at a time; upstream itself recommends --n_samples 15 for 5b_lyrics. Slow work is exactly the case where renting beats buying: NexGPU bills per second with no minimum, no setup fee and no quota request, and compute billing stops when the instance stops.

Can I use music generated by Jukebox commercially?

No. The repository ships a Noncommercial Use License that explicitly covers both the released code and the weights, and its wording is unambiguous: no portion of the software, nor any content created with the software, may be used for commercial purposes. It also asks you to clearly indicate that your content was created using OpenAI's Jukebox. Commercial music projects need a different model; Jukebox belongs in research, evaluation, personal creation and internal technical validation. Short-lived experiments like those are precisely where NexGPU's per-second billing costs least.

My artist and genre conditioning seems to be ignored — the output has nothing to do with what I asked for.

Almost always a label miss. Jukebox silently falls back to (id, name) = (0, unknown) and prints a single warning line that is easy to lose in the log spam. The two label sets differ, too: 5b and 5b_lyrics use v2 with 4,111 artists and 120 genres, lowercasing the string and replacing non-alphanumerics with underscores, and splitting the genre into a bag of words; 1b_lyrics uses v3 with 7,898 artists and 604 genres, matching lowercased strings verbatim with spaces intact (for example, pop rock). Grep jukebox/data/ids/ for your names before you commit to a run. If you want to sweep dozens of artist-genre combinations, running several V100s in parallel on NexGPU beats waiting out a serial night.

Is the HuggingFace transformers port easier to work with?

Partly. Both openai/jukebox-1b-lyrics and openai/jukebox-5b-lyrics are on the Hub, ported by Arthur Zucker, and accelerate handles device placement for you. Three caveats: the port is inference-only with no training support; the documentation itself notes that the 5B top prior takes 8 hours to generate a minute of audio on a V100; and the module was demoted into models/deprecated/jukebox, last shipped in transformers v4.57.0, and removed outright in v5.0.0 — so transformers has to be pinned to 4.x. If you want to test the original repo and the HF path side by side, just run two NexGPU instances with separate environments; an A100 PCIE 80GB is $0.824/GPU-hour and the two never collide.

More in Music generation

Every model guide

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.

Sign up freeTalk to sales

Sign up to browse live network pricing. No payment method required.