Music generation

Self-hosting MusicGen: 300M to 3.3B, one 24GB card covers it

The audiocraft docs state that inference with the medium (~1.5B) model needs a GPU with at least 16GB. In practice musicgen-large's native audiocraft checkpoint is only 6.51GB — load it on an RTX 4090 24GB ($0.540/GPU-hour) and you still have seventeen gigabytes left to spend on batch size.

MusicGen · self-hosted

MusicGen is the single-stage autoregressive Transformer from Meta FAIR's "Simple and Controllable Music Generation" (arXiv:2306.05284). There is no cascade and no upsampling stage: it models the discrete audio tokens produced by EnCodec directly — 32kHz output, 4 codebooks at 50Hz, which works out to 50 autoregressive steps per second of audio. Text goes through a frozen T5 encoder, and a codebook interleaving (delay) pattern lets all codebooks be predicted in one forward pass. That architecture is exactly why it is still pleasant to deploy: no diffusion sampler to tune, no second-stage vocoder to load separately, `generate()` hands you a waveform.

The family on Hugging Face is facebook/musicgen-{small,medium,large} (300M / 1.5B / 3.3B), musicgen-melody and musicgen-melody-large for chroma conditioning, five musicgen-stereo-* variants, plus the later additions musicgen-style (1.5B, conditioned on a 1.5–4.5 second audio excerpt) and musicgen-stem-6cb / -7cb for bass/drums/other stem generation and editing (arXiv:2501.01757). Be clear about what it will not do: vocals were deliberately stripped from the training data using tags plus source separation, so it does not sing; prompts are English-only; positional embeddings cap a single generation at 30 seconds; and the weights are CC-BY-NC 4.0, so no commercial use.

One more thing you need to know before you `pip install`: audiocraft on PyPI is frozen at 1.3.0 (2024-06-03), while the GitHub main branch changelog has moved to 1.4.0a2 and the last commit landed 2025-03-13. So `pip install audiocraft` gets you MusicGen, MAGNeT and AudioSeal — but not MusicGen-Style or JASCO, which only exist on main. The repo is quiet, but MusicGen remains the most dependable open baseline for instrumental scoring, melody-conditioned generation and stem editing: ACE-Step 3.5B (Apache-2.0) is far faster and YuE 7B actually sings, yet nothing has replaced chroma conditioning or per-stem rewriting. On NexGPU it starts at $0.188/GPU-hour, billed by the second, and stops when you stop it.

01 —

Every MusicGen variant: parameters, checkpoint size, what it's for

Sizes are the actual file sizes in the Hugging Face repos — the audiocraft native checkpoints and the transformers conversions are stored at different precisions, so don't mix the two when you size a GPU

Version	Parameters	VRAM	Context	Notes
facebook/musicgen-small	300M (decoder LM)	audiocraft state_dict 0.84GB / transformers fp32 2.36GB	30s / 1,500 tokens at 50Hz	Use it to wire up the pipeline, do bulk first-pass screening and iterate on prompt templates. Output quality is clearly below medium.
facebook/musicgen-medium	1.5B	audiocraft 3.68GB / transformers fp32 8.04GB; official docs ask for 16GB+	30s / 1,500 tokens	The audiocraft docs call medium the best quality-per-compute trade-off. Most production work should start here, not at large.
facebook/musicgen-large	3.3B	audiocraft fp16 6.51GB / transformers fp32 13.72GB (two shards)	30s / 1,500 tokens	Top of the quality range. The transformers conversion is stored in fp32 — dropping 13.7GB of weights onto a 16GB card will OOM, so pass torch_dtype=float16.
facebook/musicgen-melody / -melody-large	1.5B / 3.3B	Same class as medium / large	30s plus a melody chromagram	Condition on a hummed or reference melody. In audiocraft that's generate_with_chroma(); in transformers it is a separate class, MusicgenMelodyForConditionalGeneration — the plain Musicgen class will not load it.
facebook/musicgen-stereo-{small,medium,large,melody,melody-large}	300M – 3.3B	stereo-large fp16 safetensors 6.93GB	30s across 2 channels	Generates two sets of codebooks, one per channel, each decoded independently through EnCodec and then combined. Budget memory and decode steps as double.
facebook/musicgen-style / musicgen-stem-6cb·7cb	1.5B	stem state_dict 3.72GB plus ~0.70GB of per-stem codecs	30s; style excerpt 1.5–4.5s	Style conditions on an audio excerpt with double classifier-free guidance (cfg_coef=3 alongside cfg_coef_2=5). Stem splits bass, drums and other into independent tracks so you can regenerate just one. Both live on git main only — they are not in PyPI's 1.3.0.

02 —

Picking a card, from first run to full fine-tune

A single MusicGen clip is a batch-of-one, 1,500-step autoregressive decode — memory latency is the bottleneck, not FLOPs. Fill the batch before you reach for a more expensive GPU

Get small / medium running, stand up a Gradio demo
Tesla V100 32GB$0.188/GPU-hour
Cheapest card in the fleet, and 32GB swallows medium's 8.04GB fp32 weights or large in fp16 without thinking. Volta has fp16 tensor cores, and audiocraft's pinned torch 2.1.0 fully supports sm_70.
Batch production with musicgen-large / stereo-large in fp16
RTX 4090 24GB$0.540/GPU-hour
6.9GB of fp16 weights leaves 17GB for batch, and Ada's fp16/bf16 decode throughput is exactly what a 1,500-step autoregressive loop consumes. If you want the same 24GB cheaper, RTX 3090 24GB is $0.193/GPU-hour.
Full fine-tune of musicgen-medium 1.5B for a house style
RTX A6000 48GB$0.817/GPU-hour
At roughly 16 bytes per parameter for AdamW (weights, grads, first and second moments), 1.5B lands near 24GB. 48GB leaves room for on-the-fly EnCodec encoding and a real batch, on one card, without FSDP.
Full fine-tune of musicgen-large 3.3B, or many concurrent streams
A100 SXM4 80GB$1.088/GPU-hour
3.3B works out to about 53GB on the same accounting, which one 80GB card holds. If you do want FSDP, nodes go up to 14 GPUs and 2,152GB of node VRAM — no platform change needed.

03 —

Four steps to a running MusicGen

audiocraft is fussy about Python and PyTorch versions — nearly every failure people hit is in step two

01
Launch an instance on a PyTorch image
Pick a card in the console (Tesla V100 32GB to get going, RTX 4090 24GB to actually produce), choose one of the 2,000+ prebuilt images from the PyTorch family, and take your pick of SSH, Jupyter or the web terminal. First thing after boot: confirm the driver and the card you were given.
```
nvidia-smi && python -c "import torch; print(torch.__version__, torch.cuda.get_device_name(0))"
```
02
Install audiocraft — the torch version is pinned
The project requires Python 3.9+ and PyTorch 2.1.0. Install torch on its own first, or dependency resolution will pull a newer torch and then blow up building extensions. ffmpeg is a hard dependency for audio I/O. If you need MusicGen-Style or JASCO, swap the last install for git main — PyPI's 1.3.0 (2024-06-03) does not contain them.
```
apt-get update && apt-get install -y ffmpeg && python -m pip install 'torch==2.1.0' setuptools wheel && python -m pip install -U audiocraft   # for style/JASCO: pip install -U git+https://github.com/facebookresearch/audiocraft
```
03
Generate your first 30 seconds
set_generation_params defaults to duration=30.0, cfg_coef=3.0, top_k=250, temperature=1.0. Any CFG above 1 means each step runs both a conditional and an unconditional forward, so a 30-second clip is 1,500 steps at two forwards each — don't forget that doubling when you estimate memory or wall time. audio_write drops a 32kHz WAV straight to disk.
```
python -c 'from audiocraft.models import MusicGen; from audiocraft.data.audio import audio_write; m = MusicGen.get_pretrained("facebook/musicgen-large"); m.set_generation_params(duration=30, cfg_coef=3.0, top_k=250); w = m.generate(["warm lo-fi hip hop, dusty vinyl crackle, mellow rhodes chords"]); audio_write("out", w[0].cpu(), m.sample_rate, strategy="loudness")'
```
04
Bring up the Gradio app, or point a batch script at it
The repo ships demos/musicgen_app.py with all ten checkpoints already in the dropdown, stereo variants included. Point AUDIOCRAFT_CACHE_DIR at a data volume so a rebuilt instance doesn't re-download 6.5GB of weights. For long batch runs just write the loop — compute billing stops the moment the instance stops.
```
AUDIOCRAFT_CACHE_DIR=/workspace/ac_cache python -m demos.musicgen_app --listen 0.0.0.0 --server_port 7860
```

What a batch of 30-second cues actually costs

Say you're producing a batch of 30-second mono demos with musicgen-large. Take an RTX 4090 24GB at $0.540/GPU-hour: about 6.9GB of fp16 weights, 17GB left over for batch. Three hours = 3 × $0.540 = $1.62. The output side is exactly computable, because MusicGen always writes 32kHz — one 30-second 16-bit WAV is 32000 × 2 × 30 = 1,920,000 bytes ≈ 1.92MB. Assume those three hours produce 1,000 clips: 1.92GB total, and pulling it down costs 1.92 × $0.0081 ≈ $0.016 in egress. Weights plus output occupy 10GB; if you stop the instance but keep the volume that's 10 × $0.414 = $4.14/month, or $0 if you destroy it. Round trip: about $1.64. Spending more doesn't buy proportionally more — the same three hours on an A100 SXM4 80GB is 3 × $1.088 = $3.26, and on an H100 SXM 80GB it's 3 × $3.582 = $10.75. But a single clip is a batch-of-one 1,500-step decode bound by memory latency, so an H100 will not hand you 6.6× the throughput; filling the batch on a 24GB card is what actually saves money. Want it cheaper still? Tesla V100 32GB at $0.188/GPU-hour makes three hours $0.564, and 32GB holds large's fp16 weights with room to spare. Everything is metered per second — no minimum, no setup fee, no quota request.

04 —

FAQ

How much VRAM do I actually need to run MusicGen locally?

Depends which path you take. Native audiocraft checkpoints: small 0.84GB, medium 3.68GB, large 6.51GB (stored fp16). The transformers conversions are fp32: small 2.36GB, medium 8.04GB, large 13.72GB. On top of that comes the 236MB EnCodec 32kHz decoder, the T5 text encoder, and the doubled activations that classifier-free guidance brings. The official line in the audiocraft docs is 16GB minimum for medium inference. Translated into cards: small and medium are comfortable on a Tesla V100 32GB ($0.188/GPU-hour), and large in fp16 is happiest on an RTX 4090 24GB ($0.540/GPU-hour). Both are billed per second on NexGPU and ready the moment you launch.

Why does MusicGen stop at 30 seconds, and how do I get longer tracks?

Sinusoidal positional embeddings cap a single generation at 30 seconds — 1,503 tokens. If you use audio-prompted continuation, the input audio eats into that same budget: feed in 20 seconds and only 10 seconds of new audio remain. audiocraft's answer is a sliding window: set duration above 30 and it extends the piece in chunks using extend_stride (18 seconds by default). The cost is audible seams at the joins, so long pieces are usually better generated in sections and assembled in a DAW. That kind of work means a lot of parameter sweeps, which is exactly where per-second billing beats a monthly plan.

Can MusicGen generate vocals or singing?

No, and that's a design decision rather than a bug. The model card is explicit: across the 20K hours of training data (Meta Music Initiative Sound Collection, Shutterstock, Pond5), vocals were filtered by tag and then removed with source separation. It is an instrumental model. For songs with singing, look at YuE 7B (Apache-2.0, lyrics-to-full-song, from HKUST and M-A-P) or ACE-Step 3.5B (Apache-2.0). Both are heavier than MusicGen, and NexGPU runs everything from RTX A6000 48GB ($0.817/GPU-hour) up to H200 141GB ($6.660/GPU-hour).

Can I use MusicGen output commercially?

Not with these weights. The audiocraft code is MIT, but every MusicGen checkpoint is CC-BY-NC 4.0 — non-commercial. For production teams that's a far harder wall than any VRAM number. If commercial use is non-negotiable, either move to Apache-2.0 ACE-Step, or retrain MusicGen on a catalogue you own — the audiocraft repo does ship full MusicGen training code. Retraining is a genuine multi-GPU job: NexGPU nodes go to 14 GPUs and 2,152GB of node VRAM, with A100 SXM4 80GB from $1.088/GPU-hour and no quota request to file.

audiocraft won't install, or won't run on my RTX 5090 — what now?

Almost every install failure traces back to the pinned torch==2.1.0: install torch by itself first, keep Python in the 3.9–3.11 range, and install ffmpeg at the system level. The RTX 5090 is Blackwell (sm_120), which simply isn't in torch 2.1.0's compiled target list, so the official audiocraft path will not run there — either switch to the transformers MusicgenForConditionalGeneration path on a modern torch, or rent a card the pinned stack actually supports. On NexGPU that means RTX 4090 24GB at $0.540, RTX 3090 24GB at $0.193, A10 24GB at $0.414 or Tesla V100 32GB at $0.188, and starting from a prebuilt PyTorch image removes half the remaining friction.

Is MusicGen still worth using, or has it been superseded?

The repo has gone quiet: PyPI sits at 1.3.0 from 2024-06-03, the GitHub main changelog reads 1.4.0a2, and the last commit was 2025-03-13. On raw text-to-music speed and licensing it has been passed — ACE-Step 3.5B is Apache-2.0 and its authors report 4 minutes of music in 20 seconds on an A100. But two MusicGen capabilities still have no clean equivalent: chroma melody conditioning in the melody variants, where a hummed line constrains the generation, and musicgen-stem's per-track bass/drums/other generation and rewriting. Add that it's purely autoregressive with no sampler to tune, and it stays an easy baseline and an easy fine-tuning starting point. The honest way to decide is to run your own prompts across the candidates on identical hardware — NexGPU spans 51 countries and regions, 1,175 verified rentable nodes, 2,498 GPUs and 75 GPU models, billed by the second, so an afternoon of bake-off costs less than a coffee.

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.