How does AI separate stems from a finished song?

Modern models like Demucs are neural networks trained on thousands of songs where the isolated stems were known. They learn what vocals, drums and bass sound like — their timbre, rhythm and place in the spectrum — and estimate each source from the mixed waveform. It is a learned reconstruction, not a reversal of the mixing process.

Why can most tools only separate 4 to 6 stems?

Training data is the bottleneck. Public multitrack datasets label vocals, drums, bass and 'other' — so that's what models learn best. Six-stem models add guitar and piano, but with far less training material those stems are noticeably rougher. Beyond six, there simply isn't enough labelled data yet.

Why are guitar and piano stems worse than vocals?

Vocals, drums and bass each occupy a distinctive sonic niche. Guitars and pianos are chameleons: both are broadband, polyphonic and overlap heavily with each other, with synths and with strings — and the models have seen much less labelled guitar/piano data during training.

Is stem separation lossless — can I get the original studio multitracks?

No. The original multitracks are gone once a song is mixed to stereo; separation is an estimate. On modern, well-produced tracks the estimate is remarkably close, but reverb tails, bleed and dense arrangements always leave small artifacts.

STEMGRAB Open the tool →

Explainer · honest edition

How AI stem separation actually works

A mixed song is like a baked cake: the eggs and flour are in there, but you can't pull them back out. So how does a neural network hand you the isolated vocal from a 40-year-old record? Here's the real story — including what still doesn't work.

Before AI: the phase-cancel trick

For decades, "vocal remover" meant one thing: phase inversion. In many mixes the lead vocal is panned dead centre — identical in the left and right channel. Flip the polarity of one channel and sum the two, and everything identical in both channels cancels out. The vocal disappears.

So does everything else in the centre: the kick drum, the snare, the bass. What's left sounds hollow and phasey, like the band playing at the end of a hallway. And the trick only ran one way — it could (sort of) remove the centre, but it could never isolate the vocal by itself. It also failed completely on mono recordings, on vocals with stereo reverb, and on anything not mixed exactly down the middle. Phase-cancelling was a party trick, not a tool.

The spectrogram era: masking

The first real machine-learning separators (around 2015–2019, e.g. Open-Unmix and the original Spleeter) worked on spectrograms — an image of the song showing which frequencies are loud at each moment. A neural network was trained to look at that image and paint a mask over it: "these pixels belong to the voice, those belong to the drums." Multiply the mix by the mask, convert back to audio, done.

This was a leap — suddenly you could isolate a vocal, not just cancel it. But masking has a built-in ceiling: when a vocal and a guitar occupy the same frequency at the same moment (which happens constantly in real music), a mask has to split that energy somehow, and the result is the watery, underwater warble every early Spleeter user remembers. Spectrograms also discard phase information, which the model then has to fake on the way back to audio.

Today: hybrid transformer models

Modern separators like Demucs (the model behind StemGrab) attack the problem from two directions at once — hence "hybrid":

A waveform branch works directly on the raw audio samples, so nothing is lost to a spectrogram conversion — phase included. This is what makes modern bass and drum stems punch instead of smear.
A spectral branch still works on the frequency picture, where harmonic patterns (a voice's overtone stack, a piano chord) are easiest to spot.
A transformer core — the same architectural family as modern language models — sits across both branches and looks at long stretches of the song at once. That context matters: if the model heard an exposed vocal in the intro, it uses that knowledge to dig the same voice out of the dense final chorus.

How does it know what a "vocal" is at all? Training. The model learned from thousands of songs where the real isolated stems were available, mixing them together and grading itself on how well it recovered each part. After enough of this, it has internalised what voices, kicks, snares and bass lines sound like — well enough to estimate them from mixes it has never heard.

Key insight: separation is not "un-mixing". The original multitracks are mathematically gone. The model reconstructs what each stem most plausibly was — a very educated guess that happens to be shockingly good.

Why six stems is the current ceiling

The classic split is four stems: vocals, drums, bass, other — because that's how the standard research dataset (MUSDB18) is labelled, and models can only learn categories their training data names. Six-stem models like the one StemGrab runs add guitar and piano, carved out of "other".

Going further — separate lead from backing vocals, split the guitars, isolate the synth — mostly isn't a modelling problem, it's a data problem. There is no large public library of songs with, say, a labelled tambourine stem. Until that data exists, "everything else" stays glued together in the other track.

Why guitar and piano lag behind vocals, drums and bass

The easy stems are easy because each occupies a distinctive niche:

Vocals have a fingerprint nothing else shares — formants, vibrato, breath, consonants. Human speech is also the single best-studied sound in all of audio research.
Drums are transients: short, broadband hits with sharp attacks, rhythmically predictable. Nothing else in a mix looks like a snare hit.
Bass owns the bottom octaves nearly alone below ~120 Hz.

Guitar and piano are chameleons. Both span almost the full frequency range, both play chords and melodies, and they overlap heavily — with each other, with synth pads, with strings, with organ. A clean piano and a warm pad playing the same chord are genuinely ambiguous even to human ears. Add distortion to a guitar and it becomes broadband noise-with-pitch, easily confused with synths. And the training data for these two categories is a fraction of what exists for vocals. The result: on a sparse singer-songwriter track, the guitar stem can come out beautifully; in a dense wall-of-sound mix, expect bleed and smearing.

What still goes wrong — honestly

Reverb and echo stay ambiguous. A vocal's reverb tail is "vocal-coloured room sound" — some lands in the vocal stem, some in the instrumental. No current model resolves this cleanly.
Dense mixes cost quality. Metal, orchestral pop and heavily layered EDM force the model to make more hard calls per second — artifacts rise accordingly.
Lo-fi sources are hard. Tape hiss, vinyl crackle, mono recordings and low-bitrate rips give the model less to work with. A 128 kbps MP3 has already thrown away detail the separator would have loved to use.
Sounds between categories confuse it. Heavily vocoded or autotuned vocals may partially land in "other"; an 808 that's both a kick and a bassline gets split across drums and bass.

None of this means the tech is bad — vocals, drums and bass on a modern mix come out cleaner than anyone would have believed possible in 2018. It means the honest pitch is "remarkably good estimate", not "magic".

Hear it on your own music

The fastest way to calibrate your expectations is to run a track you know inside out. Drop a song into StemGrab — you'll get vocals, drums, bass, guitar, piano, other and a clean instrumental back in about a minute, free, no account. Then put on headphones and listen to what the model got right — and where it guessed.

Split a track free →

← StemGrab home Vocal remover Instrumental maker About Privacy