01 The data
Tracks were sampled randomly per year from the Spotify Web API and deduplicated by track ID. Spotify's popularity score (0–100) is the target, binned into deciles. Two things stand out: nothing in a random sample scores below ~25 — obscure music never surfaces — and the score is strongly recency-biased, so newer tracks rate higher by construction.
The 14 audio features, in plain words as defined by the Spotify Web API
The target, popularity, is Spotify's 0–100 score — driven mostly by how much a track has been played recently, which is why newer songs score higher.
Popularity distribution 56,652 tracks, bin width 5
Only 8 of 10 deciles occur. Chance accuracy: ~12.5% uniform, ~33% always-guess-the-majority.
Mean popularity by release year recency bias, visualised
From 43 (2008) to 66 (2019) — the popularity metric itself rewards recent releases.
02 Model leaderboard — 2026 benchmark
Honest protocol: rebuilt from the raw CSVs, stratified 80/20 split, train-fitted scaler only, balanced sample weights. Popularity deciles are ordered, so alongside accuracy we track ordinal MAE (how many deciles off, on average) and within-1-bin rate.
Accuracy vs. within-1-decile rate
The winner isn't a fancier classifier — it's a better problem formulation. Regressing the raw 0–100 score with LightGBM and binning the prediction beats every multiclass model on accuracy and lands within one decile 85.6% of the time.
Full results
| Model | Accuracy | Balanced acc | Macro F1 | Ordinal MAE ↓ | Within-1 bin |
|---|
- Accuracy
- share of test tracks whose popularity decile was predicted exactly right
- Balanced acc
- accuracy averaged per decile, so rare deciles count as much as common ones
- Macro F1
- precision–recall balance, averaged per decile (1 = perfect)
- Ordinal MAE
- how many deciles off the prediction is, on average — lower is better
- Within-1 bin
- share of predictions that land in the true decile or an adjacent one
03 2020 vs 2026 — why the numbers changed
The original coursework reported ~0.38 accuracy with CatBoost on top. The revisit reports lower headline accuracy — and that's the point: the numbers aren't comparable, because the original protocol had three rough edges the revisit fixes.
Test-set leakage
The 2020 test set was scaled with its own MinMaxScaler. The revisit fits the scaler on train only.
Distribution shift
The 2020 test set was 2019 chart-toppers only — a different population than the training sample. The revisit uses a stratified split of the same population.
Ordinal structure ignored
Deciles are ordered; plain multiclass objectives throw that away. The revisit adds regression-then-bin and ordinal metrics.
Original 2020 leaderboard balanced subset, 2019-only test set — kept for the record
CatBoost edged out XGBoost and NGBoost in 2020. Under the honest 2026 protocol the ranking reshuffles — and the formulation matters more than the model.
04 What in the sound drives popularity?
SHAP attributions from the best model (the LightGBM regressor), cross-checked with permutation importance. A feature's SHAP value is how many popularity points (on the 0–100 scale) it moved the model's prediction for a given track; the bars average its absolute size over the test set, signed by direction: green pushes popularity up, red pushes it down. Hover a bar for the feature's meaning.
Feature attribution mean |SHAP|, signed by direction of effect
Each feature moves the prediction by only ~±1 point on average — audio shapes popularity at the margins; it doesn't decide it.
The audio recipe for a popular track (2008–2019)
the single strongest driver
but not high-energy once loudness is controlled for
longer songs lose popularity
low valence beats happy-sounding
instrumental is a clear penalty; spoken/rap content helps
the explicit flag is a positive
A coherent description of streaming-era hip-hop/pop — and consistent with what the 2020 feature-importance pass hinted at with blunter tools.
05 The recipe as a decision tree
SHAP tells you which features matter; a decision tree turns that into instructions. This is a real tree — depth 3, no leaf smaller than 2% of the data — fitted on the training split under the same protocol as the benchmark, regressing the raw 0–100 popularity score. Every number below is re-scored on the held-out test set, so the leaves are out-of-sample averages, not training artifacts.
How to make a popular song leaf values = mean popularity of test tracks landing there · green path = best recipe
The tree, in plain words how to read it, no jargon
The tree is a flowchart of yes/no questions. Drop any song in at the top, answer each question, follow the branch. The number you land on is the average popularity (0–100) of every song that answered the same way.
- Is it flagged explicit? That's the first and biggest fork. Explicit songs average ~55; clean songs ~51. (It's not the swearing itself — the tag is a marker for the rap/hip-hop sound that dominated streaming.)
- If explicit: don't max the intensity. Songs that stay below the very top of the energy scale do better — loud is fine, exhausting is not.
- Keep it short. Under 3:45 is the sweet spot for explicit tracks (the best group: 57/100). Clean songs over 5:10 are the worst group (48/100).
- If clean: leave some acoustic texture in. Tracks that sound 100% computer-generated lose a few points.
The catch: a typical song scores ~52, so following the perfect recipe buys you about 5 extra points out of 100. The other 95 are decided by things the audio can't see — who the artist is, playlists, marketing, timing. The sound sets you up; it doesn't make you famous.
06 Raw trends, no model
Mean popularity across 12 quantile bins of each feature, straight from the data. Marginal trends are flatter than the SHAP effects above — SHAP controls for the other features, these don't — but the directions agree.
Explicit flag
Vocals vs instrumental
07 Conclusions
1 · Respect the ordinal structure
Popularity deciles are ordered, and plain multiclass objectives throw that away. Regressing the raw 0–100 score and binning the prediction was the single biggest improvement — best accuracy (0.370), best ordinal MAE (0.79 deciles), within one decile 86% of the time — and it needed no new data or model, just a better problem formulation.
2 · Audio features alone carry limited signal
Under an honest protocol no classifier separates adjacent deciles well (macro-F1 ≤ 0.18 with balanced weighting). Popularity is largely driven by things the audio can't see: artist fame, playlisting, marketing, release timing. The audio's contribution is real but marginal — ~±1 popularity point per feature.
3 · Evaluation honesty changes the story
The 2020 setup (own-scaler test set, 2019-only chart-toppers) made models look better and ranked them differently. Fixing the protocol didn't just lower the numbers — it changed the conclusion from "CatBoost wins" to "the formulation wins."