Fixing a Recommender System's Cold-Start Problem With a Boring Heuristic

A client's recommender system worked well for users with any meaningful interaction history and was close to useless for brand-new users — the classic cold-start problem. We tried several reasonably sophisticated fixes before landing on the one that actually worked, and the one that worked was embarrassingly simple compared to everything we tried first.

What we tried and why it underperformed

Content-based filtering using onboarding survey responses helped a little but the survey had low completion rates, so it only covered a fraction of new users. A matrix factorization approach seeded with demographic similarity to existing users produced recommendations that were technically personalized but often felt generically off, since demographic similarity is a weak proxy for actual taste. Both approaches added real engineering complexity for a modest improvement over no personalization at all.

Sophisticated cold-start solutions compete with each other on marginal personalization gains. The real gap for a brand-new user is that they have zero signal at all, and no amount of sophistication conjures signal that doesn't exist.

The fix that actually worked

We fell back to a simple, boring heuristic: show new users the most broadly popular items in their general category, weighted slightly by whatever thin signal exists (device type, referral source, time of day), and switch them over to full personalized recommendations as soon as they accumulate even a handful of real interactions — usually within the first session. Popularity-based recommendations for brand-new users outperformed both of our more sophisticated attempts on every engagement metric we tracked.

New user, 0 interactions   → popularity-weighted fallback
User has 3-5 interactions  → blend of popularity + early personalization
User has 10+ interactions  → full collaborative filtering

• Popularity-based fallback for zero-signal users outperformed demographic-proxy personalization in every metric we tracked.
• The transition to full personalization should trigger on actual interaction count, not a fixed time window.
• Simple heuristics are easier to explain, easier to debug, and easier for a non-technical stakeholder to trust than a sophisticated proxy model.
• Cold start isn't a modeling problem to be solved cleverly — it's a signal-absence problem, and the honest fix is bridging the gap cheaply until real signal arrives.

The sophisticated approaches weren't wasted effort exactly, but they were solving a problem that didn't actually need sophistication — it needed an honest, well-executed fallback and a fast handoff to the real personalization system once any real signal showed up. Sometimes the boring answer is the correct one, and it's worth trying before reaching for anything clever.