Redesigning a Customer Health Score That Was Lying to the CS Team

A SaaS client's customer health score was a weighted blend of login frequency, feature adoption breadth, and support ticket volume. On paper it looked thorough. In practice, three of their last five churned accounts had shown a 'healthy' or 'green' score in the thirty days before cancellation.

Why login frequency was the wrong anchor

Login frequency measures activity, not value. Several of the churned accounts had people logging in regularly right up to cancellation — checking a dashboard, running a report — while the underlying business reason they'd bought the product in the first place had already stopped being true. One account kept a daily login habit going for two months after the specific use case they'd purchased for had been solved by an internal tool, making the subscription redundant well before anyone noticed.

A health score built on activity measures whether someone is still opening the app. It doesn't measure whether the app is still solving their problem.

The redesign

• Replaced raw login frequency with a weighted measure of core-workflow completion — did they actually finish the task the product is for, not just open it.
• Added a decline-rate signal, since an account trending down from its own baseline matters more than an absolute activity level.
• Incorporated a renewal-conversation sentiment flag from CS notes, since qualitative signal caught two at-risk accounts the quantitative score missed entirely.
• Backtested the new score against the last eight quarters of actual churn to validate it before rolling it out.

The redesigned score correctly flagged four of the next five churned accounts at least 45 days before cancellation, enough lead time for CS to actually intervene. The original score wasn't a bad idea poorly executed — it was measuring the wrong underlying behavior entirely, and no amount of weighting tweaks would have fixed that.