Health Score Backtest Playbook 2026

One row per account whose renewal came up in the last 12 months:

• Score 90 days out: the score as it stood 90 days before the renewal date (default), not today's. Today's has already absorbed the result.
• Customer outcome: Grew (ARR up 10% or more, or upgraded plans with ARR at least unchanged), Left (canceled, or ARR down 25% or more), Stayed (everyone else who renewed). These are backtest buckets; the benchmark cohort includes only completed renewals at the same or higher revenue.
• ARR before and after from billing or the CRM.
• Usage before renewal: weekly active users and key actions over the six months ending 90 days before renewal, from PostHog, Amplitude or Mixpanel. Nothing after that date goes into the score or the benchmark.

If your score is a number, use the green, amber and red cut-offs your team already works to. If you never kept score history, there's nothing to test yet: start a weekly snapshot now and go straight to the benchmark. Under about a hundred accounts, do it all by hand in a spreadsheet.

Call an account flagged if it was amber or red 90 days out. Exclude accounts that had already given notice 90 days out; the score gets no credit for catching those. Mark each with one Backtest result:

• Caught: flagged, then left.
• Missed while green: green, then left.
• Grew while flagged: flagged, then grew – the score read growth as risk.
• Flagged, then stayed: record separately whether an intervention (a call, a save plan, an escalation) is documented before the renewal. Report those two groups apart; neither tells you whether the flag was right or the move saved the account.
• Right, no flag: green, then stayed or grew.

Then four numbers: the share of lost ARR that was green, the share of flagged accounts that left, the median days of warning, and the share of growth accounts flagged. For days of warning, take every lost account and count the days from the start of its last unbroken run of amber or red to its renewal, from the weekly score history. A lost account still green at renewal counts as zero; report accounts with missing history apart. Say 140 renewals: 46% of lost ARR was green, one flagged account in three left, the median warning was 34 days, and nine of 22 growth accounts were amber. (The numbers are illustrative.)

Write every Missed while green, Flagged, then stayed and Grew while flagged onto the company in HubSpot, with the owner's task below.

Two lines in What the score missed: what they saw at the time, and what in the product or the contract would have shown it. Before counting a miss against the score, check the records:

• Leaves usage can't show – acquired, closed, lost the budget. Exclude them, and count them apart.
• Gaps in score history – if more than one account in five has no score 90 days out (default), write "low confidence" beside the verdict.

Rule, defaults to tune, set before you look: the score fell short if more than 20% of lost ARR was green 90 days out, or fewer than one flagged account in three left, or the median warning is under 60 days. Otherwise it held up.

Record No move in the backtest review with the four numbers and a retest date a year out. A written pass gives the team a reason to trust red.

Don't reweight the inputs – replace the question. A score asks "does this account look healthy?" A benchmark asks "is this account doing what our customers who stayed and grew did, at this stage, at this price?"

Take the customers who renewed at the same or higher revenue, compare what they did in the product and what they pay with those who left, and keep only the measures that separate them. Pick the measures on earlier renewals. Test them on a later period only if it holds enough renewals and losses to judge both rules. Otherwise build on the history you have, freeze the rule, and record both scores at 90 days out from now on; review quarterly, and keep collecting outcomes while the result is too close to call. Add the owners' notes on the misses as candidate measures. The benchmark play has the steps.

Run the test on the later renewals, and score the old health score on the same renewals: 90 days before each renewal, was the account short of the benchmark? Replay both rules week by week, using only what was known that week. Short counts as flagged; on or ahead counts as green. Count the same four numbers. Recommend switching if the benchmark leaves less lost ARR unflagged and gives more warning, without doubling the flagged accounts that stayed (default). It also shows what a score can't: accounts that stayed but sat well short of their peers – the growth a risk flag never looks for.

With the two results side by side, they switch, keep the score, or run both for a quarter – recorded in the backtest review.

The weekly check compares every account in the segment with the benchmark and puts the recommended move on the HubSpot record. Retire the score from views and tasks so the team works from one list.

On the review date, answer three questions: are owners acting on the moves (activity), are gaps closing (progress), and did any account leave without being flagged first (the result)? Note what else played a part, then change it, stop it, or give it time – the eight-week review has the method. Run this backtest again next year, with the benchmark in the score's seat.