The one audience you actually shouldn't re-engage

Recently, I was talking with Jeff Lee, the Lifecycle Marketing Tech Lead at Calm, about lifecycle experiments that look smart on paper but feel very different once you see the results. And he shared a really interesting learning with me from what might just be the single most important experiment he has ever run.

The upside of re-engagement is incremental. Moving a user from one active week to two might slightly improve renewal odds — maybe worth a dollar in expected value. But the downside of triggering a cancellation is absolute. You don’t lose a dollar. You lose 100% of that user’s future value, which means you’re risking everything to gain almost nothing. The math just doesn’t add up.

But some quick context first.

For those not familiar, Calm is one of the largest mental health apps (roughly 180M downloads). Jeff has helped scale the company from ~20k notifications to over 300 million. He leads the tech and engineering foundation of the company’s entire CRM program, which accounts for another ~2 billion emails every year.

In other words, when something works or breaks in lifecycle at Calm, it shows up clearly in the data.

Like any product-first company, paid users are incredibly important to Calm. The question they set out to answer was: What do you do when your paid users start to go dormant?

The obvious answer is to try and re-engage them — or at least that’s what Jeff thought.

So the team did what most lifecycle teams would do. They sent a re-engagement message designed to bring users back into the product. And initially, the results looked promising because a small subset of users became active again.

Per standard best practice, they created a holdout group to measure the test results. And soon, something uncomfortable became very clear: users who received the re-engagement message were canceling at a substantially higher rate. They’d woken up a subset of “zombie users” who were subscribed but not actively using the product, and the message had the opposite effect they intended.

Sometimes, doing nothing is really the most effective lifecycle strategy. The point isn’t to “gotcha” users who aren’t getting value from your product, of course, but also not to nudge people in the wrong direction who were getting value in less predictable ways.

For Calm, there are all sorts of reasons people might go dormant for a bit but stay subscribed: they could spike in usage at different seasons, when they’ve dropped a meditation habit or are just starting to pick it back up, if they need it at a point of transition in their lives, etc. A misplaced re-engagement blast might just tip the scales for folks who would have returned, but instead, just didn’t want to deal with one more message at that moment.