How to Track Email Replies in Kit (So You Don't Delete Real Subscribers)

In this video I walk through the setup we built to fix that: a Zapier BCC address that tags a subscriber in Kit when you reply to them, a custom field that counts how many times they’ve replied plus a timestamp of their last reply, a rolling 30-day “replied recently” tag that resets every time they reply again, and how all of it plugs into a list cleaning sequence so people who reply stay on the list. And why we always delete cold subscribers manually, never automatically.

If you’re removing cold subscribers from your email newsletter platform based on whether they are opening your emails, you might be removing subscribers who actually want to be on your list. That almost happened today with a client of mine. Some of the subscribers replied to our “Hey, we’re about to remove you” email and asked to stay on the list. But Kit, the tool that we’re using for this client, had no record of them opening any emails.

So in this video, I’m going to show you how we set up a Kit visual automation to track replies and how we use that to decide who stays on the list.

Setting up reply tracking with a Zapier BCC address

A client of mine wanted an automation for when somebody replies to one of the emails in his newsletter. We wanted to be able to track how many replies there were. So we set up a Zap so that when my client replies to an email, he BCCs the Zapier email address, which triggers an automation that adds a tag in Kit. So when you reply and BCC the Zapier email, it adds a tag in Kit.

That tag triggers the “replied to email” automation, and you can see it’s working. We got 30 replies here. So what does the automation do? You hit reply, then we increment a counter.

Counting replies and storing a timestamp

We have a custom field called email reply count. We default it to zero, and we add one. So however many replies this subscriber has, it will now have one, two, three, four, five, whatever it is.

But we also wanted to track a timestamp. When was the last reply? Because if someone has eight replies, you might think they’re super engaged, but they might have only replied a year ago, which is not very recent engagement and not useful. If we want to prune or clean an email list based on engagement, we also track the latest reply. I always put in a timestamp, and that’s a Unix timestamp.

The rolling 30-day replied tag

We check to see if they have the “replied to email in the last 30 days” tag. If they don’t have it, it means they have not gone through this automation before. They’ve never replied before. So they fall down and get the “replied to email in the last 30 days” tag added. Then we remove the “replied to email” tag so that we can use this automation again.

Adding the “replied to email in the last 30 days” tag triggers a different visual automation called the replied to emails timer. We wait 30 days and we remove the tag. What this does is allow us to know if somebody is engaging with the emails. So you reply to an email, Zapier tags you with “replied to email,” we increment how many times you have replied, we add a timestamp, and then we add the “replied to email in the last 30 days” tag and timer.

Let’s say my client sends another email next week and that person replies again. They’ll get the “replied to email” tag added and the count will increment to two. It will set the timestamp to the most recent reply, and then it will see that they already have the “replied to email in the last 30 days” tag. If they have it, we remove that tag, wait five minutes, and then re-add the tag. Why do we do that? Because then we know it’s a rolling window of the last 30 days. This little condition makes sure that if they’ve replied, we remove the tag and reset the timer.

Why we prune cold subscribers at all

So why would we do this? Is it just kind of cool? Is it a novelty? It could just be data for data’s sake. But the reason we’re doing this is not because we like to have data. I like to make sure that any of the data we use has some sort of purpose.

We have a cold subscriber pruning sequence, and we do this for all of our clients. Regardless of the tool you’re using, whether you’re in Kit or HubSpot or Drip or Bento or whatever, we want to make sure that your list is constantly being checked, cleaned and pruned to increase email engagement.

Why opens aren’t a reliable signal

The reason we built this specific automation is because of our list pruning automation, which everybody goes through perpetually. They’re always in it. There were people who went down and failed to pass the “opened an email” condition. This checks to see if you’ve opened any emails in the last 30 days, probably. If not, they get a do not disturb, and we send them an email that says, “Hey, we’re going to be removing you from the list soon,” to get them engaged. Anyone who wants to be on the list opens the email and engages.

So I thought opening emails was going to be enough, that Kit would notice if you open an email and you’d satisfy the next “opened an email” condition. But something weird happened. We had a number of people reply to this email saying they weren’t opening the emails, but they said, “Please don’t remove me from the list.” So what was going on?

We found out that with some email providers, like Apple iOS or whatever mail client you’re using to receive emails, some of those weren’t being marked as opened even though they did open them, and they had been opening and reading them. Kit didn’t see that because some sort of cloaking or blocking mechanism was in place with whatever tool the subscriber was using, and it made Kit think this was an unengaged subscriber. If we had just removed those people, they would have stopped receiving emails they wanted to receive, because we were basing it on whether they opened the emails.

I looked into this and found out that Apple Mail has a privacy feature that loads the tracking pixel automatically. Apple users can often look like they opened an email even when they didn’t. But the problem with my client was the opposite: nothing was getting opened, despite their plea to keep them on the list because they’d apparently been reading everything. The people who weren’t showing opens were most likely reading with images turned off or using something that blocks tracking.

When you open an email, there’s a pixel in the images. When that image is loaded, it sends a notification to the email service provider, like Kit, saying this person opened the email. Opens aren’t a reliable way to decide who is engaged or cold, and that’s why we want to track replies. We needed another mechanism to make sure these people weren’t going to be removed, and that’s where the reply counter and countdown timer came in.

How replies plug into the list cleaning sequence

So now what happens? They get the cold subscriber engagement attempt: “Hey, we’re going to remove you from the list. You haven’t been opening emails.” We wait seven days. We check to see if they’ve opened an email, which they haven’t, because the email provider is not sending that data to Kit or it’s blocking it. So they get do not disturb again, which means we don’t send them broadcasts, newsletters or pitches.

Then we go down and check again to see if they’ve opened an email. They have not. So we have a condition that checks if they’ve replied to an email in the last 30 days. This is a reliable tag because it gets added whenever they last replied. My client replies with the Zapier BCC and says, “Hey, no worries. We got you. We’re going to keep you on the list.” That triggers the Zap, which triggers the automation, which marks that this person replied in the last 30 days and does the counters, the incrementers, the timestamps, all of that.

So now when they fall down to the “replied to an email in the last 30 days” condition, we add the list pruning holding tag to bring them back up to the top, and then they exit this automation. Anyone who failed that condition didn’t reply in the last 30 days, so they’ll get another cold subscriber engagement attempt. We check one more time with one more condition: do they have the “replied to email in the last 30 days” tag? Maybe they replied to the second email but not the first one. If so, we add them to the holding tag and they exit.

Why we remove cold subscribers manually

If they fail both of those, then they truly are cold. We queue them for removal so that we can manually delete those subscribers. I always make sure to delete them manually, not automatically. There’s always a scenario where the email service provider or the inbox client, like Gmail or Outlook, is not sending the right data, and we’re assuming they’re a cold subscriber. And sometimes that’s not true.