Do clickbait subject lines work?
An A/B test result
I’ve always wondered if the really obnoxious, clickbait-y subject lines actually improve open and conversion rates in emails. I had the chance to test this out myself a few months ago and when I told a colleague about it, realised there are probably a lot of us out there who have wondered if it works or not, but are too cautious to try it on our own audiences.
So here is the experiment I ran, and the results!
The objective
So this was back at Perkbox (disclaimer, my old job) and our objective was to re-engage users who had gone through the onboarding process and completed the set up of their accounts, but had never redeemed a perk. In Perkbox’s case, a perk is exactly what it sounds like - exclusive offers available to users, including things like savings at a number of local and international retailers.
We wanted them to open the email, click through to the platform, browse a curated selection of perks, and redeem one.
The experiment
We split the audience 50/50 and decided to do one that was very “corporate” and transactional, and one that was “funny” and really pushed the limit of what was acceptable to say as a brand.
I don’t remember the corporate subject line exactly, but the sentiment we went for was to pretend like it was a brand new initiative. Perkbox is provided by employers to employees, so I’m pretty sure the subject line was along the lines of “[Company name] has given you access to Perkbox”. The body of the email was then a sentence explaining that Perkbox was a [company name] initiative that all employees have access to, and instructions on how to redeem a perk.
The funny one I do remember exactly, it was “Oops, there’s an error with your account” and the body of the email was a cheeky “the error is that you haven’t redeemed a perk yet”. It then had the same selection of perks previewed in the email as the corporate one, and both emails linked out to a carefully selected curation of perks.
The curation was another experiment. There’s hundreds of perks on the platform, which can be overwhelming especially if you’re just there to look and you’re not specifically on the market for something. We narrowed this selection down to the most commonly redeemed perks that were general “oh yeah, I can see myself using that” and had positive or easy redemption mechanics.
We knew from our UK equivalents that some perks had a more difficult redemption than others (they might have had an extra step to them or something like that), and if one of those perks was the first one you ever redeemed, you were less likely to come back and redeem a second one. So we made sure we picked perks that had the smoothest and easiest redemption experiences.
The results
Highest open rate: Funny subject line
Highest click through rate: Corporate subject line
Highest perk redemption: Corporate subject line
So while the funny one did get more people to open the email, it didn’t spur them to action. Meanwhile, the corporate version not only had the highest rate/%, it also won by volume, so by all the important metrics it was the better performer.
Some anecdotes:
We actually had an admin tell us during their next customer success call that they were guilty of not having had redeemed a perk, but they redeemed one as soon as they got the email from us - they had been intending to redeem a perk for ages but this was the final push they needed to actually do it.
This email campaign actually was part of a larger push to get people who’d never redeemed to pass that milestone. The other parts of the campaign were more complex (things like competitions, big show-y promos of perks etc) but this one email was actually the most effective push. Simple things work. This was a great reminder to keep it simple, there’s no need to reinvent the wheel.
Our UK counterparts ran a very similar A/B test, and the ‘funny’ one won over there. Just remember every audience is unique and so you should never assume what the result will be.
This test is still ongoing (as far as I know - I don’t work there anymore!) and the second time we ran it, we kept the corporate one exactly as it was and then re-wrote the funny one to be less clickbait-y but still humourous. The corporate one won again, and I left not long after that so can’t tell you how V3 is going!
Housekeeping
Next week’s Mehdeeka is going to be the last ‘proper’ issue for this season! It’s going to be an accumulation of many small things that weren’t quite big enough for their own issue, little bits of information I’ve accumlated from various sources.
The issue after that will be a “best of” the season, and then I’m going on break again.
I do intend to do a season 4 of Mehdeeka, but this season I also started a new role and I have found it hard to keep up the intensity of absorbing so much new information of work and writing this newsletter on time and organising interviews. I’ve let it slip, and I don’t like doing that.
I’m going to take some time off from Mehdeeka (as usual) but it might be a longer break than usual, as I’m going to have a crack at writing the entirety of season 4 before releasing it on a schedule. I might also throw all caution to the wind and do whatever I feel like though, so don’t hold me to my word.
Either way, two more issues coming, then I’ll disappear from your inbox for a while!
Just one link this week
If like me and most other Australians, you’re back in lockdown, I found this 8 hour recipe really helped pass the time over the weekend because it literally takes all day (and includes an overnight soak). It was extremely delicious (French style pork belly, chicken, sausage, and beans slow cooked) and worth it, and the write up accompanying it was also very interesting.
To keep myself sane I’m trying to spend my evenings away from screens, so I have been spending a lot less time on the internet and thus consuming less, which explains why there’s only one link today!
Stay home!