Retention hypotheses are all tasks/experiments aimed to bring users back, for example using the site (e.g. the retargeting code installation on the website), the product (collecting the weekly summary and sending it to the user) and channels of information (creating the mailing in the email automation system). You can return users to keep using the product or other AAARRR stages of the funnel: acquisition, activation, revenue (payment or upselling).
Retention is vital for the user flow because it’s 5 times cheaper to retain users than to acquire new ones. It's a common mistake many businesses make - to focus on Acquisition step only, trying to acquire as many people as possible, while too many of them drop out down the funnel.
So let’s look at a few cases showing retention experiments in action.
Tom’s team thought that the retention increase by 5% could boost their profits by up to 95%, so they started from the top of the funnel. They have already set up the Facebook retargeting; however, they came up with the idea that could potentially improve this retargeting campaign by not showing one ad for everyone, but by showing the ads of that city, for which the user has viewed the trip page on the website. The team decided to create separate ads only for one city and test how Facebook channel Retention would change for this town.
The time of making the buying decision in Tom's business isn't more than a week, and usually, it is only a couple of days. Therefore, the team compared how a 2-day Retention changed. To measure this experiment, they just enter:
It was the Retention experiment that returned users to the Activation stage.
Onboarding emails, explaining the benefits of the service, should affect its retention. Tom’s Growth Team tests onboarding emails quite frequently. To make the right decisions about the success of experiments, it is important to understand the contribution of each of the letters to the general Retention. But how do you measure changes in the impact of the letters focused on Retention?
Very simple. It’s enough to specify what Retention you’re interested in: DayRetention, WeekRetention or MonthRetention, and specify the Campaign=onboarding_mail2, which is included in letters’ links.
This graph can be compared to those users who don’t receive these emails.
Or compared to general Retention for users who clicked links in all letters, just by specifying medium=email
However, Retention experiments can be measured not only with the Retention graph. Let’s see another example.
After placing the order, users do not always pay for it. On the last step, the Conversion Rate of the payment (comparing to the order) was 70%. The problem of the "abandoned shopping cart" can be solved with the help of the letter - by returning the user to the payment step, reminding him. The simple hypothesis "Send a letter the next day after the order with a reminder about it and a direct link to the payment" increased the Conversion Rate of Order -> Payment funnel by 84%.
To measure this experiment, just enter the funnel consisting of two actions: Order -> Payment and trace how the Conversion Rate changes after sending these letters.
Or for the greater fairness of the experiment, we can compare it to the funnel in which users have received emails and in which they haven’t.