If you have ever thought about building an in-app messaging platform for your mobile app, Lyft's experience may help you get it right. Lyft engineers applied a three-staged approach: MVP, scaling, and optimization.
Among the driving factors behind Lyft's decision to build their own in-app messaging platform were two major shortcomings in using external messaging platforms such as SMS, email, or push notifications: a potential delay in the message delivery and the disruption of the user experience caused by using a different app.
The first decision you want to make is how you are going to deliver your messages. In Lyft's case, in-app banners appeared the right way to deliver messages to riders in a non-invasive yet prominent way. Once that decision was taken, the team defined a three-stage process, starting with the implementation of a minimum viable product (MVP), followed by an adoption and scaling phase, and finally an optimization effort.
In the first stage, aiming to create an MVP, the Lyft team took a number of decisions to reduce the complexity of the problem at hand. What they wanted to have in their MVP was a way to target users based on their state, location, or marketplace context; a way to limit the rate of banners and prevent that too frequent messages could make them less relevant; a way to carry through experiments; and a way to deal with conflicts among banners.
In order to target users effectively, Lyft engineers integrated the messaging platform with the customer data platform. Initially, banners were created and targeted banners manually, in a non-scalable way. Message duplication was dealt with limiting the maximum number of times a message could be shown. To reduce the impact of conflicts, Lyft defined a simple heuristic that assigned a higher priority to those messages that targeted the smalled population. Furthermore, to further simplify things, banners could only be placed on the main screen.
With all of that in place, Lyft started experimenting with the platform and collect encouraging statistics about feature adoption, active riders, and several user engagement metrics.
In the second stage, the team attempted to remove a few bottlenecks hampering scalability, such as the manual creation of banners, and the limited placements available. So the team implemented an internal tool to create new banners without programmers' intervention and, to deal with the growing number of messages, a better ranking strategy to avoid conflicts. Specifically, instead of ranking individual messages, they created banner categories and ranked those, with so-called transactional messages having top priority and promotional messages being lower-priority. This was also the time to improve user targeting by adding support for geofencing and for event-specific messages. Finally, the team extended the number of available placements to integrate banners more effectively.
In the third stage, the Lyft team aimed to optimize the impact of messages, which required providing answers for a number of critical questions to ensure they delivered the right message to the right user at the right time.
This was solved by creating an ML model to rank messages based on relevant context data and predicting the expected reward for showing a banner to a user.
This caused another problem, known as the "long-tail problem", which means that over time only popular banners tend to be shown while less popular ones do not get a chance. The solution to avoid the long-tail was using a bandit reinforcement learning algorithm.
Bandit optimization was also leveraged to improve the way the Lyft team carried through experiments to decide which version of a banners should be shown, which shortened the time to find out the best message version.
Lyft’s article provides many more details than can be covered here, so do not miss it to get the full picture.