InfoQ Homepage A/B Testing Content on InfoQ
-
How Spotify Carries through Experiments at Scale for Spotify Home
Spotify runs more than 250 online experiments annually on its Spotify Home platform, which are used by dozen of different teams. To accomplish running experiments at such scale, Spotify uses a number of different tools, explains Spotify product manager Nik Goyle.
-
How Lyft Detects Android Memory Leaks in Production
While modern tooling for Android and iOS enable memory leak detection using local builds, this is not enough to guarantee an app shows correct memory behavior in production, where it runs on a wide range of devices in diverse conditions. For this reason, Lyft engineers combine A/B testing and memory observability to detect which features leak memory.
-
Safe Mode is Lyft's Approach to Handling Feature Flags Incidents
Feature flags are a useful tool to conduct A/B experiments and to roll out changes in a controlled way. To make sure that their use does not end up disappointing users when a change causes a crash or degrades the user experience, Lyft created Safe Mode, specifically aimed to prevent crash loops on launch.
-
How Lyft Is Improving Their Experiments beyond A/B Testing
Lyft’s product manager John Kirn recently published an article about the challenges they face when conducting experiments. Existing experimentation techniques did not fully adapt to Lyft’s real-time business nature or mitigate network effects. Lyft’s Experimentation team deployed new ones, such as time and region split testing, and improved internal experimentation norms and techniques.
-
BBC New Serverless Platform Improves Scalability and Performance
One year into the transition to their new WebCore serverless platform, the BBC has started to reap the benefits of an architecture that removes the burden on engineers to solve performance and operational challenges and allows them to focus on the value they deliver to customers.
-
Airbnb: Using Guardrails to Identify Changes with Negative Impact across Teams
Airbnb rolled out an internal Experiment Guardrails system to identify potentially negative impacts of changes across different teams. Whenever a proposed change does not pass any of the guardrails, it is escalated for further analysis by affected teams and stakeholders, explains Airbnb data scientist Tatiana Xifara.
-
GoDaddy Releases Automatic Canary Deployments Tool for Kubernetes
GoDaddy recently released an open-source tool to automate gated deployments in Kubernetes. Every time a deployment happens, the tool can run regression tests, and pull metrics from data backends like New Relic. After some time, the tool decides whether to roll back or continue with the deployment automatically. Users can run A/B tests and run experiments with a small portion of live traffic.
-
HashiCorp Releases Consul 1.6 with Layer 7 Dynamic Traffic Management and Cross-Network Connectivity
Hashicorp released version 1.6 of Consul, their service mesh application and key-value store. This release builds on the features added in version 1.5 by introducing layer 7 routing and traffic management. It additionally delivers a new feature, mesh gateway, to route service traffic across regions, platforms, and clouds.
-
Expo: Real Time A/B Testing and Monitoring with Spark Streaming and Kafka at Walmart Labs
The WalmartLabs engineering team developed a real time A/B testing tool called Expo that collects and analyzes user engagement metrics. It uses Spark Structured Streaming to process the incoming data and stores the metrics in KairosDB.
-
Q&A with Katrina Clokie on Testing in DevOps for Engineers
Wellington's DevOpsDays NZ recently closed with a keynote by Katrina Clokie on the Testing Skills and Superpower which engineers can utilise in a DevOps setting. The author of A Practical Guide to Testing in DevOps spoke with InfoQ to discuss the changes she's seen in the testing landscape and how this is further impacted by the move to embrace DevOps principles.
-
StackPointCloud Enhances Istio and Kubernetes Functionality in Cloud-Native Management Platform
StackPointCloud, provider of a cloud-native management platform for Istio and Kubernetes, has added a series of tools to help enterprises deploy and use microservices and containers. Users can now manage Istio service meshes across Kubernetes clusters using their preferred cloud providers and StackPointCloud has partnered with Packet Host to help customers use ARM with their Kubernetes clusters.
-
Apple TestFlight Now Supports A/B Testing of iOS Apps
With its recent update to TestFlight, Apple has introduced a number of features, such as multiple builds and enhanced groups, that make it possible to do A/B testing for iOS apps.
-
Large Scale Experimentation at Spotify
When you want to scale the number of A/B tests to do many experiments at the same time, you need to adopt your processes and platform, and it might also impact your culture. Doing product research with controlled experiments helps to confront your ideas about how customers will use your product in reality, and check if those ideas actually impact user behaviour.
-
Patrick Debois Bringing DevOps to Mobile Delivery
Patrick Debois, founder of DevOps Days conferences, together with AWS's Trent Peterson and Etsy's Nassim Kammah, are organizing Mobile Delivery Days, a new conference for discussing the entire mobile delivery tool chain, from building and testing to publishing automation, production support (logging, instrumentation, monitoring) and (A/B) testing.
-
Parse Announces Support for Push Notification A/B Testing
Parse has announced Push Experiments, a feature that allows to conduct A/B tests for campaigns based on the use of push notifications, which are often deployed to re-engage users in mobile apps. Push Experiments will allow to evaluate different messaging strategy and choose the one with the highest open rate, say Parse.