InfoQ Homepage Netflix Content on InfoQ
-
Inside Netflix’s Distributed Counter: Scalable, Accurate, and Real-Time Counting at Global Scale
Netflix engineers recently published a deep dive into their Distributed Counter Abstraction, a scalable service designed to track user interactions, feature usage, and business performance metrics with low latency. The system balances performance, accuracy, and cost through configurable counting modes, resilient data aggregation, and a globally distributed architecture.
-
QCon SF 2024 - Incremental Data Processing at Netflix
Jun He gave a talk at QCon SF 2024 titled Efficient Incremental Processing with Netflix Maestro and Apache Iceberg. He showed how Netflix used the system to reduce processing time and cost while improving data freshness.
-
QCon SF: Large Scale Search and Ranking Systems at Netflix
Moumita Bhattacharya spoke at QCon SF 2024 about state-of-the-art search and ranking systems. She gave an overview of the typical structure of these systems and followed with a deep dive into how Netflix created a single combined model to handle both tasks.
-
QCon SF: Using Metaflow to Support Diverse ML Systems at Netflix
At QCon SF 2024, David Berg and Romain Cledat gave a talk about how Netflix uses Metaflow, an open-source framework, to support a variety of ML systems. The pair gave an overview of Metaflow's design principles and illustrated several of Netflix's use cases, including media processing, content demand modeling, and meta-models for explaining models.
-
Netflix Adopts Virtual Threads: a Case Study on Performance and Pitfalls
Netflix, a long-time Java adopter, recently upgraded to Java 21. They are now harnessing new features such as generational ZGC and virtual threads to improve performance across their extensive microservices fleet. While virtual threads, designed for high-throughput concurrent applications, showed early promise, they also brought unique challenges in real-world scenarios.
-
Netflix Uses Elasticsearch Percolate Queries to Implement Reverse Searches Efficiently
Netflix engineers recently published how they use Elasticsearch Percolate Queries to "reverse search" entities in a connected graph. Reverse search means that instead of searching for documents that match a query, they search for queries that match a document, powering dynamic subscription scenarios where there is no direct association between the subscriber and the subscribed entities.
-
Netflix Launches bpftop Aimed at Enhancing eBPF Performance Efficiency
Netflix has recently announced bpftop, a command-line utility aimed at enhancing the optimization and monitoring of eBPF programs. bpftop provides a real-time snapshot of eBPF programs in operation. It shows metrics such as the average duration of program execution, the number of events processed every second, and an approximation of the total CPU usage percentage for each program.
-
Netflix Uses Metaflow to Manage Hundreds of AI/ML Applications at Scale
Netflix recently published how its Machine Learning Platform (MLP) team provides an ecosystem around Metaflow, an open-source machine learning infrastructure framework. By creating various integrations for Metaflow, Netflix already has hundreds of Metaflow projects maintained by multiple engineering teams.
-
QCon SF 2023: How Netflix Really Uses Java by Paul Bakker
Paul Bakker, Java Platform at Netflix, Java Champion, and co-author of "Java 9 Modularity," presented How Netflix Really Uses Java at the 2023 QCon San Francisco conference. Bakker described the evolution of the architecture behind their movie application, introduced the GraphQL Federation, and described how Java is used at Netflix that includes plans to support JDK 21.
-
QCon San Francisco 2023 Day 2: Design for Resilience, Platform Engineering, Modern ML, JVM Trends
The 17th annual QCon San Francisco conference was held at the Hyatt Regency San Francisco in San Francisco, California. This five-day event, organized by C4Media, consists of three days of presentations and two days of workshops. Day Two, scheduled on October 3rd, 2023, included a keynote address by Neha Narkhede and presentations from four conference tracks and one sponsored track.
-
Managing 238 Million Memberships of Netflix: Surabhi Diwan at QCon San Francisco
During the first day of QCon San-Francisco 2023, Surabhi Diwan, a senior software engineer at Netflix, presented on managing 238 million Memberships of Netflix. The talk is a part of the “Architectures You’ve Always Wondered About" track. Diwan's work at Netflix involves the backend work regarding membership engineering, which is critical for both signups and streaming at Netflix.
-
Implementation of Zero-Configuration Service Mesh at Netflix
In a recent blog post, Netflix described why they engaged the Envoy community and Kinvolk to implement a new feature for Envoy, the open-source proxy developed by Lyft. This new feature called On-Demand Cluster Discovery helped Netflix to implement a zero-configuration service mesh.
-
Netflix Built a Scalable Annotation Service Using Cassandra, Elasticsearch and Iceberg
Netflix recently published how it built Marken, a scalable annotation service using Cassandra, ElasticSearch and Iceberg. Marken allows storing and querying annotations, or tags, on arbitrary entities. Users define versioned schemas for their annotations, which include out-of-the-box support for temporal and spatial objects.
-
Scaling GraphQL Adoption at Netflix: Tejas Shikhare at QCon San Francisco 2022
At QCon San Francisco 2022, Tejas Shikhare, senior software engineer at Netflix, presented Scaling GraphQL Adoption at Netflix. Shikhare has been working at Netflix’s federated GraphQL platform, distributed systems, and, more recently, developer tools and education. This talk is part of the editorial track Modern APIs: Building and Evolving.
-
Netflix Builds a Custom High-Throughput Priority Queue Backed by Redis, Kafka and Elasticsearch
Netflix recently published how it built Timestone, a custom high-throughput, low-latency priority queueing system. They built it using open-source components such as Redis, Apache Kafka, Apache Flink and Elasticsearch. Engineers state that they made Timestone since they could not find an off-the-shelf solution that met all of its requirements.