Pinterest has partnered with Intel to extend gProfiler, an open-source continuous profiler that provides production performance visibility across an internet-scale fleet of infrastructure.
Pinterest claims it operates hundreds of thousands of instances supporting 640 million monthly active users, with services written in Java, Python, C++, Go, and Node.js. At that scale, even a small efficiency improvement can have significant infrastructure and cost implications. The challenge was that engineers were using separate tools such as perf, async-profiler, and py-spy, producing fragmented profiles that were difficult to correlate across languages and machines.
Rather than building another proprietary profiling platform, Pinterest worked with Intel to extend gProfiler and contribute those capabilities back to the open-source project. The collaboration adds dynamic profiling, hardware-level telemetry, wall-time profiling, and performance-counter flame graphs, while also hardening the system for large production fleets.
gProfiler acts as an orchestration layer over several existing profilers. It combines system-wide perf data with runtime-specific tools such as async-profiler for Java and eBPF-based profiling for Python, producing a unified flame graph across the processes running on a host.
This is important for modern services where application execution crosses multiple layers. A Java application may spend time inside the JVM, native libraries, or the Linux kernel, while a Python service can involve interpreter, native, and kernel activity. Instead of requiring engineers to select and manually correlate separate profilers, gProfiler combines those views.
The agent discovers processes, starts the appropriate profilers, merges their stack traces, and uploads the resulting profile without requiring application code changes.
The new capability allows engineers to select services, hosts, or clusters and trigger profiling remotely through the UI or API. Agents normally remain dormant, consuming less than 1% CPU and 100 MB of memory, and receive profiling instructions through periodic heartbeats. You can change profiling frequency, duration, and the profilers used without redeploying the agent.
This effectively turns profiling into an operational capability rather than something that requires engineers to gain shell access to individual machines.
Integration with Intel PerfSpect allows gProfiler to collect hardware-level information such as cache behaviour, memory bandwidth, and CPU stalls. Pinterest also added PMC flame graphs, which correlate functions with hardware performance events such as cache misses and branch mispredictions.
Another addition is wall-time profiling. CPU profiling shows where processor time is being spent, but a service can still be slow while consuming relatively little CPU because threads are waiting on I/O, locks, or remote procedure calls. Wall-time profiling makes those waiting periods visible.
The gProfiler architecture separates collection from analysis. Agents run on individual hosts, while gProfiler Performance Studio aggregates and visualises the resulting profiles.
Profiles are stored in S3, passed through SQS for asynchronous indexing, and stored in ClickHouse for analytical queries. A React frontend provides flame-graph exploration, while aggregated data allows engineers to compare profiles across hosts, services, and time periods.
Pinterest and Intel also hardened the platform for production, addressing issues involving agent resource consumption, disk usage, and multi-terabyte ingestion before deploying it across the fleet.
gProfiler sits in an increasingly crowded continuous-profiling landscape. Parca and Grafana Pyroscope use eBPF and related sampling technologies to provide fleet-wide profiling, while commercial platforms such as Datadog offer continuous profiling alongside broader observability capabilities.
The distinction with gProfiler is its emphasis on system-wide, multi-runtime profiling and open-source extensibility. Rather than replacing tools such as perf or async-profiler, it orchestrates them. It also adds hardware-level profiling through Intel's tooling.
Intel's VTune provides much deeper processor and application performance analysis, but it was designed primarily as a performance-engineering tool rather than a cloud-native fleet profiler. Pinterest specifically notes that this difference influenced the decision to build on gProfiler rather than adapt VTune for its environment.
At small scale, profiling is usually something an engineer enables when a performance problem appears. At Pinterest's scale, the cost of waiting for engineers to reproduce problems or manually profile machines becomes too high. Continuous profiling provides a baseline, while dynamic profiling allows teams to investigate specific anomalies when they occur.
By combining runtime profiles, system behaviour, and hardware performance data under an open-source platform, Pinterest is effectively making profiling another layer of production observability.