Recently launched, Amazon CloudWatch Omni is an AI-first observability platform designed to monitor, evaluate, and troubleshoot applications and autonomous AI agents in a unified environment.
For traditional applications, metrics such as latency, errors, CPU and availability are often sufficient to identify operational problems, says AWS senior specialist solutions architect Daniel Abib. For AI agents, an execution can succeed technically while still producing the wrong result, using the wrong tool, retrieving poor information, or taking an unnecessarily expensive path:
Agent behavior is non-deterministic: a prompt change can degrade response quality even when standard metrics show no errors. Teams spend hours manually reviewing logs across multiple systems, unable to pinpoint what changed or why.
To address this challenge, Omni captures end-to-end traces, evaluates correctness, coherence, retrieval, and tool selection, and lets you compare prompts, build test datasets from production traffic, run experiments, and detect regressions.
It supports several agent frameworks, including LangChain, LangGraph, CrewAI, OpenAI SDK, Strands, and Vercel AI SDK. It also uses open standards such as OpenInference and AWS Distro for OpenTelemetry (ADOT) and integrates with Amazon Bedrock AgentCore. For evaluations, CloudWatch Omni also supports third-party evaluators such as Braintrust, DeepEval, and Ragas.
Amazon CloudWatch Omni provide unified observability, bringing traditional microservices, cloud infrastructure, and generative AI/agentic workloads under the same umbrella for a unified view. It offers native OpenTelemetry support, allowing existing telemetry to feed into Omni without complex reconfigurations, as well as dual workspaces, which provide a standalone web experience for operators via single sign-on (SSO) outside the traditional AWS Management Console, alongside a free, local, developer-friendly IDE extension supporting both VS Code and Kiro.
CloudWatch Omni also support AI-powered investigations, allowing users to query logs, metrics, and traces using natural language to identify topology issues and pinpoint root causes.
AWS vice president Chet Kapoor summarizes the new tool on LinkedIn saying that it "helps you catch issues proactively, trace them to their root cause, and identify improvements across your agents, applications and infrastructure in one place". Former AWS principal security consultant Jorg Huser, similarly notes that "A unified view that explains why an agent acted the way it did is the only way to keep confidence intact at scale".
As a final note, Deutsche Bank lead devops engineer Florin Lungu emphasizes that CloudWatch Omni "supports open standards and built-in evaluators for quality assurance".
While CloudWatch Omni may be the natural fit for applications running on AWS, several other observability tools and platforms combine tracing, evaluation, prompt experimentation, and datasets, with a focus on agentic applications. These include LangChain LangSmith, LangFuse, Arize AI Phoenix, and others.