AWS re:Invent 2025 - Tracing the Untraceable: Full-Stack Observability for LLMs and Agents (AIM212)

Watch	Topic	Details
YouTube	Importance of LLM Observability	Observability is crucial due to the complexity of modern LLM workflows. Silent failures in any part of the workflow can disrupt the entire process. Tool augmentation and script usage in workflows require comprehensive monitoring.
YouTube	Key Metrics for LLM Observability	Token usage is critical for cost management and understanding user consumption. Response latency impacts overall API performance, especially in customer-facing applications. Error rates need to be monitored to ensure API reliability and correct model responses. LLM responses must be monitored for hallucinations and correctness.
YouTube	Challenges in LLM Observability	Rapid changes in AI implementation outpace the ability to instrument and monitor. Sensitivity of user input data poses privacy and security challenges. Uncertainty about which models are in use due to rapid deployment cycles.
YouTube	Groundcover's Approach to LLM Observability	Bring-your-own-cloud architecture ensures data privacy and residency. eBPF provides kernel-level visibility without requiring code instrumentation. Real-time insights into token usage, reasoning paths, and error rates are available. Full API requests and responses are captured for detailed analysis.
YouTube	Benefits of Groundcover's Solution	Protects LLM pipelines from data exposure by storing data on-premises. Enables performance improvement and cost management through detailed monitoring. Allows monitoring of LLM responses with context for actionable insights.
YouTube	Demo of Groundcover's Capabilities	Automatic discovery and monitoring of Bedrock APIs without manual instrumentation. Detailed visibility into API calls, models used, latency, and throughput. Correlation of LLM traces with logs and infrastructure metrics for comprehensive insights. Examples of error detection and troubleshooting without DevOps intervention. Analysis of P90 latency and token usage to inform business decisions. Setting alerts for SLA breaches and creating dashboards for ongoing monitoring.

Watch on YouTube

Description

In just a few years, LLMs have gone from research curiosities to the backbone of new software experiences. Organizations are rushing to productionize LLM workflows because of their immense value, but are doing so without any observability guardrails, introducing new layers of fragility and complexity such as performance volatility, quality drift, and security risks.
In this talk, we’ll discover how to instantly monitor and troubleshoot LLM applications with zero-instrumentation. Whether on commercial LLM stacks or AWS Bedrock - we’ll break out of the LLM blackbox and learn how to monitor token usage, response latency, data exposure prompts, and model execution failures. This presentation is brought to you by groundcover, an AWS Partner.

Learn more:
AWS re:Invent: https://go.aws/reinforce.
More AWS events: https://go.aws/3kss9CP

Subscribe:
More AWS videos: http://bit.ly/2O3zS75
More AWS events videos: http://bit.ly/316g9t4

ABOUT AWS:
Amazon Web Services (AWS) hosts events, both online and in-person, bringing the cloud computing community together to connect, collaborate, and learn from AWS experts.
AWS is the world's most comprehensive and broadly adopted cloud platform, offering over 200 fully featured services from data centers globally. Millions of customers—including the fastest-growing startups, largest enterprises, and leading government agencies—are using AWS to lower costs, become more agile, and innovate faster.

#AWSreInvent #AWSreInvent2025 #AWS