AWS re:Invent 2025 - How Netflix Connects Product Experiments to its AWS Bill (IND388)

Watch	Topic	Details
YouTube	Connecting Product Experiments to AWS Cloud Spend	Netflix has developed a framework to connect product experimentation with cloud spend. The framework treats the AWS bill as an experiment metric to make data-driven decisions. It aims to prevent hidden costs and surprises in the AWS bill by providing clear insights into the cost impact of experiments.
YouTube	Framework Components: Attribution and Estimation	The framework consists of two main components: attribution and estimation. Attribution identifies the root cause of changes in infrastructure metrics. Estimation translates these changes into quantified business impact, providing a cost projection for experiments.
YouTube	Attribution Process	Attribution involves measuring usage metrics such as request latency, payload size, volume, and error rates. It uses distributed tracing to track requests across a complex microservice architecture. Challenges include dealing with sampling, data inconsistencies, and large volumes of unstructured data.
YouTube	Estimation Process	Estimation is a three-step process: learning from production, simulating, and aggregating data. It uses machine learning models trained on historical data to predict the cost impact of experiments. The process generates a comprehensive view of the total cost delta if an experiment is launched globally.
YouTube	Lessons Learned and Technical Hurdles	Trace completeness is crucial for accurate attribution. Continuous retraining of machine learning models is necessary to adapt to evolving infrastructure. Robust fallback mechanisms are implemented to handle edge cases and ensure estimates are always available.
YouTube	Operational Excellence Use Cases	The framework enables proactive capacity planning for experiment rollouts. It allows for 'shift left' validation to ensure features do not depend on legacy or improperly scaled systems. Provides performance insights beyond business metrics, tying infrastructure impact to member experience.
YouTube	Future Roadmap	Plans to expand the framework to include storage costs and batch services. Focus on new domains like generative AI to address rising costs. Key takeaways include the importance of attribution, projecting the cost of success, and cultivating shared ownership of cost data.

Watch on YouTube

Description

How do you measure the cost of innovation? At Netflix, a "winning" A/B test can create hidden cloud infrastructure costs that offset its business value. This session details our strategy for connecting product experiments to their infrastructure costs. We'll share our framework for identifying an experiment's impact on our cloud systems and show how we translate that impact into a projected AWS cost. Learn how this enables our teams to make smarter, more cost-aware product decisions at scale.

Learn more:
AWS re:Invent: https://go.aws/reinforce.
More AWS events: https://go.aws/3kss9CP

Subscribe:
More AWS videos: http://bit.ly/2O3zS75
More AWS events videos: http://bit.ly/316g9t4

ABOUT AWS:
Amazon Web Services (AWS) hosts events, both online and in-person, bringing the cloud computing community together to connect, collaborate, and learn from AWS experts.
AWS is the world's most comprehensive and broadly adopted cloud platform, offering over 200 fully featured services from data centers globally. Millions of customers—including the fastest-growing startups, largest enterprises, and leading government agencies—are using AWS to lower costs, become more agile, and innovate faster.

#AWSreInvent #AWSreInvent2025 #AWS