Open Source Summit North America 2025

As Generative AI unfolds its transformative potential, and enterprise needs have shifted toward large-scale, distributed deployments, the requirements for inference serving have fundamentally changed. To meet these new demands, NVIDIA has introduced Dynamo— a high-throughput low-latency inference framework designed for serving generative AI and reasoning models in multi-node distributed environments.

This session provides a technical overview of Dynamo’s architecture, focusing on how its design addresses the core challenges of large-scale, distributed generative AI inference. We will walk through concrete deployment scenarios—including disaggregated serving and dynamic GPU scheduling—and examine how Dynamo manages resource allocation, request routing, and memory efficiency for high-throughput, low-latency inference.

We will also share practical implementation examples and discuss engineering best practices for optimizing workload performance, scalability, and cost using Dynamo. We’ll outline the steps and considerations for deploying Dynamo, highlighting key architectural differences and compatibility factors. By the end of the session, attendees will have a clear understanding of how to deploy and operate Dynamo in production environments to support advanced AI workloads.