Open Source Summit North America 2025

Triton Inference Server has long been a reliable tool for AI model deployment. As Generative AI unfolds its transformative potential, Triton continues to evolve, offering both time-tested features and new capabilities tailored for large language models and more complex agentic workflows.

This session explores how Triton’s core strengths continue to play a crucial role in optimizing generative AI deployments. These include its robust multi-framework support, dynamic batching, concurrent model execution, and the capability to deploy complex inference pipelines through model ensembling and business logic scripting.

We’ll also cover recent enhancements such as OpenAI compatible frontend, allowing easy integration with existing OpenAI-based applications; Python-based backends to standardize the deployment of Python models without writing a custom C++ backend; Triton CLI to simplify model deployment and management; distributed inference enhancements for Data Center scale.

Throughout the presentation, we’ll share practical examples and best practices, equipping our listeners with the knowledge to effectively use Triton Inference Server to optimize AI workloads’ performance and efficiency.

Generative AI applications, in general, excel in zero-shot and one-shot types of specific tasks. However, we live in a complicated world and we are beginning to see that today’s generative AI systems are simply not well equipped to handle the increased complexity that is found especially in business workflows and transactions. Traditional architectures often fall short in handling the dynamic nature and real-time requirements of these systems. We will also need a way to coordinate multiple components to generate coherent and contextually relevant outputs. Event-driven architectures and multi-agent systems offer a promising solution by enabling real-time processing, decentralized decision-making, and enhanced adaptability.

This presentation proposes an in-depth exploration of how event-driven architectures and multi-agent systems can be leveraged to design and implement complex workflows in generative AI. By combining the real-time responsiveness of event-driven systems with the collaborative intelligence of multi-agent architectures, we can create highly adaptive, efficient, and scalable AI systems. This presentation will delve into the theoretical and practical sides.

This session will guide attendees through the process of understanding and mitigating the carbon emissions of machine learning models and AI systems. We'll delve into methods for measuring the environmental impact of these technologies and discuss the pivotal role developers play in pioneering eco-conscious computing. Participants will gain insights into optimizing algorithms, adopting sustainable coding practices, and choosing energy-efficient tools to minimize the carbon footprint of their machine learning projects.

Additionally, we'll examine the environmental considerations of deploying AI systems in the cloud. As cloud computing becomes integral to deploying AI solutions, understanding its ecological impacts is crucial. We'll cover strategies for making environmentally responsible decisions when selecting and utilizing cloud services, aiming to maintain the eco-friendliness of AI applications.

Together, we'll explore how to balance the demands of advanced computational technologies with the urgent need for sustainability.

Join us to discover how Intuit's GenAI framework is reshaping AI development, enabling swift integration of AI functions across varied business units. We'll focus on a robust framework of reusable agents and tools derived from open-source technologies like LangChain/LangGraph, facilitating diverse functionalities from simple data retrieval to complex processes such as query generation, optimization, pipeline creation and debugging. This framework dramatically reduces the time required for data workers to operationalize data pipelines and supports diverse customer interactions through notebooks, no-code approaches, REST integrations, and Python libraries, catering to a wide range of needs including agent developers and teams in pre-production settings. Our meticulous evaluation process ensures that each tool and agent is rigorously tested against high-performance benchmarks to guarantee reliability and consistency before deployment. By centralizing these AI components, Intuit has not only accelerated development timelines but also upheld a high standard of quality, establishing a benchmark for crafting scalable, effective AI solutions in the dynamically evolving tech landscape.