Open Source Summit North America 2025

Generative AI (GenAI) is ushering in a new era of human innovation, but building your own GenAI application can feel overwhelming. Which Large Language Model (LLM) should you choose? Should you incorporate Retrieval-Augmented Generation (RAG)? How can you ensure your application runs securely and efficiently on Kubernetes, with robust observability and debugging? And how do you manage API calls and control costs for external LLMs?

This demo-driven session will guide you step by step through building, running, securing, and managing traffic for a GenAI application from scratch. Starting with a native setup, we’ll then transition to Kubernetes, simplifying the entire process. You’ll learn how to enhance your application with domain-specific knowledge using RAG and leverage cloud-native tools like Kubernetes, Prometheus, Kiali, Istio, and the Kubernetes Gateway API to run your application securely and effectively.