Open Source Summit North America 2025

Open Source Summit North America 2025

June 23 - 25, 2025
Denver, Colorado
View More Details & Registration
Note: The schedule is subject to change.

The Sched app allows you to build your schedule but is not a substitute for your event registration. You must be registered for Open Source Summit North America 2025 to participate in the sessions. If you have not registered but would like to join us, please go to the event registration page to purchase a registration.

This schedule is automatically displayed in Mountain Daylight Time (UTC/GMT -6). To see the schedule in your preferred timezone, please select from the drop-down menu to the right, above "Filter by Date."

IMPORTANT NOTE: Timing of sessions and room locations are subject to change.

11:20am MDT

EdgeLake-FL: An Automated Federated Learning Platform for the Edge - Ori Shadmon & Moshe Shadmon, AnyLog

Monday June 23, 2025 11:20am - 12:00pm MDT

Bluebird Ballroom 3E

Edge AI today relies on centralizing data from edge devices to the cloud, but this is impractical due to costs and privacy constraints. Federated Learning (FL) is a viable alternative: edge nodes collaboratively train a ML model without transferring or exposing proprietary data. Instead, only model weights are shared, allowing each entity to develop a model that outperforms what it could train independently. Despite its potential, FL is largely academic due to the complexity of integrating expertise across the technology stack. Additionally, decentralized data can be heterogeneous, requiring non-generalizable, application-specific solutions. EdgeLake-FL is a hardware-agnostic framework leveraging EdgeLake, an LF Edge project, to automate the continuous learning FL workflow. With EdgeLake as the data management layer, decentralized data appears centralized and data heterogeneity is resolved. Using EdgeLake-FL, an ML engineer publishes a training application, and Edge nodes with relevant data autonomously train, share, and aggregate models. Each node can then leverage the aggregated models for inference directly at the Edge. In this talk, I will demo EdgeLake-FL in a real use case.

Speakers

Ori Shadmon

Product Manager, AnyLog

Moshe Shadmon

CEO, AnyLog

Moshe Shadmon, CEO at Anylog. AnyLog’s Virtual Edge Data Network is a Plug & Play software, deployed at the edge, allowing real-time insight without centralizing the data. AnyLog enables deployment of applications and AI at the distributed edge. Prior to AnyLog, Moshe was the CEO... Read More →

Monday June 23, 2025 11:20am - 12:00pm MDT
Bluebird Ballroom 3E

Audience Experience Level Any

11:20am MDT

Leveraging the OPEA Platform and GraphRAG Architectures To Drive Cloud-Native AI Adoption - Stephen Chin, Neo4j & Ezequiel Lanza, Intel

Monday June 23, 2025 11:20am - 12:00pm MDT

Bluebird Ballroom 2C

In the era of cloud-native technologies, businesses are increasingly looking to integrate advanced AI capabilities into their workflows. This session explores how the Open Platform for Enterprise AI (OPEA) offers a transformative, open-source framework designed to accelerate the adoption of Generative AI (GenAI) across industries. Attendees will learn about the key benefits of adopting an open-source GenAI platform, including flexibility, community-driven improvements, and cost efficiency, as well as the advantages of deploying AI models in a cloud-native environment. We will also dive deep into the emerging GraphRAG (Graph-based Retrieval-Augmented Generation) architecture, demonstrating how it enhances the accuracy and explainability of AI-driven responses. Join us for this session to explore how the open-source OPEA platform and GraphRAG architectures are poised to redefine enterprise AI and to drive smarter, more reliable cloud-native AI applications.

Speakers

Stephen Chin

VP of Developer Relations, Neo4j

Stephen Chin is VP of Developer Relations at Neo4j, conference chair of the LF AI & Data Foundation, and author of numerous titles including the upcoming GraphRAG: The Definitive Guide for O'Reilly. He has given keynotes and main stage talks at innumerable AI and developer conferences... Read More →

Ezequiel Lanza

LF AI & Data TAC Board/Chairperson | Open Source AI Evangelist at Intel, Intel

Passionate about helping people discover the exciting world of artificial intelligence, Ezequiel is a frequent AI conference presenter and the creator of use cases, tutorials, and guides that help developers adopt open source AI tools.

Monday June 23, 2025 11:20am - 12:00pm MDT
Bluebird Ballroom 2C

Audience Experience Level Intermediate

1:30pm MDT

The Generative AI Commons: Unlocking Potential Through Collaboration and Transparency - Arnaud Le Hors, IBM

Monday June 23, 2025 1:30pm - 2:10pm MDT

Bluebird Ballroom 2C

In the rapidly evolving landscape of Generative AI, it is becoming clearer every day that open strategies are key drivers of innovation and widespread adoption. This session will present the Generative AI Commons, its activities, and deliverables to date, including the Model Openness Framework (MOF) and the Responsible Generative AI Framework,

LF AI & Data initiative dedicated to fostering the democratization, advancement and adoption of efficient, secure, reliable, and ethical Generative AI open source innovations through neutral governance, open and transparent collaboration and education.

Speakers

Arnaud Le Hors

Senior Technical Staff Member Open Technologies, IBM

Arnaud Le Hors is Senior Technical Staff Member of Open Technologies at IBM, primarily focusing on Open Source security and AI. He has been working on standards and open source for over 30 years. Arnaud was editor of several key web specifications including HTML and DOM and was a... Read More →

Monday June 23, 2025 1:30pm - 2:10pm MDT
Bluebird Ballroom 2C

Audience Experience Level Any

1:30pm MDT

Tutorial: From Planning To Production-Ready RAG With OPEA - Andreas Kollegger, Neo4j; Ezequiel Lanza, Rakshith Krishnappa & Katherine Druckman, Intel

Monday June 23, 2025 1:30pm - 3:05pm MDT

Bluebird Ballroom 3E

Enterprises struggle to integrate fragmented generative AI (GenAI) technologies. Due to its rapid evolution and diverse implementations, even top LLMs hallucinate when answering Kubernetes-related questions.

The Open Platform for Enterprise AI (OPEA), a Linux Foundation project, accelerates GenAI adoption with an orchestration framework that composes microservices via customizable blueprints to deploy or create GenAI applications.

In this hands-on tutorial, developers will deploy advanced retrieval-augmented generation (RAG) applications using Kubernetes. They’ll explore how OPEA orchestrates AI workloads via a microservices architecture and build a production-ready RAG chatbot. Attendees will go beyond deployment, enhancing vector search with knowledge graphs, customizing OPEA components, and scaling AI solutions efficiently on Kubernetes—all while integrating AI agents for more intelligent automation.

Speakers

Rakshith Krishnappa

AI Solutions Software Engineer, Intel

Ezequiel Lanza

LF AI & Data TAC Board/Chairperson | Open Source AI Evangelist at Intel, Intel

Passionate about helping people discover the exciting world of artificial intelligence, Ezequiel is a frequent AI conference presenter and the creator of use cases, tutorials, and guides that help developers adopt open source AI tools.

Katherine Druckman

Open Source Evangelist, Intel

Katherine Druckman is an Open Source Evangelist at Intel, where she enjoys sharing her passion for a variety of open source topics. She currently combines her enthusiasm for software security and emerging AI technology as the OPEA Security Working Group Lead and Co-Chair of the OpenSSF... Read More →

Andreas Kollegger

Senior Developer Advocate, Neo4j

Andreas is a technological humanist. Starting at NASA, Andreas designed systems from scratch to support science missions. Then in Zambia, he built medical informatics systems to apply technology for social good. Now with Neo4j, he is democratizing graph databases to validate and extend... Read More →

Monday June 23, 2025 1:30pm - 3:05pm MDT
Bluebird Ballroom 3E

Audience Experience Level Intermediate

2:25pm MDT

Beaconforge.org, Open Agentic AI for Hallucination Mitigation - Diego Gosmar, Voiceinteroperability.ai, LF AI and Data

Monday June 23, 2025 2:25pm - 3:05pm MDT

Bluebird Ballroom 2C

This talk will describe the emerging standard and the Beaconforge.org framework, which is being developed by the Voiceinteroperability.ai Project of the LFAI and Data Foundation, to enable conversational assistants to interact with each other. Beaconforge.org is based on a standardized set of novel agentic inter-assistant messages that utilize NLP (Natural Language Processing) Universal APIs.

We will also explore how Agentic AI frameworks like Beaconforge.org can help mitigate hallucination effects in generative AI agents. This will be demonstrated through an empirical experiment involving hundreds of diverse prompts, a chain of multiple agents, and four novel KPIs used to evaluate hallucination score mitigation.

More information about the project can be found in the following papers:
Diego Gosmar et al., 2025. Hallucination Mitigation using Agentic AI Natural Language-Based Frameworks, https://arxiv.org/abs/2501.13946

Diego Gosmar et al., 2024. Conversational AI Multi-Agent Interoperability, Universal Open APIs for Agentic Natural Language Multimodal Communications, https://arxiv.org/abs/2407.19438

Speakers

Diego Gosmar

Principal AI Advisor, Voiceinteroperability.ai, LF AI and Data

Diego Gosmar serves as Chief AI Officer specializing in Artificial Intelligence, with particular focus on Generative Conversational AI, Natural Language Processing (NLP), AI Agent interoperability, Sustainable and Ethical Conversational AI.Diego is member of the Open Voice Interoperability... Read More →

DG BForge OSS 2025 Official pdf

Monday June 23, 2025 2:25pm - 3:05pm MDT
Bluebird Ballroom 2C

Audience Experience Level Intermediate
Session Slides Yes

2:25pm MDT

Sponsored Session: Navigating Open Source AI with Guardrails - Stacey Miller, SUSE & Mandanna Appanderanda, Infosys

Monday June 23, 2025 2:25pm - 3:05pm MDT

Bluebird Ballroom 3H

If the world of open source is moving fast; the world of open source AI is moving even faster. The swift expansion of open source AI projects creates a challenge. Although these readily available technologies are attractive, they can also bring security risks and ethical output dilemmas Guardrails technology is crucial for effectively controlling model behavior in this dynamic environment. Join SUSE and Infosys to explore how to implement guardrails while staying true to open source values.

In order to facilitate networking and business relationships at the event, you may choose to visit a third party's booth or access sponsored content. You are never required to visit third party booths or to access sponsored content. When visiting a booth or participating in sponsored activities, the third party will receive some of your registration data. This data includes your first name, last name, title, company, address, email, standard demographics questions (i.e. job function, industry), consenting to receipt and use of such data by the third-party recipients, which will be subject to their own privacy policies.

Speakers

Mandanna Appanderanda

Head of Infosys Responsible AI, Americas, Infosys

Mandanna Appanderanda is the Head of Infosys Responsible AI for the Americas. He leads consulting services for AI Governance, Strategy, Safety, and Security for Infosys customers. With over 16 years of experience in various roles, Mandanna believes AI adoption increases with awareness... Read More →

Stacey Miller

Principal Product Marketing Manager, SUSE

Stacey Miller is the Principal Product Marketing Manager for SUSE AI. Stacey is responsible for helping to determine strategy and offerings and communicating their value to SUSE partners and customers. With more than 25 years of marketing with both large and small software companies... Read More →

Monday June 23, 2025 2:25pm - 3:05pm MDT
Bluebird Ballroom 3H

Open AI + Data

3:35pm MDT

The MODERN Modern Data Stack: Building an Open Distributed Data Warehouse Beyond Data Lakes - David Aronchick, Expanso

Monday June 23, 2025 3:35pm - 4:15pm MDT

Bluebird Ballroom 3E

Organizations face a critical challenge: data is growing exponentially across distributed locations, but traditional centralized processing approaches are becoming unsustainable. With the majority of enterprise data going unused, companies struggle with massive transfer costs, compliance issues, and network reliability problems when moving data to centralized infrastructure.

This talk introduces a paradigm shift: bringing compute to where data lives. Using the open-source Bacalhau project, we'll demonstrate how to:

- Deploy distributed processing jobs across clouds, edge devices, and on-premises infrastructure
- Reduce data movement costs while maintaining centralized control
- Ensure compliance by processing sensitive data in place
- Enable real-time analytics at the edge

Through real-world examples, including an energy company managing 15,000 microgrids and cities processing camera feeds, attendees will learn practical patterns for modernizing their data infrastructure. We'll explore architectural patterns, security considerations, and best practices for implementing compute-over-data architectures.

Speakers

David Aronchick

CEO, Expanso

David Aronchick is CEO of Expanso, the distributed computing company built on Bacalhau ([https://bacalhau.org](https://bacalhau.org/)). Previously, he led Compute over Data at Protocol Labs, Open Source Machine Learning Strategy at Azure, was a product management for Kubernetes... Read More →

Monday June 23, 2025 3:35pm - 4:15pm MDT
Bluebird Ballroom 3E

Audience Experience Level Intermediate

3:35pm MDT

What’s New in Valkey - Madelyn Olson, Amazon & Ping Xie, Google

Monday June 23, 2025 3:35pm - 4:15pm MDT

Bluebird Ballroom 2C

Valkey is the leading open-source database for building high performance applications that are compatible with the Redis API. It natively integrates with many popular frameworks and is commonly used for a wide range of applications such as caching, session storage, streaming, and more. In this talk, hear from members of the Valkey technical steering committee discuss some of the exciting new functionality that has been released including the new bloom filter data type, vector similarity search, and a new hash table implementation that reduces memory overhead and improves performance. We'll talk about how these functionalities enable next generation use cases and what's coming in upcoming releases.

Speakers

Madelyn Olson

Software Engineer, Amazon

I work primarily on the open source Redis project and evangelize the importance of open source software development.

Ping Xie

Senior Staff Software Engineer, Google

Ping Xie is a maintainer of Valkey and a Senior Staff Software Engineer at Google, working on GCP Memorystore. As an active contributor to Valkey, Ping focuses on core development, community engagement, and ensuring Valkey remains a reliable and adaptable solution for a wide range... Read More →

Monday June 23, 2025 3:35pm - 4:15pm MDT
Bluebird Ballroom 2C

Audience Experience Level Intermediate

4:30pm MDT

Generative AI Model Data Pre-Training on Kubernetes: A Use Case Study - Anish Asthana, Red Hat & Mohammad Nassar, IBM

Monday June 23, 2025 4:30pm - 5:10pm MDT

Bluebird Ballroom 2C

Large Language Models (LLM) require preprocessing vast amounts of data, a process that can span days due to its complexity and scale, often involving PetaBytes of data. This talk demonstrates how Kubeflow Pipelines (KFP) simplify LLM data processing with flexibility, repeatability, and scalability. These pipelines are being used daily at IBM Research to build indemnified LLMs tailored for enterprise applications.

Different data preparation toolkits are built on Kubernetes, Rust, Slurm, or Spark. How would you choose one for your own LLM experiments or enterprise use cases and why should you consider Kubernetes and KFP?

This talk describes how open source Data Prep Toolkit leverages KFP and KubeRay for scalable pipeline orchestration, e.g. deduplication, content classification, and tokenization.

We share challenges, lessons, and insights from our experience with KFP, highlighting its applicability for diverse LLM tasks, such as data preprocessing, RAG retrieval, and model fine-tuning.

Speakers

Mohammad Nassar

Research Engineer, IBM

Mohammad Nassar, a Cloud Research Engineer at IBM Haifa, specializes in AI-driven data engineering, automation, and hybrid cloud technologies. With an M.Sc. in Computer Science from Technion, his research focused on coding theory and data systems. His work spans AI-powered data preparation... Read More →

Anish Asthana

Engineering Manager, Red Hat

Anish is an engineering manager at Red Hat in the OpenShift AI organization. He is working on making machine learning easier for the wider community by building a platform out with cloud capabilities at the core. Most recently, his interests have been focused on the Distributed Workloads... Read More →

Monday June 23, 2025 4:30pm - 5:10pm MDT
Bluebird Ballroom 2C

Audience Experience Level Intermediate

4:30pm MDT

Universal AI: Execute Your Models Where Your Data (And Users) Are - David Aronchick, Expanso

Monday June 23, 2025 4:30pm - 5:10pm MDT

Bluebird Ballroom 3E

Data is exploding across distributed locations, but centralized processing is increasingly unsustainable. This talk explores "compute over data" architectures that bring ML to your data, unlocking new possibilities through real-world examples.

Speakers

David Aronchick

CEO, Expanso

David Aronchick is CEO of Expanso, the distributed computing company built on Bacalhau ([https://bacalhau.org](https://bacalhau.org/)). Previously, he led Compute over Data at Protocol Labs, Open Source Machine Learning Strategy at Azure, was a product management for Kubernetes... Read More →

Monday June 23, 2025 4:30pm - 5:10pm MDT
Bluebird Ballroom 3E

Audience Experience Level Intermediate

5:30pm MDT

Guarding the LLM Galaxy: Security, Privacy, and Guardrails in the AI Era - Jigyasa Grover, BORDO AI & Rishabh Misra, Attentive Mobile Inc

Monday June 23, 2025 5:30pm - 6:10pm MDT

Bluebird Ballroom 3E

The widespread adoption of Large Language Models (LLMs) like GPT-4, Claude, and Gemini has introduced unprecedented capabilities and equally unprecedented risks. Organizations are increasingly deploying LLMs to handle sensitive tasks, from processing medical records to analyzing financial documents. This talk examines the evolving landscape of LLM security and privacy, combining theoretical foundations with a walkthrough of example implementations.

Through real-world case studies of both attacks and defenses and practical implementation guidance using popular security tools, we'll explore critical vulnerabilities and proven defensive techniques. Special attention will be given to securing fine-tuned and domain-specific LLMs, with live examples using NVIDIA’s NeMo Guardrails, LangChain's security tools, and Microsoft's guidance library.

Speakers

Jigyasa Grover

AI Lead, Data Scientist & ML Engineer • 10x AI & Open Source Award Winner • Google Developer Expert • 'Sculpting Data for ML' Book Author

Jigyasa Grover, a 10-time AI award winner and author of book, Sculpting Data For ML, has expertise in ML engineering at Twitter, Facebook, and Faire. A Google Developer Expert and Women Techmaker Ambassador, she was featured at at Google I/O 2024 on Gemini 1.5 Pro. As a World Economic... Read More →

Rishabh Misra

Staff Machine Learning Engineer, Attentive

Author of the book "Sculpting Data for ML", I am a Staff ML Engineer & Researcher recognized by the US Government for outstanding contribution to ML research. I have extensively published and reviewed research at top AI conferences in NLP (LLMs / GenAI), Deep Learning, and Applied... Read More →

Monday June 23, 2025 5:30pm - 6:10pm MDT
Bluebird Ballroom 3E

Audience Experience Level Intermediate

11:00am MDT

Open AI (Two Words): The Only Path Forward for AI - Matt White, Linux Foundation

Tuesday June 24, 2025 11:00am - 11:40am MDT

Bluebird Ballroom 3E

The exponential growth in artificial intelligence capabilities has been fundamentally driven by open science and collaborative research. From the publication of the "Attention Is All You Need" paper that introduced the Transformer architecture to OpenAI's strategic release of GPT-2, openness has repeatedly catalyzed breakthrough innovations while enabling crucial public discourse around AI's implications.

This talk presents a compelling case for why open source development is not just beneficial but essential for the future of safe and equitable AI. We'll examine how the open-source ecosystem has democratized access to AI technology, enabled transparency and innovation, and fostered a global community of researchers working to ensure AI systems are robust and aligned with human values.

Through concrete examples, we'll demonstrate how open-source initiatives have already begun addressing critical challenges in AI development. The Model Openness Framework has established clear standards for transparency, while the pioneering OpenMDW license has created a legal framework for responsible sharing of AI artifacts.

Speakers

Matt White

Executive Director, PyTorch Foundation. GM of AI., Linux Foundation

Matt White is the Executive Director of the PyTorch Foundation and GM of AI at the Linux Foundation. He is also the Director of the Generative AI Commons. Matt has nearly 30 years of experience in applied research and standards in AI and data in telecom, media and gaming industries... Read More →

Tuesday June 24, 2025 11:00am - 11:40am MDT
Bluebird Ballroom 3E

Audience Experience Level Intermediate

11:55am MDT

The Responsible Generative AI Framework (RGAF) : An Introduction - Jim Schreckengast, HPE

Tuesday June 24, 2025 11:55am - 12:35pm MDT

Bluebird Ballroom 3E

The Responsible Generative AI Framework (RGAF) lays the foundation for ethical and transparent AI development. The RGAF was produced by the Responsible AI Workstream in the Generative AI Commons. The session will provide an introduction to the 9 Dimensions of the RGAF that provide a practical blueprint for building ethical, transparent, and sustainable AI. The session will also refer to the Responsible AI Pathways, a set of strategic directions designed to move from framework to implementation.

You can join the Generative AI Commons here.

Speakers

Jim Schreckengast (Schreck)

Strategist, Hewlett Packard Enterprise

Jim Schreckengast, better known as Schreck, has built an impressive career over 26 years at HP/HPE, where he currently serves as a strategist in the Open Program Office. In this role, he helps engineers and business leaders shape open technology strategies while leading governance... Read More →

Tuesday June 24, 2025 11:55am - 12:35pm MDT
Bluebird Ballroom 3E

Audience Experience Level Beginner

2:10pm MDT

Lightning Talk: Streaming and Processing Edge Vision Data in Real Time - Joyce Lin, Viam

Tuesday June 24, 2025 2:10pm - 2:20pm MDT

Bluebird Ballroom 3E

Edge-based computer vision gives us real-time insights, but getting that data where it needs to go without high bandwidth, lag, or hardware strain is a big challenge. Learn how to build a fast, event-driven vision pipeline using WebRTC for real-time streaming and gRPC for lightweight commands. Whether for security cameras or IoT, you'll gain a practical blueprint for a scalable, open-source vision system to stay responsive at the edge while being cost-effective, adaptable, and cloud-independent.

Speakers

Joyce Lin

Head of developer relations, Viam

Joyce Lin is the head of developer relations at Viam, a robotics platform that connects software with smart machines in the physical world. Based in San Francisco, she is also a Tiktok influencer, dog mom, cat mom, and writer.

Tuesday June 24, 2025 2:10pm - 2:20pm MDT
Bluebird Ballroom 3E

Audience Experience Level Beginner

2:25pm MDT

Lightning Talk: Serving Guardrail Detectors on Vllm - Evaline Ju, IBM

Tuesday June 24, 2025 2:25pm - 2:35pm MDT

Bluebird Ballroom 3E

With the increase in generative AI model use, there is a growing concern of how models can divulge information or generate inappropriate content. This concern is leading to the development of technologies to “guardrail” user interactions with models. Some of these guardrails models are simple classification models, while others like IBM’s Granite Guardian or Meta’s Llama Guard are themselves generative models, able to identify multiple risks. As new models appear, a variety of large language model serving solutions are being developed and optimized. An open-sourced example, vLLM, has become an increasingly popular serving engine.

In this talk I’ll discuss how we built an open-sourced adapter on top of vLLM to serve an API for guardrails models, so that models like Granite Guardian and Llama Guard can be easily applied as guardrails in generative AI workflows.

Speakers

Evaline Ju

Senior Software Engineer, IBM

Evaline is a senior engineer working on the watsonx platform engineering team of IBM Research and based in Denver, Colorado. She currently focuses on building guardrails infrastructure for large language model workflows. Her previous experience includes MLOps for IBM’s cloud ML... Read More →

serving guardrail detectors on vllm pdf

Tuesday June 24, 2025 2:25pm - 2:35pm MDT
Bluebird Ballroom 3E

Audience Experience Level Intermediate

2:40pm MDT

Lightning Talk: Future-Proofing Compliance: Leveraging Knowledge Graphs and AI in Cybersecurity - Zeyno Dodd, Conjectura R&D

Tuesday June 24, 2025 2:40pm - 2:50pm MDT

Bluebird Ballroom 3E

Traditional approaches to cybersecurity compliance are being redefined in an era marked by rapidly evolving cybersecurity threats and stringent compliance requirements. This session explores the innovative integration of Knowledge Graphs (KG) and Retrieval Augmented Generation (RAG) with Generative AI to address the ever-evolving complexities of cybersecurity frameworks like NIST CSF v2.0, NIST 800-171, and CMMC. I will briefly delve into an open-source proof-of-concept demonstrating how these technologies can automate the discovery of compliance relationships and streamline cross-framework assessments. Join me in discovering how we can significantly enhance cybersecurity measures by harnessing open-source tools and AI, reducing the resource burden, and maintaining timely and robust adherence to evolving standards.

Speakers

Zeyno Dodd

R&D Architect, Conjectura R&D

Cloud Solution Architect and Researcher with 25+ years in software development and research. Committed to leveraging AI to address complex real-world challenges with societal impact. Specializes in applying Graph Neural Networks (GNN) within Cloud/Edge/Hybrid Machine Learning frameworks... Read More →

Tuesday June 24, 2025 2:40pm - 2:50pm MDT
Bluebird Ballroom 3E

Audience Experience Level Any

3:05pm MDT

Agents in Action: Advancing Open Source AI Missions - Hema Veeradhi, Red Hat

Tuesday June 24, 2025 3:05pm - 3:45pm MDT

Bluebird Ballroom 3E

AI Agents have become a buzzword in the world of generative AI—but what exactly are agents, and how are they advancing open source AI? Agents are autonomous systems that extend the capabilities of Large Language Models (LLMs) by leveraging external tools and real-time data to perform dynamic actions for solving complex, multi-step tasks. They are at the forefront of transforming open source AI by enabling adaptability, automation and more intelligent decision-making.
In this talk, we’ll explore popular open source agent frameworks like LangChain and Haystack, which are driving rapid development and community-powered innovation in AI agents. Using real-world examples and a live demo, we’ll demonstrate how these frameworks can be leveraged to build a variety of generative AI applications, including RAG, IT operations automation and dynamic, context-aware chatbots.
Attendees will gain a clear insight into why AI agents are at the forefront of innovation and how open source collaboration is driving its evolution. Whether you’re an AI developer, OSPO leader or open source enthusiast, this session will highlight the potential of agents to power the next wave of open source AI missions.

Speakers

Hema Veeradhi

Principal Data Scientist, Red Hat

Hema Veeradhi is a Principal Data Scientist working in the Emerging Technologies team part of the office of the CTO at Red Hat. Her work primarily focuses on implementing innovative open AI and machine learning solutions to help solve business and engineering problems. Hema is a staunch... Read More →

Tuesday June 24, 2025 3:05pm - 3:45pm MDT
Bluebird Ballroom 3E

Audience Experience Level Any

4:20pm MDT

Gotta Cache 'em All: Scaling AI Workloads With Model Caching in a Hybrid Cloud - Rituraj Singh & Jin Dong, Bloomberg

Tuesday June 24, 2025 4:20pm - 5:00pm MDT

Bluebird Ballroom 3E

AI models are evolving rapidly, while also growing exponentially in size and complexity. As AI workloads become larger, it is crucial to address the challenges of rapidly scaling inference services during peak hours and how to ensure optimal GPU utilization for fine-tuning workloads. To tackle this, Bloomberg’s Data Science Platform team has implemented a “Model Cache” feature in the open source KServe project for caching large models on GPUs in a multi-cloud and multi-cluster cloud-native environment.

This talk discusses the challenges faced with hosting large models for inference and fine-tuning purposes, and how model caching can help mitigate some of these challenges by reducing load times during auto-scaling of services, improving resource utilization, and boosting data scientists’ productivity. The talk dives into how Bloomberg integrated KServe’s Model Cache into its AI workloads and built an API on top of Karmada to manage cache federation. AI infrastructure engineers will learn about the profound impact of enabling model caching and how teams can adopt this feature in their own AI infrastructure environment.

Speakers

Rituraj Singh

Software Engineer, Bloomberg LP

Rituraj Singh is a software engineer on Bloomberg’s Data Science Platform engineering team, which is focused on enabling large-scale AI model training on GPUs. Rituraj graduated from Carnegie Mellon University with a master's degree in computer engineering.

Jin Dong

Software Engineer, Bloomberg

Jin Dong is a software engineer at Bloomberg. He works on building an inference platform for machine learning with KServe.

Tuesday June 24, 2025 4:20pm - 5:00pm MDT
Bluebird Ballroom 3E

Audience Experience Level Intermediate

11:00am MDT

Dynamo: Supporting Next-Generation AI Workloads - Olga Andreeva & Ryan McCormick, NVIDIA

Wednesday June 25, 2025 11:00am - 11:40am MDT

Bluebird Ballroom 3F

As Generative AI unfolds its transformative potential, and enterprise needs have shifted toward large-scale, distributed deployments, the requirements for inference serving have fundamentally changed. To meet these new demands, NVIDIA has introduced Dynamo— a high-throughput low-latency inference framework designed for serving generative AI and reasoning models in multi-node distributed environments.

This session provides a technical overview of Dynamo’s architecture, focusing on how its design addresses the core challenges of large-scale, distributed generative AI inference. We will walk through concrete deployment scenarios—including disaggregated serving and dynamic GPU scheduling—and examine how Dynamo manages resource allocation, request routing, and memory efficiency for high-throughput, low-latency inference.

We will also share practical implementation examples and discuss engineering best practices for optimizing workload performance, scalability, and cost using Dynamo. We’ll outline the steps and considerations for deploying Dynamo, highlighting key architectural differences and compatibility factors. By the end of the session, attendees will have a clear understanding of how to deploy and operate Dynamo in production environments to support advanced AI workloads.

Speakers

Olga Andreeva

Senior Software Engineer, NVIDIA

Olga Andreeva is a senior software engineer, specializing in machine learning inferencing. With a PhD in Computer Science from the University of Massachusetts Boston and experience in both academia and industry, Olga specializes in translating cutting-edge ML research into robust... Read More →

Ryan McCormick

Senior Software Engineer, NVIDIA

Ryan McCormick is a senior software engineer working at the intersection of machine learning, systems software and distributed systems at NVIDIA. He is responsible for developing scalable and performant inference solutions, with a current focus on the Triton Inference Server and Triton... Read More →

Wednesday June 25, 2025 11:00am - 11:40am MDT
Bluebird Ballroom 3F

Audience Experience Level Any

11:55am MDT

Fast Inference, Furious Scaling: Leveraging VLLM With KServe - Rafael Vasquez, IBM

Wednesday June 25, 2025 11:55am - 12:35pm MDT

Bluebird Ballroom 3E

In this talk, we will introduce two open-source projects vLLM and KServe and explain how they can be integrated to leverage better performance and scalability for LLMs in production. The session will include a demo showcasing their integration.

vLLM is a high-performance library specifically designed for LLM inference and serving, offering cutting-edge throughput and efficiency through techniques such as PagedAttention, continuous batching, and optimized CUDA kernels, making it ideal for production environments that demand fast, large-scale LLM serving.

KServe is a Kubernetes-based platform designed for scalable model deployment. It provides robust features for managing AI models in production, including autoscaling, monitoring, and model versioning.

By combining vLLM's inference optimizations with KServe's scalability, organizations can deploy LLMs effectively in production environments, ensuring fast, low-latency inference and seamless scaling across cloud platforms.

Speakers

Rafael Vasquez

Open Source Software Developer, IBM

Rafael Vasquez is a software developer on the Open Technology team at IBM. He previously completed an MASc. working on self-driving car research and transitioned from a data scientist role in the retail field to his current role where he continues to grow his passion for MLOps and... Read More →

Wednesday June 25, 2025 11:55am - 12:35pm MDT
Bluebird Ballroom 3E

Audience Experience Level Beginner

11:55am MDT

Harnessing Event-Driven and Multi-Agent Architectures for Complex Workflows in Generative AI System - Mary Grygleski, Callibrity

Wednesday June 25, 2025 11:55am - 12:35pm MDT

Bluebird Ballroom 3F

Generative AI applications, in general, excel in zero-shot and one-shot types of specific tasks. However, we live in a complicated world and we are beginning to see that today’s generative AI systems are simply not well equipped to handle the increased complexity that is found especially in business workflows and transactions. Traditional architectures often fall short in handling the dynamic nature and real-time requirements of these systems. We will also need a way to coordinate multiple components to generate coherent and contextually relevant outputs. Event-driven architectures and multi-agent systems offer a promising solution by enabling real-time processing, decentralized decision-making, and enhanced adaptability.

This presentation proposes an in-depth exploration of how event-driven architectures and multi-agent systems can be leveraged to design and implement complex workflows in generative AI. By combining the real-time responsiveness of event-driven systems with the collaborative intelligence of multi-agent architectures, we can create highly adaptive, efficient, and scalable AI systems. This presentation will delve into the theoretical and practical sides.

Speakers

Mary Grygleski

Director, Emerging Technologies, Callibrity

Mary is a Technical Advocate, Java Champion, and the Director of Emerging Technologies at Callibrity. She started as an engineer in Unix/C, then transitioned to Java around 2000 and has never looked back since then. After 20+ years of being a software engineer and technical architect... Read More →

Wednesday June 25, 2025 11:55am - 12:35pm MDT
Bluebird Ballroom 3F

Audience Experience Level Intermediate

2:10pm MDT

Building Your (Local) LLM Second Brain - Olivia Buzek, IBM

Wednesday June 25, 2025 2:10pm - 2:50pm MDT

Bluebird Ballroom 3E

LLMs are hotter than ever, but most LLM-based solutions available to us require you to use models trained on data with unknown provenance, send your most important data off to corporate-controlled servers, and use prodigious amounts of energy every time you write an email.

What if you could design a “second brain” assistant with OSS technologies, that lives on your laptop?

We’ll walk through the OSS landscape, discussing the nuts and bolts of combining Ollama, LangChain, OpenWebUI, Autogen and Granite models to build a fully local LLM assistant. We’ll also discuss some of the particular complexities involved when your solution involves a local quantized model vs one that’s cloud-hosted.

In this talk, we'll build on the lightning talk to include complexities like:
* how much latency are you dealing with when you're running on a laptop?
* does degradation from working with a 7-8b model reduce effectiveness?
* how do reasoning + multimodal abilities help the assistant task?

Speakers

Olivia Buzek

STSM watsonx.ai - IBM Research, IBM

Olivia has been building machine learning and natural language processing models since before it was cool. She's spent several years at IBM working on opening up Watson tech, around the country and around the world.

Wednesday June 25, 2025 2:10pm - 2:50pm MDT
Bluebird Ballroom 3E

Audience Experience Level Intermediate

3:05pm MDT

AI Pipelines With OPEA: Best Practices for Cloud Native ML Operations - Ezequiel Lanza, Intel & Melissa McKay, DevSecOps & MLOps Author/Speaker

Wednesday June 25, 2025 3:05pm - 3:45pm MDT

Bluebird Ballroom 3E

The Open Platform for Enterprise AI (OPEA) is an open source project intended to assist organizations with the realities of enterprise-grade deployments of GenAI apps. Beginning from scratch is a costly endeavor, and the ability to quickly iterate on a solution and determine its viability for your organization is essential to ensure you are making the best moves forward.

During this session, Ezequiel and Melissa will introduce you to the OPEA platform and how to empower your team to build, deploy, and manage AI pipelines more effectively. Attendees will gain insights into best practices for handling complex AI/ML workloads, automating dependency management, and integrating Kubernetes for efficient resource utilization. With a focus on real-world applications, this talk not only showcases the transformative potential of these tools but also encourages attendees to explore new ways to contribute, innovate, and collaborate in driving the future of AI adoption in enterprise environments.

Speakers

Ezequiel Lanza

LF AI & Data TAC Board/Chairperson | Open Source AI Evangelist at Intel, Intel

Passionate about helping people discover the exciting world of artificial intelligence, Ezequiel is a frequent AI conference presenter and the creator of use cases, tutorials, and guides that help developers adopt open source AI tools.

Melissa McKay

DevSecOps & MLOps Author/Speaker

Melissa is passionate about Java, DevSecOps, CI/CD, and MLOps. She has served on the TSC of the Open Platform for Enterprise AI (OPEA) and the CNCF Governing Board, and regularly shares her knowledge with the community as a developer, speaker, and author. She has been recognized as... Read More →

Wednesday June 25, 2025 3:05pm - 3:45pm MDT
Bluebird Ballroom 3E

Audience Experience Level Intermediate

4:20pm MDT

Accelerating GenAI Innovation: Lessons From Intuit's Agents and Tools Framework - Shradha Ambekar & Conrad De Peuter, Intuit

Wednesday June 25, 2025 4:20pm - 5:00pm MDT

Bluebird Ballroom 3F

Join us to discover how Intuit's GenAI framework is reshaping AI development, enabling swift integration of AI functions across varied business units. We'll focus on a robust framework of reusable agents and tools derived from open-source technologies like LangChain/LangGraph, facilitating diverse functionalities from simple data retrieval to complex processes such as query generation, optimization, pipeline creation and debugging. This framework dramatically reduces the time required for data workers to operationalize data pipelines and supports diverse customer interactions through notebooks, no-code approaches, REST integrations, and Python libraries, catering to a wide range of needs including agent developers and teams in pre-production settings. Our meticulous evaluation process ensures that each tool and agent is rigorously tested against high-performance benchmarks to guarantee reliability and consistency before deployment. By centralizing these AI components, Intuit has not only accelerated development timelines but also upheld a high standard of quality, establishing a benchmark for crafting scalable, effective AI solutions in the dynamically evolving tech landscape.

Speakers

Conrad De Peuter

Senior Staff AI Scientist, Intuit

Conrad De Peuter is a Senior Staff AI Scientist and Manager at Intuit. He has worked on deep learning models in the document understanding space, delivering reusable AI services from a central platform, and most recently as the lead for a portfolio of product-focused R&D projects... Read More →

Shradha Ambekar

Senior Staff Software Engineer, Intuit

Shradha Ambekar is a senior staff software engineer with the Data Platform Group at Intuit. She is an experienced technologist and has led projects working with GENAI, Spark, Kafka, Presto, Athena, Cassandra and Vertica. She has made numerous open-source contributions to presto, calcite... Read More →

Wednesday June 25, 2025 4:20pm - 5:00pm MDT
Bluebird Ballroom 3F

Audience Experience Level Beginner

4:20pm MDT

Scalable and Efficient LLM Serving With the VLLM Production Stack - Junchen Jiang, University of Chicago & Yue Zhu, IBM Research

Wednesday June 25, 2025 4:20pm - 5:00pm MDT

Bluebird Ballroom 3E

Large Language Models (LLMs) are reshaping how we build applications; however, efficiently serving them at scale remains a major challenge.

The vLLM serving engine, historically focused on single-node deployments, is now being extended into a full-stack inference system through our open-source project, **vLLM Production Stack**. This extension enables any organization to deploy vLLM at scale with high reliability, high throughput, and low latency.
Code: https://github.com/vllm-project/production-stack

At a high level, the vLLM Production Stack project allows users to easily deploy to their Kubernetes cluster through a single command. vLLM Production Stack's optimizations include KV cache sharing to speed up inference (https://github.com/LMCache/LMCache), prefix-aware routing that directs inference queries to vLLM instances holding the corresponding KV caches, and robust observability features for monitoring engine status and autoscaling.

Attendees will discover best practices and see real-time demonstrations of how these optimizations work together to enhance LLM inference performance.

Speakers

Junchen Jiang

Assistant Professor, University of Chicago

Junchen Jiang is an Assistant Professor of CS at the University of Chicago. His research pioneers new approaches to LLM inference systems (https://github.com/vllm-project/production-stack and https://github.com/LMCache/LMCache). He received his Ph.D. from CMU in 2017 and his bachelor’s... Read More →

Yue Zhu

Staff Research Scientist, IBM Research

Yue Zhu is a Staff Research Scientist specializing in foundation model systems and distributed storage systems. Yue obtained a Ph.D. in Computer Science from Florida State University in 2021 and has consistently contribute to sustainability for foundation models and scalable and efficient... Read More →

Wednesday June 25, 2025 4:20pm - 5:00pm MDT
Bluebird Ballroom 3E

Audience Experience Level Intermediate