Open Source Summit North America 2025

The exponential growth in artificial intelligence capabilities has been fundamentally driven by open science and collaborative research. From the publication of the "Attention Is All You Need" paper that introduced the Transformer architecture to OpenAI's strategic release of GPT-2, openness has repeatedly catalyzed breakthrough innovations while enabling crucial public discourse around AI's implications.

This talk presents a compelling case for why open source development is not just beneficial but essential for the future of safe and equitable AI. We'll examine how the open-source ecosystem has democratized access to AI technology, enabled transparency and innovation, and fostered a global community of researchers working to ensure AI systems are robust and aligned with human values.

Through concrete examples, we'll demonstrate how open-source initiatives have already begun addressing critical challenges in AI development. The Model Openness Framework has established clear standards for transparency, while the pioneering OpenMDW license has created a legal framework for responsible sharing of AI artifacts.

The Responsible Generative AI Framework (RGAF) lays the foundation for ethical and transparent AI development. The RGAF was produced by the Responsible AI Workstream in the Generative AI Commons. The session will provide an introduction to the 9 Dimensions of the RGAF that provide a practical blueprint for building ethical, transparent, and sustainable AI. The session will also refer to the Responsible AI Pathways, a set of strategic directions designed to move from framework to implementation.

You can join the Generative AI Commons here.

Edge-based computer vision gives us real-time insights, but getting that data where it needs to go without high bandwidth, lag, or hardware strain is a big challenge. Learn how to build a fast, event-driven vision pipeline using WebRTC for real-time streaming and gRPC for lightweight commands. Whether for security cameras or IoT, you'll gain a practical blueprint for a scalable, open-source vision system to stay responsive at the edge while being cost-effective, adaptable, and cloud-independent.

With the increase in generative AI model use, there is a growing concern of how models can divulge information or generate inappropriate content. This concern is leading to the development of technologies to “guardrail” user interactions with models. Some of these guardrails models are simple classification models, while others like IBM’s Granite Guardian or Meta’s Llama Guard are themselves generative models, able to identify multiple risks. As new models appear, a variety of large language model serving solutions are being developed and optimized. An open-sourced example, vLLM, has become an increasingly popular serving engine.

In this talk I’ll discuss how we built an open-sourced adapter on top of vLLM to serve an API for guardrails models, so that models like Granite Guardian and Llama Guard can be easily applied as guardrails in generative AI workflows.

Traditional approaches to cybersecurity compliance are being redefined in an era marked by rapidly evolving cybersecurity threats and stringent compliance requirements. This session explores the innovative integration of Knowledge Graphs (KG) and Retrieval Augmented Generation (RAG) with Generative AI to address the ever-evolving complexities of cybersecurity frameworks like NIST CSF v2.0, NIST 800-171, and CMMC. I will briefly delve into an open-source proof-of-concept demonstrating how these technologies can automate the discovery of compliance relationships and streamline cross-framework assessments. Join me in discovering how we can significantly enhance cybersecurity measures by harnessing open-source tools and AI, reducing the resource burden, and maintaining timely and robust adherence to evolving standards.

AI Agents have become a buzzword in the world of generative AI—but what exactly are agents, and how are they advancing open source AI? Agents are autonomous systems that extend the capabilities of Large Language Models (LLMs) by leveraging external tools and real-time data to perform dynamic actions for solving complex, multi-step tasks. They are at the forefront of transforming open source AI by enabling adaptability, automation and more intelligent decision-making.
In this talk, we’ll explore popular open source agent frameworks like LangChain and Haystack, which are driving rapid development and community-powered innovation in AI agents. Using real-world examples and a live demo, we’ll demonstrate how these frameworks can be leveraged to build a variety of generative AI applications, including RAG, IT operations automation and dynamic, context-aware chatbots.
Attendees will gain a clear insight into why AI agents are at the forefront of innovation and how open source collaboration is driving its evolution. Whether you’re an AI developer, OSPO leader or open source enthusiast, this session will highlight the potential of agents to power the next wave of open source AI missions.

AI models are evolving rapidly, while also growing exponentially in size and complexity. As AI workloads become larger, it is crucial to address the challenges of rapidly scaling inference services during peak hours and how to ensure optimal GPU utilization for fine-tuning workloads. To tackle this, Bloomberg’s Data Science Platform team has implemented a “Model Cache” feature in the open source KServe project for caching large models on GPUs in a multi-cloud and multi-cluster cloud-native environment.

This talk discusses the challenges faced with hosting large models for inference and fine-tuning purposes, and how model caching can help mitigate some of these challenges by reducing load times during auto-scaling of services, improving resource utilization, and boosting data scientists’ productivity. The talk dives into how Bloomberg integrated KServe’s Model Cache into its AI workloads and built an API on top of Karmada to manage cache federation. AI infrastructure engineers will learn about the profound impact of enabling model caching and how teams can adopt this feature in their own AI infrastructure environment.