Open Source Summit North America 2025

Confidential AI leveraging GPUs can bring AI to the masses without sacrificing the privacy of end users. Individual open source technologies already exist to configure, deploy, and manage confidential TEEs. However, clobbering a multitude of components into a coherent, secure, and efficient solution is challenging with many pitfalls. For example, depending on use cases and involved parties (cloud/model/service owners), attestation and key management methodology can vary drastically. In addition, for TEEs with confidential GPUs, complexity extends to increased load times, affecting services that serve multiple models.

This talk will go through key components and design decisions needed to enable confidential AI. Specifically: i) implications of different trust models on the solution and (ii) performance tradeoff considerations. To concretize the discussion, we will present a detailed end-to-end 'how to', for deploying an inference service on Nvidia H100 GPUs and AMD-based TEE with a focus on protecting the model and the user input. The audience will be able to appreciate why there can be no one size fit all confidential AI solution and understand what design works for them.

Take one single application compiled to WebAssembly and split it into pieces at deployment time. Run these pieces in different Kubernetes deployments, different clouds, or even split across edge and cloud.

This code-forward talk will show how to write an application using Wasm components and a combination of Rust and TypeScript. We'll show how to use the CNCF project Spin for developing apps, and then use Kubernetes, Helm, SpinKube, and other open source tools to deploy this application in multiple locations.

Conceptually, we'll tie this new development pattern to microservice architecture and distributed systems to show how WebAssembly's Component Model is paving the way for a new class of application.

As the fast-paced AI-driven landscape of computing continues to diversify, the importance of multi-arch container images cannot be overstated. Applications are no longer confined to data centers but extend across multiple platforms, devices, and appliances.

Wouldn’t it be great if we could build images for every architecture from just one machine? It would be even more amazing if we could do that without the slowness of emulation! This is where Podman farm comes in. Podman farm is a new feature that allows you to 'farm' out builds to groups of machines you have access to, enabling you to easily build multi-architecture images with a single command. In this talk, we will highlight the challenges of multi-architecture builds and demonstrate how Podman farm addresses them, keeping performance and usability in mind.

Container images that run seamlessly across different architectures ensure consistency, reduce complexity, and accelerate the development cycle. This session will empower attendees to develop on one architecture and deploy confidently on another.

Why do some requests take so much longer than others? A major contributor, memory-related contention between containers, was shown to increase latency by 4-13x. It can be triggered by garbage collection, and existing observability cannot even detect it! Current collectors just show high CPU utilization, and the standard mitigation is to scale out and run at low utilization: expensive, and does not solve the response time problem.

We set out to build a new detector, but found that measuring every few seconds (current practice for collectors) is inadequate. Servers quickly jump between intense resource competition and under-utilization, so averaging over seconds does not show any contention. We needed measurements at millisecond frequency.

This session first examines real-world patterns that trigger interference and surveys methods for detecting memory interference, including findings from Google, Alibaba, and Meta's production environments. We'll then discuss the design of the OSS collector, and how it combines CPU performance counters, eBPF and high-resolution timers to identify noisy neighbors. We close with future directions and opportunities to get involved.

As confidential virtual machines become mainstream in confidential computing, the Arm Confidential Computing Architecture (CCA) was introduced as a key innovation of Arm v9 in 2021. Linaro has been deeply involved in integrating CCA into open-source projects over the past years.
In this presentation, we'll share the progress of our open-source enablement efforts. This includes the current status of fundamental software support and the next-stage plan for projects such as TF - A, Kernel, and Qemu. We'll also talk about container runtime adoption in Kata containers and Confidential containers. For instance, we'll detail the work on supporting CCA in Kata container runtimes with Qemu backend, like in kata-deploy. The support for guest-components and Trustee in Confidential containers will be covered too.
Remote attestation is another crucial aspect that can't be overlooked. To reduce solution fragmentation in open-source projects for production, Arm and Linaro are collaborating on an end-to-end experimental attestation platform using Veraison project components. We'll present a case study from the Confidential Containers project to show the practical adoption of these technologies.