IT&I – Information Technology & Interoperability

The OSDF Cookbook: Reproducible Recipes for Streaming Scientific Data

Presented by: Harsha Hampapura, National Center for Atmospheric Research

Abstract: Have you ever wondered why streaming a movie on Netflix is effortless, yet accessing large-scale scientific data can feel so cumbersome? When you watch a movie online, you don’t need to download the entire file in advance — you stream it, thanks to Content Distribution Networks (CDNs) that cache popular content close to users. This approach minimizes latency, improves reliability, and enables near-instant access. Now imagine bringing the same streaming efficiency to science. The Open Science Data Federation (OSDF)—an NSF-funded cyberinfrastructure—makes this possible by providing a CDN for scientific data. OSDF caches data from major science facilities and open repositories at strategic locations, enabling researchers to stream only the data they need, when they need it. In this talk, we introduce the OSDF Cookbook, a growing collection of example workflows that demonstrate how scientists can leverage OSDF for distributed, scalable and reproducible data analysis.

Upcoming

Check out www.esipfed.org/collaboration-areas/iti-committee for more info on upcoming IT&I events.

• Stay in the loop for all ESIP events: esipfed.org/subscribe
• Learn more about all ESIP Collaboration Areas: esipfed.org/collaborate