BONUS: Zarr as HDF5 Cloud Format? – Aleksandar Jelenak, The HDF Group

The HDF Group's Call the Doctor

Jan 2 2024 • 24 mins

While we're off for this week, please enjoy this audio recording from the 2023 HDF5 Users Group meeting held in Ohio.

This session was presented by Aleksandar Jelenak, The HDF Group.

Zarr is a fairly recent format for multidimensional data arrays specifically targeting storage systems with key-value interface. Some scientific communities interested in implementing scalable cloud-native data analysis are considering Zarr as their chosen data format because of its straightforward implementation in cloud object stores. HDF Group had developed its own cloud-native HDF5 format, called HSDS schema, about the same time as Zarr. Only HDF Group’s developed software, HSDS, currently creates data in the HSDS schema. Since both Zarr and HSDS schema share the same design approach, it would be worthwhile to consider whether Zarr could serve as the cloud-native HDF5 format. The currently developed Zarr version 3 specification introduces the concept of extensions as a way to add more storage features. The goal of this session is to discuss pros and cons of using Zarr v3 to formulate a new cloud-native HDF5 format. Some technical information will be provided with aim to open up discussion among all attendees.

If you'd like, you can watch this session online.

Call the Doctor is a series of weekly, unscripted, live events! The HDF Group’s staff members will answer attendee questions and, for example, go over the previous week’s HDF Forum posts. The HDF Clinics are free sessions intended to help users tackle real-world HDF problems from a common cold to severe headaches and offer relief where that’s possible. As time permits, we will include how-tos, offer advice on tool usage, review your code samples, teach you survival in the documentation jungle, and discuss what’s new or just around the corner in the land of HDF.

Join us every Tuesday at 12:20 p.m. central (US/Canada.) on Zoom!

You Might Like

The Daily
The Daily
The New York Times
The Dan Bongino Show
The Dan Bongino Show
Cumulus Podcast Network | Dan Bongino
WSJ What’s News
WSJ What’s News
The Wall Street Journal
Serial
Serial
Serial Productions & The New York Times
Pod Save America
Pod Save America
Crooked Media
Mark Levin Podcast
Mark Levin Podcast
Cumulus Podcast Network
The Glenn Beck Program
The Glenn Beck Program
Blaze Podcast Network
The Rachel Maddow Show
The Rachel Maddow Show
Rachel Maddow, MSNBC
Morning Joe
Morning Joe
Joe Scarborough and Mika Brzezinski, MSNBC
The Ben Shapiro Show
The Ben Shapiro Show
The Daily Wire
WSJ Your Money Briefing
WSJ Your Money Briefing
The Wall Street Journal
Morning Wire
Morning Wire
The Daily Wire
The Matt Walsh Show
The Matt Walsh Show
The Daily Wire