Large numerical forecast datasets are commonly used for atmospheric research with dataset sizes exceeding several terabytes.Watch part 3 of the interview with adult film star Sara Jay as she talks about the dreaded 'A' word.dtype, chunk size, compression type) is stored inside the zarr dataset too. Other libraries such as xarray, Dask and TensorStore have good interoperability with Zarr. Zarr supports multiple backend stores, your data could also live in a zip file, or even a remote server or S3 bucket.
