These notebooks show some of the conversion patterns used in the hackathon datasets. The focus is on file layout and metadata, not on deciding the scientifically correct reprojection or resampling method for every variable. Those choices remain dataset-specific.
The examples cover:
| Format | Best for | Why it matters in object storage |
|---|---|---|
| Zarr | Multidimensional arrays such as time/depth/y/x cubes | Chunked reads let analysis load only the pieces it needs. |
| Cloud Optimized GeoTIFF (COG) | Single rasters or raster stacks exposed as files | Internal tiling and overviews enable efficient range reads and quicklooks. |
| GeoParquet | Vector geometries and tabular point data | Columnar storage, embedded geospatial metadata, and fast filtering. |
Metadata¶
For example datatype we also show how we genereated its associated metadata. Rich metadata ensures that the datasets are more accessible - https://esa-earthcode.github.io/documentation/Community and Best Practices/FAIR and Open Science Best Practices/Data.
All the metadata generated for the datasets and hosted on the Open Science Catalog is in STAC format. The SpatioTemporal Asset Catalog (STAC) specification was designed to establish a standard, unified language to talk about geospatial data, allowing it to be more easily searchable and queryable. For more information and a introduction to STAC, see https://
Shared Local Data Directory¶
The conversion notebooks write examples under downloaded_data/, which is ignored by Git. The cell below downloads the small set of source files used by the format tutorials.
download_urls = [
"https://data.catds.fr/cecsm/Cryo_products/IceSheet_InternalTemperature/SM_TEST_MIR_ITUDP4_20130101T000000_20141231T000000_200_001_0.nc",
"https://zenodo.org/records/5642755/files/WAIS_Max_Extent.zip",
"https://s3.waw4-1.cloudferro.com/EarthCODE/OSCAssets/Polar_Ice_Shelves/antarctic-ice-shelf-melt-rates/basal_melt_map_racmo_firn_air_corrected.tif",
"https://s3.waw4-1.cloudferro.com/EarthCODE/OSCAssets/polar_cube_datasets/albatross/datacollection_ALBATROSS_tidal_amplitude_phase_tide_gauges.nc",
]
for url in download_urls:
!wget -P ../downloaded_data/ {url}--2026-06-23 17:39:18-- https://data.catds.fr/cecsm/Cryo_products/IceSheet_InternalTemperature/SM_TEST_MIR_ITUDP4_20130101T000000_20141231T000000_200_001_0.nc
Resolving data.catds.fr (data.catds.fr)... 134.246.232.82
Connecting to data.catds.fr (data.catds.fr)|134.246.232.82|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 17142684 (16M) [application/x-netcdf]
Saving to: ‘../downloaded_data/SM_TEST_MIR_ITUDP4_20130101T000000_20141231T000000_200_001_0.nc’
SM_TEST_MIR_ITUDP4_ 100%[===================>] 16.35M 3.40MB/s in 4.3s
2026-06-23 17:39:23 (3.80 MB/s) - ‘../downloaded_data/SM_TEST_MIR_ITUDP4_20130101T000000_20141231T000000_200_001_0.nc’ saved [17142684/17142684]
--2026-06-23 17:39:23-- https://zenodo.org/records/5642755/files/WAIS_Max_Extent.zip
Resolving zenodo.org (zenodo.org)... 188.185.48.75, 137.138.153.219, 188.184.98.114, ...
Connecting to zenodo.org (zenodo.org)|188.185.48.75|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 17472257 (17M) [application/octet-stream]
Saving to: ‘../downloaded_data/WAIS_Max_Extent.zip’
WAIS_Max_Extent.zip 100%[===================>] 16.66M 1.96MB/s in 8.5s
2026-06-23 17:39:32 (1.97 MB/s) - ‘../downloaded_data/WAIS_Max_Extent.zip’ saved [17472257/17472257]
--2026-06-23 17:39:32-- https://s3.waw4-1.cloudferro.com/EarthCODE/OSCAssets/Polar_Ice_Shelves/antarctic-ice-shelf-melt-rates/basal_melt_map_racmo_firn_air_corrected.tif
Resolving s3.waw4-1.cloudferro.com (s3.waw4-1.cloudferro.com)... 194.146.52.75
Connecting to s3.waw4-1.cloudferro.com (s3.waw4-1.cloudferro.com)|194.146.52.75|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 157567824 (150M) [image/tiff]
Saving to: ‘../downloaded_data/basal_melt_map_racmo_firn_air_corrected.tif’
basal_melt_map_racm 100%[===================>] 150.27M 9.89MB/s in 18s
2026-06-23 17:39:50 (8.31 MB/s) - ‘../downloaded_data/basal_melt_map_racmo_firn_air_corrected.tif’ saved [157567824/157567824]
--2026-06-23 17:39:50-- https://s3.waw4-1.cloudferro.com/EarthCODE/OSCAssets/polar_cube_datasets/albatross/datacollection_ALBATROSS_tidal_amplitude_phase_tide_gauges.nc
Resolving s3.waw4-1.cloudferro.com (s3.waw4-1.cloudferro.com)... 194.146.52.75
Connecting to s3.waw4-1.cloudferro.com (s3.waw4-1.cloudferro.com)|194.146.52.75|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 16893 (16K) [application/x-netcdf]
Saving to: ‘../downloaded_data/datacollection_ALBATROSS_tidal_amplitude_phase_tide_gauges.nc’
datacollection_ALBA 100%[===================>] 16.50K --.-KB/s in 0s
2026-06-23 17:39:51 (252 MB/s) - ‘../downloaded_data/datacollection_ALBATROSS_tidal_amplitude_phase_tide_gauges.nc’ saved [16893/16893]