Convert A Shapefile To GeoParquet

This notebook converts the supraglacial-lakes shapefile archive into GeoParquet. GeoParquet keeps geometry and CRS metadata with the table and is much easier to read from object storage than a multi-file shapefile.

from pathlib import Path


def find_repo_root(start: Path = Path.cwd()) -> Path:
    """Find the repository root when the kernel starts in a subfolder."""
    start = start.resolve()
    for path in (start, *start.parents):
        if (path / ".git").exists() or (path / "downloaded_data").exists():
            return path
    return start


REPO_ROOT = find_repo_root()
DATA_DIR = REPO_ROOT / "downloaded_data"
DATA_DIR.mkdir(exist_ok=True)

import zipfile
import geopandas as gpd

ZIP_PATH = DATA_DIR / "WAIS_Max_Extent.zip"
TARGET_FILE = "WAIS_Max_Extent/WAIS_Jan_2017_Polygons.shp"
PARQUET_FILE = DATA_DIR / "WAIS_Jan_2017_Polygons.parquet"

Read The Shapefile Directly From The ZIP Archive¶

uri = f"zip://{ZIP_PATH}!{TARGET_FILE}"
gdf = gpd.read_file(uri)

gdf.head()

Write GeoParquet And Read It Back¶

Schema version 1.1.0 is preferred when the installed GeoPandas/pyarrow stack supports it. The fallback keeps the notebook runnable on older environments.

# To get effective geo parquet you have to use atleast schema=1.1.0 + sort the data spatially
# Infact if you have more  than 2gbs worth of data its a good idea to partition it spatially, not just sort it
# more info on best practices here: - https://geoparquet.io/
gdf.sort_values('geometry',  inplace=True, ignore_index=True)

gdf.to_parquet(
    PARQUET_FILE,
    engine='pyarrow',
    compression='zstd',
    write_covering_bbox=True, 
    schema_version='1.1.0'
)