2.2 Product files - self-hosted

To add your data to the Open Science Catalog, you have to generate a STAC items that describes your files and associated documentation.

We have many tutorials available from the EarthCODE Portal and executable from a designated workspace. The tutorials examples how to generate the STAC Items from most commonly used data formats like: netcdf, tiff and zarr files. Note, that the code in the examples/tutorials and below does not generalise fully. You will have to tailor the code to your data.

2.1.1 Generating STAC items

Below you can see an example of a more manual approach to creating STAC items and adding them to an existing OSC product (created and saved in the previous stage.) You can fill the information for a single file item and create the STAC item this way for each file. If you have multiple files, you will have to update the code accordingly.

We can support you through this all stages of this process, just contact us or post in the FORUM!

from earthcode.git_add import save_item_to_product_collection
from earthcode.validator import validate_catalog
from pathlib import Path
from earthcode.metadata_input_definitions import ItemMetadata
from earthcode.static import create_item
import json
import shapely
import pandas as pd

catalog_root = str(Path("../open-science-catalog-metadata").resolve())

product_id = ""

# Define the relevant links to complete your data description.
access_link = ""
documentation_link = ""
license_link = ""
item_link = ""
item_title = ""

# Item metadata (for adding one item to the product collection)
item_id = ""
item_bbox = [-180.0, -90.0, 180.0, 90.0]
item_datetime = ""
item_license = ""
item_description = ""
item_data_url = ""
item_data_mime_type = ""
item_data_title = ""
item_extra_fields = {}

catalog_root = Path(catalog_root)

## generate a STAC item associated with a specific collection
geometry = json.loads(json.dumps(shapely.box(*item_bbox).__geo_interface__))
item_metadata = ItemMetadata(
    itemid=item_id,
    geometry=geometry,
    data_time=pd.to_datetime(item_datetime),
    bbox=item_bbox,
    product_id=product_id,
    license=item_license,
    description=item_description,
    data_url=item_data_url,
    data_mime_type=item_data_mime_type,
    data_title=item_data_title,
    extra_fields=item_extra_fields,
)
item = create_item(item_metadata)

2.1.2 Adding STAC items to existing OSC product collections

If the files are remote, and you’ve generated the items use the below function to read all items and connect them to the product collection.

# save item to collection and add backlinks
save_item_to_product_collection(item, product_id, catalog_root)

Run validation to make sure everything works.

## run validation
errors, error_files = validate_catalog(catalog_root)
if errors or error_files:
    raise AssertionError(f"Catalog validation failed. errors={len(errors)} files={len(error_files)}")