EarthCODE Documentation

Appearance

For Workflows ​

FAIR Workflows Best Practices EarthCODE ​

The research software and algorithms you publish to EarthCODE is referred to as workflows. Workflows may take different forms, including a Jupyter Notebook, OpenEO Graph, CWL, or containerized workflow. A workflow in EarthCODE typically includes:

  • source code, stored on Github
  • environment dependancies
  • an executable specification (e.g. OpenEO Graph, CWL, image).
  • Documentation of methods and code

Each workflow is described using OGC API Records metadata, which, similar to Products captures the workflow definition, its project, related themes, variables and EO missions, related products and other related links.

alt text

Workflow Components
  • Workflow Metadata: The central description, following OGC, STAC and OSC extensions, that ties together all other elements.
  • Projects: The ESA-funded research initiative under which the workflow was created.
  • Documentation: The Papers, handbook and/or documentation that describe the methods, implementation and requeired inputs.
  • Source Code: The repository (GitHub) where the code for the workflow is maintained.
  • Dependencies/Environment: Configuration files (e.g. environment.yml, Dockerfile) specifying libraries, software, and system requirements.
  • Packaged Application: A runnable artifact such as a Jupyter Notebook, CWL workflow, or containerized image.
  • Variables: The scientific or environmental parameters processed or modeled by the workflow.
  • Themes: ESA’s strategic science challenges to which the workflow contributes (e.g. climate, cryosphere).
  • EO Missions: The satellite missions or instruments used as inputs to the workflow.

FAIR Checklist ​

Use this checklist to prepare your Product for publication in EarthCODE.

FAIR EarthCODE Workflows Standards ​

EarthCODE's standards for FAIR workflows follow the principles from Applying the FAIR Principles to computational workflows and Introducing the FAIR Principles for research software.

In short, Findability implies that people and machines can easily discover a workflow and the metadata that goes with it. Accessibility means that you can get to a process and its information via standard protocols. Interoperability indicates that a process can work with other workflows and that workflow components may work with each other by sharing data and/or metadata, which are prescribed by standards. Lastly, Reusability indicates that a workflow may be used (performed) and reused (understood, changed, built upon, or added to other workflows):

Findable (F)Accessible (A)

F1 — Workflows and experiments are assigned globally unique and persistent identifiers (e.g., DOIs)

F1.2 — Versions of workflows are explicitly identified, for example: WorldCereal workflow version: 2 with link to GitHub tagged release.

F2 — Rich metadata describing workflow type, dependencies, environment, using OGC API Records

F3 — Metadata clearly includes the identifier of the workflow it describes.

F4 — Workflows and experiments are searchable and indexed in the ESA Open Science Catalog.

A1 — Workflows are retrievable via standardized protocols (HTTPS, Git, OGC APIs)

A1.1 — Protocols are open and free (HTTPS, GitHub).

A1.2 — Protocols allow for authentication/authorization where needed.

A2 — Metadata remains accessible in the Open Science Catalog even if github repository becomes unavailable

Interoperable (I)

I1 — Workflows use formal, accessible representations (CWL, openEO, Jupyter), e.g. WorldCereal workflow defined as an openEO Process Graph.

I2 — Metadata uses controlled vocabularies (OSC themes, OSC variables, etc..).

I3 — Workflows declare qualified links to related data/products and experiments.

Reusable (R)

R1 — Workflows include detailed attributes (execution environment), e.g. ESA CCI Permafrost notebook declares jupyter:kernel and environment.yml for reproducibility.

R1.1 — Workflows include a clear, accessible license.

R1.2 — Provenance is recorded, linking workflow code, execution, and project. For example WorldCereal record links to Git source repository and to its generated experiment.

R2 — Workflows reference related software and datasets via experiments.

R3 — Workflows align with EO and Earth Science standards (OpenEO, CWL, STAC, OGC API Records).

EarthCODE FAIR Experiment Example ​

The ESA CCI Permafrost workflow is published as an OGC API Record (F2) with rich metadata describing its type (jupyter-notebook), environment (deepesdl-xcube-1.9.1, Python 3.11), theme (cryosphere), and project (DeepESDL). It has a persistent identifier and versioning via its GitHub repository (F1, F1.2), is retrievable through open protocols (A1), and remains indexed in the ESA Open Science Catalog (F4, A2). It is interoperable through standards (Jupyter, OGC API, OSC vocabularies) (I1, I2) and links to related experiments, and the DeepESDL platform (I3). Reusability is ensured by declaring its execution environment and provenance (linked Git, environment.yml, experiment record) (R1, R1.2) and by aligning with EO standards (R3).

Worflow Example OGC API - Records
json
{
  "id": "esa-cci-permafrost",
  "type": "Feature",
  "conformsTo": [
    "http://www.opengis.net/spec/ogcapi-records-1/1.0/req/record-core",
    "https://stac-extensions.github.io/application/v0.1.0/schema.json",
    "https://raw.githubusercontent.com/EOEPCA/metadata-profile/refs/heads/1.0/schemas/application-type-jupyter-notebook"
  ],
  "title": "ESA CCI permafrost",
  "geometry": null,
  "properties": {
    "created": "2025-03-17T14:30:23.650003+00:00",
    "updated": "2025-04-25T14:00:00Z",
    "type": "workflow",
    "title": "ESA CCI permafrost",
    "description": "cube generation workflow for esa-cci-permafrost",
    "application:type": "jupyter-notebook",
    "application:container": true,
    "application:language": "Python",
    "keywords": ["Earth Science"],
    "contacts": [
      {
        "name": "Tejas Morbagal Harish",
        "organization": "Brockmann Consult GmbH",
        "links": [
          {
            "rel": "about",
            "type": "text/html",
            "href": "https://www.brockmann-consult.de/"
          }
        ],
        "roles": ["principal investigator"]
      }
    ],
    "themes": [
      {
        "concepts": [
          {
            "id": "cryosphere"
          }
        ],
        "scheme": "https://github.com/stac-extensions/osc#theme"
      }
    ],
    "formats": [],
    "license": "proprietary",
    "osc:project": "deep-earth-system-data-lab"
  },
  "linkTemplates": [],
  "links": [
    {
      "rel": "root",
      "href": "../../catalog.json",
      "type": "application/json",
      "title": "Open Science Catalog"
    },
    {
      "rel": "parent",
      "href": "../catalog.json",
      "type": "application/json",
      "title": "Workflows"
    },
    {
      "rel": "self",
      "href": "https://esa-earthcode.github.io/open-science-catalog-metadata/workflows/esa-cci-permafrost/record.json",
      "type": "application/json"
    },
    {
      "rel": "related",
      "href": "../../projects/deep-earth-system-data-lab/collection.json",
      "type": "application/json",
      "title": "Project: DeepESDL"
    },
    {
      "rel": "child",
      "href": "../../experiments/esa-cci-permafrost/record.json",
      "type": "application/json",
      "title": "ESA CCI permafrost"
    },
     {
      "rel": "vcs",
      "title": "Git source repository",
      "href": "https://github.com/deepesdl/cube-gen.git",
      "vcs:type": "git",
      "vcs:branch": "main"
    },
    {
      "rel": "related",
      "href": "../../themes/cryosphere/catalog.json",
      "type": "application/json",
      "title": "Theme: Cryosphere"
    },
    {
      "rel": "vcs",
      "title": "Git source repository",
      "href": "https://github.com/deepesdl/cube-gen.git",
      "vcs:type": "git",
      "vcs:branch": "main"
    },
    {
      "rel": "application",
      "title": "Jupyter Notebook: Create CCI Permafrost cube",
      "href": "https://github.com/deepesdl/cube-gen/blob/main/Permafrost/Create-CCI-Permafrost-cube-EarthCODE.ipynb",
      "file:local_path": "Permafrost/Create-CCI-Permafrost-cube-EarthCODE.ipynb",
      "type": "application/x-ipynb+json",
      "application:type": "jupyter-notebook",
      "application:container": true,
      "application:language": "Python",
      "jupyter:kernel": {
        "name": "deepesdl-xcube-1.9.1",
        "pythonVersion": 3.11,
        "envFile": "https://github.com/deepesdl/cube-gen/blob/main/Permafrost/environment.yml"
      }
    },
    {
      "rel": "application-originating-platform",
      "title": "DeepESDL platform",
      "href": "https://deep.earthsystemdatalab.net/",
      "type": "text/html",
      "application:platform_supports": ["jupyter-notebook"],
      "application:preferred_app": "JupyterLab"
    },
    {
      "rel": "related",
      "href": "https://deep.earthsystemdatalab.net/hub/user-redirect/git-pull?repo=https%3A%2F%2Fgithub.com%2Fdeepesdl%2Fcube-gen&urlpath=lab%2Ftree%2Fcube-gen%2FPermafrost%2FCreate-CCI-Permafrost-cube-EarthCODE.ipynb&branch=main",
      "type": "text/html",
      "title": "Open notebook on the DeepESDL platform"
    }
  ]
}

Open Source & Licensing ​

To qualify as Open, your source code must follow the OSI Open Source Definition, allowing anyone to use, modify, and redistribute them under certain conditions.

You should consider the following when publishing a workflow as Open:

  • License is mandatory. Without it, workflows cannot be reused.
  • Others must be able to copy and share the workflow.
  • Adaptation and extension must be permitted.
  • The license applies regardless of packaging of the software

A licence is the permission slip for reuse. In Earth Science, this ensures others can validate, extend, and integrate your work into new experiments and platforms.

Open Source licenses fall into several families: Copyleft (e.g., GPL) enforces “share-alike” obligations and requires derivatives to remain open; Permissive (e.g., MIT, BSD) allows wide reuse with attribution and minimal restrictions; Creative Commons offers building blocks such as attribution, share-alike, non-commercial, and no-derivatives clauses; and Public Domain effectively removes restrictions altogether, though liability questions may remain.

EarthCODE workflows are typically licensed under OSI-approved licenses, most commonly:

LicenceDescription & Typical Use in EarthCODENotes
Apache 2.0permissive, allows reuse, redistribution, and patent protection.Encourages wide adoption.
MITSimple and permissive; common in scientific notebooks and lightweight workflows.Maximum flexibility but no patent protection.
BSD 3-ClauseAcademic-friendly, permissive with attribution and no-endorsement clause.Common in research software.
EUPLEU-backed open source license, ensuring strong copyleft and EU compatibility.Increasingly used in European public-sector projects.

In practice:

For EarthCODE workflows, the default recommendation is Apache 2.0 or MIT for maximum reuse and interoperability.

In OGC API Records or EarthCODE metadata, workflows should declare the license explicitly:

json
{
  "id": "example-workflow",
  "type": "workflow",
  "license": "Apache-2.0",
  "links": [
    { "rel": "license", "href": "https://www.apache.org/licenses/LICENSE-2.0" },
  ]
}

Workflows in EarthCODE should clearly document the input data they require and the outputs they generate. This is typically captured through an associated Experiment, which links workflows to datasets, configurations, and results.

In some cases, input data itself may be a candidate for publication in EarthCODE—for example, smaller datasets such as in-situ measurements, training/labelled data, or other derived collections. Large, publicly available archives (e.g., Sentinel or Landsat) are not re-uploaded but referenced directly.

Validate Before Publishing ​

Before publishing, make sure your workflow executes successfully in the declared environment (e.g., include an environment.yml or list of dependencies). If the workflow processes large datasets, test it first on a small sample to confirm it runs as expected.

Where possible, re-run the workflow and create an Experiment to allow for verification and reproducibility.

DOI Assignment ​

In Development

A Digital Object Identifier (DOI) provides a persistent, citable link to your EarthCODE Workflow, making it reliably referenceable in publications, catalogs, and research outputs. DOIs are essential for making workflows Findable under the FAIR principles.

EarthCODE can mint and assign a DOI to your workflow for free during publishing. It will be recorded in the OGC API Record metadata and shown in the ESA Open Science Catalog. Simply request a DOI in your pull request or by email to earth-code@esa.int.

If your workflow already has a DOI, include it in your metadata (e.g., using the Scientific Citation Extension):

json
"sci:doi": "https://doi.org/10.57780/s3d-83ad619"

ESA – European Space Agency © 2020-2025