Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Invoke SNAP on master/slave pairs store to readable format (Zarr, Geotiff) #1034

Open
EmileSonneveld opened this issue Feb 4, 2025 · 2 comments
Assignees

Comments

@EmileSonneveld
Copy link
Contributor

cloudinsar/s1-workflows#3

@EmileSonneveld
Copy link
Contributor Author

EmileSonneveld commented Feb 6, 2025

Not all PVCs need to be s3 mounted in local tests. This might give better performance on CDSE staging:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: calrissian-input-data
  namespace: calrissian-demo-project
spec:
  accessModes:
    - ReadWriteOnce
    - ReadOnlyMany
  resources:
    requests:
      storage: 1Gi
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: calrissian-tmpout
  namespace: calrissian-demo-project
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: calrissian-output-data
  namespace: calrissian-demo-project
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  storageClassName: csi-s3-calrissian

This might avoid errors like: cwltool.errors.WorkflowException: ../calrissian/input-data/r-25020610512247ea8b-cal-inp-46968e7e.cwl:18:7: Did not find output file with glob pattern: '['output.txt']' (for openEO request r-25020610512247ea8b4c994a5e78a650 )

@EmileSonneveld
Copy link
Contributor Author

Example script on how to select bursts and preprocess with calrissian.

import openeo
import openeo.processes
import openeo.internal

url = "https://openeo-staging.dataspace.copernicus.eu/"
# url = "https://openeo.dev.warsaw.openeo.dataspace.copernicus.eu/"  # needs VPN
# url = "http://127.0.0.1:8080"
connection = openeo.connect(url).authenticate_oidc()

datacube = openeo.rest.datacube.DataCube(
    openeo.rest.datacube.PGNode(
        "_cwl_insar",
        arguments={
            "spatial_extent": {
                "west": 10.751,
                "south": 46.741,
                "east": 10.759,
                "north": 46.749
            },
            "temporal_extent": [
                "2024-08-14",
                "2024-08-26"
            ]
        },
    ),
    connection=connection,
)

datacube.download("cwl_result.json")  # TODO: Export as datacube

EmileSonneveld added a commit that referenced this issue Feb 11, 2025
EmileSonneveld added a commit to cloudinsar/s1-workflows that referenced this issue Feb 11, 2025
EmileSonneveld added a commit to cloudinsar/s1-workflows that referenced this issue Feb 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant