SENTINEL2_L2A with mask_scl_dilation gives empty data

[EmileSonneveld]

An issue raised by Stijn. Requesting this simplified process graph results in an empty data cube.
When giving it a bigger date range, there are correct looking results.

import openeo
import json

connection = openeo.connect("openeo-dev.vito.be").authenticate_oidc()

job_options = {
    "executor-memory": "8G",
    "executor-memoryOverhead": "8G",
    "executor-cores": "4",
}

true = True
false = False
null = None
job = connection.create_job(
    title="Large",
    additional=job_options,
    process_graph={
        "process_graph": {
            "loadcollection1": {
                "arguments": {
                    "bands": [
                        "B04",
                        "B08",
                        "SCL"
                    ],
                    "id": "SENTINEL2_L2A",
                    "properties": {
                        "eo:cloud_cover": {
                            "process_graph": {
                                "lte1": {
                                    "arguments": {
                                        "x": {
                                            "from_parameter": "value"
                                        },
                                        "y": 95
                                    },
                                    "process_id": "lte",
                                    "result": true
                                }
                            }
                        }
                    },
                    "spatial_extent": {
                        "east": 5.080232620239258,
                        "north": 51.234084900561015,
                        "south": 51.20946446493662,
                        "west": 5.034656524658203
                    },
                    "temporal_extent": [
                        # "2021-07-21", # Smaller range gives results 
                        # "2021-08-01"  # Smaller range gives results 
                        "2021-04-12", # Big range gives empty result
                        "2021-10-20"  # Big range gives empty result
                    ]
                },
                "process_id": "load_collection"
            },
            "maskscldilation1": {
                "arguments": {
                    "data": {
                        "from_node": "loadcollection1"
                    },
                    "erosion_kernel_size": 3,
                    "kernel1_size": 17,
                    "kernel2_size": 77,
                    "mask1_values": [2,4,5,6,7],
                    "mask2_values": [3,8,9,10,11],
                    "scl_band_name": "SCL"
                },
                "process_id": "mask_scl_dilation"
            },
            "saveresult1": {
                "arguments": {
                    "data": {
                        "from_node": "maskscldilation1"
                    },
                    "format": "gtiff"
                },
                "process_id": "save_result",
                "result": true
            }
        }
    })
job.start_and_wait()
job.get_results().download_files(job.job_id)

[EmileSonneveld]

There is an error message in the empty case: Error communicating with MapOutputTracker. This could be related. I tried running the task with more memory, but the problem is still there.

[jdries]

This specific error shows up more often and is not necessarily a reason for data dissappearing. (Spark is robust, so if data is lost, it actually fails.)

[EmileSonneveld]

Totally not sure about this, but could this be related to the ephemeral-volumes? That some temporary data gets lost before giving the result to the user?

[EmileSonneveld]

This might also be the issue. When the code falls back or not to sentinelhub:

[jdries]

Maybe the shorter timeframe does work simply because it gets processed on terrascope?

[jdries]

I didn't know that the user was setting custom params, but without them, I do get results...

[EmileSonneveld]

The custom params are indeed not needed here. I added them to experiment with the memory, but finally it did not have effect Sent from Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ From: Jeroen Dries ***@***.***> Sent: Friday, March 31, 2023 9:33:15 PM To: Open-EO/openeo-geopyspark-driver ***@***.***> Cc: Emile ***@***.***>; Author ***@***.***> Subject: Re: [Open-EO/openeo-geopyspark-driver] SENTINEL2_L2A with mask_scl_dilation gives empty data (Issue #385) I didn't know that the user was setting custom params, but without them, I do get results... — Reply to this email directly, view it on GitHub<#385 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AARREOSVMRUEYSNZF5OFB6TW64WPXANCNFSM6AAAAAAWOYJHIA>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[jdries]

It's mostly the kernel1 size and erosion kernel size that trigger the issue.
For fast debuggin:

• set max cloud cover to a lower value
• use synchronous jobs
• reduce to 2 bands (e.g. only B04)

[jdries]

Sentinelhub and creodias load sentinel-2 data as unsigned short, while terrascope used signed short.
The scl masking process was depending on being able to have negative values at some point (specific to new erosion feature), which did not work well with unsigned type.