Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Processing Request]: OPERA RTC-S1 for Dist-S1 R&D #39

Open
2 tasks
cmarshak opened this issue Mar 12, 2024 · 6 comments
Open
2 tasks

[Processing Request]: OPERA RTC-S1 for Dist-S1 R&D #39

cmarshak opened this issue Mar 12, 2024 · 6 comments

Comments

@cmarshak
Copy link

Venue

PST

Product

RTC-S1

SAS Version

No response

SDS Version

No response

Input Data

This is a specific request for generating time-series of a given bursts that overlaps 300 sites around the globe.

The relevant table is here: https://github.com/OPERA-Cal-Val/dist-s1-research/blob/dev/marshak/3_dist_sites/slcs_for_processing.csv.zip

In terms of processing, the relevant information in the table is: (s1_slc_id, burst_id) --> (uniquely) JPL burst RTC-S1 product

This amounts to approximately 2 TBs of data assuming there is about ~30 MBs files per burst product generate (though is an overestimate I believe).

Share Results

  • Google Earth Engine (GEE)
  • NASA Distributed Active Archive Centers (DAAC) User Acceptance Testing (UAT)

Additional Notes

If you do not have the capability to limit to specified burst, we will need to talk as this generates orders of magnitude more data.

@cmarshak
Copy link
Author

cmarshak commented Mar 12, 2024

Would like to process a few bursts in the list and see them delivered to UAT before proceeding to the entire list.

This volume is not possible to deliver to UAT - so hoping this can be stored on our public PST clone's s3 bucket. There will be an excess of data generated, but we can work with you to delete the data that is not required for our development.

As noted above, we likely need 1-2 bursts per all the SLCs requested (which can have 20 - 40 bursts per product!). I am taking the highest end of all ranges for the approximate estimates above.

@cmarshak
Copy link
Author

@niarenaw and @LucaCinquini - we have updated the request.

The SLCs span a subset of sites and only go from 2020-10-01 to 2022-12-01 (a little over 2 years).

There should be 8,703 SLCs in https://github.com/OPERA-Cal-Val/dist-s1-research/blob/dev/marshak/3_dist_sites/slcs_for_processing.csv.zip. The request will cost ~$3,050 based on Nick's estimate of $0.35 / SLC.

Let me know if there are any questions.

@niarenaw
Copy link
Contributor

niarenaw commented May 1, 2024

All SLCs processed except the following:
S1A_IW_SLC__1SDV_20221117T060948_20221117T061015_045930_057EE2_89A5 S1A_IW_SLC__1SDV_20221117T153336_20221117T153403_045936_057F16_AD44 S1A_IW_SLC__1SDV_20221117T153516_20221117T153543_045936_057F16_AB2D S1A_IW_SLC__1SDV_20221117T171630_20221117T171657_045937_057F23_824A
Further investigation needed on why these can't be downloaded from ASF.

@cmarshak
Copy link
Author

cmarshak commented May 1, 2024

@niarenaw
Copy link
Contributor

niarenaw commented May 6, 2024

Products should be complete now.

@cmarshak
Copy link
Author

cmarshak commented May 8, 2024

Thanks, Nick!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants