Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvements regarding signed URLs #380

Merged
merged 2 commits into from
Apr 15, 2021
Merged

Improvements regarding signed URLs #380

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4724ed5
Improvements for issue #379
m-mohr Apr 14, 2021
85b7377
Feedback from code review
m-mohr Apr 14, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- `GET /credentials/oidc` can provide a set of default client ids for OpenID Connect. [#366](https://github.com/Open-EO/openeo-api/pull/366)
- `experimental` and `deprecated` flags added for file formats, service types, udf runtimes, udf runtime versions, udf runtime libraries and all related parameters and schemas. [#354](https://github.com/Open-EO/openeo-api/issues/354)
- `GET /jobs/{job_id}/logs` and `GET /services/{service_id}/logs`: `time` property added. [#330](https://github.com/Open-EO/openeo-api/issues/330)
- Added error `FileExpired`. [#379](https://github.com/Open-EO/openeo-api/issues/379)
m-mohr marked this conversation as resolved.
Show resolved Hide resolved

### Changed

Expand All @@ -21,6 +22,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Clarified how process exceptions should be used. [#352](https://github.com/Open-EO/openeo-api/issues/352)
- Clarified that billing plans, service names and file formats must be accepted case-insensitive. [#371](https://github.com/Open-EO/openeo-api/issues/371)
- Clarified that the first provider listed at `GET /credentials/oidc` is the default provider for OpenID Connect.
- Clarified that `GET /jobs/{job_id}/results` should always return valid signed URLs and the endpoint can be used to renew the signed URLs. [#379](https://github.com/Open-EO/openeo-api/issues/379)
- Fixed casing of potential endpoints `GET /collections/{collection_id}/items` and `GET /collections/{collection_id}/items/{feature_id}`.

## 1.0.1 - 2020-12-07
Expand Down
8 changes: 8 additions & 0 deletions errors.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,14 @@
"Batch Jobs"
]
},
"LinkExpired": {
m-mohr marked this conversation as resolved.
Show resolved Hide resolved
"description": "The signed URLs for batch job results have expired. Please send a request to `GET /jobs/{job_id}/results` to refresh the links.",
"message": "The links to the batch job results have expired. Please request the results again.",
m-mohr marked this conversation as resolved.
Show resolved Hide resolved
"http": 410,
"tags": [
"Batch Jobs"
]
},
"PaymentRequired": {
"description": "The budget required to fulfil the request is insufficient.",
"message": "The budget required to fulfil the request is not sufficient. A payment is required first.",
Expand Down
13 changes: 7 additions & 6 deletions openapi.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2980,26 +2980,27 @@ paths:
get:
summary: Download results for a completed batch job
operationId: list-results
description: >-
description: |-
Lists signed URLs pointing to the processed files after the batch job
has finished. Back-ends may also point to intermediate results after the
job has stopped due to an error.


The response includes additional metadata. It is a valid
[STAC Item (version 0.9.0)](https://github.com/radiantearth/stac-spec/tree/v0.9.0/item-spec)
[STAC Item](https://github.com/radiantearth/stac-spec/tree/v0.9.0/item-spec)
if it has spatial and temporal references included.


URL signing is a way to protect files from unauthorized access with a
key in the URL instead of HTTP header based authorization. The URL
signing key is similar to a password and its inclusion in the URL allows
to download files using simple GET requests supported by a wide range of
programs, e.g. web browsers or download managers. Back-ends are
responsible to generate the URL signing keys and to manage their
appropriate expiration. The back-end MAY indicate an expiration time by
setting the `expires` property.

setting the `expires` property in the reponse. Requesting this endpoint
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add a recommendation that the expiry time should be order of minutes/hours "to give clients enough time to download all assets, while still mitigating the security risk.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm... I'm not sure how high the security risk is actually. Google uses signed URLs for Docs, Spreadsheets etc. and they seem to not expire at all.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a difference is that the google doc user can stop sharing a doc (invalidate the signed url)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that is what issue #341 has been opened for.

SHOULD always return valid URLs. If the links generated for a previous
request have expired, the signed URLs SHOULD be renewed.
Once the signed URLs have expired, the signed URLs MAY return the openEO
error `LinkExpired`.
m-mohr marked this conversation as resolved.
Show resolved Hide resolved

If processing has not finished yet requests to this endpoint MUST be
rejected with openEO error `JobNotFinished`.
Expand Down