Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wait for CreateMPU before returning from put_object #1192

Merged
merged 6 commits into from
Dec 18, 2024

Conversation

passaro
Copy link
Contributor

@passaro passaro commented Dec 9, 2024

S3CrtClient::put_object was originally implemented so that it would complete immediately and return a PutObjectRequest implementation. Any error from the S3 request would only be returned on calling write or complete on the PutObjectRequest. With this change, we modify put_object to await for the initial CreateMultipartUpload request to complete and only then either return a PutObjectRequest or propagate the error from the request. This is analogous to what done for get_object in #1171 and addresses an issue where errors were not propagated correctly (#1007).

At the file handle level, however, we still want the open operation to complete quickly, without waiting for CreateMultipartUpload to complete. In order to preserve the previous behavior, upload::atomic was adapted to spawn a concurrent task in the background when calling put_object.

Does this change impact existing behavior?

Yes.

Does this change need a changelog entry?

Yes, for mountpoint-s3-client. No user-visible changes in mountpoint-s3.


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and I agree to the terms of the Developer Certificate of Origin (DCO).

@passaro passaro requested a deployment to PR integration tests December 9, 2024 13:48 — with GitHub Actions Waiting
@passaro passaro requested a deployment to PR integration tests December 9, 2024 13:48 — with GitHub Actions Waiting
@passaro passaro requested a deployment to PR integration tests December 9, 2024 13:48 — with GitHub Actions Waiting
@passaro passaro requested a deployment to PR integration tests December 9, 2024 13:48 — with GitHub Actions Waiting
@passaro passaro requested a deployment to PR integration tests December 9, 2024 13:48 — with GitHub Actions Waiting
@passaro passaro requested a deployment to PR integration tests December 9, 2024 13:48 — with GitHub Actions Waiting
@passaro passaro requested a deployment to PR integration tests December 9, 2024 13:48 — with GitHub Actions Waiting
@passaro passaro requested a deployment to PR integration tests December 9, 2024 13:53 — with GitHub Actions Waiting
@passaro passaro requested a deployment to PR integration tests December 9, 2024 13:53 — with GitHub Actions Waiting
@passaro passaro requested a deployment to PR integration tests December 9, 2024 13:53 — with GitHub Actions Waiting
@passaro passaro requested a deployment to PR integration tests December 9, 2024 13:53 — with GitHub Actions Waiting
@passaro passaro requested a deployment to PR integration tests December 9, 2024 13:53 — with GitHub Actions Waiting
@passaro passaro requested a deployment to PR integration tests December 9, 2024 13:53 — with GitHub Actions Waiting
@passaro passaro requested a deployment to PR integration tests December 9, 2024 13:53 — with GitHub Actions Waiting
@passaro passaro temporarily deployed to PR integration tests December 10, 2024 10:46 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests December 10, 2024 10:46 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests December 10, 2024 10:46 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests December 10, 2024 10:46 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests December 10, 2024 10:46 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests December 10, 2024 10:46 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests December 10, 2024 10:46 — with GitHub Actions Inactive
@passaro passaro marked this pull request as ready for review December 10, 2024 11:23
Signed-off-by: Alessandro Passaro <[email protected]>
Signed-off-by: Alessandro Passaro <[email protected]>
Signed-off-by: Alessandro Passaro <[email protected]>
@passaro passaro temporarily deployed to PR integration tests December 12, 2024 08:17 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests December 12, 2024 08:17 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests December 12, 2024 08:17 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests December 16, 2024 15:18 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests December 16, 2024 15:18 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests December 16, 2024 15:18 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests December 16, 2024 15:18 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests December 16, 2024 15:18 — with GitHub Actions Inactive
@passaro passaro requested a review from monthonk December 16, 2024 16:52
Copy link
Contributor

@monthonk monthonk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only have a few comments, otherwise lgtm!

mountpoint-s3/src/upload/atomic.rs Outdated Show resolved Hide resolved
mountpoint-s3/src/async_util.rs Outdated Show resolved Hide resolved
Signed-off-by: Alessandro Passaro <[email protected]>
Signed-off-by: Alessandro Passaro <[email protected]>
@passaro passaro temporarily deployed to PR integration tests December 17, 2024 16:22 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests December 17, 2024 16:23 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests December 17, 2024 16:23 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests December 17, 2024 16:23 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests December 17, 2024 16:23 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests December 17, 2024 16:23 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests December 17, 2024 16:23 — with GitHub Actions Inactive
@passaro passaro requested a review from monthonk December 17, 2024 18:02
@passaro passaro added this pull request to the merge queue Dec 18, 2024
Merged via the queue into awslabs:main with commit 3ee6fbc Dec 18, 2024
26 checks passed
@passaro passaro deleted the async-put-object branch December 18, 2024 10:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance PRs to run benchmarks on
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants