Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S3 default integrity protection change and server-side encryption compatibility #5830

Closed
1 task done
msi591 opened this issue Jan 28, 2025 · 4 comments
Closed
1 task done
Assignees
Labels
bug This issue is a bug. p1 This is a high priority issue potential-regression Marking this issue as a potential regression to be checked by team member response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 10 days.

Comments

@msi591
Copy link

msi591 commented Jan 28, 2025

Describe the bug

From version 2.30.0 (#5802), our putObject requests no longer work with server-side encryption (SSE-C).

Should they be compatible, I didn't find a clear answer in AWS documentation?

The problem goes away setting requestChecksumCalculation and responseChecksumValidation parameters to WHEN_REQUIRED.

Regression Issue

  • Select this option if this issue appears to be a regression.

Expected Behavior

PutObject requests upload files with sse-c encryption

Current Behavior

S3Exception: Invalid Argument (Service: S3, Status Code: 400):

2025-01-27T11:09:35.982+01:00 DEBUG 45225 ---   s.a.a.c.i.ExecutionInterceptorChain      : Creating an interceptor chain that will apply interceptors in the following order: [software.amazon.awssdk.core.internal.interceptor.HttpChecksumValidationInterceptor@44eae19c, software.amazon.awssdk.awscore.interceptor.HelpfulUnknownHostExceptionInterceptor@236c9462, software.amazon.awssdk.awscore.eventstream.EventStreamInitialRequestInterceptor@2e26c67d, software.amazon.awssdk.awscore.interceptor.TraceIdExecutionInterceptor@1d021fc8, software.amazon.awssdk.services.s3.auth.scheme.internal.S3AuthSchemeInterceptor@ef7ebf5, software.amazon.awssdk.services.s3.endpoints.internal.S3ResolveEndpointInterceptor@49ffb1cd, software.amazon.awssdk.services.s3.endpoints.internal.S3RequestSetEndpointInterceptor@51f283a1, software.amazon.awssdk.services.s3.internal.handlers.StreamingRequestInterceptor@5c832a08, software.amazon.awssdk.services.s3.internal.handlers.CreateBucketInterceptor@6d1ada2e, software.amazon.awssdk.services.s3.internal.handlers.CreateMultipartUploadRequestInterceptor@59df6f99, software.amazon.awssdk.services.s3.internal.handlers.DecodeUrlEncodedResponseInterceptor@1cf50088, software.amazon.awssdk.services.s3.internal.handlers.GetBucketPolicyInterceptor@59e3aa74, software.amazon.awssdk.services.s3.internal.handlers.AsyncChecksumValidationInterceptor@6213c0cc, software.amazon.awssdk.services.s3.internal.handlers.SyncChecksumValidationInterceptor@4012c8b0, software.amazon.awssdk.services.s3.internal.handlers.EnableTrailingChecksumInterceptor@74d7617a, software.amazon.awssdk.services.s3.internal.handlers.ExceptionTranslationInterceptor@3023c59e, software.amazon.awssdk.services.s3.internal.handlers.GetObjectInterceptor@4cca66f6, software.amazon.awssdk.services.s3.internal.handlers.ObjectMetadataInterceptor@1acb7097]
2025-01-27T11:09:35.984+01:00 DEBUG 45225 ---   s.a.a.c.i.ExecutionInterceptorChain      : Interceptor 'software.amazon.awssdk.services.s3.endpoints.internal.S3RequestSetEndpointInterceptor@51f283a1' modified the message with its modifyHttpRequest method.
2025-01-27T11:09:35.984+01:00 DEBUG 45225 ---   s.a.a.c.i.ExecutionInterceptorChain      : Interceptor 'software.amazon.awssdk.services.s3.internal.handlers.StreamingRequestInterceptor@5c832a08' modified the message with its modifyHttpRequest method.
2025-01-27T11:09:35.985+01:00 DEBUG 45225 ---   s.a.awssdk.retries.LegacyRetryStrategy   : Request attempt 1 token acquired (backoff: 0ms, cost: 0, capacity: 500/500)
2025-01-27T11:09:35.985+01:00 DEBUG 45225 ---   software.amazon.awssdk.request           : Sending Request: DefaultSdkHttpFullRequest(httpMethod=PUT, protocol=https, host=*****, encodedPath=/file, headers=[amz-sdk-invocation-id, Content-Length, Content-Type, Expect, User-Agent, x-amz-sdk-checksum-algorithm, x-amz-server-side-encryption-customer-algorithm, x-amz-server-side-encryption-customer-key, x-amz-server-side-encryption-customer-key-MD5], queryParameters=[])
2025-01-27T11:09:35.985+01:00 DEBUG 45225 ---   s.a.a.c.i.h.p.stages.SigningStage        : Using SelectedAuthScheme: aws.auth#sigv4
2025-01-27T11:09:35.988+01:00 DEBUG 45225 ---   s.a.a.h.a.a.i.s.DefaultV4RequestSigner   : AWS4 Canonical Request: PUT
/file

amz-sdk-invocation-id:b3b2c4e2-95b0-f0b3-348f
amz-sdk-request:attempt=1; max=4
content-encoding:aws-chunked
content-length:363003
content-type:application/octet-stream
host:*****
x-amz-content-sha256:STREAMING-UNSIGNED-PAYLOAD-TRAILER
x-amz-date:20250127T100935Z
x-amz-decoded-content-length:362940
x-amz-sdk-checksum-algorithm:CRC32
x-amz-server-side-encryption-customer-algorithm:AES256
x-amz-server-side-encryption-customer-key:*****
x-amz-server-side-encryption-customer-key-md5:*****
x-amz-trailer:x-amz-checksum-crc32

amz-sdk-invocation-id;amz-sdk-request;content-encoding;content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-decoded-content-length;x-amz-sdk-checksum-algorithm;x-amz-server-side-encryption-customer-algorithm;x-amz-server-side-encryption-customer-key;x-amz-server-side-encryption-customer-key-md5;x-amz-trailer
STREAMING-UNSIGNED-PAYLOAD-TRAILER
2025-01-27T11:09:35.988+01:00 DEBUG 45225 ---   s.a.a.h.a.a.i.s.DefaultV4RequestSigner   : AWS4 Canonical Request Hash: fafb86
2025-01-27T11:09:35.988+01:00 DEBUG 45225 ---   s.a.a.h.a.a.i.s.DefaultV4RequestSigner   : AWS4 String to sign: AWS4-HMAC-SHA256
20250127T100935Z
20250127/region/s3/aws4_request
fafb86
2025-01-27T11:09:36.007+01:00 DEBUG 45225 ---   s.a.a.c.i.ExecutionInterceptorChain      : Creating an interceptor chain that will apply interceptors in the following order: [software.amazon.awssdk.core.internal.interceptor.HttpChecksumValidationInterceptor@44eae19c, software.amazon.awssdk.awscore.interceptor.HelpfulUnknownHostExceptionInterceptor@236c9462, software.amazon.awssdk.awscore.eventstream.EventStreamInitialRequestInterceptor@2e26c67d, software.amazon.awssdk.awscore.interceptor.TraceIdExecutionInterceptor@1d021fc8, software.amazon.awssdk.services.s3.auth.scheme.internal.S3AuthSchemeInterceptor@ef7ebf5, software.amazon.awssdk.services.s3.endpoints.internal.S3ResolveEndpointInterceptor@49ffb1cd, software.amazon.awssdk.services.s3.endpoints.internal.S3RequestSetEndpointInterceptor@51f283a1, software.amazon.awssdk.services.s3.internal.handlers.StreamingRequestInterceptor@5c832a08, software.amazon.awssdk.services.s3.internal.handlers.CreateBucketInterceptor@6d1ada2e, software.amazon.awssdk.services.s3.internal.handlers.CreateMultipartUploadRequestInterceptor@59df6f99, software.amazon.awssdk.services.s3.internal.handlers.DecodeUrlEncodedResponseInterceptor@1cf50088, software.amazon.awssdk.services.s3.internal.handlers.GetBucketPolicyInterceptor@59e3aa74, software.amazon.awssdk.services.s3.internal.handlers.AsyncChecksumValidationInterceptor@6213c0cc, software.amazon.awssdk.services.s3.internal.handlers.SyncChecksumValidationInterceptor@4012c8b0, software.amazon.awssdk.services.s3.internal.handlers.EnableTrailingChecksumInterceptor@74d7617a, software.amazon.awssdk.services.s3.internal.handlers.ExceptionTranslationInterceptor@3023c59e, software.amazon.awssdk.services.s3.internal.handlers.GetObjectInterceptor@4cca66f6, software.amazon.awssdk.services.s3.internal.handlers.ObjectMetadataInterceptor@1acb7097]
2025-01-27T11:09:36.008+01:00 DEBUG 45225 ---   s.a.a.h.a.i.conn.SdkTlsSocketFactory     : Connecting socket to ***** with timeout 2000
2025-01-27T11:09:36.008+01:00 DEBUG 45225 ---   s.a.a.c.i.ExecutionInterceptorChain      : Interceptor 'software.amazon.awssdk.services.s3.endpoints.internal.S3RequestSetEndpointInterceptor@51f283a1' modified the message with its modifyHttpRequest method.
2025-01-27T11:09:36.008+01:00 DEBUG 45225 ---   s.a.awssdk.retries.LegacyRetryStrategy   : Request attempt 1 token acquired (backoff: 0ms, cost: 0, capacity: 500/500)
2025-01-27T11:09:36.009+01:00 DEBUG 45225 ---   software.amazon.awssdk.request           : Sending Request: DefaultSdkHttpFullRequest(httpMethod=GET, protocol=https, host=*****, encodedPath=, headers=[amz-sdk-invocation-id, User-Agent], queryParameters=[list-type])
2025-01-27T11:09:36.009+01:00 DEBUG 45225 ---   s.a.a.c.i.h.p.stages.SigningStage        : Using SelectedAuthScheme: aws.auth#sigv4
2025-01-27T11:09:36.009+01:00 DEBUG 45225 ---   s.a.a.h.a.a.i.s.DefaultV4RequestSigner   : AWS4 Canonical Request: GET
/
list-type=2
amz-sdk-invocation-id:eef43b32-5638-7c1e-9c26
amz-sdk-request:attempt=1; max=4
host:*****
x-amz-content-sha256:UNSIGNED-PAYLOAD
x-amz-date:20250127T100936Z

amz-sdk-invocation-id;amz-sdk-request;host;x-amz-content-sha256;x-amz-date
UNSIGNED-PAYLOAD
2025-01-27T11:09:36.009+01:00 DEBUG 45225 ---   s.a.a.h.a.a.i.s.DefaultV4RequestSigner   : AWS4 Canonical Request Hash: 86484ac
2025-01-27T11:09:36.009+01:00 DEBUG 45225 ---   s.a.a.h.a.a.i.s.DefaultV4RequestSigner   : AWS4 String to sign: AWS4-HMAC-SHA256
20250127T100936Z
20250127/region/s3/aws4_request
86484ac
2025-01-27T11:09:36.018+01:00 DEBUG 45225 ---   s.a.a.h.a.i.conn.SdkTlsSocketFactory     : Enabled protocols: [TLSv1.3, TLSv1.2]
2025-01-27T11:09:36.018+01:00 DEBUG 45225 ---   s.a.a.h.a.i.conn.SdkTlsSocketFactory     : Enabled cipher suites:[TLS_AES_256_GCM_SHA384, TLS_AES_128_GCM_SHA256, TLS_CHACHA20_POLY1305_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_RSA_WITH_CHACHA20_POLY1305_SHA256, TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
2025-01-27T11:09:36.018+01:00 DEBUG 45225 ---   s.a.a.h.a.i.conn.SdkTlsSocketFactory     : socket.getSupportedProtocols(): [TLSv1.3, TLSv1.2, TLSv1.1, TLSv1, SSLv3, SSLv2Hello], socket.getEnabledProtocols(): [TLSv1.3, TLSv1.2]
2025-01-27T11:09:36.018+01:00 DEBUG 45225 ---   s.a.a.h.a.i.conn.SdkTlsSocketFactory     : Starting handshake
2025-01-27T11:09:36.029+01:00 DEBUG 45225 ---   software.amazon.awssdk.requestId         : Received successful response: 200, Request ID: 1737972109504208, Extended Request ID: 12937990
2025-01-27T11:09:36.029+01:00 DEBUG 45225 ---   software.amazon.awssdk.request           : Received successful response: 200, Request ID: 1737972109504208, Extended Request ID: 12937990
2025-01-27T11:09:36.030+01:00 DEBUG 45225 ---   s.a.awssdk.retries.LegacyRetryStrategy   : Request attempt 1 succeeded (cost: -1, capacity: 500/500)
2025-01-27T11:09:36.043+01:00 DEBUG 45225 ---   s.a.a.h.a.i.conn.SdkTlsSocketFactory     : Secure session established
2025-01-27T11:09:36.043+01:00 DEBUG 45225 ---   s.a.a.h.a.i.conn.SdkTlsSocketFactory     :  negotiated protocol: TLSv1.3
2025-01-27T11:09:36.043+01:00 DEBUG 45225 ---   s.a.a.h.a.i.conn.SdkTlsSocketFactory     :  negotiated cipher suite: TLS_AES_256_GCM_SHA384
2025-01-27T11:09:36.043+01:00 DEBUG 45225 ---   s.a.a.h.a.i.conn.SdkTlsSocketFactory     :  peer principal: CN=*****
2025-01-27T11:09:36.043+01:00 DEBUG 45225 ---   s.a.a.h.a.i.conn.SdkTlsSocketFactory     :  peer alternative names: [*****]
2025-01-27T11:09:36.043+01:00 DEBUG 45225 ---   s.a.a.h.a.i.conn.SdkTlsSocketFactory     :  issuer principal: CN=*****
2025-01-27T11:09:36.043+01:00 DEBUG 45225 ---   s.a.a.h.a.internal.net.SdkSslSocket      : created: *****
2025-01-27T11:09:36.060+01:00 DEBUG 45225 ---   .a.h.a.a.i.s.c.ChunkedEncodedInputStream : Reading next chunk.
2025-01-27T11:09:36.086+01:00 DEBUG 45225 ---   .a.h.a.a.i.s.c.ChunkedEncodedInputStream : Reading next chunk.
2025-01-27T11:09:36.104+01:00 DEBUG 45225 ---   .a.h.a.a.i.s.c.ChunkedEncodedInputStream : Reading next chunk.
2025-01-27T11:09:36.105+01:00 DEBUG 45225 ---   s.a.a.c.i.io.SdkLengthAwareInputStream   : Specified InputStream length of 362940 has been reached. Returning EOF.
2025-01-27T11:09:36.111+01:00 DEBUG 45225 ---   .a.h.a.a.i.s.c.ChunkedEncodedInputStream : Reading next chunk.
2025-01-27T11:09:36.111+01:00 DEBUG 45225 ---   s.a.a.c.i.io.SdkLengthAwareInputStream   : Specified InputStream length of 362940 has been reached. Returning EOF.
2025-01-27T11:09:36.111+01:00 DEBUG 45225 ---   .a.h.a.a.i.s.c.ChunkedEncodedInputStream : End of backing stream reached. Reading final chunk.
2025-01-27T11:09:36.151+01:00 DEBUG 45225 ---   software.amazon.awssdk.requestId         : Received failed response: 400, Request ID: 1737972095169022, Extended Request ID: 12937990
2025-01-27T11:09:36.151+01:00 DEBUG 45225 ---   software.amazon.awssdk.request           : Received failed response: 400, Request ID: 1737972095169022, Extended Request ID: 12937990
2025-01-27T11:09:36.156+01:00 DEBUG 45225 ---   s.a.awssdk.retries.LegacyRetryStrategy   : Request attempt 1 encountered non-retryable failure

software.amazon.awssdk.services.s3.model.S3Exception: Invalid Argument (Service: S3, Status Code: 400, Request ID: 1737972095169022, Extended Request ID: 12937990)
	at software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handleErrorResponse(AwsXmlPredicatedResponseHandler.java:155)
	at software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handleResponse(AwsXmlPredicatedResponseHandler.java:107)
	at software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handle(AwsXmlPredicatedResponseHandler.java:84)
	at software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handle(AwsXmlPredicatedResponseHandler.java:42)
	at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler$Crc32ValidationResponseHandler.handle(AwsSyncClientHandler.java:93)
	at software.amazon.awssdk.core.internal.handler.BaseClientHandler.lambda$successTransformationResponseHandler$7(BaseClientHandler.java:279)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:50)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:38)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:74)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:43)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:79)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:41)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:55)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:39)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.executeRequest(RetryableStage.java:93)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:56)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:36)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:53)
	at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:35)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:82)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:62)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:43)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:50)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:32)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)
	at software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:210)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:103)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:173)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:80)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:182)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:74)
	at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
	at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:53)
	at software.amazon.awssdk.services.s3.DefaultS3Client.putObject(DefaultS3Client.java:11169)
	
2025-01-27T11:09:36.191+01:00 DEBUG 45225 ---   s.a.a.c.i.ExecutionInterceptorChain      : Creating an interceptor chain that will apply interceptors in the following order: [software.amazon.awssdk.core.internal.interceptor.HttpChecksumValidationInterceptor@44eae19c, software.amazon.awssdk.awscore.interceptor.HelpfulUnknownHostExceptionInterceptor@236c9462, software.amazon.awssdk.awscore.eventstream.EventStreamInitialRequestInterceptor@2e26c67d, software.amazon.awssdk.awscore.interceptor.TraceIdExecutionInterceptor@1d021fc8, software.amazon.awssdk.services.s3.auth.scheme.internal.S3AuthSchemeInterceptor@ef7ebf5, software.amazon.awssdk.services.s3.endpoints.internal.S3ResolveEndpointInterceptor@49ffb1cd, software.amazon.awssdk.services.s3.endpoints.internal.S3RequestSetEndpointInterceptor@51f283a1, software.amazon.awssdk.services.s3.internal.handlers.StreamingRequestInterceptor@5c832a08, software.amazon.awssdk.services.s3.internal.handlers.CreateBucketInterceptor@6d1ada2e, software.amazon.awssdk.services.s3.internal.handlers.CreateMultipartUploadRequestInterceptor@59df6f99, software.amazon.awssdk.services.s3.internal.handlers.DecodeUrlEncodedResponseInterceptor@1cf50088, software.amazon.awssdk.services.s3.internal.handlers.GetBucketPolicyInterceptor@59e3aa74, software.amazon.awssdk.services.s3.internal.handlers.AsyncChecksumValidationInterceptor@6213c0cc, software.amazon.awssdk.services.s3.internal.handlers.SyncChecksumValidationInterceptor@4012c8b0, software.amazon.awssdk.services.s3.internal.handlers.EnableTrailingChecksumInterceptor@74d7617a, software.amazon.awssdk.services.s3.internal.handlers.ExceptionTranslationInterceptor@3023c59e, software.amazon.awssdk.services.s3.internal.handlers.GetObjectInterceptor@4cca66f6, software.amazon.awssdk.services.s3.internal.handlers.ObjectMetadataInterceptor@1acb7097]
2025-01-27T11:09:36.192+01:00 DEBUG 45225 ---   s.a.a.c.i.ExecutionInterceptorChain      : Interceptor 'software.amazon.awssdk.services.s3.endpoints.internal.S3RequestSetEndpointInterceptor@51f283a1' modified the message with its modifyHttpRequest method.
2025-01-27T11:09:36.192+01:00 DEBUG 45225 ---   s.a.awssdk.retries.LegacyRetryStrategy   : Request attempt 1 token acquired (backoff: 0ms, cost: 0, capacity: 500/500)


Reproduction Steps

S3Client s3Client = S3Client.builder()
    .endpointOverride(uri)
    .credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create(accessKeyId, accessKeySecret)))
    .region(region)
    // .requestChecksumCalculation(RequestChecksumCalculation.WHEN_REQUIRED)
    // .responseChecksumValidation(ResponseChecksumValidation.WHEN_REQUIRED)
    .build();

PutObjectRequest putObjectRequest = PutObjectRequest.builder()
    .bucket(destination)
    .key(fileName)
    .sseCustomerAlgorithm(sseCustomerAlgorithm)
    .sseCustomerKey(sseCustomerKey)
    .sseCustomerKeyMD5(sseCustomerKeyMd5)
    .build();
s3Client.putObject(putObjectRequest, RequestBody.fromFile(path));

Possible Solution

No response

Additional Information/Context

No response

AWS Java SDK version used

2.30.6

JDK version used

openjdk 21.0.5 2024-10-15 LTS

Operating System and version

Ubuntu 24.04.1 LTS

@msi591 msi591 added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Jan 28, 2025
@bhoradc bhoradc added p1 This is a high priority issue potential-regression Marking this issue as a potential regression to be checked by team member and removed needs-triage This issue or PR still needs to be triaged. labels Jan 28, 2025
@bhoradc bhoradc self-assigned this Jan 28, 2025
@bhoradc
Copy link

bhoradc commented Jan 28, 2025

Hello @msi591,

Thank you for reporting the issue.

  • Sample code suggests that you are overriding the endpoint url. Are you seeing this error for a third-party S3-compatible service or proxy or while connecting to AWS S3?
  • I am unable to reproduce the error using AWS S3 with below code sample. The program uploads correctly to S3 (validated on Java SDK v2.29.52 and above).
public class Main {
    public static void main(String[] args) throws NoSuchAlgorithmException {

        S3Client s3Client = null;
        try {
            final var bucket = "<<bucketname>>";
            final var key = "<<key>>";
            final var path = new File("/Users/*****/<<key>>.txt");

            s3Client = S3Client.builder()
                    .region(Region.US_EAST_1)
                    .build();

            KeyGenerator keyGen = KeyGenerator.getInstance("AES");
            keyGen.init(256);
            SecretKey secretKey = keyGen.generateKey();
            String sseCustomerKey = Base64.getEncoder().encodeToString(secretKey.getEncoded());

            MessageDigest md = MessageDigest.getInstance("MD5");
            byte[] md5Hash = md.digest(secretKey.getEncoded());
            String sseCustomerKeyMd5 = Base64.getEncoder().encodeToString(md5Hash);

            PutObjectRequest putObjectRequest = PutObjectRequest.builder()
                    .bucket(bucket)
                    .key(key)
                    .sseCustomerAlgorithm("AES256")
                    .sseCustomerKey(sseCustomerKey)
                    .sseCustomerKeyMD5(sseCustomerKeyMd5)
                    .build();

            s3Client.putObject(putObjectRequest, RequestBody.fromFile(path));
        } catch (NoSuchAlgorithmException e) {
            System.err.println("Cryptographic algorithm is not available: " + e.getMessage());
        } catch (S3Exception e) {
            System.err.println("Error during S3 operation: " + e.getMessage());
        } catch (Exception e) {
            System.err.println("Unexpected error occurred: " + e.getMessage());
            e.printStackTrace();
        } finally {
            if (s3Client != null) {
                s3Client.close();
            }
        }
    }
}

Let me know if you have further inputs on this.

Regards,
Chaitanya

@bhoradc bhoradc added the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 10 days. label Jan 28, 2025
@msi591
Copy link
Author

msi591 commented Jan 28, 2025

Hi bhoradc!

Yes sorry, I forgot to specify that I'm connecting to a third-party S3-compatible service.

So I understand that default checksum calculation and sse-c encryption should be compatible.

For my part, downgrading java sdk to v2.29.50 solved the problem (as well as setting requestChecksumCalculation and responseChecksumValidation parameters to WHEN_REQUIRED with v2.30.6).

@bhoradc
Copy link

bhoradc commented Jan 28, 2025

Thank you for the confirmation. Hence closing the issue.

@bhoradc bhoradc closed this as completed Jan 28, 2025
Copy link

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a bug. p1 This is a high priority issue potential-regression Marking this issue as a potential regression to be checked by team member response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 10 days.
Projects
None yet
Development

No branches or pull requests

2 participants