Override RegionSet in EnpointResolverInterceptor after fetching the Signing Properties from Endpoint rules #5825

joviegas · 2025-01-26T05:49:44Z

Motivation and Context

Currently, sigv4a is is resolved as floows

In finalizeAwsConfiguration() of AwsClientOption.AWS_SIGV4A_SIGNING_REGION_SET we resolveSigv4aRegionSet in folllowing order

    private Set<String> resolveSigv4aRegionSet(LazyValueSource config) {
        Supplier<ProfileFile> profileFile = config.get(SdkClientOption.PROFILE_FILE_SUPPLIER);
        String profileName = config.get(SdkClientOption.PROFILE_NAME);
        return Sigv4aSigningRegionSetProvider.builder()
                                             .profileFile(profileFile)
                                             .profileName(profileName)
                                             .build()
                                             .resolveRegionSet();
    }

In AwsExecutionContextBuilder we update AwsExecutionAttribute.AWS_SIGV4A_SIGNING_REGION_SET from AwsClientOption.AWS_SIGV4A_SIGNING_REGION_SET
Then in ServiceAuthSchemeInterceptor we update the authSchemeParams from above and then can call resolveAuthScheme where the Auth option is updated with this option
Then In ServiceResolveEndpointInterceptor will then resolve based on this PR

Modifications

Modified the code where we override the Auth Scheme properties to make sure we follow above table for Sigv4a
Also added changes to support new mult-auth launched services which will not have deprecated "authType" trait.

Testing

Codegen changes to verify Generated interceptors are generated as required.
Added signer test to make sure properties get overridden after Endpoint resolution

Screenshots (if appropriate)

Appendix

The Sigv4a signing region set should be resolved based on below specifications

Sigv4a signing region set resolution

Configuration follows the same resolution order as other client configuration
options, where more specific configuration is preferred over less specific. The
following table shows cases that demonstrate this. Each column except for
Result shows a source for the signing region set, and Result is the value
that should be used. Note that * indcates the request is valid in all regions.

Code	Environment	Config File	Endpoints 2.0 Metadata	Endpoint Region	Result
n/a	n/a	n/a	n/a	us-west-2	us-west-2
n/a	n/a	n/a	*	us-west-2	*
n/a	*	n/a	n/a	us-west-2	*
n/a	n/a	*	n/a	us-west-2	*
*	n/a	n/a	n/a	us-west-2	*
us-west-2	us-west-2	n/a	*	us-west-2	us-west-2
us-west-2	n/a	us-west-2	*	us-west-2	us-west-2
us-west-2	n/a	n/a	*	us-west-2	us-west-2
us-west-2	*	us-west-2	n/a	us-west-2	us-west-2
us-west-2	*	n/a	n/a	us-west-2	us-west-2
us-west-2	n/a	*	n/a	us-west-2	us-west-2
n/a	n/a	n/a	n/a	us-west-2	us-west-2
us-west-2, us-east-1	n/a	n/a	us-west-2, us-east-1	us-west-2	us-west-2, us-east-1
us-west-2, us-east-1	us-west-2, us-east-1	n/a	n/a	us-west-2	us-west-2, us-east-1
us-west-2, us-east-1	n/a	us-west-2, us-east-1	n/a	us-west-2	us-west-2, us-east-1
us-west-2, us-east-1	n/a	n/a	n/a	us-west-2	us-west-2, us-east-1

License

I confirm that this pull request can be released under the Apache 2 license

…igning Properties from Endpoint rules

joviegas · 2025-01-26T21:07:11Z

...rc/test/java/software/amazon/awssdk/services/endpointproviders/EndpointInterceptorTests.java

@@ -187,8 +201,7 @@ public void async_endpointProviderReturnsHeaders_appendedToExistingRequest() {
        assertThat(interceptor.context.httpRequest().matchingHeaders("TestHeader")).containsExactly("TestValue", "TestValue0");
    }

-    // TODO(sra-identity-auth): Enable for useSraAuth=true
-    /*
+


Uncommented these test cases

millems · 2025-01-27T22:17:57Z

codegen/src/main/java/software/amazon/awssdk/codegen/AddMetadata.java

@@ -74,7 +74,8 @@ public static Metadata constructMetadata(ServiceModel serviceModel,
                .withJsonVersion(serviceMetadata.getJsonVersion())
                .withEndpointPrefix(serviceMetadata.getEndpointPrefix())
                .withSigningName(serviceMetadata.getSigningName())
-                .withAuthType(AuthType.fromValue(serviceMetadata.getSignatureVersion()))
+                .withAuthType(serviceMetadata.getSignatureVersion() != null ?


When can this be null?

The specifications states

The authType and signatureVersion traits will be deprecated in favor of the auth and unsignedPayload traits.

Thus for new services which will be completed based on multi-auth signatureVersion will be null.
I added a test case for in codegen-tst when a new service is added with just multi-auth supporting only sigv4a

millems · 2025-01-27T22:55:24Z

...src/main/java/software/amazon/awssdk/codegen/poet/rules/EndpointResolverInterceptorSpec.java

@@ -219,7 +227,11 @@ private MethodSpec modifyRequestMethod(String endpointAuthSchemeStrategyFieldNam
                       SelectedAuthScheme.class, SdkInternalExecutionAttribute.class);
        b.beginControlFlow("if (endpointAuthSchemes != null && selectedAuthScheme != null)");


Is selectedAuthScheme ever null nowadays? It feels like this should be a dead code branch now?

Removed this check

My bad , we need this check , the S3 integ test failed , will revert this

java.lang.NullPointerException: Cannot invoke "software.amazon.awssdk.core.SelectedAuthScheme.authSchemeOption()" because "selectedAuthScheme" is null at software.amazon.awssdk.services.s3.endpoints.internal.S3ResolveEndpointInterceptor.authSchemeWithEndpointSignerProperties(S3ResolveEndpointInterceptor.java:1383) at software.amazon.awssdk.services.s3.endpoints.internal.S3ResolveEndpointInterceptor.modifyRequest(S3ResolveEndpointInterceptor.java:177) at software.amazon.awssdk.core.interceptor.ExecutionInterceptorChain.modifyRequest(ExecutionInterceptorC

millems · 2025-01-27T22:58:29Z

...src/main/java/software/amazon/awssdk/codegen/poet/rules/EndpointResolverInterceptorSpec.java

+        if (multiAuthSigv4a) {
+            code.beginControlFlow("if (!hasRegionSet(selectedAuthScheme) && v4aAuthScheme.signingRegionSet() != null)");
+        } else {
+            code.beginControlFlow("if (v4aAuthScheme.signingRegionSet() != null)");
+        }


hasRegionSet seems like a relatively small method. Same with updateAuthSchemeWithRegionSet. Do we want to just always generate those methods, so that we don't have to do this branching in the code generator? It feels simpler to have a single code path regardless of sigv4a or sigv4 (both in the code generator, and in the generated code).

I have made updateAuthSchemeWithRegionSet and hasRegionSet inline. The reason I added a specific check for multiAuthSigv4a is because for non-Sigv4a existing features, that check is dead code and might confuse readers of generated packages while debugging or during code walks.

millems · 2025-01-30T18:39:07Z

...ware/amazon/awssdk/codegen/poet/rules/endpoint-resolve-interceptor-with-multiauthsigv4a.java

+                if (!hasRegionSet(selectedAuthScheme) && v4aAuthScheme.signingRegionSet() != null) {
+                    RegionSet regionSet = RegionSet.create(v4aAuthScheme.signingRegionSet());
+                    option.putSignerProperty(AwsV4aHttpSigner.REGION_SET, regionSet);
+                }


If I'm remembering the priorities right from the discussion with Alex, the endpoint provider's setting takes precedence over the auth scheme resolver's setting. This looks like we're favoring the auth scheme resolver. Am I misreading or misremembering?

The discussion covered two distinct concepts: Signing Region and RegionSet. The current implementation specifically addresses SigV4a RegionSet, where Endpoint Parameters have the lowest precedence according to the Multi-auth specification. The precedence order is documented in the attached table.

The table doesn't include the auth scheme resolver. In the discussion with Alex, when we added that, endpoint resolver came out higher than auth scheme resolver.

Updated a comment

sonarqubecloud · 2025-01-31T20:38:28Z

Quality Gate failed

Failed conditions
7.6% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud

Override RegionSet in EnpointResolverInterceptor after fetching the S…

082a218

…igning Properties from Endpoint rules

joviegas requested a review from a team as a code owner January 26, 2025 05:49

joviegas added 2 commits January 25, 2025 21:57

Endpoint Resolver Spec test added

81cca37

Add test case for service with MultiAuth only and not using Sigv4

5953fdf

joviegas force-pushed the joviegas/resolve-regioset-after-endpoints branch from e5fb5b7 to 5953fdf Compare January 26, 2025 20:26

checkstyle fixed

94f3951

joviegas commented Jan 26, 2025

View reviewed changes

millems reviewed Jan 30, 2025

View reviewed changes

joviegas added 2 commits January 30, 2025 21:22

Handled comments

9bd5a75

Updated after Integ test failure and some comments

e517fc4

millems approved these changes Jan 31, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Override RegionSet in EnpointResolverInterceptor after fetching the Signing Properties from Endpoint rules #5825

Override RegionSet in EnpointResolverInterceptor after fetching the Signing Properties from Endpoint rules #5825

joviegas commented Jan 26, 2025 •

edited

Loading

joviegas Jan 26, 2025

millems Jan 27, 2025

joviegas Jan 31, 2025

millems Jan 27, 2025

joviegas Jan 31, 2025

joviegas Jan 31, 2025

millems Jan 27, 2025

joviegas Jan 31, 2025

millems Jan 30, 2025

joviegas Jan 31, 2025

millems Jan 31, 2025

joviegas Jan 31, 2025

sonarqubecloud bot commented Jan 31, 2025

		@@ -219,7 +227,11 @@ private MethodSpec modifyRequestMethod(String endpointAuthSchemeStrategyFieldNam
		SelectedAuthScheme.class, SdkInternalExecutionAttribute.class);
		b.beginControlFlow("if (endpointAuthSchemes != null && selectedAuthScheme != null)");

Override RegionSet in EnpointResolverInterceptor after fetching the Signing Properties from Endpoint rules #5825

Are you sure you want to change the base?

Override RegionSet in EnpointResolverInterceptor after fetching the Signing Properties from Endpoint rules #5825

Conversation

joviegas commented Jan 26, 2025 • edited Loading

Motivation and Context

Modifications

Testing

Screenshots (if appropriate)

Appendix

Sigv4a signing region set resolution

License

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sonarqubecloud bot commented Jan 31, 2025

Quality Gate failed

joviegas commented Jan 26, 2025 •

edited

Loading