Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: use sdk-partitions.json from trebuchet #1667

Closed
wants to merge 7 commits into from
+1,186 −1,595
Closed

feat: use sdk-partitions.json from trebuchet #1667

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
8e279c1
use sdk-partitions.json from trebuchet
dayaffe Aug 7, 2024
62193ef
Fix AWSClientRuntime source & resource paths; correct Bundle reference
jbelkins Aug 7, 2024
798c256
Merge branch 'main' into day/partition-file
dayaffe Aug 7, 2024
803e397
add thread-safe caching
dayaffe Aug 8, 2024
fefde75
Merge branch 'main' into day/partition-file
dayaffe Aug 8, 2024
1a3cbb9
make init async
dayaffe Aug 22, 2024
07392cc
remove code star
dayaffe Aug 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
4 changes: 2 additions & 2 deletions AWSSDKSwiftCLI/Sources/AWSSDKSwiftCLI/Resources/Package.Base.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,9 +83,9 @@ let package = Package(
.awsSDKHTTPAuth,
.awsSDKIdentity
],
path: "Sources/Core/AWSClientRuntime/Sources",
path: "Sources/Core/AWSClientRuntime/Sources/AWSClientRuntime",
resources: [
.process("AWSClientRuntime/Resources")
.process("Resources"),
]
),
.target(
Expand Down
6 changes: 5 additions & 1 deletion Package.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,9 +83,13 @@ let package = Package(
.awsSDKHTTPAuth,
.awsSDKIdentity
],
path: "Sources/Core/AWSClientRuntime/Sources",
path: "Sources/Core/AWSClientRuntime/Sources/AWSClientRuntime",
resources: [
<<<<<<< HEAD
.process("Resources"),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lol pick this

=======
.process("AWSClientRuntime/Resources")
>>>>>>> 8389b4dba (add thread-safe caching)
]
),
.target(
Expand Down
259 changes: 37 additions & 222 deletions ...ces/Core/AWSClientRuntime/Sources/AWSClientRuntime/Endpoints/AWSPartitionDefinition.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,232 +7,47 @@

import Foundation

// Partition definitions are embedded as a static resource in this project, for now.
// When Trebuchet integration is performed, partitions should be obtained from Trebuchet for every
// build instead of being loaded from a static definition.
public let awsPartitionJSON = """
{
"version": "1.1",
"partitions": [
{
"id": "aws",
"regionRegex": "^(us|eu|ap|sa|ca|me|af|il)-\\\\w+-\\\\d+$",
"regions": {
"af-south-1": {
"description": "Africa (Cape Town)"
},
"ap-east-1": {
"description": "Asia Pacific (Hong Kong)"
},
"ap-northeast-1": {
"description": "Asia Pacific (Tokyo)"
},
"ap-northeast-2": {
"description": "Asia Pacific (Seoul)"
},
"ap-northeast-3": {
"description": "Asia Pacific (Osaka)"
},
"ap-south-1": {
"description": "Asia Pacific (Mumbai)"
},
"ap-south-2": {
"description": "Asia Pacific (Hyderabad)"
},
"ap-southeast-1": {
"description": "Asia Pacific (Singapore)"
},
"ap-southeast-2": {
"description": "Asia Pacific (Sydney)"
},
"ap-southeast-3": {
"description": "Asia Pacific (Jakarta)"
},
"ap-southeast-4": {
"description": "Asia Pacific (Melbourne)"
},
"aws-global": {
"description": "AWS Standard global region"
},
"ca-central-1": {
"description": "Canada (Central)"
},
"ca-west-1": {
"description": "Canada West (Calgary)"
},
"eu-central-1": {
"description": "Europe (Frankfurt)"
},
"eu-central-2": {
"description": "Europe (Zurich)"
},
"eu-north-1": {
"description": "Europe (Stockholm)"
},
"eu-south-1": {
"description": "Europe (Milan)"
},
"eu-south-2": {
"description": "Europe (Spain)"
},
"eu-west-1": {
"description": "Europe (Ireland)"
},
"eu-west-2": {
"description": "Europe (London)"
},
"eu-west-3": {
"description": "Europe (Paris)"
},
"il-central-1": {
"description": "Israel (Tel Aviv)"
},
"me-central-1": {
"description": "Middle East (UAE)"
},
"me-south-1": {
"description": "Middle East (Bahrain)"
},
"sa-east-1": {
"description": "South America (Sao Paulo)"
},
"us-east-1": {
"description": "US East (N. Virginia)"
},
"us-east-2": {
"description": "US East (Ohio)"
},
"us-west-1": {
"description": "US West (N. California)"
},
"us-west-2": {
"description": "US West (Oregon)"
enum PartitionLoadingError: Error {
case fileNotFound
case readError(String)
}

public class PartitionData {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think if you just make this type an actor, it will enforce exclusive calls to loadAWSPartitionJSON().

We should not be using DispatchQueue & other GCD types in new code, except where necessary to interact with pre-Swift concurrency code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made PartitionDataCache an actor which is used by this and removed DispatchQueue

private static var cachedPartitionJSON: String?
// Use a dispatch queue to synchronize access to the cache
private static let cacheQueue = DispatchQueue(label: "awsClientRuntime.partitionCacheQueue", attributes: .concurrent)

Check warning on line 18 in Sources/Core/AWSClientRuntime/Sources/AWSClientRuntime/Endpoints/AWSPartitionDefinition.swift

View workflow job for this annotation

GitHub Actions / swiftlint 

Line should be 120 characters or less; currently it has 121 characters (line_length)

// Loading trebuchet-provided sdk-partitions.json
public static func loadAWSPartitionJSON() throws -> String {
// Read from cache if available
if let cached = getCachedPartitionJSON() {
return cached
}
},
"outputs": {
"name": "aws",
"dnsSuffix": "amazonaws.com",
"dualStackDnsSuffix": "api.aws",
"supportsFIPS": true,
"supportsDualStack": true,
"implicitGlobalRegion": "us-east-1"
}
},
{
"id": "aws-cn",
"regionRegex": "^cn\\\\-\\\\w+\\\\-\\\\d+$",
"regions": {
"aws-cn-global": {
"description": "AWS China global region"
},
"cn-north-1": {
"description": "China (Beijing)"
},
"cn-northwest-1": {
"description": "China (Ningxia)"

guard let url = Bundle.module.url(forResource: "sdk-partitions", withExtension: "json") else {
throw PartitionLoadingError.fileNotFound
}
},
"outputs": {
"name": "aws-cn",
"dnsSuffix": "amazonaws.com.cn",
"dualStackDnsSuffix": "api.amazonwebservices.com.cn",
"supportsFIPS": true,
"supportsDualStack": true,
"implicitGlobalRegion": "cn-northwest-1"
}
},
{
"id": "aws-us-gov",
"regionRegex": "^us\\\\-gov\\\\-\\\\w+\\\\-\\\\d+$",
"regions": {
"aws-us-gov-global": {
"description": "AWS GovCloud (US) global region"
},
"us-gov-east-1": {
"description": "AWS GovCloud (US-East)"
},
"us-gov-west-1": {
"description": "AWS GovCloud (US-West)"

do {
let partitionJSON = try String(contentsOf: url, encoding: .utf8)
setCachedPartitionJSON(partitionJSON)
return partitionJSON
} catch {
throw PartitionLoadingError.readError("Failed to read the partitions file: \(error)")
}
},
"outputs": {
"name": "aws-us-gov",
"dnsSuffix": "amazonaws.com",
"dualStackDnsSuffix": "api.aws",
"supportsFIPS": true,
"supportsDualStack": true,
"implicitGlobalRegion": "us-gov-west-1"
}
},
{
"id": "aws-iso",
"regionRegex": "^us\\\\-iso\\\\-\\\\w+\\\\-\\\\d+$",
"regions": {
"aws-iso-global": {
"description": "AWS ISO (US) global region"
},
"us-iso-east-1": {
"description": "US ISO East"
},
"us-iso-west-1": {
"description": "US ISO WEST"
}

private static func getCachedPartitionJSON() -> String? {
var cached: String?
cacheQueue.sync {
cached = cachedPartitionJSON
}
},
"outputs": {
"name": "aws-iso",
"dnsSuffix": "c2s.ic.gov",
"dualStackDnsSuffix": "c2s.ic.gov",
"supportsFIPS": true,
"supportsDualStack": false,
"implicitGlobalRegion": "us-iso-east-1"
}
},
{
"id": "aws-iso-b",
"regionRegex": "^us\\\\-isob\\\\-\\\\w+\\\\-\\\\d+$",
"regions": {
"aws-iso-b-global": {
"description": "AWS ISOB (US) global region"
},
"us-isob-east-1": {
"description": "US ISOB East (Ohio)"
return cached
}

private static func setCachedPartitionJSON(_ json: String) {
cacheQueue.async(flags: .barrier) {
self.cachedPartitionJSON = json
}
},
"outputs": {
"name": "aws-iso-b",
"dnsSuffix": "sc2s.sgov.gov",
"dualStackDnsSuffix": "sc2s.sgov.gov",
"supportsFIPS": true,
"supportsDualStack": false,
"implicitGlobalRegion": "us-isob-east-1"
}
},
{
"id": "aws-iso-e",
"regionRegex": "^eu\\\\-isoe\\\\-\\\\w+\\\\-\\\\d+$",
"regions": { },
"outputs": {
"name": "aws-iso-e",
"dnsSuffix": "cloud.adc-e.uk",
"dualStackDnsSuffix": "cloud.adc-e.uk",
"supportsFIPS": true,
"supportsDualStack": false,
"implicitGlobalRegion": "eu-isoe-west-1"
}
},
{
"id": "aws-iso-f",
"regionRegex": "^us\\\\-isof\\\\-\\\\w+\\\\-\\\\d+$",
"regions": { },
"outputs": {
"name": "aws-iso-f",
"dnsSuffix": "csp.hci.ic.gov",
"dualStackDnsSuffix": "csp.hci.ic.gov",
"supportsFIPS": true,
"supportsDualStack": false,
"implicitGlobalRegion": "us-isof-south-1"
}
}
]
}
"""
4 changes: 2 additions & 2 deletions Sources/Services/AWSACM/Sources/AWSACM/Endpoints.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@

// Code generated by smithy-swift-codegen. DO NOT EDIT!

import class AWSClientRuntime.PartitionData
import class ClientRuntime.EndpointsRequestContext
import let AWSClientRuntime.awsPartitionJSON
import protocol ClientRuntime.EndpointsRequestContextProviding
import struct ClientRuntime.DefaultEndpointResolver
import struct ClientRuntime.StaticEndpointResolver
Expand Down Expand Up @@ -62,7 +62,7 @@ extension DefaultEndpointResolver {
private static let ruleSet = "{\"version\":\"1.0\",\"parameters\":{\"Region\":{\"builtIn\":\"AWS::Region\",\"required\":false,\"documentation\":\"The AWS region used to dispatch the request.\",\"type\":\"String\"},\"UseDualStack\":{\"builtIn\":\"AWS::UseDualStack\",\"required\":true,\"default\":false,\"documentation\":\"When true, use the dual-stack endpoint. If the configured endpoint does not support dual-stack, dispatching the request MAY return an error.\",\"type\":\"Boolean\"},\"UseFIPS\":{\"builtIn\":\"AWS::UseFIPS\",\"required\":true,\"default\":false,\"documentation\":\"When true, send this request to the FIPS-compliant regional endpoint. If the configured endpoint does not have a FIPS compliant endpoint, dispatching the request will return an error.\",\"type\":\"Boolean\"},\"Endpoint\":{\"builtIn\":\"SDK::Endpoint\",\"required\":false,\"documentation\":\"Override the endpoint used to send this request\",\"type\":\"String\"}},\"rules\":[{\"conditions\":[{\"fn\":\"isSet\",\"argv\":[{\"ref\":\"Endpoint\"}]}],\"rules\":[{\"conditions\":[{\"fn\":\"booleanEquals\",\"argv\":[{\"ref\":\"UseFIPS\"},true]}],\"error\":\"Invalid Configuration: FIPS and custom endpoint are not supported\",\"type\":\"error\"},{\"conditions\":[{\"fn\":\"booleanEquals\",\"argv\":[{\"ref\":\"UseDualStack\"},true]}],\"error\":\"Invalid Configuration: Dualstack and custom endpoint are not supported\",\"type\":\"error\"},{\"conditions\":[],\"endpoint\":{\"url\":{\"ref\":\"Endpoint\"},\"properties\":{},\"headers\":{}},\"type\":\"endpoint\"}],\"type\":\"tree\"},{\"conditions\":[{\"fn\":\"isSet\",\"argv\":[{\"ref\":\"Region\"}]}],\"rules\":[{\"conditions\":[{\"fn\":\"aws.partition\",\"argv\":[{\"ref\":\"Region\"}],\"assign\":\"PartitionResult\"}],\"rules\":[{\"conditions\":[{\"fn\":\"booleanEquals\",\"argv\":[{\"ref\":\"UseFIPS\"},true]},{\"fn\":\"booleanEquals\",\"argv\":[{\"ref\":\"UseDualStack\"},true]}],\"rules\":[{\"conditions\":[{\"fn\":\"booleanEquals\",\"argv\":[true,{\"fn\":\"getAttr\",\"argv\":[{\"ref\":\"PartitionResult\"},\"supportsFIPS\"]}]},{\"fn\":\"booleanEquals\",\"argv\":[true,{\"fn\":\"getAttr\",\"argv\":[{\"ref\":\"PartitionResult\"},\"supportsDualStack\"]}]}],\"rules\":[{\"conditions\":[],\"endpoint\":{\"url\":\"https://acm-fips.{Region}.{PartitionResult#dualStackDnsSuffix}\",\"properties\":{},\"headers\":{}},\"type\":\"endpoint\"}],\"type\":\"tree\"},{\"conditions\":[],\"error\":\"FIPS and DualStack are enabled, but this partition does not support one or both\",\"type\":\"error\"}],\"type\":\"tree\"},{\"conditions\":[{\"fn\":\"booleanEquals\",\"argv\":[{\"ref\":\"UseFIPS\"},true]}],\"rules\":[{\"conditions\":[{\"fn\":\"booleanEquals\",\"argv\":[{\"fn\":\"getAttr\",\"argv\":[{\"ref\":\"PartitionResult\"},\"supportsFIPS\"]},true]}],\"rules\":[{\"conditions\":[{\"fn\":\"stringEquals\",\"argv\":[{\"fn\":\"getAttr\",\"argv\":[{\"ref\":\"PartitionResult\"},\"name\"]},\"aws-us-gov\"]}],\"endpoint\":{\"url\":\"https://acm.{Region}.amazonaws.com\",\"properties\":{},\"headers\":{}},\"type\":\"endpoint\"},{\"conditions\":[],\"endpoint\":{\"url\":\"https://acm-fips.{Region}.{PartitionResult#dnsSuffix}\",\"properties\":{},\"headers\":{}},\"type\":\"endpoint\"}],\"type\":\"tree\"},{\"conditions\":[],\"error\":\"FIPS is enabled but this partition does not support FIPS\",\"type\":\"error\"}],\"type\":\"tree\"},{\"conditions\":[{\"fn\":\"booleanEquals\",\"argv\":[{\"ref\":\"UseDualStack\"},true]}],\"rules\":[{\"conditions\":[{\"fn\":\"booleanEquals\",\"argv\":[true,{\"fn\":\"getAttr\",\"argv\":[{\"ref\":\"PartitionResult\"},\"supportsDualStack\"]}]}],\"rules\":[{\"conditions\":[],\"endpoint\":{\"url\":\"https://acm.{Region}.{PartitionResult#dualStackDnsSuffix}\",\"properties\":{},\"headers\":{}},\"type\":\"endpoint\"}],\"type\":\"tree\"},{\"conditions\":[],\"error\":\"DualStack is enabled but this partition does not support DualStack\",\"type\":\"error\"}],\"type\":\"tree\"},{\"conditions\":[],\"endpoint\":{\"url\":\"https://acm.{Region}.{PartitionResult#dnsSuffix}\",\"properties\":{},\"headers\":{}},\"type\":\"endpoint\"}],\"type\":\"tree\"}],\"type\":\"tree\"},{\"conditions\":[],\"error\":\"Invalid Configuration: Missing Region\",\"type\":\"error\"}]}"

init() throws {
try self.init(partitions: AWSClientRuntime.awsPartitionJSON, ruleSet: Self.ruleSet)
try self.init(partitions: AWSClientRuntime.PartitionData.loadAWSPartitionJSON(), ruleSet: Self.ruleSet)
}
}

Expand Down
4 changes: 2 additions & 2 deletions Sources/Services/AWSACMPCA/Sources/AWSACMPCA/Endpoints.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@

// Code generated by smithy-swift-codegen. DO NOT EDIT!

import class AWSClientRuntime.PartitionData
import class ClientRuntime.EndpointsRequestContext
import let AWSClientRuntime.awsPartitionJSON
import protocol ClientRuntime.EndpointsRequestContextProviding
import struct ClientRuntime.DefaultEndpointResolver
import struct ClientRuntime.StaticEndpointResolver
Expand Down Expand Up @@ -62,7 +62,7 @@ extension DefaultEndpointResolver {
private static let ruleSet = "{\"version\":\"1.0\",\"parameters\":{\"Region\":{\"builtIn\":\"AWS::Region\",\"required\":false,\"documentation\":\"The AWS region used to dispatch the request.\",\"type\":\"String\"},\"UseDualStack\":{\"builtIn\":\"AWS::UseDualStack\",\"required\":true,\"default\":false,\"documentation\":\"When true, use the dual-stack endpoint. If the configured endpoint does not support dual-stack, dispatching the request MAY return an error.\",\"type\":\"Boolean\"},\"UseFIPS\":{\"builtIn\":\"AWS::UseFIPS\",\"required\":true,\"default\":false,\"documentation\":\"When true, send this request to the FIPS-compliant regional endpoint. If the configured endpoint does not have a FIPS compliant endpoint, dispatching the request will return an error.\",\"type\":\"Boolean\"},\"Endpoint\":{\"builtIn\":\"SDK::Endpoint\",\"required\":false,\"documentation\":\"Override the endpoint used to send this request\",\"type\":\"String\"}},\"rules\":[{\"conditions\":[{\"fn\":\"isSet\",\"argv\":[{\"ref\":\"Endpoint\"}]}],\"rules\":[{\"conditions\":[{\"fn\":\"booleanEquals\",\"argv\":[{\"ref\":\"UseFIPS\"},true]}],\"error\":\"Invalid Configuration: FIPS and custom endpoint are not supported\",\"type\":\"error\"},{\"conditions\":[{\"fn\":\"booleanEquals\",\"argv\":[{\"ref\":\"UseDualStack\"},true]}],\"error\":\"Invalid Configuration: Dualstack and custom endpoint are not supported\",\"type\":\"error\"},{\"conditions\":[],\"endpoint\":{\"url\":{\"ref\":\"Endpoint\"},\"properties\":{},\"headers\":{}},\"type\":\"endpoint\"}],\"type\":\"tree\"},{\"conditions\":[{\"fn\":\"isSet\",\"argv\":[{\"ref\":\"Region\"}]}],\"rules\":[{\"conditions\":[{\"fn\":\"aws.partition\",\"argv\":[{\"ref\":\"Region\"}],\"assign\":\"PartitionResult\"}],\"rules\":[{\"conditions\":[{\"fn\":\"booleanEquals\",\"argv\":[{\"ref\":\"UseFIPS\"},true]},{\"fn\":\"booleanEquals\",\"argv\":[{\"ref\":\"UseDualStack\"},true]}],\"rules\":[{\"conditions\":[{\"fn\":\"booleanEquals\",\"argv\":[true,{\"fn\":\"getAttr\",\"argv\":[{\"ref\":\"PartitionResult\"},\"supportsFIPS\"]}]},{\"fn\":\"booleanEquals\",\"argv\":[true,{\"fn\":\"getAttr\",\"argv\":[{\"ref\":\"PartitionResult\"},\"supportsDualStack\"]}]}],\"rules\":[{\"conditions\":[],\"endpoint\":{\"url\":\"https://acm-pca-fips.{Region}.{PartitionResult#dualStackDnsSuffix}\",\"properties\":{},\"headers\":{}},\"type\":\"endpoint\"}],\"type\":\"tree\"},{\"conditions\":[],\"error\":\"FIPS and DualStack are enabled, but this partition does not support one or both\",\"type\":\"error\"}],\"type\":\"tree\"},{\"conditions\":[{\"fn\":\"booleanEquals\",\"argv\":[{\"ref\":\"UseFIPS\"},true]}],\"rules\":[{\"conditions\":[{\"fn\":\"booleanEquals\",\"argv\":[{\"fn\":\"getAttr\",\"argv\":[{\"ref\":\"PartitionResult\"},\"supportsFIPS\"]},true]}],\"rules\":[{\"conditions\":[{\"fn\":\"stringEquals\",\"argv\":[{\"fn\":\"getAttr\",\"argv\":[{\"ref\":\"PartitionResult\"},\"name\"]},\"aws-us-gov\"]}],\"endpoint\":{\"url\":\"https://acm-pca.{Region}.amazonaws.com\",\"properties\":{},\"headers\":{}},\"type\":\"endpoint\"},{\"conditions\":[],\"endpoint\":{\"url\":\"https://acm-pca-fips.{Region}.{PartitionResult#dnsSuffix}\",\"properties\":{},\"headers\":{}},\"type\":\"endpoint\"}],\"type\":\"tree\"},{\"conditions\":[],\"error\":\"FIPS is enabled but this partition does not support FIPS\",\"type\":\"error\"}],\"type\":\"tree\"},{\"conditions\":[{\"fn\":\"booleanEquals\",\"argv\":[{\"ref\":\"UseDualStack\"},true]}],\"rules\":[{\"conditions\":[{\"fn\":\"booleanEquals\",\"argv\":[true,{\"fn\":\"getAttr\",\"argv\":[{\"ref\":\"PartitionResult\"},\"supportsDualStack\"]}]}],\"rules\":[{\"conditions\":[],\"endpoint\":{\"url\":\"https://acm-pca.{Region}.{PartitionResult#dualStackDnsSuffix}\",\"properties\":{},\"headers\":{}},\"type\":\"endpoint\"}],\"type\":\"tree\"},{\"conditions\":[],\"error\":\"DualStack is enabled but this partition does not support DualStack\",\"type\":\"error\"}],\"type\":\"tree\"},{\"conditions\":[],\"endpoint\":{\"url\":\"https://acm-pca.{Region}.{PartitionResult#dnsSuffix}\",\"properties\":{},\"headers\":{}},\"type\":\"endpoint\"}],\"type\":\"tree\"}],\"type\":\"tree\"},{\"conditions\":[],\"error\":\"Invalid Configuration: Missing Region\",\"type\":\"error\"}]}"

init() throws {
try self.init(partitions: AWSClientRuntime.awsPartitionJSON, ruleSet: Self.ruleSet)
try self.init(partitions: AWSClientRuntime.PartitionData.loadAWSPartitionJSON(), ruleSet: Self.ruleSet)
}
}

Expand Down
Loading
Loading