Attribute-based Semantic Equality

[austinvalle]

Module version

github.com/hashicorp/terraform-plugin-framework v1.13.0

Background

Currently, the primary method of handling normalization from remote APIs in terraform-plugin-framework is type-based, using a custom type that implements semantic equality. This is a reasonable place to handle semantic equality, especially for data like JSON strings that need to ignore whitespace or IPv6 addresses that have multiple representations of equivalent data.

Semantic equality logic is used to compare a "prior value" to a "proposed new value", and if the two are determined to be "semantically equal", then the prior value can be kept. This logic is executed at the following points:

• During ApplyResourceChange, after the Create method is called. Prior value is PlannedState, proposed new value is NewState returned by the Create method.
• During ApplyResourceChange, after the Update method is called. Prior value is PlannedState, proposed new value is NewState returned by the Update method.
• During ReadResource, after the Read method is called. Prior value is CurrentState, proposed new value is NewState returned by the Read method.

Problem

There are some edge cases where using a custom type for semantic equality is not preferred. For example, in the case of the AWS provider, custom types are heavily used to facilitate their reflection flex package with data handling (mapping from TF to AWS APIs):

• Custom types: https://github.com/hashicorp/terraform-provider-aws/tree/main/internal/framework/types
• Flex: https://github.com/hashicorp/terraform-provider-aws/tree/main/internal/framework/flex
• Example Usage in Schema: https://github.com/hashicorp/terraform-provider-aws/blob/bcc77522c1b5d2b8ca21464df041d4ca37b3a7cf/internal/service/lexv2models/intent.go#L601

In situations where generic custom types like this are already being used for data handling, if semantic equality logic is also needed, it becomes more difficult to introduce logic on a per-attribute-basis to solve unwanted drift issues like these:

• [Bug]: aws_lexv2models_slot resource cannot create slot terraform-provider-aws#36845 (comment)
• [Bug]: lexv2models/intent: confirmation_setting produces diffs for prompt_attempts_specification terraform-provider-aws#35346

Workarounds

• As with all semantic equality/drift/normalization problems, it's possible for providers to implement the logic manually in the Create/Update/Read methods
• In the case of using a custom type, it's possible to statically define the *SemanticEquals method, then optionally pass a function when defining the custom type:

resp.Schema = schema.Schema{
  Attributes: map[string]schema.Attribute{
    "example_attr": schema.StringAttribute{
      CustomType: customtypes.ExampleCustomType{
        SemanticEqualityFunc: HandleSemanticEquality,
      },
      Optional: true,
    },
  },
}

func HandleSemanticEquality(ctx context.Context, priorVal basetypes.StringValuable, proposedVal basetypes.StringValuable) (bool, diag.Diagnostics) {
  // Do conditional semantic equality here
  return true, nil
}

Passing this function along from the type to the value:

func (v ExampleCustomValue) StringSemanticEquals(ctx context.Context, value basetypes.StringValuable) (bool, diag.Diagnostics) {
  if v.SemanticEqualityFunc != nil {
    return v.SemanticEqualityFunc(ctx, v.StringValue, value)
  }

  // No logic defined, so the values can't be semantically equal.
  return false, nil
}

While this workaround will achieve the equivalent of something like attribute-based semantic equality it's certainly not a straightforward implementation.

Proposal

TBD on possible SDK design

In addition to type-based semantic equality, which would still be the preferred route, we could also implement attribute-based semantic equality, to allow defining equivalent semantic equality logic directly on the attributes in resource/schema and datasource/schema. This was originally considered during #70, but was abandoned due to the increased complexity, lack of concrete use-cases, and the confusion of implementing multiple competing ways to solve the problem.

Considerations

There are some internal refactoring hurdles that we'll need to clear if we want to support both, notably, all of our semantic equality logic is written based off the value itself, and refactoring that logic to be similar to plan modification/validation will be non-trivial, see comment.

• We'll need to determine how nested/collection value type semantic equality logic will interact with nested attribute/collection attribute logic (order, precedence, etc.)

Documentation will also need to be updated, possibly splitting semantic equality into it's own page so we can properly describe both approaches and which to prefer (type-based should always be preferred to attribute-based).

Refs

• Classifying normalization vs. drift #70
• Type-Based Semantic Equality #737

[chrismarget-j]

Attribute-based expressions of semantic equality behavior sound pretty interesting to me as a provider developer. I don't have any use cases in mind, but expect I'd find 'em if it were available.