Broker class based defaults

[Leo6Leo]

Fixes #5992

Proposed Changes

• Allowing users to set the default configuration for the namespaces adds more flexibility, in case they want to have the different configuration settings for different namespaces.
  Corner cases - what will happen if the user do these:

Pre-review Checklist

• At least 80% unit test coverage
• E2E tests for any new behavior
• Docs PR for any user-facing impact
• Spec PR for any new API feature
• Conformance test for any change to the spec

Release Note

Docs

knative/docs#6069

[knative-prow]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[codecov]

Codecov Report

Attention: Patch coverage is 85.00000% with 9 lines in your changes missing coverage. Please review.

Project coverage is 67.88%. Comparing base (332d974) to head (0e62b00).
Report is 53 commits behind head on main.

Files	Patch %	Lines
pkg/apis/config/defaults.go	79.54%	4 Missing and 5 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7631      +/-   ##
==========================================
- Coverage   67.92%   67.88%   -0.04%     
==========================================
  Files         367      368       +1     
  Lines       17323    17586     +263     
==========================================
+ Hits        11767    11939     +172     
- Misses       4823     4897      +74     
- Partials      733      750      +17     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Leo6Leo]

Steps to test

1. Apply the following default br config

# Copyright 2020 The Knative Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

apiVersion: v1
kind: ConfigMap
metadata:
  name: config-br-defaults
  namespace: knative-eventing
  labels:

data:
  default-br-config: |
    clusterDefault:
      brokerClass: default-cluster-class
      apiVersion: v1
      kind: ConfigMap
      name: config-default-cluster-class
      namespace: knative-eventing
      delivery:
        retry: 3
        deadLetterSink:
          ref:
            apiVersion: serving.knative.dev/v1
            kind: Service
            name: mt-handle-error
            namespace: knative-eventing
        backoffPolicy: exponential
        backoffDelay: PT0.2S
      brokerClasses:
        cluster-class-2:
          delivery:
            retry: 3
            deadLetterSink:
              ref:
                apiVersion: serving.knative.dev/v1
                kind: Service
                name: mt-handle-error
                namespace: knative-eventing
            backoffPolicy: exponential
            backoffDelay: PT0.2S

          apiVersion: v1
          kind: ConfigMap
          name: config-cluster-class-2
          namespace: knative-eventing

        shared-class:
          delivery:
            retry: 3
            deadLetterSink:
              ref:
                apiVersion: serving.knative.dev/v1
                kind: Service
                name: kafka-handle-error
                namespace: knative-eventing
            backoffPolicy: exponential
            backoffDelay: PT0.2S

          apiVersion: v1
          kind: ConfigMap
          name: config-shared-class
          namespace: knative-eventing


    namespaceDefaults:
      namespace-1:
        brokerClass: namespace-1-class
        apiVersion: v1
        kind: ConfigMap
        name: config-namespace-1-class
        namespace: namespace-1
        delivery:
          retry: 5
          deadLetterSink:
            ref:
              apiVersion: serving.knative.dev/v1
              kind: Service
              name: someother-handle-error
              namespace: knative-eventing
          backoffPolicy: linear
          backoffDelay: PT0.2S
        brokerClasses:
          namespace-1-class-2:
            delivery:
              retry: 3
              deadLetterSink:
                ref:
                  apiVersion: serving.knative.dev/v1
                  kind: Service
                  name: mt-handle-error
                  namespace: knative-eventing
              backoffPolicy: exponential
              backoffDelay: PT0.2S

            apiVersion: v1
            kind: ConfigMap
            name: config-namespace-1-class-2
            namespace: knative-eventing

          shared-class:
            delivery:
              retry: 3
              deadLetterSink:
                ref:
                  apiVersion: serving.knative.dev/v1
                  kind: Service
                  name: kafka-handle-error
                  namespace: knative-eventing
              backoffPolicy: exponential
              backoffDelay: PT0.2S

            apiVersion: v1
            kind: ConfigMap
            name: config-shared-class-in-namespace-1
            namespace: knative-eventing

      namespace-2:
        brokerClass: default-namespace-2-class


      namespace-3:

2. Create brokers and then describe it to see whether the correct broker config is used.

[Leo6Leo]

/unhold
The PR is ready for review.

[Cali0707]

@Leo6Leo is there a plan to add a rekt test for this feature? Maybe in another PR?

[Cali0707]

Does it make sense to return an error here instead? At this point we know there is no config for the requested broker class, so we might get weird behaviour for brokers with a random config that doesn't match their class. @pierDipi @creydr any ideas?

[Cali0707]

Ping @Leo6Leo - wdyt here? From the previous comment, my intuition is that this should return an error, but I'm not sure

[Leo6Leo]

If the requested brokerClass lacks a specific configuration, the system should fallback to using the default brokerClass and its configuration, rather than throwing an error. This behavior aligns with the purpose of having a default configuration in place in my opinion. But returning the error also makes sense to me.

[Cali0707]

Ping @Leo6Leo - wdyt here? From the previous comment, my intuition is that this should return an error, but I'm not sure

[Leo6Leo]

Note: Thanks to @Cali0707 's suggestion , now trying to gradually deprecate the deafult broker class and config here.

Current behavior:

• When the same broker class appears in both brokerClass and brokerClasses:
  • The brokerClasses map takes precedence and overwrites the default.

Future plan:

• We will transition to using only the brokerClasses map for storing default configurations for different broker classes.

Rationale:

• In cases where a broker class is defined in both places, it's more intuitive and maintainable to have a single source of truth.
• Using only the brokerClasses map allows for consistent handling of default configurations across all broker classes.

This change simplifies the configuration model and reduces potential confusion or conflicts between the two ways of specifying default broker configurations.

[Cali0707]

@Leo6Leo I think it's worth still having a "default" class available, just maybe not also a config. Then the default class is just referring to the one of the classes in the brokerClasses map. So, basically the default config would become brokerClasses[defaultClassName]

[Leo6Leo]

The documentation is written here in this PR knative/docs#6069

[Cali0707]

As far as I can tell this looks good, but since I was involved in a lot of the code maybe @creydr or @matzew could provide a final review here?

Looks Good

[pierDipi]

/lgtm
/approve

[knative-prow]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Leo6Leo, pierDipi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [Leo6Leo,pierDipi]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment