Multiple schedule in disruption budgets don't work

[nantiferov]

Description

Hi,

I'm not sure if this is expected behavior, but I don't see it mentioned in docs https://karpenter.sh/v1.0/concepts/disruption/

Basically, I want to invert logic, described here https://karpenter.sh/docs/concepts/nodepools/

    budgets:
    - nodes: 10%
    - schedule: "0 9 * * mon-fri" # On Weekdays during business hours, don't do any deprovisioning.
      duration: 8h
      nodes: "0"

To have disruptions only during working hours. I tried to achieve this with something like:

    budgets:
    - nodes: 10%
    - schedule: 0 16 * * mon-fri
      duration: 16h
      nodes: "0"
     
    - schedule: 0 0 * * sat-sun
      duration: 24h
      nodes: "0"

Observed Behavior:

Errors in karpenter logs: {"level":"ERROR","time":"2024-12-05T14:34:31.529Z","logger":"controller","message":"detected disruption budget errors","commit":"a2875e3","controller":"disruption","namespace":"","name":"","reconcileID":"ca5962e8-d939-4453-a8be-4ca11fdf9bf4","error":"invalid disruption budgets in nodepool xxx, invariant violated, invalid cron %!s(<nil>)"}

Expected Behavior:

Maybe it could be achieved differently, but end goal with this - to have disruptions only during working hours.

Reproduction Steps (Please include YAML):

Provided before.

Versions:

• Chart Version: 1.0.8
• Kubernetes Version (kubectl version): 1.30

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[k8s-ci-robot]

This issue is currently awaiting triage.

If Karpenter contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[nantiferov]

Hmm, I might be wrong, but after some tests I think this actually provides what I want:

    budgets:
    - schedule: "0 9 * * mon-fri" # On Weekdays during business hours, don't do any deprovisioning.
      duration: 8h
      nodes: 10%

Basically nodes: 10% moved to schedule and nodes: "0" removed

[nantiferov]

Hmm, tests shows that schedule I mentioned in #1866 (comment) actually doesn't work and disruptions are happening any time.

Maybe I'm missing something?

[jigisha620]

HI @nantiferov,
The standard cron syntax expects day in numerical form. The error that you have shared indicates incorrect syntax. You can use -

budgets:
    - schedule: "0 9 * * 1-5" # On Weekdays during business hours, don't do any deprovisioning.
      duration: 8h
      nodes: 0

[nantiferov]

Hmm, thank you, will double check with 1-5, but I remember vaguely I tested this as well.

Btw, this syntax is listed in docs https://karpenter.sh/docs/reference/settings/#feature-gates

[nantiferov]

But from logic perspective, am I right, that this configuration should prevent Karpenter to do any disruptions outside of schedule time (working hours)?

  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized
    consolidateAfter: 0s
    budgets:
      - schedule: 0 8 * * 1-5 # working days and hours
        duration: 8h
        nodes: 10%

[jigisha620]

The configuration that you have shared will allow 10% nodes to get disrupted between Mon-Fri starting at 08:00 up to 8 hours. If you don't want any kind of disruption happening in that period you want to set nodes: 0.

[jigisha620]

Also sorry for the confusion I didn't mean that you can't specify "mon-fri", I was just trying to say that the standard is to use number. But even then there's something wrong in the syntax and that's why you are seeing this error -
invariant violated, invalid cron %!s(<nil>). I think you need to specify cron as a string.

[nantiferov]

The configuration that you have shared will allow 10% nodes to get disrupted between Mon-Fri starting at 08:00 up to 8 hours.

But will it do any disruptions outside of these hours?
My goal is to invert logic shown in docs, as I mentioned initially. I.e. allow disruptions only during working hours.

Basically, I want to invert logic, described here https://karpenter.sh/docs/concepts/nodepools/

And so far it looks at least reasons: Empty disruptions are happening at any time.

I think you need to specify cron as a string.

Thank you, will try this