KEDA Scaling Issue: Jobs Continuously Created Until Karpenter Finishes Node Provisioning

[davidt-gh]

Hi all,

I’m encountering an issue with KEDA autoscaling while integrating it with Selenium Grid and Karpenter for node provisioning.

Scenario

• I’m running Selenium tests and want KEDA to create exactly 3 jobs when I request 3 sessions.
• However, KEDA continuously creates new jobs every few seconds until Karpenter finishes provisioning new nodes.

This results in more jobs being created than the actual number of Selenium sessions required.

Current Setup

Here is part of my values.yaml configuration:

selenium-grid:
  chromeNode:
    scaledOptions:
      minReplicaCount: 0
      maxReplicaCount: 15
      pollingInterval: 10
  autoscaling:
    enabled: true
    scalingType: job

Issue
• KEDA seems to poll the conditions too frequently (pollingInterval: 10) and creates new jobs when nodes are not yet ready.
• This causes KEDA to overscale beyond the requested session count.

Goal

I want KEDA to:
1. Create exactly 3 jobs for 3 sessions (no more, no less).
2. Avoid continuously spawning jobs while Karpenter provisions new nodes.

Proposed Solution

I believe the following changes might help resolve the issue:

1. Increase pollingInterval and add a cooldownPeriod to prevent rapid scaling:

pollingInterval: 30
cooldownPeriod: 300

2. Limit maxReplicaCount to the exact number of sessions requested:

maxReplicaCount: 3

3. Adjust KEDA triggers to scale based on exact pending Selenium sessions, possibly using Prometheus metrics.

Questions
1. How can I fine-tune KEDA’s behavior to strictly match the requested number of sessions without overscaling?
2. Are there best practices to integrate KEDA with Karpenter to ensure smooth node scaling and job creation?

Any guidance or insights would be greatly appreciated!

Thank you!

[VietND96]

Thanks for your detailed description.
In case Until Karpenter finishes provisioning new nodes, pods scaled up and kept status as Pending only, right?

[davidt-gh]

Yes.
But if I sent 3 jobs (in total), until karpenter provide new nodes, I have 8 jobs == 8 8 pods, but only 3 are running.
Then after few minutes all of them cool down and removed.

[VietND96]

In case new pods scaled up with status Pending for a while, and how will KEDA core behave, which I am not sure much. I will recheck and let you know if I can find the correct answer.
In case pods scaled up and running. However, it still having overate scales, this one I am still working on improving the scaler logic. Hopefully, the upcoming KDEA core 2.16.1 to be released will have those improvements.

[davidt-gh]

Thanks!
I've thought it will already implemented when I selected Job, which suppose to create relevant jobs per needed sessions.
When I ask for X session it should create X jobs, and they need to wait in Pending mode until K8s complete them. Creating new jobs have no value in here.. Just saying.