Updated container garbage collection with accurate information #48001

T-Lakshmi · 2024-09-19T13:32:29Z

k8s-ci-robot · 2024-09-19T13:32:35Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign princesso for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

content/en/docs/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

netlify · 2024-09-19T13:41:53Z

✅ Pull request preview available for checking

Built without sensitive environment variables

Name	Link
🔨 Latest commit	`e661ab0`
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-io-main-staging/deploys/66ecf727b594800007af69ee
😎 Deploy Preview	https://deploy-preview-48001--kubernetes-io-main-staging.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

T-Lakshmi · 2024-09-19T13:44:46Z

/sig node
/kind documentation

T-Lakshmi · 2024-09-19T13:48:59Z

Preview page

Hi @dipesh-rawat, @Princesso,
Pinging for review, PTAL!

bart0sh · 2024-09-19T16:47:46Z

/cc @haircommander @esotsal

haircommander · 2024-09-19T16:55:25Z

content/en/docs/concepts/architecture/garbage-collection.md

+### Container garbage collection {#container-image-garbage-collection} (deprecated)
+
+{{< caution >}}
+Container garbage collection is deprecated infavour of `--eviction-hard` or


in favour *

haircommander · 2024-09-19T16:55:43Z

content/en/docs/concepts/architecture/garbage-collection.md

+
+{{< caution >}}
+Container garbage collection is deprecated infavour of `--eviction-hard` or
+`--eviction-soft` and Will be removed in a future version. For more details on


haircommander · 2024-09-19T16:59:06Z

thanks! content wise it looks good, I just have a couple of stylistic nits

T-Lakshmi · 2024-09-20T04:19:35Z

@haircommander,
Thanks for the review and suggestions.
I have updated the content with your suggestions, PTAL!

esotsal · 2024-09-20T09:54:02Z

Thanks
/lgtm

T-Lakshmi · 2024-09-20T12:52:08Z

Thnks for the review @haircommander, @esotsal

/assign @Princesso
Assigning to @Princesso for approval, PTAL!

sftim

This does need an update before we can merge it, to make sure the heading is right.

/lgtm cancel

sftim · 2024-09-20T12:52:54Z

content/en/docs/concepts/architecture/garbage-collection.md

+### Container garbage collection {#container-image-garbage-collection} (deprecated)
+
+{{< caution >}}
+Container garbage collection is deprecated in favour of `--eviction-hard` or
+`--eviction-soft` and will be removed in a future version. For more details on
+eviction policy, see [node-pressure-eviction](/docs/concepts/scheduling-eviction/node-pressure-eviction/).
+{{< /caution >}}


These command line arguments are deprecated, and the format isn't right.
We also avoid statements about the future, although we do make reasonable exceptions to that general guidance.

Try this:

Suggested change

### Container garbage collection {#container-image-garbage-collection} (deprecated)

{{< caution >}}

Container garbage collection is deprecated in favour of `--eviction-hard` or

`--eviction-soft` and will be removed in a future version. For more details on

eviction policy, see [node-pressure-eviction](/docs/concepts/scheduling-eviction/node-pressure-eviction/).

{{< /caution >}}

### Legacy garbage collection for containers (deprecated) {#container-image-garbage-collection}

{{< feature-state for_k8s_version="v1.9" state="deprecated" >}}

{{< note >}}

Container garbage collection is deprecated, in favor of using the kubelet configuration file

entries `evictionHard` and `evictionSoft` to manage overall Pod

{{< glossary_tooltip text="eviction" term_id="eviction" >}}.

{{< /note >}}

ℹ️ I am not sure about which version deprecated setting these command line arguments.

Am I right that eviction is always at the Pod level, whereas these deprecated settings apply at the container level?

sftim · 2024-09-20T12:52:56Z

content/en/docs/concepts/architecture/garbage-collection.md

 The kubelet garbage collects unused containers based on the following variables,
 which you can define:

-* `MinAge`: the minimum age at which the kubelet can garbage collect a
-  container. Disable by setting to `0`.
-* `MaxPerPodContainer`: the maximum number of dead containers each Pod 
-  can have. Disable by setting to less than `0`.
+* `MinAge`: the minimum age at which the kubelet can garbage collect a container.
+  Disable by setting a kubelet config flag `--minimum-container-ttl-duration` to `0`.
+* `MaxPerPodContainer`: the maximum number of dead containers each Pod can have.
+  Disable by setting a kubelet config flag `--maximum-dead-containers-per-container`
+  to less than `0`.
 * `MaxContainers`: the maximum number of dead containers the cluster can have.
-  Disable by setting to less than `0`.
+  Disable by setting a kubelet config flag `--maximum-dead-containers`
+  to less than `0`.


Is this right?

Suggested change

The kubelet garbage collects unused containers based on the following variables,

which you can define:

* `MinAge`: the minimum age at which the kubelet can garbage collect a

container. Disable by setting to `0`.

* `MaxPerPodContainer`: the maximum number of dead containers each Pod

can have. Disable by setting to less than `0`.

* `MinAge`: the minimum age at which the kubelet can garbage collect a container.

Disable by setting a kubelet config flag `--minimum-container-ttl-duration` to `0`.

* `MaxPerPodContainer`: the maximum number of dead containers each Pod can have.

Disable by setting a kubelet config flag `--maximum-dead-containers-per-container`

to less than `0`.

* `MaxContainers`: the maximum number of dead containers the cluster can have.

Disable by setting to less than `0`.

Disable by setting a kubelet config flag `--maximum-dead-containers`

to less than `0`.

The kubelet garbage collects dead containers within existing Pods. This garbage collection is

based on the values for several related command line arguments, that are all deprecated:

`--minimum-container-ttl-duration`

: the minimum age at which the kubelet can garbage collect a dead container. Disable by setting to 0. Setting a value for `--minimum-container-ttl-duration` is deprecated.

`--maximum-dead-containers`

: limits the total number of dead containers for this node. The default is unlimited. You can set this to an integer that is zero or higher in order to set legacy dead container limits, node-wide. Setting a value for `--maximum-dead-containers` is deprecated.

`--maximum-dead-containers-per-container`: the maximum number of dead containers each Pod

can have. The default value for this setting is 1; however, you can specify unlimited dead containers by setting this value to 0. Note that setting a value for `--maximum-dead-containers-per-container` is deprecated.

yeah I agree we should refer folks to the kubelet configuration options as we're redirecting folks away from using the CLI

WDYT ^ @T-Lakshmi ?

@haircommander,
Yes, I also agree to mention about CLI usage of these arguments.

But my concern is the below statement is true or not?

Container garbage collection is deprecated in favour of --eviction-hard or --eviction-soft and will be removed in a future version.

also when container GC got deprecated and what are the recommended way to use as alternate to container GC?

Also here it mentioned old logs are stored outside of container's context.

Is that true? if true, in which location old logs are storing now?
Please share your inputs.

T-Lakshmi · 2024-09-24T13:56:29Z

I think sig-node leaders may be able to clarify this.

Hello @kubernetes/sig-node-leads, @kubernetes/sig-node-bugs,
we would appreciate your feedback to ensure the accuracy of these proposed changes. Thanks!

tengqm · 2024-09-25T03:29:31Z

content/en/docs/concepts/architecture/garbage-collection.md

+
+{{< caution >}}
+Container garbage collection is deprecated in favour of `--eviction-hard` or
+`--eviction-soft` and will be removed in a future version. For more details on


container gc and eviction are different things ...

Hello @tengqm,

container gc and eviction are different things ...

Can you clarify on the below points of container gc?

Is --minimum-container-ttl-duration, --maximum-dead-containers and --maximum-dead-containers-per-container related to container gc?

In which version container gc got deprecated?

Is there any alternate recommended way for container garbage collection? What it is?

Container GC is a task performed by kubelet regularly. The command line flags you mentioned are about container GC. The things that got deprecated, IIRC, are the command line flags. The intent of the deprecation was to reduce knobs for users to tune.
With the mechanism still there in kubelet, container GC won't go away any time soon. I'm not aware of anyone pushing this forward. In other words, the "deprecation" itself may get rotten (or deprecated).

Thanks for sharing feedback @tengqm.

What others say on this? Please share your opinion.

Container GC is a task performed by kubelet regularly. The command line flags you mentioned are about container GC. The things that got deprecated, IIRC, are the command line flags.

That looks right. Most kubelet command line arguments are deprecated (you should use the configuration file), and also the settings for container / container image GC were a bit legacy anyway.

T-Lakshmi · 2024-10-01T09:36:28Z

/cc @kubernetes/sig-node-pr-reviews

k8s-ci-robot · 2024-10-01T09:36:30Z

@T-Lakshmi: GitHub didn't allow me to request PR reviews from the following users: kubernetes/sig-node-pr-reviews.

Note that only kubernetes members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

/cc @kubernetes/sig-node-pr-reviews

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot · 2024-10-01T09:38:38Z

@T-Lakshmi: GitHub didn't allow me to request PR reviews from the following users: kubernetes/sig-node-pr-reviews.

Note that only kubernetes members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

/cc @kubernetes/sig-node-pr-reviews

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

esotsal · 2024-10-19T10:52:54Z

/unassign

esotsal · 2024-10-19T10:54:26Z

I think sig-node leaders may be able to clarify this.

Hello @kubernetes/sig-node-leads, @kubernetes/sig-node-bugs, we would appreciate your feedback to ensure the accuracy of these proposed changes. Thanks!

/cc @SergeyKanzhelev @haircommander WDYT ?

T-Lakshmi · 2024-10-21T04:32:01Z

Thanks @esotsal.

/cc @mrunalp

looping @mrunalp also for feedback.

T-Lakshmi · 2024-10-29T07:09:26Z

Added PR to sig node weekly meeting Agenda.
https://docs.google.com/document/d/1Ne57gvidMEWXR70OxxnRkYquAoMpt56o75oZtg-OeBg/edit?tab=t.0

@SergeyKanzhelev @mrunalp @haircommander Please have a look!

SergeyKanzhelev · 2024-11-05T22:58:01Z

Added PR to sig node weekly meeting Agenda. https://docs.google.com/document/d/1Ne57gvidMEWXR70OxxnRkYquAoMpt56o75oZtg-OeBg/edit?tab=t.0

@SergeyKanzhelev @mrunalp @haircommander Please have a look!

Sorry, I wasn't looking at the deprecations around this in a while. It will simplify reviews if you can provide easy to check links on what and where we announced was deprecated.

I will likely have time to look at this only after KubeCon

esotsal · 2024-11-11T17:17:45Z

It will simplify reviews if you can provide easy to check links on what and where we announced was deprecated.

This PR was created as response to this comment discussed in sig-node some time ago. Googling i found deprecation is mentioned here With git blame found deprecation comment has been commited 7 years ago in 1.9.0 here

updated container garbage policy current state

7f6ab53

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Sep 19, 2024

k8s-ci-robot added the language/en Issues or PRs related to English language label Sep 19, 2024

k8s-ci-robot requested review from dipesh-rawat and Princesso September 19, 2024 13:32

k8s-ci-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Sep 19, 2024

k8s-ci-robot added sig/node Categorizes an issue or PR as relevant to SIG Node. kind/documentation Categorizes issue or PR as related to documentation. labels Sep 19, 2024

T-Lakshmi mentioned this pull request Sep 19, 2024

Unclear with Container garbage collection configuration kubernetes/kubernetes#127157

Open

k8s-ci-robot requested review from esotsal and haircommander September 19, 2024 16:47

haircommander reviewed Sep 19, 2024

View reviewed changes

Corrected stylish nits

e661ab0

k8s-ci-robot assigned esotsal Sep 20, 2024

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 20, 2024

k8s-ci-robot assigned Princesso Sep 20, 2024

sftim reviewed Sep 20, 2024

View reviewed changes

k8s-ci-robot assigned sftim Sep 20, 2024

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 20, 2024

k8s-ci-robot added the kind/bug Categorizes issue or PR as related to a bug. label Sep 24, 2024

tengqm reviewed Sep 25, 2024

View reviewed changes

k8s-ci-robot unassigned esotsal Oct 19, 2024

k8s-ci-robot requested a review from mrunalp October 21, 2024 04:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updated container garbage collection with accurate information #48001

Updated container garbage collection with accurate information #48001

T-Lakshmi commented Sep 19, 2024

k8s-ci-robot commented Sep 19, 2024

netlify bot commented Sep 19, 2024 •

edited

Loading

T-Lakshmi commented Sep 19, 2024

T-Lakshmi commented Sep 19, 2024

bart0sh commented Sep 19, 2024

haircommander Sep 19, 2024

haircommander Sep 19, 2024

haircommander commented Sep 19, 2024

T-Lakshmi commented Sep 20, 2024

esotsal commented Sep 20, 2024

T-Lakshmi commented Sep 20, 2024

sftim left a comment

sftim Sep 20, 2024 •

edited

Loading

sftim Sep 20, 2024 •

edited

Loading

haircommander Oct 29, 2024

haircommander Oct 31, 2024

T-Lakshmi Nov 4, 2024

T-Lakshmi Nov 4, 2024

T-Lakshmi commented Sep 24, 2024 •

edited

Loading

tengqm Sep 25, 2024

T-Lakshmi Oct 1, 2024

tengqm Oct 1, 2024

T-Lakshmi Nov 4, 2024

sftim Nov 14, 2024

T-Lakshmi commented Oct 1, 2024

k8s-ci-robot commented Oct 1, 2024

k8s-ci-robot commented Oct 1, 2024

esotsal commented Oct 19, 2024

esotsal commented Oct 19, 2024

T-Lakshmi commented Oct 21, 2024

T-Lakshmi commented Oct 29, 2024

SergeyKanzhelev commented Nov 5, 2024

esotsal commented Nov 11, 2024 •

edited

Loading

-### Container garbage collection {#container-image-garbage-collection} (deprecated)
-{{< caution >}}
-Container garbage collection is deprecated in favour of `--eviction-hard` or
-`--eviction-soft` and will be removed in a future version. For more details on
-eviction policy, see [node-pressure-eviction](/docs/concepts/scheduling-eviction/node-pressure-eviction/).
-{{< /caution >}}
+### Legacy garbage collection for containers (deprecated) {#container-image-garbage-collection}
+{{< feature-state for_k8s_version="v1.9" state="deprecated" >}}
+{{< note >}}
+Container garbage collection is deprecated, in favor of using the kubelet configuration file
+entries `evictionHard` and `evictionSoft` to manage overall Pod
+{{< glossary_tooltip text="eviction" term_id="eviction" >}}.
+{{< /note >}}

Updated container garbage collection with accurate information #48001

Are you sure you want to change the base?

Updated container garbage collection with accurate information #48001

Conversation

T-Lakshmi commented Sep 19, 2024

k8s-ci-robot commented Sep 19, 2024

netlify bot commented Sep 19, 2024 • edited Loading

✅ Pull request preview available for checking

T-Lakshmi commented Sep 19, 2024

T-Lakshmi commented Sep 19, 2024

bart0sh commented Sep 19, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

haircommander commented Sep 19, 2024

T-Lakshmi commented Sep 20, 2024

esotsal commented Sep 20, 2024

T-Lakshmi commented Sep 20, 2024

sftim left a comment

Choose a reason for hiding this comment

sftim Sep 20, 2024 • edited Loading

Choose a reason for hiding this comment

sftim Sep 20, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

T-Lakshmi commented Sep 24, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

T-Lakshmi commented Oct 1, 2024

k8s-ci-robot commented Oct 1, 2024

k8s-ci-robot commented Oct 1, 2024

esotsal commented Oct 19, 2024

esotsal commented Oct 19, 2024

T-Lakshmi commented Oct 21, 2024

T-Lakshmi commented Oct 29, 2024

SergeyKanzhelev commented Nov 5, 2024

esotsal commented Nov 11, 2024 • edited Loading

netlify bot commented Sep 19, 2024 •

edited

Loading

sftim Sep 20, 2024 •

edited

Loading

sftim Sep 20, 2024 •

edited

Loading

T-Lakshmi commented Sep 24, 2024 •

edited

Loading

esotsal commented Nov 11, 2024 •

edited

Loading