Konnectivity is not offered, and therefore no eBPF for Cilium

[almereyda]

What happened

In creating a user cluster, no Konnectivity is offered, despite it is now default kubermatic/kubermatic#8129

whyfore Cilium does not offer eBPF mode.

(ref. image and description in #4533 (comment))

This results in the deployment of OpenVPN as SDN

Expected behavior

Konnectivity is available for selection, and Cilium can be set to eBPF mode.

[almereyda]

This could be traced back to the missing feature gate for Konnectivity. I thought it defaults to true nowadays, and does not need to be set manually. Yet Konnectivity only appeared after following the dedicated setup instructions from

• CNI (Container Network Interface) & Cluster Network Configuration - KKP Documentation

Maybe the by default recommended configuration of the KubermaticConfiguration object should still activate Konnectivity explicitly, in so kubermatic/kubermatic#8129 was not implemented only on the presentation layer, but also in the services?

[rastislavs]

Hi @almereyda
Konnectivity is still not default in KKP - in other words, it first needs to be enabled by the feature gate.
Once the feature gate is enabled, it will be marked (checked) by default in KKP UI for new user clusters. But the feature gate is still required, as there are still some issues in Konnectivity that we track upstream (mainly resource leaks).

[almereyda]

Am I right in concluding then, that this is rather a documentation issue for now, and should be closed here, relocated into docs? There is no perceivable technical regression within the dashboard after all, as this was a configuration issue with the underlying KKP deployment.

[embik]

Before we move the issue around, can you please clarify what problem you see with the current documentation? https://docs.kubermatic.com/kubermatic/v2.20/tutorials_howtos/networking/cni_cluster_network/#enabling-konnectivity-in-kubermaticconfiguration seems to point out that you need the feature gate to use Konnectivity.

[rastislavs]

also:

The most of the Cilium CNI features can be utilized when the ebpf Proxy Mode is used (Cilium kube-proxy-replacement is enabled). This can be done by selecting ebpf for Proxy Mode in the Cluster Network Configuration. Please note that this option is available only of Konnectivity is enabled.

https://docs.kubermatic.com/kubermatic/v2.20/tutorials_howtos/networking/cni_cluster_network/#cilium-cni

[almereyda]

Sorry that I'm sometimes having a really hard time following the documentation. This page did not come into my eyes when following the installation instructions, until I encountered my perceived error from the OP. The steps are sometimes spread accross many places, outdated kubermatic/docs#1126, and not linearily written. One jumps across many areas of the documentation when setting up the whole stack.

You are very right that the requested information is there in the documentation, for me it was just not easily discoverable.

I think I'm easily distracted by the always present sidebar, a little lost on individual documents without a scrolling ToC and sometimes don't realise which links and backlinks exist within pages: which section asks me to follow another section first, before proceeding, from where are sections linked, and in which order do the steps need to be followed. Sometimes colourful info boxes could help my eyes to find rest between the interchangeable code snippets, screenshots and text deserts.

And to be even more frank: I think I was misguided by kubermatic/kubermatic#8129 and didn't understand that it talks about enabling Konnectivity on newly create user clusters by default within the dashboard and always thought that this ticket also enabled Konnectivity within the seed cluster deployment by default.

I think the Metrics, Logging & Aggregation plus the Networking sections from within the Tutorials & How-tos actually need to be part of Installation instead, because they are required for a fully working installation.

[embik]

Thanks a lot for the feedback, this is super valuable as an outside perspective. I think the idea behind this being on separate pages is that it's optional features you can also add to your KKP installation later on if you want/need them (not saying this is the only way to do it though). Making every optional choice during the initial setup feels very confusing to be honest.

I think the Metrics, Logging & Aggregation plus the Networking sections from within the Tutorials & How-tos actually need to be part of Installation instead, because they are required for a fully working installation.

Hope it's okay for me to quizz you a bit (more feedback helps us better pinpoint the problem) - for MLA I see your point about maybe being in Installation, but can you point out the sections of Networking you needed for a fully working installation? Is it the Konnectivity part? Because I don't think we'd consider that a requirement for a fully working installation right now. Things hidden behind a feature gate never should be integral to a functional KKP installation.

[almereyda]

Actually I should have been aware of this, yet due to the spread issues everywhere, I didn't remember the one that I already answered to:

• Make Konnectivity generally available kubermatic#8536
• Remove Konenctivity feature gate and make it GA and enabled by default kubermatic#10154

Also I'm having issues on the cluster right now using nodeport-proxy (envoy), Cilium and Konnectivity together. So the rationale to not activate it right away might not be the worst.

I'll document that case in the appropriate location at the Kubermatic issue tracker, and come back here to think more about how to improve the documentation from my perspective. Indeed there are many degrees of freedom involved here, so drawing a line between mandatory and optional components is indeed tricky. Maybe some kind of architecture diagram could help evaluating the perimeters.

[rastislavs]

Thanks again for the feedback @almereyda

I understand that multiple issues about Konnectivity may be misleading, but that is how we are trying to organise our work - one of the issues that you listed is an Epic and the other one the last remaining subtask of that epic.

Anyways - it probably would not be an issue if we proceeded with making it default already. As we postponed that, I should at least mention the current state in those issues.

[almereyda]

@embik The whole monitoring, metering and MLA space is a little confusing right now, to be honest. The parts work, but their interplay and distinction is not the clearest. Why I have recorded

• Wondering about the relation of `monitoring`, `logging` and `minio` charts to MLA kubermatic#10727

Additionally it is almost undocumented how to investigate and repair a broken Konnectivity configuration, which ultimately turns it into a SPOF from our point of view. Which I have documented in:

• Broken nodeport-proxy brings cluster into unusable state kubermatic#10739

Closing here, as the OP was resolved, and follow up discussions had been refactored.