[teraslice, e2e] Add Kubernetes role restrictions to teraslice master

[sotojn]

This PR makes the following changes:

• Assigns a new, dedicated and restricted kubernetes role to the Teraslice master pod, enhancing secure access control
• Removes the default ServiceAccount usage, which granted all pods in the namespace full access to the Kubernetes API
• Updated documentation to reflect changes

Ref to issue #3251

[sotojn]

I've tested the new access controls by running a teraslice job with the new role configuration.

I exec'd into the master pod, installed kubectl and tried listing all resources:

teraslice-master-747fc6767f-kjb8c:/app/source#  kubectl -n ts-dev1 get all
NAME                                                     READY   STATUS    RESTARTS   AGE
pod/teraslice-master-747fc6767f-kjb8c                    1/1     Running   0          56m
pod/ts-exc-data-to-noop-b63e8588-6f2c-c6rbc              1/1     Running   0          55m
pod/ts-wkr-data-to-noop-b63e8588-6f2c-546bb44856-7x8rh   1/1     Running   0          55m

NAME                       TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
service/teraslice-master   NodePort   10.96.204.194   <none>        5678:30678/TCP   56m

NAME                                                READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/teraslice-master                    1/1     1            1           56m
deployment.apps/ts-wkr-data-to-noop-b63e8588-6f2c   1/1     1            1           55m

NAME                                                           DESIRED   CURRENT   READY   AGE
replicaset.apps/teraslice-master-747fc6767f                    1         1         1       56m
replicaset.apps/ts-wkr-data-to-noop-b63e8588-6f2c-546bb44856   1         1         1       55m

NAME                                          COMPLETIONS   DURATION   AGE
job.batch/ts-exc-data-to-noop-b63e8588-6f2c   0/1           55m        55m
Error from server (Forbidden): replicationcontrollers is forbidden: User "system:serviceaccount:ts-dev1:master-service-account" cannot list resource "replicationcontrollers" in API group "" in the namespace "ts-dev1"
Error from server (Forbidden): daemonsets.apps is forbidden: User "system:serviceaccount:ts-dev1:master-service-account" cannot list resource "daemonsets" in API group "apps" in the namespace "ts-dev1"
Error from server (Forbidden): statefulsets.apps is forbidden: User "system:serviceaccount:ts-dev1:master-service-account" cannot list resource "statefulsets" in API group "apps" in the namespace "ts-dev1"
Error from server (Forbidden): horizontalpodautoscalers.autoscaling is forbidden: User "system:serviceaccount:ts-dev1:master-service-account" cannot list resource "horizontalpodautoscalers" in API group "autoscaling" in the namespace "ts-dev1"
Error from server (Forbidden): cronjobs.batch is forbidden: User "system:serviceaccount:ts-dev1:master-service-account" cannot list resource "cronjobs" in API group "batch" in the namespace "ts-dev1"

Then I did the same thing in an execution pod and got this result:

ts-exc-data-to-noop-b63e8588-6f2c-c6rbc:/app/source# kubectl -n ts-dev1 get all
Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:ts-dev1:default" cannot list resource "pods" in API group "" in the namespace "ts-dev1"
Error from server (Forbidden): replicationcontrollers is forbidden: User "system:serviceaccount:ts-dev1:default" cannot list resource "replicationcontrollers" in API group "" in the namespace "ts-dev1"
Error from server (Forbidden): services is forbidden: User "system:serviceaccount:ts-dev1:default" cannot list resource "services" in API group "" in the namespace "ts-dev1"
Error from server (Forbidden): daemonsets.apps is forbidden: User "system:serviceaccount:ts-dev1:default" cannot list resource "daemonsets" in API group "apps" in the namespace "ts-dev1"
Error from server (Forbidden): deployments.apps is forbidden: User "system:serviceaccount:ts-dev1:default" cannot list resource "deployments" in API group "apps" in the namespace "ts-dev1"
Error from server (Forbidden): replicasets.apps is forbidden: User "system:serviceaccount:ts-dev1:default" cannot list resource "replicasets" in API group "apps" in the namespace "ts-dev1"
Error from server (Forbidden): statefulsets.apps is forbidden: User "system:serviceaccount:ts-dev1:default" cannot list resource "statefulsets" in API group "apps" in the namespace "ts-dev1"
Error from server (Forbidden): horizontalpodautoscalers.autoscaling is forbidden: User "system:serviceaccount:ts-dev1:default" cannot list resource "horizontalpodautoscalers" in API group "autoscaling" in the namespace "ts-dev1"
Error from server (Forbidden): cronjobs.batch is forbidden: User "system:serviceaccount:ts-dev1:default" cannot list resource "cronjobs" in API group "batch" in the namespace "ts-dev1"
Error from server (Forbidden): jobs.batch is forbidden: User "system:serviceaccount:ts-dev1:default" cannot list resource "jobs" in API group "batch" in the namespace "ts-dev1"
ts-exc-data-to-noop-b63e8588-6f2c-c6rbc:/app/source# 

Currently in the master branch, any pod in the namespace can see and interact with the k8s api. It's even possible to exec into a teraslice worker pod and inside the pod, exec into the master pod.

[busma13]

Looks good, I just had a few docs suggestions.

[busma13]

LGTM

[godber]

I think there's a minor but important improvement that can be made here, see my comment.

[godber]

master pod configuration

is probably too vague, do you mean in the deployment, or is this set somewhere else?

[sotojn]

I've refactored the whole thing to be more specific. We provide a deployment so I specified it's location within the repo and reined more specifically where it could be set.