Security Policies

In order to secure the Kubernetes cluster, some entries in resources are restricted. The constraints are checked in the cluster with Gatekeeper and if necessary, resources are rejected.

Creating erroneous resources results in error messages during creation. For resources like pods that should be automatically generated from ReplicaSets, these error messages can be found in the events. A few constraints are not enforced but only reported as a warning; those are marked as such below.

Wherever a value has to be “agreed upon with the administrators”, it is enabled per namespace. Host names can also be enabled as wildcard patterns, for example *.example.uni-muenster.de.

The following describes the constraints for the various resources.

VirtualService

The .spec.gateways entry must always contain at least one gateway. These gateways must not contain /.

Also, the gateway mesh is only allowed in combination with .spec.exportTo: ["."].

If the wwu.io/nic_node annotation is used in a VirtualService to export a host to the NIC, the entries in .spec.hosts are limited to the DNS entries agreed upon with the administrators.

Certificate

If .spec.issuerRef.kind is set to ClusterIssuer or DFNClusterIssuer, all entries in .spec.commonName and .spec.dnsNames must be agreed upon with the administrators.

Ingress

All hosts in the Ingress resources, both in .spec.rules[*].host and .spec.tls[*].hosts must be cleared with the administrators and enabled individually.

Gateway

The .spec.selector is limited to the explicit entries {"istio": "ingressgateway"}, {"istio": "egressgateway"} and {"istio": "egressgateway-ng"}.

If ingressgateway is selected in the selector, all hosts .spec.servers[*].hosts must start with either ./ or with <namespace>/.

Also, in this case, only the DNS entries that have been previously agreed upon with the administrators are allowed as hosts.

Every secret referenced by a gateway — .spec.servers[*].tls.credentialName, each entry of .spec.servers[*].tls.credentialNames and .spec.servers[*].tls.caCertCredentialName — must be prefixed with the namespace name, that is <namespace>--<secret-name>. Istio resolves those references in the namespace of the ingress gateway and not in the namespace of the Gateway, so the prefix is what keeps one namespace from picking up another one’s certificate. The secrets are copied there for you under that name.

In addition, .spec.servers[*].tls.caCertCredentialName must end in -cacert, because Istio only recognises a secret as CA material with that suffix.

DNSEndpoint

All DNS names in .spec.endpoints[*].dnsName must be agreed upon with the administrators and enabled individually.

Service

Services of the type LoadBalancer are not allowed per se and must be permitted by the administrators for the namespace.

For the default (OpenStack Octavia) load balancers, one of the annotations loadbalancer.openstack.org/keep-floatingip: "true" or service.beta.kubernetes.io/openstack-internal-load-balancer: "true" must be set, and the IP in .spec.loadBalancerIP must be discussed with the administrators and enabled individually.

For load balancers with .spec.loadBalancerClass: io.cilium/bgp-control-plane no annotation is required. Here the IPs actually assigned to the service (.status.loadBalancer.ingress[*].ip) must be among the IPs enabled for the namespace.

No entries are allowed in .spec.externalIPs.

All ports exported to Istio with the annotation networking.istio.io/exportTo must either have appProtocol set in each .spec.ports or the name must start with the protocol, that is one of http, http2, https, tcp, tls, grpc, grpc-web, mongo, mysql, redis, udp. See the documentation for details.

PodDisruptionBudget

A PodDisruptionBudget must not block voluntary disruptions such as node drains. Therefore

• .spec.maxUnavailable must not be 0, and
• .spec.minAvailable must be lower than the number of replicas of the Deployment or StatefulSet it selects.

Deployments and StatefulSets selected by such a PodDisruptionBudget are rejected as well.

Pod

When istio containers are used as sidecars, all other containers should not run as user 1337. That is .spec.securityContext.runAsUser and .spec.containers[name!=istio].securityContext.runAsUser should not be 1337. This constraint currently only produces a warning and does not reject the pod.

Also, the allowed .spec.tolerations on the pods are restricted and must be discussed with the administrators. For example certain tolerations are required to run pods on GPU nodes or worker nodes. The tolerations for node.kubernetes.io/not-ready and node.kubernetes.io/unreachable (NoExecute, Exists), which Kubernetes adds to every pod, are always allowed.

If a pod tolerates role.node.kubernetes.io/worker-vgpu (NoSchedule, Exists) to be scheduled on a GPU node, the sum of the .spec.containers[*].resources.limits.nvidia.com/gpu and .spec.initContainers[*].resources.limits.nvidia.com/gpu values must be at least 1 and at most 4. This is probably illustrated more clearly in this example.

If a seccompProfile is set — either at pod level in .spec.securityContext.seccompProfile or on a container — its type must be RuntimeDefault. Not setting a profile at all is allowed.

Every container must set resources.limits.ephemeral-storage and it must not exceed 32Gi. The istio containers injected into your pods (istio-proxy, istio-validation) are handled automatically and do not need a limit of their own.

Also, for pods, we have complex security profiles that further restrict the pods. These are explained in detail below.

Security Profile

Currently there is only one security profile available for tenants, called default, which restricts the access to privileged features of the cluster.

Default in detail

This is the profile for all customers. It protects the cluster from unauthorized access.

Application

ArgoCD Applications in your namespace are restricted as well.

.spec.project must be set explicitly and may only reference one of the ArgoCD projects that have been enabled for your namespace.

Config management plugins are currently not allowed, that is neither .spec.source.plugin nor .spec.sources[*].plugin may be set.