Security Policies

To make sure deployments in the WWU Kube are secure, security policies restricting the permitted ressources are in effect.

In order to secure the Kubernetes cluster, some entries in resources are restricted. The constraints are checked in the cluster with Gatekeeper and if necessary, resources are rejected.

Creating erroneous resources results in error messages during creation. For resources like pods that should be automatically generated from ReplicaSets, these error messages can be found in the events.

The following describes the constraints for the various resources.

VirtualService

The .spec.gateways entry must always contain at least one gateway. These gateways must not contain /.

Also, the gateway mesh is only allowed in combination with .spec.exportTo: ["."].

If the wwu.io/nic_node annotation is used in a VirtualService to export a host to the NIC, the entries in .spec.hosts are limited to the DNS entries agreed upon with the administrators.

Certificate

If .spec.issuerRef.kind is set to ClusterIssuer, all entries in .spec.commonName and .spec.dnsNames must be agreed upon with the administrators.

Ingress

All hosts in the Ingress resources, both in .spec.rules[*].host and .spec.tls[*].hosts must be cleared with the administrators and enabled individually.

Gateway

The .spec.selectors are limited to the explicit {"istio": "ingressgateway"} and {"istio": "egressgateway"} entries.

If ingressgateway is selected in the selector, all hosts .spec.servers[*].hosts must start with either ./ or with <namespace>/.

Also, in this case, only the DNS entries that have been previously agreed upon with the administrators are allowed as hosts.

All .spec.servers[*].tls.credentialName must have as a prefix the namespace name, that is: <namespace>--<secret-name>.

DNSEndpoint

All DNS names in .spec.endpoints[*].dnsName must be agreed upon with the administrators and enabled individually.

Service

Services of the type Loadbalancer are not allowed per se and must be permitted by the administrators.

If loadbalancers are allowed, the annotations loadbalancer.openstack.org/keep-floatingip=true or service.beta.kubernetes.io/openstack-internal-load-balancer: "true" must be set and each IP in .spec.loadBalancerIP must be discussed with the administrators and enabled individually.

No entries are allowed in spec.ExternalIPs.

All ports exported to Istio with the annotation networking.istio.io/exportTo must either have appProtocol set in each .spec.ports or the name must start with the protocol See the documentation for details.

Pod

When istio containers are used as sidecars, all other containers must not run as user 1337. That is .spec.securityContext.runAsUser and .spec.containers[name!=istio].securityContext.runAsUser must not be 1337.

Also, the allowed spec.tolerations on the pods are restricted and must be discussed with the administrators. For example certain tolerations are required to run pods on GPU nodes or worker nodes.

If GPUs are used, the sum of the .spec.containers[*].resource.limits.nvidia.com/gpu be equal to 1. That means, one of the containers must have a 1 there and for the others this value must either not be set or must be explicitly set to 0. This is probably illustrated more clearly in this example.

Also, for pods, we have complex security profiles that further restrict the pods. These are explained in detail below.

Security Profile

Currently there is only one security profile available for tenants, called default, which restricts the access to privileged features of the cluster.

Default in detail

This is the profile for all customers. It protects the cluster from unauthorized access.

NameCRDRestrictions
default-psp-capabilitiesK8sPSPCapabilitiesIt is not allowed to add capabilities via securityContext.capabilities.add capabilities
default-psp-flexvolume-driversK8sPSPCapabilitiesIt is not allowed to use flex volumes
default-psp-forbidden-systclsK8sPSPForbiddenSysctlsAll sysctl calls are prohibited
default-psp-host-filesystemK8sPSPHostFilesystemNo hostPaths are allowed
default-psp-host-namespaceK8sPSPHostNamespaceIt is not allowed to share the process namespace with the host
default-psp-host-network-portsK8sPSPHostNetworkingPortsIt is not allowed to use the host’s network
default-psp-privileged-containerK8sPSPPrivilegedContainerPrivileged pods are prohibited
default-psp-proc-mountK8sPSPProcMountOnly the default procMountType is allowed
default-psp-volume-typesK8sPSPVolumeTypesThe following volume types are allowed: emptyDir, downwardAPI, configMap, secret, persistentVolumeClaim, projected
default-container-must-have-limitsK8sContainerLimitsEach container must have resources.limits.cpu (<= 8), resources.limits.memory (<= 32Gi) and resources.limits.ephemeral-storage (<= 32Gi) set.