# Security Policies

> To make sure deployments in the Kube are secure, security policies restricting the permitted ressources are in effect.

---

Site [llms.txt](/llms.txt)

---

_This page was drafted with AI assistance and has been fully reviewed by a human before publishing._

In order to secure the Kubernetes cluster, some entries in resources are restricted. The constraints are checked in the cluster with [Gatekeeper](https://github.com/open-policy-agent/gatekeeper) and if necessary, resources are rejected.

Creating erroneous resources results in error messages during creation. For resources like pods that should be automatically generated from ReplicaSets, these error messages can be found in the events. A few constraints are not enforced but only reported as a warning; those are marked as such below.

Wherever a value has to be "agreed upon with the administrators", it is enabled per namespace. Host names can also be enabled as wildcard patterns, for example `*.example.uni-muenster.de`.

The following describes the constraints for the various resources.

## Application

ArgoCD `Applications` in your namespace are restricted as well.

`.spec.project` must be set explicitly and may only reference one of the ArgoCD projects that have been enabled for your namespace.

Config management plugins are currently not allowed, that is neither `.spec.source.plugin` nor `.spec.sources[*].plugin` may be set.

## Certificate {#SecurityPolicies-Certificate}

If `.spec.issuerRef.kind` is set to `ClusterIssuer` or `DFNClusterIssuer`, all entries in `.spec.commonName` and `.spec.dnsNames` must be agreed upon with the administrators.

## DNSEndpoint

All DNS names in `.spec.endpoints[*].dnsName` must be agreed upon with the
administrators and enabled individually.

## Gateway

The `.spec.selector` is limited to the explicit entries `{"istio": "ingressgateway"}`, `{"istio": "egressgateway"}` and `{"istio": "egressgateway-ng"}`.

If `ingressgateway` is selected in the selector, all hosts `.spec.servers[*].hosts` must start with either `./` or with `<namespace>/`.

Also, in this case, only the DNS entries that have been previously agreed upon with the administrators are allowed as hosts.

Every secret referenced by a gateway — `.spec.servers[*].tls.credentialName`, each entry of `.spec.servers[*].tls.credentialNames` and `.spec.servers[*].tls.caCertCredentialName` — must be prefixed with the namespace name, that is `<namespace>--<secret-name>`. Istio resolves those references in the namespace of the ingress gateway and not in the namespace of the `Gateway`, so the prefix is what keeps one namespace from picking up another one's certificate. The secrets are [copied there for you](/docs/kubernetes/network/ingressgateway/#tls-certificates) under that name.

In addition, `.spec.servers[*].tls.caCertCredentialName` must end in `-cacert`, because Istio only recognises a secret as CA material with that suffix.

## Ingress

All hosts in the Ingress resources, both in `.spec.rules[*].host` and `.spec.tls[*].hosts` must be cleared with the administrators and enabled individually.

## Pod

When istio containers are used as sidecars, all other containers should not run as user `1337`. That is `.spec.securityContext.runAsUser`
and `.spec.containers[name!=istio].securityContext.runAsUser` should not be `1337`. This constraint currently only produces a warning and does not reject the pod.

Also, the allowed `.spec.tolerations` on the pods are restricted
and must be discussed with the administrators. For example
certain tolerations are required to run pods on GPU nodes or worker nodes. The tolerations for `node.kubernetes.io/not-ready` and `node.kubernetes.io/unreachable` (`NoExecute`, `Exists`), which Kubernetes adds to every pod, are always allowed.

If a pod tolerates `role.node.kubernetes.io/worker-vgpu` (`NoSchedule`, `Exists`) to be scheduled on a GPU node, the sum of the
`.spec.containers[*].resources.limits.nvidia.com/gpu` and `.spec.initContainers[*].resources.limits.nvidia.com/gpu` values must be at least `1` and at most `4`. This is probably illustrated more clearly in [this example](https://zivgitlab.uni-muenster.de/wwu-it/kube/examples/-/tree/main/gpu).

A `seccompProfile`, either at pod level in `.spec.securityContext.seccompProfile` or on a container, is required and its `type` must be `RuntimeDefault`. You don't have to set it yourself: if it's missing at pod level, it is filled in automatically.

Every container must set `resources.limits.ephemeral-storage` and it must not exceed `32Gi`. The istio containers injected into your pods (`istio-proxy`, `istio-validation`) are handled automatically and do not need a limit of their own.

Also, for pods, we have complex **security profiles** that further restrict the pods. These are explained in detail below.

### Security Profile

Two security profiles are available for tenants: `default` and `hardened_default`. Which profile applies to a namespace is controlled by the namespace label `constraints.gatekeeper.sh/securityprofile`. Switching a namespace to `hardened_default` must currently be agreed upon with the administrators, the same way as other settings in this document. For a new project, `hardened_default` can be requested directly in the [project request form](../project_request_form.md).

If a `Deployment`'s pods violate the active security profile — `default` or `hardened_default` — the `Deployment` itself will not show an error. The rejection is only visible as an event on the generated `ReplicaSet`, the same as for other constraints described at the top of this page. You can see it with, for example, `kubectl --context kube-staging-ms1 -n my-namespace get events` or `kubectl --context kube-staging-ms1 -n my-namespace describe replicaset <name>`.

#### Default in detail

This is the baseline profile. It protects the cluster from unauthorized access and applies to every namespace, whether it is set to `default` or `hardened_default`.

| Name                               | CRD                       | Restrictions                                                                                                                                                                                                                                                                                                                                                                                                          |
| ---------------------------------- | ------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| default-psp-apparmor               | K8sPSPAppArmor            | If `appArmorProfile` is set, only `runtime/default` is allowed. This constraint currently only produces a warning and does not reject the pod, since AppArmor is not yet enabled on the cluster.                                                                                                                                                                                                                      |
| default-psp-capabilities           | K8sPSPCapabilities        | Capabilities may only be added via `securityContext.capabilities.add` from the Kubernetes Pod Security Standards baseline allow-list: `AUDIT_WRITE`, `CHOWN`, `DAC_OVERRIDE`, `FOWNER`, `FSETID`, `KILL`, `MKNOD`, `NET_BIND_SERVICE`, `SETFCAP`, `SETGID`, `SETPCAP`, `SETUID`, `SYS_CHROOT`                                                                                                                         |
| default-psp-flexvolume-drivers     | K8sPSPFlexVolumes         | It is not allowed to use flex volumes                                                                                                                                                                                                                                                                                                                                                                                 |
| default-psp-forbidden-systcls      | K8sPSPForbiddenSysctls    | All sysctls are prohibited, except the Kubernetes Pod Security Standards baseline allow-list: `kernel.shm_rmid_forced`, `net.ipv4.ip_local_port_range`, `net.ipv4.ip_unprivileged_port_start`, `net.ipv4.tcp_syncookies`, `net.ipv4.ping_group_range`, `net.ipv4.ip_local_reserved_ports`, `net.ipv4.tcp_keepalive_time`, `net.ipv4.tcp_fin_timeout`, `net.ipv4.tcp_keepalive_intvl`, `net.ipv4.tcp_keepalive_probes` |
| default-psp-host-filesystem        | K8sPSPHostFilesystem      | No hostPaths are allowed                                                                                                                                                                                                                                                                                                                                                                                              |
| default-psp-host-namespace         | K8sPSPHostNamespace       | It is not allowed to share the process (`hostPID`) or IPC (`hostIPC`) namespace with the host                                                                                                                                                                                                                                                                                                                         |
| default-psp-host-network-ports     | K8sPSPHostNetworkingPorts | It is not allowed to use the host's network (`hostNetwork`) or host ports                                                                                                                                                                                                                                                                                                                                             |
| default-psp-host-probes-lifecycle  | K8sPSPHostProbesLifecycle | It is not allowed to set a `host` field on `livenessProbe`, `readinessProbe`, `startupProbe` or on the `postStart`/`preStop` lifecycle hooks                                                                                                                                                                                                                                                                          |
| default-psp-privileged-container   | K8sPSPPrivilegedContainer | Privileged pods are prohibited                                                                                                                                                                                                                                                                                                                                                                                        |
| default-psp-proc-mount             | K8sPSPProcMount           | Only the default procMountType is allowed                                                                                                                                                                                                                                                                                                                                                                             |
| default-psp-selinux                | K8sPSPSELinuxV2           | If `seLinuxOptions` is set, only the following types are allowed: `container_t`, `container_init_t`, `container_kvm_t`, `container_engine_t`; a custom `user`, `role` or `level` is not allowed. This constraint currently only produces a warning and does not reject the pod, since SELinux is not yet enabled on the cluster.                                                                                      |
| default-psp-volume-types           | K8sPSPVolumeTypes         | The following volume types are allowed: `emptyDir`, `downwardAPI`, `configMap`, `secret`, `persistentVolumeClaim`, `projected`, `ephemeral`, `csi`, `image`. `csi` is allowed here so that a namespace on `hardened_default` that drops `hardened-psp-volume-types` (see below) still keeps a restricted volume-type list instead of losing this restriction entirely.                                                |
| default-container-must-have-limits | K8sContainerLimits        | Each container must have `resources.limits.cpu` (<= 8) and `resources.limits.memory` (<= 32Gi) set.                                                                                                                                                                                                                                                                                                                   |

#### Hardened default in detail

Namespaces set to `hardened_default` get the full `default` rule set above, plus the full `hardened-*` rule set below, applied on top. Where a `hardened-*` rule's restriction is identical to its `default-*` counterpart, it still exists as a separately named rule so it can be dropped independently (see below).

| Name                                              | CRD                                     | Restriction                                                                                                                                                             |
| ------------------------------------------------- | --------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| hardened-psp-apparmor                             | K8sPSPAppArmor                          | Same restriction as `default-psp-apparmor` above (warn-only for now).                                                                                                   |
| hardened-psp-capabilities                         | K8sPSPCapabilities                      | Every container must drop the `ALL` capability (`securityContext.capabilities.drop: ["ALL"]`); set automatically if missing. Only `NET_BIND_SERVICE` may be added back. |
| hardened-psp-flexvolume-drivers                   | K8sPSPFlexVolumes                       | It is not allowed to use flex volumes                                                                                                                                   |
| hardened-psp-forbidden-systcls                    | K8sPSPForbiddenSysctls                  | Same Kubernetes Pod Security Standards baseline allow-list as `default-psp-forbidden-systcls` above.                                                                    |
| hardened-psp-host-filesystem                      | K8sPSPHostFilesystem                    | No hostPaths are allowed                                                                                                                                                |
| hardened-psp-host-namespace                       | K8sPSPHostNamespace                     | It is not allowed to share the process (`hostPID`) or IPC (`hostIPC`) namespace with the host                                                                           |
| hardened-psp-host-network-ports                   | K8sPSPHostNetworkingPorts               | It is not allowed to use the host's network (`hostNetwork`) or host ports                                                                                               |
| hardened-psp-host-probes-lifecycle                | K8sPSPHostProbesLifecycle               | Same restriction as `default-psp-host-probes-lifecycle` above.                                                                                                          |
| hardened-psp-privileged-container                 | K8sPSPPrivilegedContainer               | Privileged pods are prohibited                                                                                                                                          |
| hardened-psp-proc-mount                           | K8sPSPProcMount                         | Only the default procMountType is allowed                                                                                                                               |
| hardened-psp-selinux                              | K8sPSPSELinuxV2                         | Same restriction as `default-psp-selinux` above (warn-only for now).                                                                                                    |
| hardened-psp-volume-types                         | K8sPSPVolumeTypes                       | The following volume types are allowed: `emptyDir`, `downwardAPI`, `configMap`, `secret`, `persistentVolumeClaim`, `projected`, `ephemeral`                             |
| hardened-container-must-have-limits               | K8sContainerLimits                      | Each container must have `resources.limits.cpu` (<= 8) and `resources.limits.memory` (<= 32Gi) set.                                                                     |
| hardened-psp-pods-allowed-user-ranges             | K8sPSPAllowedUsers                      | Containers must run as a non-root user; group, supplemental groups and fsGroup must be in the range 1-65535.                                                            |
| hardened-psp-allow-privilege-escalation-container | K8sPSPAllowPrivilegeEscalationContainer | Privilege escalation is forbidden (`securityContext.allowPrivilegeEscalation: false`); set automatically if missing.                                                    |
| hardened-psp-readonlyrootfilesystem               | K8sPSPReadOnlyRootFilesystem            | The container's root filesystem must be read-only (`securityContext.readOnlyRootFilesystem: true`); set automatically if missing.                                       |

Individual `hardened-*` rules can be disabled for a single pod by setting the label `constraints.gatekeeper.sh/drop-<constraint-name>` on that pod, for example `constraints.gatekeeper.sh/drop-hardened-psp-capabilities`. Unlike switching the namespace profile itself, this label can be set by tenants without involving the administrators.

Dropping a `hardened-*` rule normally falls back to the matching `default-*` rule rather than removing the restriction outright: dropping `hardened-psp-capabilities` falls back to `default-psp-capabilities`, so the container may add capabilities from the wider Pod Security Standards baseline allow-list instead of just `NET_BIND_SERVICE`, and dropping is no longer required for `ALL`. Similarly, dropping `hardened-psp-volume-types` falls back to `default-psp-volume-types`, which additionally allows `csi` and `image` volumes. The exceptions are `hardened-psp-pods-allowed-user-ranges`, `hardened-psp-allow-privilege-escalation-container` and `hardened-psp-readonlyrootfilesystem`, which have no `default` counterpart and are fully disabled once dropped.

## PodDisruptionBudget

A `PodDisruptionBudget` must not block voluntary disruptions such as node drains. Therefore

- `.spec.maxUnavailable` must not be `0`, and
- `.spec.minAvailable` must be lower than the number of replicas of the `Deployment` or `StatefulSet` it selects.

`Deployments` and `StatefulSets` selected by such a `PodDisruptionBudget` are rejected as well.

## Service

Services of the type `LoadBalancer` are not allowed per se and must be
permitted by the administrators for the namespace.

For the default (OpenStack Octavia) load balancers, one of the annotations
`loadbalancer.openstack.org/keep-floatingip: "true"` or
`service.beta.kubernetes.io/openstack-internal-load-balancer: "true"` must be set, and the IP in `.spec.loadBalancerIP` must be discussed with the administrators and enabled individually.

For load balancers with `.spec.loadBalancerClass: io.cilium/bgp-control-plane` no annotation is required. Here the IPs actually assigned to the service (`.status.loadBalancer.ingress[*].ip`) must be among the IPs enabled for the namespace.

No entries are allowed in `.spec.externalIPs`.

All ports exported to Istio with the annotation `networking.istio.io/exportTo` must either have `appProtocol` set in each `.spec.ports` or the `name` must start with the protocol, that is one of `http`, `http2`, `https`, `tcp`, `tls`, `grpc`, `grpc-web`, `mongo`, `mysql`, `redis`, `udp`. See [the documentation](https://istio.io/latest/docs/ops/configuration/traffic-management/protocol-selection/#explicit-protocol-selection) for details.

## VirtualService

The `.spec.gateways` entry must always contain at least one gateway. These gateways must not contain `/`.

Also, the gateway `mesh` is only allowed in combination with `.spec.exportTo: ["."]`.

If the `wwu.io/nic_node` annotation is used in a `VirtualService` to export a host to the NIC, the entries in `.spec.hosts` are limited to the DNS entries agreed upon with the administrators.