Rules for Kubernetes or OpenShift Cluster
This section assumes the following:
Kubernetes or OpenShift cluster nodes and infrastructure Pods are activated and managed.
Labels have been assigned to each workload and container workload.
All cluster nodes and infrastructure Pods are in the same application group, which means they have been assigned the same application, environment, and location labels.
Kubernetes
Create a ruleset for the Kubernetes cluster and control plane Pods. The labels assigned to all of the Kubernetes nodes and control Pod workloads should fall within the scope.
Add the following lines of policy to the ruleset.
Intra-Scope Rules
Providers | Services | Consumers | Notes |
|---|---|---|---|
docker.io (IP List) myregistry.example.com (IP List) | All Services | All Workloads | Containerized environments depend on various external resources to perform basic operations such as pulling a docker image. Illumio has determined that the listed FQDNs are essential to Kubernetes deployments. Each deployment varies and may have dependencies on additional resources. If your container infrastructure has requirements for FQDNs not mentioned in this document, then you should include those FQDNs in this policy line. |
Illumio PCE (IP List) | 8443 TCP | Kubelink | Kubelink sends context about the Kubernetes cluster to the PCE over TCP 8443 port. |
All Workloads | 53 TCP 53 UDP | Kubernetes Pod Network (IP List) | The Kubernetes cluster provides internal DNS services to the pods (using coreDNS in this example). This policy enables internal DNS resolution for these tasks. |
All Workloads (Uses Virtual Services and Workloads) | All Services | All Workloads | Any communication across all managed Kubernetes nodes or managed infrastructure pods which will be permitted by this policy. |
Kubernetes Pod Network (IP List) | All Services | All Workloads | Communications across initiated by any workload which pass through service front ends will be allowed by this policy. It also covers other IP addresses on the Kubernetes pod network which are not discovered by the PCE. Critical for infrastructure functions including but not limited to liveness probes and infrastructure service front ends (Kubernetes). |
Extra-scope Rules
Providers | Services | Consumers | Notes |
|---|---|---|---|
All Workloads | 6443 TCP 22 TCP | Any 0.0.0.0/0 (IP List) | Optional: Opens up ports which are purposed for remote management. For example, TCP 22 to provide SSH services to Kubernetes admins. TCP 6443 provides Kubernetes admins with dashboard services. The Dashboard may vary across Kubernetes deployments. The ports can be modified to what is used in your environment and consuming IP list can be changed to corporate subnets or jump servers. |
Worker | 80 TCP 443 TCP | Any 0.0.0.0/0 (IP List) | This policy assumes Ingress Controllers exist on Worker nodes. If the ingress controllers exist on other nodes, then modify the provider to the host where the Ingress controllers reside. This rule opens default front end ports which are used to access containerized applications from external IP addresses. |
OpenShift
Create a ruleset for the OpenShift cluster and control plane Pods. The labels assigned to all of the OpenShift nodes and control Pod workloads should fall within the scope.
Add the following lines of policy to the ruleset.
Note
The IP lists referenced in this ruleset are commonly used public registries (e.g., docker.io) for container environments. If you have confirmed that your OpenShift environment does not depend on a public registry shown below, then it is recommended that you remove the IP lists from the ruleset.
Intra-scope Rules
Providers | Services | Consumers | Notes |
|---|---|---|---|
docker.io (IP List) registry.access.redhat.com (IP List) registry.webscaleone.info (IP LIst) access.redhat.com (IP List) subscription.rhsm.redhat.com (IP List) | All Services | All Workloads | Containerized environments depend on various external resources to perform basic operations such as pulling a docker image. Illumio has determined that the listed FQDNs are essential to OpenShift deployments. Each deployment varies and may have dependencies on additional resources. If your container infrastructure has requirements for FQDNs not mentioned in this doc, then you should include those FQDNs in this policy line. |
Illumio PCE (IP List) | 8443 TCP | Kubelink | Kubelink sends context about the OpenShift cluster to the PCE over TCP 8443 port. |
All Workloads | 53 TCP 53 UDP | OpenShift Pod Network (IP List) | The OpenShift cluster in this example uses DNSmasq meaning each cluster node listens on port 53 and provides internal DNS services to the pods. This policy enables internal DNS resolution for these tasks. |
All Workloads (Uses Virtual Services and Workloads) | All Services | All Workloads | Any communication across all managed OpenShift nodes or managed infrastructure pods which will be permitted by this policy. |
OpenShift Pod Network (IP List) OpenShift Service Network (IP List) | All Services | All Workloads | Communications across initiated by any workload which pass through service front ends will be allowed by this policy. It also covers other IP addresses on the OpenShift pod network which are not discovered by the PCE. Critical for infrastructure functions including but not limited to liveness probes and infrastructure service front ends (Kubernetes). |
Extra-Scope Rules
Providers | Services | Consumers | Notes |
|---|---|---|---|
All Workloads | 8443 TCP 22 TCP | Any 0.0.0.0/0 (IP List) | Optional: Opens up ports which are purposed for remote management. For example, TCP 22 to provide SSH services to OpenShift admins. TCP 8443 provides OpenShift admins with webconsole services. Webconsole may vary across OpenShift deployments. The ports can be modified to server other remote management services and consuming IP list can be changed to corporate subnets or jump servers. |
Infra (Role) | TCP 80 TCP 443 | Any 0.0.0.0/0 (IP List) | This policy assumes the router exists only on dedicated Infra nodes. If the router exists on other nodes, then modify the provider to the host where the router resides. This rule opens default front end router ports which are used to access containerized applications from external IP addresses. As you start to open up application pods to the outside world, you will need to add the application's exposed port to this policy's list of services. For example, you spin up a httpd server and expose that server on TCP 8080. The first step to allow access to the httpd server from outside is to add TCP 8080 to this line of policy. |
Note
The IP lists referenced in the rulesets are commonly used public registries (for example, docker.io) for container environments. If you have confirmed that your Kubernetes or OpenShift environment does not depend on the public registries mentioned above, then it is recommended that you remove the IP lists from the ruleset.