Container Clusters
The Illumio Core uses three groups of APIs to manage container clusters:
Container Cluster API (GET, POST, PUT, DELETE)
Container Cluster Workload Profiles API (GET, POST, PUT, DELETE)
Container Cluster Service Backend API (GET)
Container Cluster API
A container cluster object stores all the information about a Kubernetes cluster in the PCE by collecting telemetry from Kubelink. Each Kubernetes cluster maps to one container cluster object in the PCE.
Use these methods to get, create, update, or delete container clusters:
Functionality | HTTP | URI |
---|---|---|
Get the list of container clusters. |
|
|
Get the specified container cluster. |
|
|
Create a container cluster. |
|
|
Update the specified container cluster. |
|
|
Delete the specified container cluster. |
|
|
Container Cluster Workload Profiles
When you install an Illumio VEN on a container cluster, all pods are unmanaged or not visible in the PCE. However, Kubelink reports all namespaces on the container clusters, which are then visible via the Container Workload Profiles API.
Each container workload profile maps to a Kubernetes namespace and can be managed or unmanaged. The default state for a profile is unmanaged.
Use these methods to get, create, update, or delete container cluster workload profiles:
Functionality | HTTP | URI |
---|---|---|
Get the list of container cluster workload profiles. |
|
|
Create container cluster workload profiles. |
|
|
Update the specified container cluster workload profile. |
|
|
Supports the UI feature for bulk update of container workload profiles. |
|
|
Delete the specified container cluster workload profile. |
|
|
Label Restrictions
Kubernetes pods and services running in a namespace (Kubernetes) or project (OpenShift) must be labeled (RAEL) to be included in the policy within Illumio Core. The container workload profile defines how labels will be assigned to pods and services within a namespace.
Using annotations, Illumio labels can be statically assigned from the PCE or defined in the Kubernetes manifest files. For each label key (RAEL), the PCE administrator can define four options:
No label will be assigned.
One label will be assigned from PCE.
Using annotations, Kubernetes can assign a restricted list of labels. Label restrictions prevent the misuse of Illumio labels by the people managing the Kubernetes platform and ensure that the labels inherit the policy they should receive.
Any label can be assigned from Kubernetes.
You can set role labels for the following APIs:
PUT /api/v2/orgs/:xorg_id/container_clusters/<:cluster_id>/ container_workload_profiles
POST /api/v2/orgs/:xorg_id/container_clusters/<:cluster_id>/ container_workload_profiles
Label Assignment Configuration
To clear the label assignment option and go back to the default option (any labels passed at runtime using Kubernetes annotations will be allowed), two options:
Option 1: explicit statement
{ "labels": [ { "key": "role", "restriction": [] } ] }
Option 2: empty payload
{ "labels": [] }
Backend Services Associated with Container Clusters
The Illumio policy model represents Kubernetes services as virtual services. For these services, Kubelink creates virtual services in the PCE and reports the list of Replication Controllers, Daemon Sets, and Replica Sets responsible for managing the pods that support them.
When a match between the Replication Controller and ReplicaSet managing a pod, the PCE creates a binding between the virtual service and the container workload.
The Service Backend matches a virtual service and an application type, such as Deployment or ReplicaSet.
Use this method to get the service backend:
Functionality | HTTP | URI |
---|---|---|
Get data about the service backend. |
|
|
Kubernetes APIs
Kubernetes Workload Endpoints
Customers must see the details of Kubernetes workloads in PCE to write policies and troubleshoot any issues.
Two new endpoints have been created for Kubernetes workloads:
GET /api/v2/orgs/:xorg_id/kubernetes_workloads
This API lists all new Kubernetes Workloads in a separate tab/page with separate sorts and filters.
It contains required properties such as
name
,kind
,namespace
, as well as optional properties:href
,labels
,enforcement_mode
,visibility_level
,container_workload_profile
,container_cluster
,security_policy_applied_at
,security_policy_sync_state
,created_at
,k8s_label
, andk8s_annotations
.GET /api/v2/orgs/:xorg_id/kubernetes_workloads
For this API, these changes have been made in release 23.5.0:
two arrays have been removed,
k8s_labels
and sk8s_annotation
, and replaced with propertymetadata
.HREF description has been changed from
URI of the container workload
, toURI of the kubernetes workload
.
"metadata": { "$ref": " ../common/kubernetes_workloads_metadata.schema.json"
GET /api/v2/orgs/:xorg_id/kubernetes_workloads/:kubernetes_workload_uuid
This API provides a detailed page for the specified Kubernetes workload with custom K8S attributes.
common non_empty_label_scopes.schema.json
This new common schema provides a collection of assigned list of labels. Minimum number is one.
common kubernetes_workloads_metadata
The new common schema
kubernetes_workloads_metadata
was added in release 23.5.0, referenced fromkubernetes_workload_get
.It provides Kubernetes properties such as labels, annotations, and the external service's UID.
Container Clusters Reference
This topic covers properties, parameters, and examples for container clusters.
Parameters
GET Method
Use the following required and optional parameters:
Parameter | Description | Type | Required |
---|---|---|---|
| URI of the container cluster. | String | Yes |
| User-assigned name of the container cluster. | String | Yes |
| User-assigned description of the container cluster. | String | Yes |
| Array | No | |
| This parameter has the following property:
| Object String | Yes |
| Manager of the container cluster (and version). | String | No |
| Type of network. | String | No |
| Date-time format. | String | No |
| Online: true/false. | Boolean | No |
| The object
| Array Object String Array String String | No |
| Kubelink software version. | String | No |
| PCE FQDN is used for this container cluster only in Supercluster. | String | No |
| The new property | String |
POST and PUT methods
Use the following parameters:
Parameter | Description | Type | Required |
---|---|---|---|
| User-assigned name of the cluster | String | Yes |
| User-assigned description of the cluster | String | No |
Curl Examples
Curl Command for GET
curl --request GET --url https://pce.my-company.com:8443/api/v2/orgs/1/container_clusters --header 'authorization: Basic YXBpXzE2YjBkYjI0MjJhZGNkYWU5OjA5ZmRjNjA4MDhiMzExZTc2Y2UyNzNmOWNiN2ZhMTA5OTdkMWNlMDAzZmMzOTQ1ZGMxYzEwZGzlmZjM='
Example Response for GET
[ { "href":"/orgs/1/container_clusters/445bfa9b-4de4-4c09-9705-496eb04b190f", "pce_fqdn":null, "name":"k8s2", "description":"", "manager_type":"Kubernetes v1.16.2", "last_connected":"2019-10-28T22:48:31.228Z", "kubelink_version":"2.0.0-master.96e58b", "online":true, "nodes": [ { "name":"node1", "pod_subnet":"10.233.64.0/24" }, { "name":"node2", "pod_subnet":"10.233.65.0/24" }, { "name":"node3", "pod_subnet":"10.233.66.0/24" } ], "errors":[] }, { "href":"/orgs/1/container_clusters/ad678193-8e2f-402b-a864-4947dcc0c6d7", "pce_fqdn":null, "name":"Openshift 3.11", "description":"", "manager_type":"Openshift v3.11.43", "last_connected":"2019-10-28T22:50:30.201Z", "kubelink_version":"1.0.0-master.a81280", "online":true, "nodes": [ { "name":"ip-172-31-19-198.us-west-2.compute.internal", "pod_subnet":"10.128.0.0/23" }, { "name":"ip-172-31-20-168.us-west-2.compute.internal", "pod_subnet":"10.131.0.0/23" }, { "name":"ip-172-31-22-56.us-west-2.compute.internal", "pod_subnet":"10.130.0.0/23" }, { "name":"ip-172-31-27-241.us-west-2.compute.internal", "pod_subnet":"10.129.0.0/23" } ], "errors":[] }, { "href": "/orgs/1/container_clusters/bef57e90-97d4-4744-a129-5d35aa12b21b", "pce_fqdn":null, "name":"k8s3 Cluster", "description":"Flannel Vx Lan", "manager_type":"Kubernetes v1.13.2", "last_connected":"2019-10-28T22:47:59.122Z", "kubelink_version":"EYE-60264", "online":true, "nodes": [ { "name":"k8s3master", "pod_subnet":"10.244.0.0/24" }, { "name":"k8s3minion1", "pod_subnet":"10.244.2.0/24" }, { "name":"k8s3minion2", "pod_subnet":"10.244.1.0/24" } ], "errors":[] }, { "href":"/orgs/1/container_clusters/d7d62400-7650-4407-ae9b-71803dbb1324", "pce_fqdn":null, "name":"k8s1 v4", "description":"", "manager_type":"Kubernetes v1.12.4", "last_connected":"2019-10-24T23:58:55.795Z", "kubelink_version":"EYE-61567", "online":false, "nodes": [ { "name":"k8s1master", "pod_subnet":"10.244.0.0/24" }, { "name":"k8s1minion1", "pod_subnet":"10.244.2.0/24" }, { "name":"k8s1minion2", "pod_subnet":"10.244.1.0/24" } ], "errors":[] } ]
Curl Example for POST
curl --request POST --url https://pce.my-company.com:8443/api/v2/orgs/1/container_clusters --header 'authorization: Basic jI0MjJhZGNkYWU5OjA5ZmRjNjA4MDhiMzExZTc2Y2UyNzNmOWNiN2ZhMTA5OTdkMWNlMDAzZmMzOTQ1ZGMxYzEwZGJhZTg5NzlmZjM=' --header 'content-type: application/json' --data '{"name": "test","description": "test"}'
Curl Example for PUT
curl --request PUT --url https://pce.my-company.com:8443/api/v2/orgs/1/container_clusters/1b851d4b-f22d-47be-b744-f3c2dca490a0 --header 'authorization: Basic YXBpXzE2YjBkYjI0MjJhZGNkYWU5OjA5ZmRjNjA4MDhiMzExZTc2Y2UyNzNmOWNiN2ZhMTA5OTdkMWNlMDAzZmMzOTQ1ZGMxYzEwZGJhZTg5NzlmZjM=' --header 'content-type: application/json' --data '{"name": "test","description": "test"}'
Example Response for POST
{ "href": "/orgs/1/container_clusters/1b851d4b-f22d-47be-b744-f3c2dca490a0", "pce_fqdn": null, "name": "test", "description": "test", "manager_type": null, "last_connected": null, "kubelink_version": null, "online": false, "nodes": [], "errors": [], "container_cluster_token": "1_0dfec0acb8e4bc53e052874874da0c24e7ac98da3b3954e3c9ea6f9860722e84" }
Parameters for container workload methods
Parameter | Description | Type | Required |
---|---|---|---|
| Organization ID | Integer | Yes |
| Cluster UUID | String | Yes |
| (GET) List of lists of label URIs, encoded as a JSON string (POST, PUT) Assigned labels | String | No No |
| (GET) Filter by enforcement mode. (PUT) workload enforcement mode | String | No No |
| Filter by linked container workload profiles. | Boolean | No |
| Filter by managed state | Boolean | No |
| Maximum number of container workloads to return. | Integer | No |
| (GET) Name string to match. Supports partial matches. (POST) A friendly name given to a profile if the namespace is not user-friendly | String | No YES |
| Namespace string to match. Supports partial matches. | String | No |
| Filter by visibility level | String | No |
Properties for container workload methods
Property | Description | Type |
---|---|---|
|
| String |
| Reference to | |
| If the namespace is managed or not | Boolean |
| Maximum number of container workloads to return. | Integer |
| A friendly name given to a profile if the namespace is not user-friendly | String, Null |
| Namespace | String, Null |
| Container workload profile UUID | String |
| Labels to assign to the workload that matches the namespace. Reference to |
Curl Examples
Curl example for GET
curl --request GET --url https://pce.my-company.com:8443/api/v2/orgs/1/containermeters iun one table and verified with the Quick Reference._clusters/445bfa9b-4de4-4c09-9705-496eb04b190f/container_workload_profiles --header 'authorization: Basic NjA4MDhiMzExZTc2Y2UyNzNmOWNiN2ZhMTA5OTdkMWNlMDAzZmMzOTQ1ZGMxYzEwZGJhZTg5NzlmZjM=' --header 'content-type: application/json'
Curl Example for POST
curl --request POST --url https://pce.my-company.com:8443/api/v2/orgs/1/container_clusters/445bfa9b-4de4-4c09-9705-496eb04b190f/container_workload_profiles --header 'authorization: Basic A5ZmRjNjA4MDhiMzExZTc2Y2UyNzNmOWNiN2ZhMTA5OTdkMWNlMDAzZmMzOTQ1ZGMxYzEwZGJhZTg5NzlmZjM=' --header 'content-type: application/json' --data '{"name": "test","description": "test","assign_labels": [{"href": "/orgs/1/labels/1"}],"mode": "full","log_traffic": true}'
Curl Example for PUT
curl --request PUT --url https://pce.my-company.com:8443/api/v2/orgs/1/container_clusters/445bfa9b-4de4-4c09-9705-496eb04b190f/container_workload_profiles/219b49c3-3bb5-4fc0-9913-b76398105e35 --header 'authorization: Basic mRjNjA4MDhiMzExZTc2Y2UyNzNmOWNiN2ZhMTA5OTdkMWNlMDAzZmMzOTQ1ZGMxYzEwZGJhZTg5NzlmZjM=' --header 'content-type: application/json' --data '{"name": "test","description": "test","assign_labels": [{"href": "/orgs/1/labels/1"}],"mode": "full","log_traffic": true}'
Example Response for GET
[ { "href": "/orgs/10/container_clusters/974aec34-e8e7-478d-9ca2-90ebb3642edc/container_workload_profiles/5454cc84-d6be-4e6c-ac62-465f9504fac0", "namespace": "openshift-host-network", "enforcement_mode": "visibility_only", "visibility_level": "flow_summary", "managed": true, "assign_labels": [ { "href": "/orgs/10/labels/128" }, { "href": "/orgs/10/labels/225" } ], "labels": [ { "key": "loc", "assignment": { "href": "/orgs/10/labels/128", "value": "AWS" } }, { "key": "env", "assignment": { "href": "/orgs/10/labels/225", "value": "OCP4.6" } } ], "linked": true, "created_at": "2021-08-25T18:11:52.665Z", "created_by": { "href": "/orgs/10/container_clusters/974aec34-e8e7-478d-9ca2-90ebb3642edc" }, "updated_at": "2021-08-25T18:11:52.665Z", "updated_by": { "href": "/orgs/10/container_clusters/974aec34-e8e7-478d-9ca2-90ebb3642edc" } } ]
Examples for container_workload_profiles/update
Request
{ "container_workload_profiles": [ { "href": "url_to_some_container_workload_profile" }, { "href": "url_to_other_container_workload_profile" } ], "labels": [ { "key": "role", "assignment": { "href": "url_to_label" } } ], "enforcement_mode": 2, "visibility_level": "flow_summary", "managed": true }
Example Response
For success: Response code 204; Response body: none
If an error occurred on any of the input records:
Response code 406;
Response body:
[ { "token": "input_validation_error", "message": "....., record_index=>1, ..., unmanaged_container_workload_profile_labels, ..." # message contains index of failed record and specific error message }, ... ]
Examples for label restrictions
Set an empty Role label.
{ "labels": [ {"key": "role", "assign": {} } ] }
Set a Location label.
PUT /api/v2/orgs/1/container_clusters/65d1f197-938a-49ef-9343-6f55ec76fd90/container_workload_profiles/afe4661a-03ef-462f-ada6-ce7334aa9704 { "labels": [ { "key": "loc", "restriction": {"href": "/orgs/1/labels/221"} } ] }
Set an allow list for the Environment label.
Allow a list of Environment labels to be assigned using Kubernetes:
PUT /api/v2/orgs/1/container_clusters/65d1f197-938a-49ef-9343-6f55ec76fd90/container_workload_profiles/afe4661a-03ef-462f-ada6-ce7334aa9704 { "labels": [ { "key": "env", "restriction": [{"href": "/orgs/1/labels/176"}, {"href": "/orgs/1/labels/302"}, {"href": "/orgs/1/labels/303"}] } ] }
Allow any value for the Application label.
PUT /api/v2/orgs/1/container_clusters/65d1f197-938a-49ef-9343-6f55ec76fd90/container_workload_profiles/afe4661a-03ef-462f-ada6-ce7334aa9704 { "labels": [ { "key": "app", "restriction": [] } ] }
Multiple ways to assign or allow labels used together in one Container Workload Profile
PUT /api/v2/orgs/1/container_clusters/65d1f197-938a-49ef-9343-6f55ec76fd90/container_workload_profiles/afe4661a-03ef-462f-ada6-ce7334aa9704 { "labels": [ {"key": "role", "assign": {} }, {"key": "app", "restriction": [] }, {"key": "env", "restriction": [{"href": "/orgs/1/labels/176"}, {"href": "/orgs/1/labels/302"}, {"href": "/orgs/1/labels/303"}] }, {"key": "loc", "assign": {"href": "/orgs/1/labels/221"} } ] }
Result for the above example:
role
: No label will be set; it is an explicit statement (you don't want arole
label to be assigned).app
: Any value can be set in the annotations for theapp
label key (provided the value exists in PCE).env
: Only the values specified in the allowlist can be set in the annotations for theenv
label key.loc
: The value of theloc
label key is assigned to the value defined in the payload.
Properties
Backend services associated with container clusters
Property | Description | Type | Required |
---|---|---|---|
| The name of the container cluster backend. | String | Yes |
| The type (or kind) of the container cluster backend. | String | Yes |
| The time (rfc339 timestamp) at which the container cluster backend was updated. | String | Yes |
| The time (rfc339 timestamp) at which the container cluster backend was created. | String | Yes |
| Includes the following properties:
| Object String String | Yes |
Curl Example for GET
curl --request GET --url https://pce.my-company.com:8443/api/v2/orgs/1/container_clusters/445bfa9b-4de4-4c09-9705-496eb04b190f/service_backends --header 'authorization: Basic YzE2YjBkYjI0MjJhZGNkYWU5OjA5ZmRjNjA4MDhiMzExZTc2Y2UyNzNmOWNiN2ZhMTA5OTdkMWNlMDAzZmMzOTQ1ZGMxYzEwZGJhZTg5NzlmZjM='
Example Response for GET
[ { "name": "58687784f9", "kind": "replicasethash", "namespace": "kube-system", "updated_at": "2020-10-25T20:07:39.741Z", "created_at": "2020-10-25T20:07:39.741Z", "virtual_service": { "href": "/orgs/1/sec_policy/draft/virtual_services/926c2f63-bcd8-42f1-8811-165b34f84334", "name": "coredns-k8s2-kube-system" } }, { "name": "556b9ff8f8", "kind": "replicasethash", "namespace": "kube-system", "updated_at": "2020-10-25T20:07:39.768Z", "created_at": "2020-10-25T20:07:39.768Z", "virtual_service": { "href": "/orgs/1/sec_policy/draft/virtual_services/58b0df03-1151-464e-8352-069e3ad0d7ed", "name": "kubernetes-dashboard-k8s2-kube-system" } } ]
GET /api/v2/orgs/:xorg_id/kubernetes_workloads
{ "$schema": "http://json-schema.org/draft-04/schema#", "type": "object", "required": [ "name", "kind", "namespace" ], "properties": { "href": { "description": "URI of the container workload", "type": "string" }, "name": { "description": "Container workload name", "type": "string" }, "namespace": { "description": "k8s namespace where this k8s Workload belongs to", "type": "string" }, "kind": { "description": "k8s resource kind, e.g. Deployment", "type": "string" }, "labels": { "type": "array", "items": { "$ref": "../common/label_optional_key_value.schema.json" } }, "enforcement_mode": { "$ref": "../common/workload_enforcement_mode.schema.json" }, "visibility_level": { "$ref": "../common/workload_visibility_level.schema.json" }, "container_workload_profile": { "$ref": "container_clusters_container_workload_profiles_get.schema.json" }, "container_cluster": { "$ref": "container_clusters_get.schema.json" }, "security_policy_applied_at": { "description": "Last reported time when policy was processed by CLAS to the k8s workload (UTC)", "type": [ "string", "null" ], "format": "date-time" }, "security_policy_sync_state": { "description": "Current state of security policy", "type": "string" }, "created_at": { "description": "RFC 3339 timestamp at which this record was created", "format": "date-time", "type": "string" }, "updated_at": { "description": "RFC 3339 timestamp at which this record was updated", "format": "date-time", "type": "string" }, "k8s_labels": { "type": "array", "items": { "type": "object", "required": [ "key", "value" ], "properties": { "key": { "type": "string" }, "value": { "type": "string" } } } }, "k8s_annotations": { "type": "array", "items": { "type": "object", "required": [ "key", "value" ], "properties": { "key": { "type": "string" }, "value": { "type": "string" } } } } } }
GET /api/v2/orgs/:xorg_id/kubernetes_workloads/:kubernetes_workload_uuid
{ "$schema": "http://json-schema.org/draft-04/schema#", "description": "Collection of assigned list of labels", "type": "array", "items": { "$ref": "labels.schema.json", "minItems": 1 }, "uniqueItems": true, "minItems": 1 }
common kubernetes_workloads_metadata
{ "$schema": "http://json-schema.org/draft-04/schema#", "description": "k8s object metadata", "additionalProperties": false, "type": "object", "properties": { "labels": { "description": "k8s key/value pairs attached to object that specify identifying attributes", "type": "object" }, "annotations": { "description": "k8s key/value pairs representing arbitrary non-identifying metadata of object", "type": "object" }, "external_service_uid": { "description": "k8s object uid of external traffic service (NodePort or LoadBalancer)", "type": "string" } } }
For more information, see Illumio Core for Kubernetes and OpenShift.