Explorer

The Public Experimental Explorer APIs search and analyze PCE traffic flows for auditing, reporting, and troubleshooting. You can search for traffic flows between workloads or hosts, labeled workloads, or IP addresses, and you can restrict the search by specific port numbers and protocols.

There are three APIs for the traffic flows search:

Traffic Analysis Queries

This was the basic traffic analyzer for queries that is now deprecated.

Functionality	HTTP	URI
Search the PCE traffic data database to discover traffic patterns and write policy.	`POST`	`[api_version][org_href]/traffic_flows/traffic_analysis_queries`

The maximum of returned results when using POST [api_version][org_href]/traffic_flows/traffic_analysis_queries was 100,000, which is a reasonable number a user can view in the UI. However, when Explorer is used for capturing all traffic flows into a CSV file to build rules offline, the queries take longer to return, traffic data contains more than 100,000 rows, and so on. Explorer queries are required to support both the single-node and multi-node Explorer in the SuperCluster environment. Therefore, the limitation of 100,000 results was raised to 200,000 to better support SuperCluster environments in Explorer.

Note

This API is now DEPRECATED and replaced with Asynchronous Queries for Traffic Flows, where the max-results limit is raised from 100,000 to 200,000.

Asynchronous Queries for Traffic Flows

Async Queries API Methods

Functionality	HTTP	URI
Create a new async traffic query.	`POST`	`[api_version][org_href]traffic_flows/async_queries`
Get a collection of async traffic queries.	`GET`	`[api_version][org_href]traffic_flows/async_queries`
Download the completed async traffic query results.	`GET`	`[api_version][org_href]traffic_flows_async/queries/:uuid/dowload`
Update an async traffic query (request cancellation of the queued async query)	`PUT`	`[api_version][org_href]traffic_flows/async_queries/:uuid`
Delete the completed async traffic query.	`DELETE`	`[api_version][org_href]traffic_flows/async_queries/:uuid`

Create New Async Traffic Queries

Parameters for POST [api_version][org_href] traffic_flows/async_queries:

Property	Description	Type	Req
`query_name`	Name of the query	String	Yes
`start_date`	Starting date for the query. If left empty, the default interpretation is “today,” which is “now” minus 24 hours.	Date-time string (`YYYY-MM-DDTHH:MM:SS`)	No
`end_date`	Ending date for the query. If left empty, the default interpretation is	Data-time string (`YYYY-MM-DDTHH:MM:SS`)	No
`sources`	Source labels, workloads, or IP addresses to include or exclude in the search. The response can contain up to five matching IP addresses. NOTE: The response returns `sources` as `consumers`. Sources are treated as consumers for the purposes of the request; the response returns the source of an individual flow as src. Sub-properties: `include`: Targets that can be included are workloads, labels, or IP addresses identified by their HREF and structured as an array of JSON objects. If this property is left empty, then `include` means consider “ALL” or “ANY” of the object type. `exclude`: Targets that can be excluded are workloads, labels, or IP addresses identified by their HREF and structured as a JSON array. When IP List is present in the consumer part of a traffic query, traffic from workloads that belong to that IP List will not be returned by default. If users want to see that traffic, they need to set `exclude_workloads_ from_ip_list_query: false` When IP List is present in the provider part of traffic query, traffic to workloads that belong to that IP List will not be returned by default. If the user wishes to see that traffic, they need to set `exclude_workloads_from_ip_ list_query: false`	Object	Yes Yes Yes
`destinations`	Target labels, workloads, or IP addresses to include or exclude in the search. The response returns `targets` as `providers`. Required sub-properties: `include`: Targets that can be included are workloads, labels, or IP addresses identified by their HREF and structured as an array of JSON objects. If this property is left empty, then `include` means consider “ALL” or “ANY” of the object type. `exclude`: Targets that can be excluded are workloads, labels, or IP addresses identified by their HREF and structured as a JSON array. If this property is left empty, then `exclude` means exclude “NONE” of the object types.	Object	Yes Yes
`services`	Services (5-tuple of port/to_port/proto/process/service) to include or exclude. Not all properties of the service subobjects are required. Required properties: `include`: List of included services (5-tuple of port/to_port/proto/process/service) `exclude:` List of excluded services (5-tuple of port/to_port/proto/process/service)", Properties of the `include`and `exclude` subobjects: `port`: Port Number (integer 0-65535). Also the starting port when specifying a range. `to_port`: High end of port range inclusive if specifying a range. If not specifying a range then don't send this: `proto`: Protocol number. For the expected proto values see IANA Protocol Numbers. `process_name`: name of the process `windows_service_name`: name of the Windows serviece		Yes
`policy_decisions`	List of policy decisions. Allows you to filter the query based on policy decision: `allowed`: Allowed traffic. `potentially_blocked`: Allowed but potentially blocked traffic. `blocked`: Blocked traffic. `unknown`	Array of strings	Yes
`boundary_decisions`	List of boundary decisions `blocked`: blocked due to boundary `override_deny_rule`: overridden deny rule `blocked_non_illumio_rule`: Deny rule not written by Illumio	Array
`max_results`	Maximum number of flows to return. Limit is 200,000	Integer	Yes
`exclude_workloads_ from_ip_list_query`	Exclude workload traffic when IP List is provided either in consumer or provider part of the traffic query	Boolean	Default is: `true`

Download Completed async traffic Query Results

Properties for GET [api_version][org_href] traffic_flows/async_queries_download

Poperty	Description	Type	Req
`src`	Reference to `traffic_flows_endpoint.schema.json`		Yes
`dst`	Reference to `traffic_flows_endpoint.schema.json`		Yes
`service`	Reference to `traffic_flows_service.schema.json`		Yes
`num_connections`	The number of times this flow was seen	Integer	Yes
`policy_decision`	Policy decision made	String	Yes
`draft_policy_decision`	The draft policy decision of the flow (added in release 23.2.10)	String	No
`timestamp_range`	Timestamp ranges for the flow detected. Required properties are: `first_detected` `last_detected`	Object	Yes
`caps`	Reference to `rbac_permission_types.schema.json`		Yes
`client_type`	The type of client that reported this flow	String	No

Introducing Illumination Plus

The API GET [api_version][org_href]traffic_flows_async/queries/:uuid/dowload has a new property: caps.

------------------------------------------------------------				
},
	"caps": {
		"description": "Array of permissions for the flow for the current user",
		"type": "array",
		"items": {
		"$ref": "rbac_permission_types.schema.json"
	}

The caps property references the common schema rbac_permission_types.schema.json, which indicates the RBAC permission that is used: write .

rbac_permission_types.schema.json

{  
	"$schema":"http://json-schema.org/draft-04/schema#",
	"type": "string",
		"description": "RBAC Permission types",
		"enum": ["write", "provision"]
}

In Illumination Plus, the type provision is not used to avoid additional delays when checking the permissions of each flow. Therefore, only permission write is used and further verification is handled on the UI side.

Examples

Example Async Explorer Queries

Curl command for POST traffic_flows_async_queries

curl -i -u api_1195cf055cf8a834c:148afd87ecc980900eaf10d6c54e6c0f607b22e0dbf768dd007e51e731096282 https://devtest0.ilabs.io:8443/api/v2/orgs/1/traffic_flows/async_queries -H "Content-Type: application/json" -X POST -d '{"sources":{"include":[[{"workload":{"href":"/orgs/1/workloads/a3ffb374-f6c6-4cce-ac57-642c66f1498f"}}]],"exclude":[]},"destinations":{"include":[[]],"exclude":[]},"services":{"include":[],"exclude":[]},"sources_destinations_query_op":"and","start_date":"2016-01-29T17:04:03.149Z","end_date":"2021-01-29T17:06:03.151Z","policy_decisions":[],"max_results":1000,"query_name":"worklaod test"}'

Response:

 HTTP/1.1 202 Accepted
	content-location: 7734501b-74a2-47a4-9ded-77bf4ceea938
	content-type: application/json
	content-length: 615
	x-request-id: 00c8fa00-dbd8-4a28-a5c7-354fb5ae3886
	cache-control: no-store
	x-frame-options: DENY
	x-xss-protection: 1; mode=block
	x-content-type-options: nosniff
{"status":"queued","href":"/orgs/1/traffic_flows/async_queries/7734501b-74a2-47a4-9ded-77bf4ceea938","created_by":{"href":"/users/1"},"query_parameters":{"sources":{"include":[[{"workload":{"href":"/orgs/1/workloads/a3ffb374-f6c6-4cce-ac57-642c66f1498f"}}]],"exclude":[]},"destinations":{"include":[[]],"exclude":[]},"services":{"include":[],"exclude":[]},"sources_destinations_query_op":"and","start_date":"2016-01-29T17:04:03.149Z","end_date":"2021-01-29T17:06:03.151Z","policy_decisions":[],"max_results":1000,"query_name":"worklaod test"},"created_at":"2021-04-09T20:50:30Z","updated_at":"2021-04-09T20:50:30Z"}

Curl command for GET traffic_flows/async_queries

This query gets the collection of all async jobs for the current user, including anything that was already submitted.

curl -i -u api_1195cf055cf8a834c:148afd87ecc980900eaf10d6c54e6c0f607b22e0dbf768dd007e51e731096282 https://devtest0.ilabs.io:8443/api/v2/orgs/1/traffic_flows/async_queries

Response

HTTP/1.1 200 OK
	content-type: application/json
	content-length: 1510
	x-request-id: fcf065e5-e465-4161-ba98-542182734c38
	cache-control: no-store
	x-frame-options: DENY
	x-xss-protection: 1; mode=block
	x-content-type-options: nosniff
[{"matches_count":1984,"flows_count":1000,"status":"completed","href":"/orgs/1/traffic_flows/async_queries/88675fbd-a88e-44bd-b358-2d6f2fc4f95a","result":"/orgs/1/traffic_flows/async_queries/88675fbd-a88e-44bd-b358-2d6f2fc4f95a/download","created_by":{"href":"/users/1"},"query_parameters":{"sources":{"include":[[{"workload":{"href":"/orgs/1/workloads/a3ffb374-f6c6-4cce-ac57-642c66f1498f"}}]],"exclude":[]},"destinations":{"include":[[]],"exclude":[]},"services":{"include":[],"exclude":[]},"sources_destinations_query_op":"and","start_date":"2016-01-29T17:04:03.149Z","end_date":"2021-01-29T17:06:03.151Z","policy_decisions":[],"max_results":1000,"query_name":"worklaod tesrrrrrt"},"created_at":"2021-04-09T20:50:19Z","updated_at":"2021-04-09T20:50:27Z"},{"matches_count":1984,"flows_count":1000,"status":"completed","href":"/orgs/1/traffic_flows/async_queries/7734501b-74a2-47a4-9ded-77bf4ceea938","result":"/orgs/1/traffic_flows/async_queries/7734501b-74a2-47a4-9ded-77bf4ceea938/download","created_by":{"href":"/users/1"},"query_parameters":{"sources":{"include":[[{"workload":{"href":"/orgs/1/workloads/a3ffb374-f6c6-4cce-ac57-642c66f1498f"}}]],"exclude":[]},"destinations":{"include":[[]],"exclude":[]},"services":{"include":[],"exclude":[]},"sources_destinations_query_op":"and","start_date":"2016-01-29T17:04:03.149Z","end_date":"2021-01-29T17:06:03.151Z","policy_decisions":[],"max_results":1000,"query_name":"worklaod test"},"created_at":"2021-04-09T20:50:30Z","updated_at":"2021-04-09T20:50:32Z"

Curl command for GET traffic_flows/async_queries/:uuid

This query gets a specific job included in the collection.

curl -i -u $KEY:$TOKEN https://devtest0.ilabs.io:8443/api/v2/orgs/1/traffic_flows/async_queries/88675fbd-a88e-44bd-b358-2d6f2fc4f95a

Response

HTTP/1.1 200 OK
	content-type: application/json
	content-length: 756
	x-request-id: f328b845-8542-4b96-a128-43aefdf7ba5a
	cache-control: no-store
	x-frame-options: DENY
	x-xss-protection: 1; mode=block
	x-content-type-options: nosniff
{"matches_count":1984,"flows_count":1000,"status":"completed",
"href":"/orgs/1/hanges for22.4.0 from the Wj/async_queries/88675fbd-a88e-44bd-b358-2d6f2fc4f95a",
"result":"/orgs/1/traffic_flows/async_queries/88675fbd-a88e-44bd-b358-2d6f2fc4f95a/download",
"created_by":{"href":"/users/1"},"query_parameters":{"sources":{"include":[[{"workload":{"href":"/orgs/1/workloads/a3ffb374-f6c6-4cce-ac57-642c66f1498f"}}]],"exclude":[]},"destinations":{"include":[[]],"exclude":[]},"services":{"include":[],"exclude":[]},"sources_destinations_query_op":"and","start_date":"2016-01-29T17:04:03.149Z","end_date":"2021-01-29T17:06:03.151Z","policy_decisions":[],"max_results":1000,"query_name":"worklaod tesrrrrrt"},"created_at":"2021-04-09T20:50:19Z","updated_at":"2021-04-09T20:50:27Z"}

Response for GET traffic_flows/async_queries/:uuid_download

{ 		
   "dst": {
	"ip": "10.244.0.1",
	"workload": {
		"href": "/orgs/1/workloads/35d8efea-f230-4027-a8ee-5f20626c4d21",
		"name": "wl3",
		"labels": [
			{
			"key": "env"reserpine for 
			"href": "/orgs/1/labels/7",
			"value": "Production"
			},
			{
			"key": "loc",
			"href": "/orgs/1/labels/11",
			"value": "Amazon"
			},
			{
			"key": "role",
			"href": "/orgs/1/labels/3",
			"value": "API"
			},
			{
			"key": "B-label",
			"href": "/orgs/1/labels/15",
			"value": "b_label_2"
			}
		],
		"managed": false,
		"os_type": "linux",
		"endpoint": false,
		"hostname": "",
		"enforcement_mode": "visibility_only"
		}
	},
    "src": {
	"ip": "10.0.2.15",
	"workload": {
		"href": "/orgs/1/workloads/fc3801b8-05ec-4954-a957-7f5673123389",
		"name": "wl2",
		"labels": [
		{
			"key": "env",
			"href": "/orgs/1/labels/7",
			"value": "Production"
			},
			{
			"key": "loc",
			"href": "/orgs/1/labels/11",
			"value": "Amazon"
			},
			{
			"key": "role",
			"href": "/orgs/1/labels/3",
			"value": "API"
			}
		],
		"managed": false,
		"os_type": "linux",
		"endpoint": false,
		"hostname": "",
		"enforcement_mode": "visibility_only"
		}
	},
    "caps": [],
    "state": "snapshot",
    "dst_bi": 0,
    "dst_bo": 0,
    "seq_id": 2,
    "network": {
	"href": "/orgs/1/networks/fbeeb98d-4ed6-428d-9f71-69f542bfd8fd",
	"name": "Corporate"
	},
    "service": {
	"port": 3306,
	"proto": 6
	},
    "flow_direction": "outbound",
    "num_connections": 1,
    "policy_decision": "unknown",
    "timestamp_range": {
	"last_detected": "2022-09-01T20:35:22Z",
	"first_detected": "2022-09-01T20:35:22Z"
	}
}

Filter for Managed Services

This API allows you to filter all managed services, such as workloads, virtual services, and so on.

Functionality	HTTP	URI
Get a list of Virtual Servers	`GET`	`[api_version][org_href]/sec_policy/:version/virtual_servers`
Get a specified Virtual Server	`GET`	`[api_version][org_href]/sec_policy/:version/virtual_servers/:uuid`

Database Metrics

The API Database Metrics provide organization-specific insight into the current traffic database. It allows you to monitor how big the traffic database is and how much data you can store. It also provides information about how many days of data is available.

The API database_metrics was expanded to include additional optional endpoint metrics: server, backlog , and endpoint.

These objects define the server's or endpoint's total flow data per organization for the total number of days, limit on the total number of days, oldest days, size in gigabytes, and so on.

Database Metrics API Methods

Functionality	HTTP	URI
Returns the organization database usage metrics. Provides customers organization-specific insight into current traffic database size (#days, #GB).	`GET`	`[api_version][org_href]traffic_flows/database_metrics`

Parameters for Database Usage Metrics

The organization flow Database Usage Metrics has the following required parameters:

Parameters	Description	Type	Required
`flows_days`	Organization's total number of days of flow data	Integer	Yes
`flows_days_limit`	Organization's limit on the total number of days of flow data The limit was increased from 90 to 97	Integer	Yes
`flows_oldest_day`	Organization's oldest day of flow data (yyyy-mm-dd)	String	No
`flows_size_gb`	Organization's limit on the total number of gigabytes of flow data	Number	Yes
`flows_size_gb_limit`	Organization's limit on the total number of gigabytes of flow data	Number	Yes
`server`	Define the server's total flow data per organization for the total numberof days, limit on the total number of days, oldest days, size in gigabytes, and so on.	Object	No
`endpoint`	Organization's total number of days of endpoint flow data.	Object	No
`backlog`	Total gigabytes used to store flow data input files	Object	No
`updated_at`	Timestamp in UTC when these flow metrics were generated	String, date	No

Parameters for `server`

Parameters	Description	Type
`num_flows_days`	Organization's total number of days of the server flow data	Integer
`num_flows_days_limit`	Organization's limit on the total number of days of server flow data	Integer
`flows_oldest_day`	Organization's oldest day of server flow data (yyyy-mm-dd)	String, date
`flows_size_gb`	Organization's limit on the total number of gigabytes of server flow data	Number
`flows_size_gb_limit`	Organization's limit on the total number of gigabytes of server flow data	Number
`num_daily_tables`	The number of server daily tables, including FlowLink and Cloud, counted once for each unique day.	Number
`num_weekly_tables`	The number of server weekly tables, including FlowLink and Cloud, counted once for each unique week.	Number

Parameters for `endpoint`

Parameters	Description	Type
`num_flows_days`	Organization's total number of days of the endpoint flow data	Integer
`num_flows_days_limit`	Organization's limit on the total number of days of endpoint flow data	Integer
`flows_oldest_day`	Organization's oldest day of endpoint flow data (yyyy-mm-dd)	String, date
`flows_size_gb`	Organization's limit on the total number of gigabytes of endpoint flow data	Number
`flows_size_gb_limit`	Organization's limit on the total number of gigabytes of endpoint flow data	Number
`num_daily_tables`	The number of endpoint daily tables is counted once for each unique day.	Number
`num_weekly_tables`	The number of endpoint weekly tables is counted once for each unique week.	Number

Parameters for `backlog`

Parameters	Description	Type
`total_disk_used_gb`	Total gigabytes used to store flow data input files	Number
`total_file_count`	Total number of flow data input files	Integer

An example response looks such as the following:

{
	"org_id":1,
	"server":{
		"flows_size_gb":2.53228759765625,
		"num_flows_days":95,
		"flows_oldest_day":"2023-02-06",
		"num_daily_tables":7,
		"num_weekly_tables":13,
		"flows_size_gb_limit":26,
		"num_flows_days_limit":90
	},
	"endpoint":{
		"flows_size_gb":0.34337615966796875,
		"num_flows_days":6,
		"flows_oldest_day":"2023-05-11",
		"num_daily_tables":6,
		"num_weekly_tables":0,
		"flows_size_gb_limit":26,
		"num_flows_days_limit":14
	},
	"flows_days":95,
	"flows_size_gb":2.8644485473632812,
	"flows_days_limit":90,
	"flows_oldest_day":"2023-02-06",
	"flows_per_second":0.0,
	"flows_size_gb_limit":26,
	"updated_at":"2023-05-16T22:36:25Z"
}

REST APIs for 23.5