Skip to main content

REST APIs for 22.2

Explorer

The Public Experimental Explorer APIs search and analyze PCE traffic flows for auditing, reporting, and troubleshooting. You can search for traffic flows between workloads or hosts, labeled workloads, or IP addresses, and you can restrict the search by specific port numbers and protocols.

There are three APIs for the traffic flow search:

Traffic Analysis Queries

This was the basic traffic analyzer for queries that is now deprecated:

Functionality

HTTP

URI

Search the PCE traffic data database to discover traffic patterns and write policies.

POST

[api_version][org_href]/traffic_flows/traffic_analysis_queries

Note

This API is now DERECATED and replaced with Asynchronous Queries for Traffic Flows, which has an additional parameter, query_name, and the max-results limit is raised from 100,000 to 200,000.

Create Traffic Analysis Queries
POST [api_version][org_href]/traffic_flows/traffic_analysis_queries
Properties

Property

Description

Type

Req

start_date

Starting date for the query. If left empty,

the default interpretation is “today,” which is “now” minus 24 hours.

Date-time string

(YYYY-MM-DDTHH:MM:SS)

Yes

end_date

Ending date for the query.

If left empty, the default interpretation is “today,” which is “now” plus 24 hours.

Data-time string

(YYYY-MM-DDTHH:MM:SS)

Yes

sources

Source labels, workloads, or IP addresses to include or exclude in the search.

The response can contain up to five matching IP addresses.

NOTE: The response returns sources as consumers.

Sources are treated as consumers for the purposes of the request; the response returns the source of an individual flow as src.

Sub-properties:

  • include: Targets that can be included are workloads, labels, or IP addresses identified by their HREF and structured as an array of JSON objects.

    If this property is left empty, then include means consider “ALL” or “ANY” of the object type.

  • exclude: Targets that can be excluded are workloads, labels, or IP addresses identified by their HREF and structured as a JSON array.

    • When IP List is present in the consumer part of a traffic query, traffic from workloads that belong to that IP List will not be returned by default.

      If users want to see that traffic, they need to set exclude_workloads_ from_ip_list_query: false

    • When IP List is present in the provider part of traffic query, traffic to workloads that belong to that IP List will not be returned by default.

      If the user wishes to see that traffic, they need to set

      exclude_workloads_from_ip_ list_query: false

Object

Yes

Yes

destinations

Target labels, workloads, or IP addresses to include or exclude in the search.

The response returns targets as providers.

Required sub-properties:

  • include: Targets that can be included are workloads, labels, or IP addresses identified by their HREF and structured as an array of JSON objects.

    If this property is left empty, then include means consider “ALL” or “ANY” of theobject type.

  • exclude: Targets that can be excluded are workloads, labels, or IP addresses identified by their HREF and structured as a JSON array.

    If this property is left empty, then exclude means exclude “NONE” of the object types.

Object

Yes

Yes

services

Services (5-tuple of port/to_port/proto/process/service) to include or exclude. Not all properties of the service subobjects are required.

Required properties:

  • include: Destination ports that must be matched in the traffic flows to be returned. An empty value is interpreted as “match any or all port values.”

  • exclude: Traffic flows with ports in this set are excluded from the final result.

    An empty value is interpreted as “exclude no ports” from the search.

Properties of the includeand exclude subobjects:

  • port: Port Number (integer 0-65535). Also the starting port when specifying a range.

  • to_port: High end of port range inclusive if specifying a range.

    If not specifying a range, then don't send this:

  • proto: Protocol number. For the expected proto values see IANA Protocol Numbers.

  • process_name

  • windows_service_name

Yes

Yes

policy_decisions

Allows you to filter the query based on policy decision:

  • allowed: Allowed traffic.

  • potentially_blocked: Allowed but potentially blocked traffic.

    This type of traffic occurs when a workload VEN is in the test policy state.

  • blocked: Blocked traffic.

  • unknown

Array of strings

Yes

max_results

Maximum number of flows to return

Integer

No

exclude_workloads _from_ip_list_query

Exclude workload traffic when the IP List is provided either in the consumer or provider part of the traffic query.

Boolean

Default is: true

Response Properties

Property

Description

Type

Required

src

Workloads (Workload), IP addresses (ip), or labels (label) that are consuming the service of the traffic flow returned in the response.

workload

ip

label

String

String

JSON list of strings

Yes

dst

Workloads (workload), IP addresses (ip), or labels (label) that are providing the service of the traffic connection in the response.

workload

ip

label

String

String

JSON list of strings

Yes

service

Port, protocol, process, service name, and user_name of this flow

Additional required properties:

  • port: Destination port

  • proto :IANA protocol number

  • process_name: Process Name for this flow

  • windows_service_name: Windows Service Name for this flow

  • user_name: User Name for this flow

Object

Integer

Integer

String

String

String

Yes

num_connections

The number of traffic flows reported in the connection is as follows:

Integer

Yes

policy_decisions

If there is a rule for the returned traffic data, this property indicates the policy decision for the flow. Indicates if the traffic flow was allowed, potentially blocked (but allowed), or blocked.

  • allowed: Allowed traffic

  • potentially_blocked: Allowed but potentially blocked traffic

  • blocked: Blocked traffic

  • unknown

String

Yes

flow_direction

Flow direction is inbound or outbound.

String

Yes

transmission

The transmission type is broadcast or multicast.

String

No

timestamp_range

The time range during which the flows were reported in date-time format. Includes both of these sub-properties:

  • first_detected: The first time this flow was detected within the time range specified by the query

  • last_detected: The last time this flow was detected within the time range specified by the query.

String

Yes

state

State of the flow, which can be one of the following values:

  • active

  • closed

  • timed-out

  • snapshot

  • new

  • incomplete

String

No

dst_bo

Bytes sent till now by the destination over the flow during the interval.

Integer

dst_bi

Bytes have been received by the destination over the flow during the interval.

Integer

icmp_type

ICMP type for the flow

Integer

icmp_code

ICMP code for the flow

Integer

network

PCE network on which this flow was observed.

Additional properties:

  • name (network name)

  • href (network href)

Object

String

String

Yes

Filter for Managed Services

This API allows you to filter all managed services, such as workloads, virtual services, and so on.

Functionality

HTTP

URI

Get a list of Virtual Servers.

GET

[api_version][org_href]/sec_policy/:version/virtual_servers

Get a specified Virtual Serve.r

GET

[api_version][org_href]/sec_policy/:version/virtual_servers/:uuid

Example Explorer Search

One preliminary method of creating policy is to make sure that different data center environments are segmented from each other, such as separating development and testing environments from production environments. Before writing policy rules to allow or block this traffic, determine if there are any traffic flows between them.

With Explorer, you can ask the PCE:

Were there any traffic flows between my development and production environments over any port besides port 80, excluding any Workloads that have a "Domain Controller" role label?

To construct this query, specify the following information:

  • sources: In the JSON request, include the HREF of the “Development” environment label and exclude the HREF of the “Domain Controller” role label.

  • targets: In the JSON request, include the HREF of the “Production” environment label and exclude any workloads that do not have a role label assigned to them.

  • ports: Search for traffic over any port except port 80.

Curl Command to Search Traffic Data with Explorer

curl -i -X POST https://pce.my-company.com:8443/api/v2/orgs/7/traffic_flows/traffic_analysis_queries -H "Content-Type: application/json" -u $KEY:$TOKEN -T explorer-example.json

The above command lists the contents of explorer-example.json to specify the query.

The example should be similar to the following:

explorer-example.json

{
  "sources": {
    "include": [
      [
        {
          "label": {
            "href": "/orgs/1/labels/20"
          }
        }
      ]
    ],
    "exclude": [
      {
        "label": {
          "href": "/orgs/1/labels/21"
        }
      }
    ]
  },
  "destinations": {
    "include": [
      [
        {
          "label": {
            "href": "/orgs/1/labels/202"
          }
        }
      ]
    ],
    "exclude": [
      {
        "label": {
          "href": "/orgs/1/labels/205"
        }
      }
    ]
  },
  "services": {
    "include": [
      {
        "port": 80
      }
    ],
    "exclude": [
      {
        "port": 888
      }
    ]
  },
  "policy_decisions": ["allowed"], 
  "max_results": 2
}

Response

The response shows a single traffic flow between your Development and Production Environment, the consumer of which is a workload identified by its IP address and labels, communicating with an unidentified IP address on TCP (number 6) over port 80.

{
	"src": {
	    "ip": "10.2.3.161",
            "workload": {
	        "href": "/orgs/1/workloads/3a5f2482-1188-4449-b035-0c7880486a4f",
		"hostname": "johns-test-client5",
		"name": null,
		"os_type": linux,
		 "labels": [{
		    "href": "/orgs/1/labels/144",
		     "key": "app",
		     "value": "test_app_1"  
        }, 
        {
		  "href": "/orgs/1/labels/143",
			"key": "role",
			"value": "test_access_1"
		}]
	    }
	},
	"dst": {
		"ip": "10.2.3.165",
		"workload": {
		    "href": "/orgs/1/workloads/87a10385-7466-488f-bce4-899de08974a0",
		    "hostname": "johns-test-server-4",
		    "name": null,
	            "os_type": linux,
		    "labels": [{
			"href": "/orgs/1/labels/145",
			"key": "app",
			"value": "test_app_2"
		}, {
			"href": "/orgs/1/labels/151",
			"key": "env",
			"value": "test_env_2"
		}, {
			"href": "/orgs/1/labels/149",
			"key": "loc",
			"value": "test_place_1"
		}, {
			"href": "/orgs/1/labels/143",
			"key": "role",
			"value": "test_access_1"
		}]
	}
    },
	"port": 14000,
	"proto": 17,
	"num_connections": 2,
	"policy_decision": "allowed",
	"timestamp_range": {
		"last_detected": "2020-03-03T00:38:12Z",
		"first_detected": "2020-03-03T00:38:12Z"
	}
}, {
	"src": {
	    "ip": "10.2.3.161",
	    "workload": {
		    "href": "/orgs/1/workloads/3a5f2482-1188-4449-b035-0c7880486a4f",
		    "hostname": "johns-test-client5",
		    "name": null,
		    "os_type": linux,
		    "labels": [{
			"href": "/orgs/1/labels/144",
			"key": "app",
			"value": "test_app_1"
			.........
		{
			"href": "/orgs/1/labels/143",
			"key": "role",
			"value": "test_access_1"
		}]
	}
    },
	"dst": {
		"ip": "10.2.3.165",
		"workload": {
		    "href": "/orgs/1/workloads/87a10385-7466-488f-bce4-899de08974a0",
		    "hostname": "johns-test-server-4",
		    "name": null,
		    "os_type": linux,
		    "labels": [{
			"href": "/orgs/1/labels/145",
			"key": "app",
			"value": "test_app_2"
			..........
			{
  			"href": "/orgs/1/labels/143",
			"key": "role",
			"value": "test_access_1"
			}]
		}
	},
	"port": 14000,
	"proto": 6,
	"num_connections": 6,
	"policy_decision": "allowed",
	"timestamp_range": {
		"last_detected": "2020-03-03T00:38:10Z",
		"first_detected": "2020-03-03T00:38:10Z"
	}
}

Asynchronous Queries for Traffic Flows

This new API is very similar to the existing one, Traffic Analysis Query, but the new one has a new parameter, query_name, while the max_results limit is raised to 200,000. Currently, there is no change to POST[api_version][org_href]/traffic_flows/traffic_analysis_queries, which will eventually be deprecated.

The maximum number of returned results in POST [api_version][org_href]/traffic_flows/traffic_analysis_queries is 100,000, which is a reasonable number a user can view in the UI. However, when Explorer captures all traffic flows into a CSV file to build rules offline, the queries take longer to return, traffic data contains more than 100,000 rows, and so on.

Explorer queries are required to support both the single-node and multi-node Explorer in the SuperCluster environment. Therefore, the limitation of 100,000 results was raised to 200,000 to support SuperCluster environments in Explorer better.

Async Queries API Methods

Functionality

HTTP

URI

Create a new async traffic query.

POST

[api_version][org_href]traffic_flows/async_queries

Get a collection of async traffic queries.

GET

[api_version][org_href]traffic_flows/async_queries

Download the completed async traffic query results.

GET

api_version][org_href]traffic_flows_async/queries/:uuid/dowload

Update an async traffic query (request cancellation of the queued async query).

PUT

[api_version][org_href]traffic_flows/async_queries/:uuid

Delete the completed async traffic quer.y

DELETE

[api_version][org_href]traffic_flows/async_queries/:uuid

Note

This new API is very similar to the existing one, Traffic Analysis Query, but the new one has a new parameter, query_name, and the max-results limit is raised to 200,000.

Parameters for GET [api_version][org_href] traffic_flows/async_queries

Property

Description

Type

Required

org_id

Organization

Integer

Yes

Parameters for POST [api_version][org_href] traffic_flows/async_queries

Property

Description

Type

Required

org_id

Organization

Integer

Yes

query_name

Name of the query.

String

Yes

sources

Source labels, workloads, and IP addresses to include or exclude.

Object

Yes

services

Services (5-tuple of port/to_port/proto/process/service) to include or exclude.

Object

Yes

max_results

Maximum number of flows to return.

"minimum": 0,

"maximum": 200000

Integer

No

status

Query status

String

Yes

start_date

Starting date for query

String/date

No

end_date

Ending date for query

String/date

No

ip_list

Object

Yes

ip_address

IP address value

String

Yes

boundary_decisions

List of boundary decisions

String

Example Async Explorer Queries

Curl command for traffic_flows_async_queries_post

curl -i -u api_1195cf055cf8a834c:148afd87ecc980900eaf10d6c54e6c0f607b22e0dbf768dd007e51e731096282 https://devtest0.ilabs.io:8443/api/v2/orgs/1/traffic_flows/async_queries -H "Content-Type: application/json" -X POST -d '{"sources":{"include":[[{"workload":{"href":"/orgs/1/workloads/a3ffb374-f6c6-4cce-ac57-642c66f1498f"}}]],"exclude":[]},"destinations":{"include":[[]],"exclude":[]},"services":{"include":[],"exclude":[]},"sources_destinations_query_op":"and","start_date":"2016-01-29T17:04:03.149Z","end_date":"2021-01-29T17:06:03.151Z","policy_decisions":[],"max_results":1000,"query_name":"worklaod test"}'

Response:

 HTTP/1.1 202 Accepted
	content-location: 7734501b-74a2-47a4-9ded-77bf4ceea938
	content-type: application/json
	content-length: 615
	x-request-id: 00c8fa00-dbd8-4a28-a5c7-354fb5ae3886
	cache-control: no-store
	x-frame-options: DENY
	x-xss-protection: 1; mode=block
	x-content-type-options: nosniff
{"status":"queued","href":"/orgs/1/traffic_flows/async_queries/
		7734501b-74a2-47a4-9ded-77bf4ceea938","created_by":
		{"href":"/users/1"},"query_parameters":{"sources":{"include":[
		[{"workload":{"href":"/orgs/1/workloads/a3ffb374-f6c6-4cce-ac57-642c66f1498f"}}]],
		"exclude":[]},"destinations":{"include":[[]],"exclude":[]},"services":
		{"include":[],"exclude":[]},"sources_destinations_query_op":"and","start_date":
		"2016-01-29T17:04:03.149Z","end_date":"2021-01-29T17:06:03.151Z",
		"policy_decisions":[],"max_results":1000,"query_name":"worklaod test"},
		"created_at":"2021-04-09T20:50:30Z","updated_at":"2021-04-09T20:50:30Z"}

Curl command for traffic_flows/async_queries_get

This query gets the collection of all async jobs for the current user, including anything that was already submitted.

curl -i -u api_1195cf055cf8a834c:148afd87ecc980900eaf10d6c54e6c0f607b22e0dbf768dd007e51e731096282 https://devtest0.ilabs.io:8443/api/v2/orgs/1/traffic_flows/async_queries

Response

HTTP/1.1 200 OK
	content-type: application/json
	content-length: 1510
	x-request-id: fcf065e5-e465-4161-ba98-542182734c38
	cache-control: no-store
	x-frame-options: DENY
	x-xss-protection: 1; mode=block
	x-content-type-options: nosniff
[{"matches_count":1984,"flows_count":1000,"status":"completed","href":"/orgs/1/traffic_flows/async_queries/88675fbd-a88e-44bd-b358-2d6f2fc4f95a","result":"/orgs/1/traffic_flows/async_queries/88675fbd-a88e-44bd-b358-2d6f2fc4f95a/download","created_by":{"href":"/users/1"},"query_parameters":{"sources":{"include":[[{"workload":{"href":"/orgs/1/workloads/a3ffb374-f6c6-4cce-ac57-642c66f1498f"}}]],"exclude":[]},"destinations":{"include":[[]],"exclude":[]},"services":{"include":[],"exclude":[]},"sources_destinations_query_op":"and","start_date":"2016-01-29T17:04:03.149Z","end_date":"2021-01-29T17:06:03.151Z","policy_decisions":[],"max_results":1000,"query_name":"worklaod tesrrrrrt"},"created_at":"2021-04-09T20:50:19Z","updated_at":"2021-04-09T20:50:27Z"},{"matches_count":1984,"flows_count":1000,"status":"completed","href":"/orgs/1/traffic_flows/async_queries/7734501b-74a2-47a4-9ded-77bf4ceea938","result":"/orgs/1/traffic_flows/async_queries/7734501b-74a2-47a4-9ded-77bf4ceea938/download","created_by":{"href":"/users/1"},"query_parameters":{"sources":{"include":[[{"workload":{"href":"/orgs/1/workloads/a3ffb374-f6c6-4cce-ac57-642c66f1498f"}}]],"exclude":[]},"destinations":{"include":[[]],"exclude":[]},"services":{"include":[],"exclude":[]},"sources_destinations_query_op":"and","start_date":"2016-01-29T17:04:03.149Z","end_date":"2021-01-29T17:06:03.151Z","policy_decisions":[],"max_results":1000,"query_name":"worklaod test"},"created_at":"2021-04-09T20:50:30Z","updated_at":"2021-04-09T20:50:32Z"

Curl command for traffic_flows/async_queries/:uuid_get

This query gets a specific job included in the collection.

curl -i -u $KEY:$TOKEN https://devtest0.ilabs.io:8443/api/v2/orgs/1/traffic_flows/async_queries/88675fbd-a88e-44bd-b358-2d6f2fc4f95a

Response

HTTP/1.1 200 OK
	content-type: application/json
	content-length: 756
	x-request-id: f328b845-8542-4b96-a128-43aefdf7ba5a
	cache-control: no-store
	x-frame-options: DENY
	x-xss-protection: 1; mode=block
	x-content-type-options: nosniff
{"matches_count":1984,"flows_count":1000,"status":"completed",
"href":"/orgs/1/traffic_flows/async_queries/88675fbd-a88e-44bd-b358-2d6f2fc4f95a",
"result":"/orgs/1/traffic_flows/async_queries/88675fbd-a88e-44bd-b358-2d6f2fc4f95a/download",
"created_by":{"href":"/users/1"},"query_parameters":{"sources":{"include":[[{"workload":{"href":"/orgs/1/workloads/a3ffb374-f6c6-4cce-ac57-642c66f1498f"}}]],"exclude":[]},"destinations":{"include":[[]],"exclude":[]},"services":{"include":[],"exclude":[]},"sources_destinations_query_op":"and","start_date":"2016-01-29T17:04:03.149Z","end_date":"2021-01-29T17:06:03.151Z","policy_decisions":[],"max_results":1000,"query_name":"worklaod tesrrrrrt"},"created_at":"2021-04-09T20:50:19Z","updated_at":"2021-04-09T20:50:27Z"}
Asynchronous Queries

Explorer provides a view to query traffic data to help users build segmentation / intra / extra scope rules. Previously, it caped the maximum number of returned results to 100,000, which is enough for the number of results a user can view in the UI. However, when you use Explorer to capture all traffic flows into a CSV file to build rules offline, the queries take longer to return, traffic data contains more than 100,000 rows, and so on.

To better support SuperCluster environments in Explorer, the limitation of 100,000 results requires a new approach to using Explorer queries. Explorer queries are required to support both the single-node and multi-node Explorer in the SuperCluster environment.

New APIs for asynchronous queries are introduced while there is no change to the existing APIs, which will be later deprecated.

Async Queries API Methods

Functionality

HTTP

URI

Create a new async traffic query.

POST

[api_version][org_href]traffic_flows_async_queries

Get a collection of async traffic queries.

GET

[api_version][org_href]traffic_flows_async_queries

Get an async traffic query. Asynchronous explorer query status.

GET

[api_version][org_href]traffic_flows_async_query/:uuid

Update an async traffic query (request cancellation of the queued async query)

PUT

[api_version][org_href]traffic_flows_async_queries/:uuid

Delete the completed async traffic query.

DELETE

[api_version][org_href]traffic_flows_async_queries/:uuid

Properties

Property

Description

Type

Required

href

(GET) Query URI

String

Yes

status

(GET) Current query status

String

Yes

creted_at

Timestamp in UTC when this query was created

date/time

Yes

created_by

The user who created this query

String

Format URI

Yes

query_parameters

Explorer query parameters

Object

Yes

query_name

(GET, POST) Name of the query.

String

Yes

sources

(GET, POST) Source labels, workloads, or IP addresses to include or exclude in the search.

Sub-properties:

  • include: Targets that can be included

  • exclude: Targets that

Object

Yes

destinations

(GET, POST)Target labels, workloads, IP addresses, domain names, and transmission to include or exclude

Yes

services

(GET, POST) Services (5-tuple of port/to_port/proto/process/service) to include or exclude

Object

Yes

policy_decisions

(GET, POST) List of policy decisions

Array

Yes

result

(GET) Result from download URI, available only if the status is completed

String

No

updated_at

Timestamp in UTC when this async query was last updated

date/time

No

start_date

(GET, POST) Starting date for the query

Data-time string (YYYY-MM-DDTHH:MM:SS)

No

end_date

(GET, POST) Ending date for the query.

Data-time string (YYYY-MM-DDTHH:MM:SS)

No

matches_count

Query result count

Integer

No

flows_count

Result count after query limits and RBAC filtering are applied.

Integer

No

sources_destinations_query_op

(GET, POST) Query logical operator between sources and destinations.

Default is "and"

String

No

regions

Region-specific response metadata. Required properties:

pce_fqdn

responded

Integer

No

max_results

(GET, POST) Maximum number of flows to return.

"minimum": 0,

"maximum": 200000

Integer

No

exclude_workloads _from_ip_list_query

(GET, POST) Exclude workload traffic when the IP List is provided either in the consumer or provider part of the traffic query. Default is TRUE

Boolean

No

boundary_decisions

List of boundary decisions

String

Database Metrics

This Public Experimental endpoint provides organization-specific insight into the current traffic database. It gives you the ability to monitor how big the traffic database is and how much data you can store. It also provides information about how many days of data is available.

Database Metrics API Method

Functionality

HTTP

URI

Returns the organization database usage metrics. Provides customers organization-specific insight into current traffic database size (#days, #GB).

GET

[api_version][org_href]traffic_flows/database_metrics

Parameters for Database Metrics

Property

Description

Type

Required

flows_days

Organization's total number of days of flow data

Integer

Yes

flows_days_limit

Organization's limit on the total number of days of flow data

Integer

Yes

flows_oldest_day

Organization's oldest day of flow data (yyyy-mm-dd)

String, date

No

flows_size_gb

Organization's limit on the total number of gigabytes of flow data

Number

Yes

flows_size_gb_limit

Organization's limit on the total number of gigabytes of flow data

Number

Yes

updated_at

Timestamp in UTC when these flow metrics were generated

String,

date/time

Yes

Curl Command to get Database Metrics
curl -i -u  api_1f0c3ebf44dcafb19:6de325a25e7e4383de1dea9d8c7ce230773ad4d8a5cbb62980e7d176254f263d https://test.mylab.io:8443/api/v2/orgs/1/traffic_flows/database_metrics

Response

{"flows_days":2,"flows_size_gb":0.2695770263671875,"flows_days_limit":90,"flows_size_gb_limit":26,"updated_at":"2021-11-17T20:50:45Z"}%
See also: