Skip to main content

Cloud

  • Cloud
  • Get Started
  • Set up flow logs in your CSP environment

Set up flow logs in your CSP environment

Learn how to setup flow logs for use by Illumio Segmentation for the Cloud.

Illumio Segmentation for the Cloud uses flow logs to provide visual diagrams and analysis of traffic between sources and destinations. In order for these features to work, after you set up flow logs, you need to grant Illumio Segmentation for the Cloud access to these flow logs through your CSP. The sequence is as follows:

  1. Set up flow logs as described here.

  2. Review destinations before granting flow log access. See Review destinations before granting flow log access.

  3. Review prerequisites for granting flow log access. See Prerequisites for granting flow log access to your CSPs

  4. Grant flow log access. For AWS, Azure, and GCP, see Grant flow log access to your CSPs. For OCI, see Grant OCI flow log access.

Note

Do you have central log storage or cross-account storage? This is where flow logs generated from one account (such as "account A1") are stored in a destination belonging to another account (such as "account A2").

You can onboard the accounts in any order but if you onboard "account A1" first, you will not see its flow logs until you onboard "account A2." See Onboarding AWS Cloud, Onboarding Azure, Onboarding GCP, and Onboarding OCI.

If you onboard the account storing the flow logs ("account A2") first, you will not see the traffic of "account A1" because it has not yet been onboarded. You also must onboard the account that is generating flow logs ("account A1"). Otherwise, Illumio Segmentation for the Cloud does not properly map the flow logs.

In short, you must onboard both.

Set up flow logs in AWS

You can set up flow logs in AWS using the console, a CloudFormation template, or the command line. You must do this before you Grant flow log access to your CSPs. Note that for AWS, Illumio Segmentation for the Cloud can read flows from S3 buckets only, so it is important to configure these accordingly.

For multi-account AWS customers, Illumio recommends deploying a centralized log storage strategy. This allows you to configure a single S3 bucket destination for all cross-VPC and cross-account flow logs. For details on configuring the necessary roles and permissions, see the AWS website.

Set up AWS flow logs using the console

To configure flow logs for a VPC in the AWS console:

  1. Go to the VPC console at https://console.aws.amazon.com/vpc/ and select the region to which the VPC belongs.

  2. Select the VPC for which flow logs are to be enabled.

  3. Under the VPC details page, select the Flow logs page and click the Create flow log button.

  4. Provide the following details in the flow log configuration page:

    • Name for the flow log config

    • Type of traffic to be filtered. For more insights, select All.

    • The time interval can be set to 10 minutes

  5. Select Send to an Amazon S3 bucket and paste the ARN of the S3 bucket. It also provides the option to create a new S3 bucket from there.

  6. For log record format, select any value. For more details, select Custom format and select all attributes. Use defaults for all other values.

  7. After entering the required information click Create flow log.

Using the CloudFormation template

To enabled flowlogs for a VPC using the CloudFormation template:

  1. Go to the VPC console page at https://console.aws.amazon.com/vpc/, select the VPC for which the flow logs are to be enabled, and copy the VPC ID.

  2. Go to the S3 console page at https://console.aws.amazon.com/s3/ and select the bucket in which the flow logs are to be stored. Under the Properties tab, copy the name.

  3. Save the following CloudFormation Template to a file named enabling-vpc-flowlogs.yaml.

AWSTemplateFormatVersion: "2010-09-09"

Description: "Enable Flow logs for a vpc"

Parameters:

VpcId:

Type: String

Description: VPC Id for which flow logs are to be enabled

BucketName:

Type: String

Description: Name of the bucket in which flow logs are to be stored.

Resources:

FlowLog:

Type: AWS::EC2::FlowLog

Properties:

ResourceId: !Ref VpcId

ResourceType: "VPC"

TrafficType: "ALL"

LogDestination: !Join

- ""

- ["arn:aws:s3:::", !Ref BucketName]

LogDestinationType: "s3"

LogFormat: "${version} ${account-id} ${interface-id} ${srcaddr} ${dstaddr} ${srcport} ${dstport} ${protocol} ${packets} ${bytes} ${start} ${end} ${action} ${log-status}"

MaxAggregationInterval: 600

Tags:

- Key: "Name"

Value: "FlowLogsForIllumioCloud"

- Key: "Purpose"

Value: "Alltrafficvizualizationmap"

Outputs:

FlowLogArn:

Description: The ARN of the created flow log

Value: !Ref FlowLog

For more information, see the AWS website.

Running the CloudFormation template

  1. Go to AWS CloudFormation service and use the template file to create a new stack with new resources (standard).

  2. Select Template is Ready and then Upload a template file. Upload the enabling-vpc-flowlogs.yaml file.

  3. In the next page, enter a desired stack name followed by the bucket name and VPC ID you copied before.

  4. Click Next and leave default values in the successive pages. In the final page click Create stack.

After the stack creation is complete, go to the VPC console and verify the flow logs being created.

Note

The template must be run in the same region in which the VPC belongs. Choose the appropriate region on top right before running CloudFormation template.

Using the command line

See the AWS website.

What's next for AWS flow logs

Now you can Grant flow log access to your CSPs.

Set up flow logs in Azure

You must do this before you Grant flow log access to your CSPs. For VNETs and NSGs, see this Azure website. For firewalls, see this Azure website.

What's next for Azure flow logs

Now you can Grant flow log access to your CSPs.

Set up flow logs in GCP

You must do this before you Grant flow log access to your CSPs. For VPC and Firewalls, use the following steps. In this example, you'll set up flow logs for a VPC.

Note

Illumio Segmentation for the Cloud supports the Compute Engine API. It does not support the Network Management API at time of writing.

First, create a flow log configuration.

  1. Login to your Google Cloud shell and go to the VPC networks page.

  2. Click on your VPC to open its details panel.

  3. Click the Subnets tab, select the subnets for which you wish to enable flow logs, and click Manage flow logs.

  4. In the dropdown menu that appears, click Add new configuration.

  5. Select Compute Engine. Illumio recommends that you use the default settings for the rest of the fields.

  6. Click Save.

  7. Browse to the VPC Flow Logs page > Subnet table to verify that you created the flow log configuration.

    For more information on using flow logs and enabling subnets, see the Google website.

Now, set up a log router to tell the GCP log explorer to redirect the logs to Illumio Segmentation for the Cloud.

  1. Type 'log' in the search bar and select the Logs Explorer result.

  2. In the Logs Explorer, go to the left-hand navigation and click Log Router.

  3. Click Create sink.

  4. In Sink details, enter a name and optional description for your sink, and click Next.

  5. In Sink destination, select Cloud Pub/Sub topic for your sink service, and select an existing topic or create a new one. Click Next.

  6. In Choose logs to include in sink and Choose logs to filter out of sink, create your inclusion and exclusion filters. That way the log router will redirect only the logs that match the filter. For example, you might include resource.type="gce_subnetwork" or something similar. You can put 'NOT' in front of an inclusion filter entry to exclude it. Note that at time of writing, Illumio Segmentation for the Cloud processes only VPC and Firewall flow logs, so you may want to include only those or exclude all other logs.

    Note

    Illumio supports the following log sink filters. Note that at least one log sink for each of your topics must have at least one of these filters applied. Otherwise, Illumio will not display the topics and log sinks on the flow access page.

    VPC Flow Logs:

    • projects/<PROJ-ID>/logs/compute.googleapis.com%2Fvpc_flows

    • folders/<FOLDER-ID>/logs/compute.googleapis.com%2Fvpc_flows

    • organizations/<ORG-ID>/logs/compute.googleapis.com%2Fvpc_flows

    • projects/<PROJ-ID>/logs/networkmanagement.googleapis.com%2Fvpc_flows

    • folders/<FOLDER-ID>/logs/networkmanagement.googleapis.com%2Fvpc_flows

    • organization/<ORG-ID>/logs/networkmanagement.googleapis.com%2Fvpc_flows

    Firewall Logs:

    • projects/<PROJ-ID>/logs/compute.googleapis.com%2Ffirewall

    • folders/<FOLDER-ID>/logs/compute.googleapis.com%2Ffirewall

    • organizations/<ORG-ID>/logs/compute.googleapis.com%2Ffirewall

    • projects/<PROJ-ID>/logs/networkmanagement.googleapis.com%2Ffirewall

    • folders/<FOLDER-ID>/logs/networkmanagement.googleapis.com%2Ffirewall

    • organizations/<ORG-ID>/logs/networkmanagement.googleapis.com%2Ffirewall

  7. Click Update Sink.

Now, verify that the flow logs are being redirected to Illumio Segmentation for the Cloud.

  1. Go to the Logs Explorer as previously described.

  2. Click Log Router, and look in the Name column for the name of the log router you created.

  3. Look in the Destination column to verify that the destination contains the topic you selected or created when choosing your sink destination.

  4. Type 'topic' in the search bar and select the Topics Pub/Sub result.

  5. In the list of topics, click the topic that you just verified in the Log Router.

  6. Click on your subscription, click the Messages tab, and click Pull to verify that your sink is sending messages to the subscription.

What's next for GCP flow logs

Now you can Grant flow log access to your CSPs.

Set up flow logs in OCI

You can set up flow logs in the console. You must do this before you Grant flow log access to your CSPs.

This guide provides a concise step-by-step process for you to enable Virtual Cloud Network (VCN) flow logs in Oracle Cloud Infrastructure (OCI) and create a service connector to store these logs in an Object Storage bucket. Observe the following prerequisites:

  • OCI Access: Ensure that you have the necessary permissions to manage Networking, Logging, Service Connector Hub, and Object Storage services.

  • Existing VCN: Identify the VCN for which you want to enable flow logs.

  • Bucket: Create a bucket to store flow logs.

Enable VCN Flow Logs

  1. Browse to Networking > Virtual Cloud Networks.

  2. Select the compartment and choose your VCN.

  3. Browse to Networking > Resources > Flow Logs on the OCI console.

  4. Configure the Flow Log with the following:

    • Name: Enter a name for the flow log.

    • Compartment: Select the appropriate compartment.

    • Enablement Point: Select VCN.

    • Flow logs that need to be enabled: Select the appropriate VCN for which flow logs need to be enabled.

    • Flow Log Type: Choose All Traffic with 100% sampling rate.

    • Log Group: Select an existing log group or create a new one.

    • Log: Provide a name for the log.

  5. Click Create Flow Log.

Store Logs

  1. If an object storage bucket is not already available, browse to Storage > Buckets.

  2. Click Create Bucket, provide a name, and set the desired configurations.

  3. Navigate to Analytics and AI > Messaging > Service Connectors.

  4. Configure the Service Connector with the following:

    • Name: Enter a name for the connector.

    • Source:

      • Select Logs

      • Configure the source to use the Log Group and Log from your flow log

    • Target:

      • Select Object Storage and choose the bucket you created.

      • (Optional) Set the Object Name Prefix, Batch Rollover Size, and Batch Time Interval.

  5. Click Create Service Connector.

  6. Navigate to your Object Storage Bucket and confirm that log files are being generated and stored. This usually takes some time to show up, as connector will start streaming from the logging service.

What's next for OCI flow logs

Now you can Grant flow log access to your CSPs.