Collect Flow Records from F5
The example listed in the following steps uses a virtual edition of the F5 BIG-IP appliance in AWS and the Illumio Flowlink application to gather and parse flow data.
Important
IPFIX and NetFlow have slightly different configuration steps depending on which flow record standard you choose.
Requirements
Flowlink (flow collector)
F5 BIG-IP system with LTM
A virtual server configured on F5 box
Note
F5 must have a self-IP interface. The flows are sent out of this interface. When Flowlink is not in the same subnet as the self-IP, you must know the default gateway IP of the self-IP interface.
Create a Pool for Flow Collector
To create a pool of flow collectors to receive the flow record messages from the F5 system:
In the F5 UI, click Main > Local Traffic > Pools > Pool Lists > Create.
Enter a unique name in the Name field, which represents the flow collector.
A Health Monitor is not required. If you want to see if the F5 system can reach the flow collector, select
gateway_icmpand move it to the Active box.In the New Member section, configure the collector IP address.
Click Add.
If you are using IPFIX, use the following configuration:
Field
Value
Node Name
Enter the Collector IP address
Service Port
4739
If you are using NetFlow, use the following configuration:
Field
Value
Node Name
Enter the Collector IP address
Service Port
2055
Click Finished.
The below example shows two (2) different nodes configured in one pool. Both nodes have the IP address. However, one is for IPFIX and one is for NetFlow. Even though F5 allows two nodes in the pool, it is recommended to only have one node enabled (either 2055 or 4739).
Example with NetFlow enabled and IPFIX disabled:
Create a Log Destination
To create a log destination to stream the logs in either IPFIX or NetFlow V9 format to the Pool:
In the F5 UI, click Main > System > Logs > Configuration > Log Destinations > Create.
Enter a unique name in the Name field, which represents the flow collector.
In the Type field, select IPFIX.
Configure the IPFIX Settings.
If you are using IPFIX, use the following configuration:
Field
Value
Protocol
Select IPFIX
Pool Name
Select the pool created earlier
Transport Profile
UDP
If you are using NetFlow, use the following configuration:
Field
Value
Protocol
Select NetFlow V9
Pool Name
Select the pool created earlier
Transport Profile
UDP
Click Finished.
Example of a Log Destination configuration with NetFlow:
Create a Log Publisher
To create a log publisher to send logs to the specified log destination:
In the F5 UI, click Main > System > Logs > Configuration > Log Publishers > Create.
Enter a unique name in the Name field, which represents the flow collector.
In the Destination field, move your log destination from Available to Selected.
Click Finished.
Create an iRule
To create an iRule to which it parses network traffic and sends flow records to the specified log publisher:
Go to Main > iRules > iRule List > Create.
Enter a unique name in the Name field, which represents the flow collector.
In the Definition text field, enter the rules for parsing traffic. Ensure the iRule points to the log publisher created earlier.
Click Finished.
Important
In the iRule example shown below, replace <insert_log_publisher_name_here> with the name of the log publisher.
when RULE_INIT {
set static::http_rule1_dest ""
set static::http_rule1_tmplt ""
}
# CLIENT_ACCEPTED event to initiate IPFIX destination and template
when CLIENT_ACCEPTED {
set start [clock clicks -milliseconds]
if { $static::http_rule1_dest == ""} {
# open the logging destination if it has not been opened yet
set static::http_rule1_dest [IPFIX::destination open -publisher /Common/
<insert_log_publisher_name_here>]
}
if { $static::http_rule1_tmplt == ""} {
# if the template has not been created yet, create the template
set static::illumio_flowlink_POC_tmplt [IPFIX::template create "flowStartMilliseconds
sourceIPv4Address sourceIPv6Address destinationIPv4Address destinationIPv6Address
sourceTransportPort destinationTransportPort protocolIdentifier octetTotalCount
packetTotalCount octetDeltaCount packetDeltaCount postNATSourceIPv4Address
postNATSourceIPv6Address postNATDestinationIPv4Address postNATDestinationIPv6Address
postNAPTSourceTransportPort postNAPTDestinationTransportPort postOctetTotalCount
postPacketTotalCount postOctetDeltaCount postPacketDeltaCount flowEndMilliseconds "]
}
set rule1_msg1 [IPFIX::msg create $static::http_rule1_tmplt]
}
# SERVER_CONNECTED event to initiate flow data to specified log publisher and populate 5
tuples
when SERVER_CONNECTED {
set client_closed_flag 0
set server_closed_flag 0
IPFIX::msg set $rule1_msg1 flowStartMilliseconds $start
IPFIX::msg set $rule1_msg1 protocolIdentifier [IP::protocol]
# Clientside
if { [clientside {IP::version}] equals "4" } {
# Client IPv4 address
IPFIX::msg set $rule1_msg1 sourceIPv4Address [IP::client_addr]
# BIG-IP IPv4 VIP address
IPFIX::msg set $rule1_msg1 destinationIPv4Address [clientside {IP::local_addr}]
} else {
# Client IPv6 address
IPFIX::msg set $rule1_msg1 sourceIPv6Address [IP::client_addr]
# BIG-IP IPv6 VIP address
IPFIX::msg set $rule1_msg1 destinationIPv6Address [clientside {IP::local_addr}]
}
# Client port
IPFIX::msg set $rule1_msg1 sourceTransportPort [TCP::client_port]
# BIG-IP VIP port
IPFIX::msg set $rule1_msg1 destinationTransportPort [clientside {TCP::local_port}]
# Serverside
if { [serverside {IP::version}] equals "4" } {
# BIG-IP IPv4 self IP address
IPFIX::msg set $rule1_msg1 postNATSourceIPv4Address [IP::local_addr]
# Server IPv4 IP address
IPFIX::msg set $rule1_msg1 postNATDestinationIPv4Address [IP::server_addr]
} else {
# BIG-IP IPv6 self IP address
IPFIX::msg set $rule1_msg1 postNATSourceIPv6Address [IP::local_addr]
# Server IPv6 IP address
IPFIX::msg set $rule1_msg1 postNATDestinationIPv6Address [IP::server_addr]
}
# BIG-IP self IP port
IPFIX::msg set $rule1_msg1 postNAPTSourceTransportPort [TCP::local_port]
# Server port
IPFIX::msg set $rule1_msg1 postNAPTDestinationTransportPort [TCP::server_port]
}
# SERVER_CLOSED event to collect IP pkts and bytes count on serverside
when SERVER_CLOSED {
set server_closed_flag 1
# when flow is completed, BIG-IP to server REQUEST pkts and bytes count
IPFIX::msg set $rule1_msg1 octetTotalCount [IP::stats bytes out]
IPFIX::msg set $rule1_msg1 packetTotalCount [IP::stats pkts out]
# when flow is completed, server to BIG-IP RESPONSE pkts and bytes count
IPFIX::msg set $rule1_msg1 octetDeltaCount [IP::stats bytes in]
IPFIX::msg set $rule1_msg1 packetDeltaCount [IP::stats pkts in]
IPFIX::destination send $static::http_rule1_dest $rule1_msg1
}
# CLIENT_CLOSED event to collect IP pkts and bytes count on clientside
when CLIENT_CLOSED {
set client_closed_flag 1
# when flow is completed, client to BIG-IP REQUEST pkts and bytes octetDeltaCount
IPFIX::msg set $rule1_msg1 postOctetTotalCount [IP::stats bytes in]
IPFIX::msg set $rule1_msg1 postPacketTotalCount [IP::stats pkts in]
# when flow is completed, BIG-IP to client RESPONSE pkts and bytes count
IPFIX::msg set $rule1_msg1 postOctetDeltaCount [IP::stats bytes out]
IPFIX::msg set $rule1_msg1 postPacketDeltaCount [IP::stats pkts out]
# record the client closed time in ms
IPFIX::msg set $rule1_msg1 flowEndMilliseconds [clock click -milliseconds]
# send the IPFIX log
IPFIX::destination send $static::http_rule1_dest $rule1_msg1
}Apply the iRule to a Virtual Server
To apply the iRule to a virtual server whose traffic you want to parse:
Go to Main > Virtual Server > Virtual Server List.
Select the virtual server you want to monitor.
Click the Resources tab. In the iRule section, click Manage.
Select the iRule that you previously created and move the iRule from Available to Enable.
Click Finished.
Example of a Virtual Server Resources page with the new iRule applied:
Create a Route Entry
By default, all traffic is sent out of the management interface. However, F5 does not support flow exports via the management NIC. You must add a route to force traffic, which is destined to the flow collector to leave a self-IP interface.
To create a route entry, if the F5 self-IP is unable to reach the flow collector:
In the F5 UI, click Main > Network > Routes > Add.
In the Properties section, create a route entry to send the flow records from F5 to the external flow collector IP address.
For Resource, select the Use Gateway option.
