Back Up Supercluster
You need to perform regular backups on all PCEs in the Supercluster.
Different data is backed up depending on whether you run the backup from the Supercluster leader or a member:
Leader backup: Contains all Supercluster replicated data, including workloads, labels, rulesets, rules, services, organization events, workload traffic data, and Supercluster configuration data.
Member backup: Contains the member's local data, including login information, workload traffic data, and Supercluster configuration data.
All PCE nodes' runtime environment file: The
runtime_env.yml
is not included in the backup and must be backed up separately for each node. The default location of the PCE Runtime Environment File is/etc/illumio-pce/runtime_env.yml
. When the location is different on your system, you can find it by checking the value of theILLUMIO_RUNTIME_ENV
environment variable.Traffic database: The traffic database dump can be very large, depending on the traffic datastore size. Therefore, the Supercluster database dump on leader and member PCEs does not include the traffic data. A separate procedure is provided. See "Back Up the Traffic Database" in PCE Administration Guide.
When to Back Up
Follow your own organization's policies and procedures for backup, including frequency (such as, hourly, daily, or weekly) and retention of backups offsite or on a system other than any of the Supercluster nodes.
Illumio recommends taking backups in the following situations:
Before and after a PCE version upgrade
After pairing a large number of VENs
After updating a large number of workloads (such as changing workload policy state or applying labels)
After provisioning major policy changes
After making major changes in your environment that affect workload information (such as an IP address changes)
Before and after adding new PCEs to your Supercluster
After you assign a new leader
On-demand backups before the procedures documented in this guide, such as migration and upgrade
Determine the Data Node of All PCEs
The data node is the node that runs the agent_traffic_redis_server
service. To determine the data node, run the following command:
sudo -u ilo-pce illumio-pce-ctl cluster-status
Expected output:
SERVICES (runlevel: 5) NODES (Reachable: 1 of 1) ====================== =========================== agent_background_worker_service 192.168.33.90 agent_service NOT RUNNING agent_slony_service 192.168.33.90 agent_traffic_redis_cache 192.168.33.90 agent_traffic_redis_server 192.168.33.90 <=== Run backup command on this node agent_traffic_service NOT RUNNING ...
Note
Check for agent_traffic_redis_server
on a data node before every backup, because this service can be running on either data node.
Back-Up Each PCE's Data
For the leader and every member PCE in your Supercluster, perform these steps:
Log into the node running the
agent_traffic_redis_server
service.Create a directory for the backup file that is not one of the PCE software's installation directories.
Grant permissions to both the ilo-pce user and the user who will run the backup command Read and Writeectory.
Run the following command:
sudo -u ilo-pce install_root/illumio-pce-db-management supercluster-data-dump --file desired_location_of_backup_file
Repeat these steps for every PCE in the Supercluster.
Copy Leader Backup to Members
Copy the backup file that you just made on the leader PCE to the data0 node of each member PCE. The leader's data is readily available to every member, so you can more quickly restore the entire Supercluster. You can copy the file to any file system location of the member data0 node, except for the PCE software's installation directories.
Back-Up Leader and Member Runtime Environment Files
Store a copy of each node's runtime_env.yml
file on a system not part of the Supercluster.
By default, the PCE Runtime Environment File is stored in /etc/illumio-pce/runtime_env.yml
. When the location is different on your system, locate the file by checking the ILLUMIO_RUNTIME_ENV
environment variable.