Expand and Migrate a PCE Supercluster

You can expand a standalone PCE to a supercluster and migrate an existing supercluster.

Expand Standalone PCE to Supercluster

To expand your existing standalone PCE to a Supercluster, follow the same steps as for installing a new Supercluster, with additional checks of the standalone PCE before the expansion.

The general workflow for expanding an existing PCE deployment into a Supercluster follows these steps:

Change the pce_fqdn on your standalone PCE; then log into the standalone PCE's web console to verify that the standalone PCE is healthy and working correctly.
See Change FQDN and Verify PCE Health for information.
Verify network connectivity to the database nodes. See Network Connectivity from PCEs to Data Nodes for information.
Initialize your existing PCE as the Supercluster leader. See Initialize Supercluster Leader for information.
Install and configure the new PCEs that will become members of the new Supercluster. See the PCE Installation and Upgrade Guide for information.
Join members to the Supercluster. See Join Each Member to Supercluster for information.

Illumio recommends that you perform each of these operations during different change windows.

After your Supercluster is operational, you can reassign workloads connected to the leader to a different PCE in the Supercluster.

Change FQDN and Verify PCE Health

See How to Rename the PCE in the Illumio Knowledge Base for information. (Login required)

Warning

After changing the pce_fqdn parameter and before proceeding with the expansion, you must log into the standalone PCE's web console to verify that it is healthy and working correctly.

Network Connectivity from PCEs to Data Nodes

Before expansion of a Supercluster, ensure that every data node in the standalone cluster can connect to the database nodes via the Supercluster FQDN.

To verify the connections, use telnet or the nc (netcat) utility, which is part of the NMAP set of tools. If not already installed, install NMAP with the following command:

yum install nmap

Important

Required runlevels

Be sure that the PCEs are set to the following runlevels before checking connectivity:

On the PCE from which you run the check: Runlevel 2
On the PCEs in other regions that you are checking: Runlevel 2 or higher

For example, you have three regions. With the following nc commands on data0 and data1 in each region, test the connection to the other regions by connecting to port 5432 for the other regions' data0 and data1 nodes.

From region 1: Set the PCE from which you are testing to runlevel 2:
```
nc -zv region2_data0_ip 5432
nc -zv region3_data0_ip 5432
```
From region 2: Set the PCE from which you are testing to runlevel 2:
```
nc -zv region1_data0_ip 5432
nc -zv region3_data0_ip 5432
```
From region 3: Set the PCE from which you are testing to runlevel 2:
```
nc -zv region1_data0_ip 5432
nc -zv region2_data0_ip 5432
```

Migrate to a New Supercluster

When you need to migrate your existing Supercluster to a new set of machines, follow these general steps:

On the PCE being migrated, pre-configure the new Supercluster's IP addresses in the runtime_env.yml file. For information, see Pre-configure New IP Addresses .
Back up the current Supercluster. See Back Up Supercluster for information.
Restore the old Supercluster configuration and data to the new systems. If you are migrating all PCEs in the Supercluster, see Restore an Entire Supercluster. If you are migrating just one PCE in a Supercluster, see Restore a Single PCE in a Supercluster.

Pre-configure New IP Addresses

Before the migration: When you use DNS-based load balancing (such as round-robin DNS) and are using new IP addresses for the restored PCE, be sure to record those new IP addresses in the runtime_env.yml file on all Supercluster core nodes. This allows VENs to continue to communicate with the PCEs after migration.

Note

When you use traffic-based load balancing, such as with the F5, you do not need to add the new IP addresses to runtime_env.yml. The VENs communicate exclusively with the traffic load balancers' virtual IP addresses, not directly with the PCEs.

To update runtime_env.yml with additional IP addresses:

If you are migrating more than one PCE, repeat the following steps for each PCE one at a time.

On all nodes of the PCE being migrated, edit the runtime_env.yml file. Under the cluster_public_ips.cluster_fqdn parameter, add the new IP addresses of all of the core nodes in this PCE:

cluster_public_ips:
  cluster_fqdn:
    - <old IP address>
    - <old IP address>
    - <new IP address>
    - <new IP address>
  cluster_event_service_fqdn:
    - <old IP address>
    - <old IP address>
    - <new IP address>
    - <new IP address>

Restart the PCE that was migrated to send the configuration update to the other PCEs:
```
sudo -u ilo-pce illumio-pce-ctl restart
```
Wait for all services to start on the restarted PCE.
```
sudo -u ilo-pce illumio-pce-ctl cluster-status
```
The command output should show RUNNING.
Restart the other PCEs in the Supercluster using the same commands in steps 2 and 3.
If you are migrating multiple PCEs in the Supercluster, repeat these steps for each PCE one at a time.

Illumio Install, Configure, and Upgrade Guide 24.2.20