In a recent customer deployment I came across an interesting issue when configuration a new interface on the vSphere Replication Appliance.
Since I last worked with the appliance the administration interface has been updated, it’s a good thing right, we now have a nice crisp easy to work with Clarity theme.
However, previously when adding another interface to the vSphere Replication appliance, the only required configuration parameters were an IP Address and Subnet mask (assuming the assignment of a static IP address). As reflected in the below screenshot.
The Clarity UI though, this requires and insists on a Gateway being set for the interface as well.
The GUI even states that only a single Gateway is supported.
Once the interface has been configured, the route table on the appliance is updated and the new default gateway on what should be the replication traffic interface now has a higher weighting in terms of the metric than the management interface.
Meaning the default gateway for the appliance is now on eth1 and not eth0, where it needs to be.
By happenstance, I discovered that if eth1 is connected to a network with DHCP running on there, this is given a lower weighting in the route table as shown below.
Based on this I then manually configured the metric on the interface, so this had a lower weighting than that of eth0, matching the same value as what was assigned when the interface was given a DHCP address.
Then after restarting systemd-networkd or a reboot of the appliance the route on eth0 has a higher priority.
While this worked, any changes to the network interface via the administration UI, overwrote this value. While it is unlikely that this would happen, short of a change to the replication network I wanted to verify this was a supported method by VMware.
Following conversations with VMware and a review of the release notes for vSphere Replication, it was clear that this was a known issue with vSphere Replication 8.5.
When adding a dedicated interface for the replication traffic, documentation details the addition of a static route (assuming that the replication is routed and not a stretched network on the physical fabric). https://docs.vmware.com/en/vSphere-Replication/8.5/com.vmware.vsphere.replication-admin.doc/GUID-7235331C-2377-44B1-9BF2-086D9AE14702.html#GUID-7235331C-2377-44B1-9BF2-086D9AE14702
Continue following the documentation, and add a static route to the interface configuration.
The route table looks much better, with only the one default route.
Then as per the known issues within the release notes, the following error will be present on the summary page of the administration interface.
As per the documentation the workaround is as follows: the NIC is correctly configured and you can discard the error. You can verify if the replication traffic uses the correct IP address by running the following command: cat /etc/vmware/hbrsrv-nic.xml
