Enhanced vMotion Compatibility VM Migration in VMware Environments

VMware has one of the coolest technologies in something called Enhanced vMotion Compatibility. It helps ensure that you can migrate virtual machines across different hosts with differen CPU generation configurations. With the enhanced vMotion compatibility feature, organizations can keep their CPU features consistent across all hosts in the cluster, even with dissimilar hosts. Let’s look at how to enable it and how it affects CPU compatibility and cluster management.

What is Enhanced vMotion Compatibility (EVC)?

First of all, what is enhanced vMotion compatibility? The enhanced vMotion compatibility, or EVC mode, allows you to keep a standard set of CPU features across ESXi hosts in a cluster. When CPUs are different between hosts in the cluster, the differences are masked so the VMs only see a consistent CPU feature set. With this, VMs can freely migrate across hosts without compatibility issues.

How does the technology work? It works by enabling a “baseline feature set”, or EVC baseline, for the hosts that exist in the cluster so these have identical CPU features. How does it know what to enable? The features are selected based on the CPU features available with the hosts in the cluster.

With this, it allows VMs to run without running into CPU compatibility issues. So, this comes in extremely handy when you add newer hosts to an EVC cluster in vCenter Server inventory that may have potentially different CPU generation configurations from slightly older hosts.

Advanced Features

There are a few advanced features to know about with EVC:

• Automatic EVC Mode Selection: Modern versions of VMware automatically select the best EVC mode based on the CPU features of the hosts
• Enhanced CPU Feature Masking: Each new vSphere release has better masking techniques to make sure that CPU features are supported across different generations and vendors.
• Future releases: The EVC enhancements are constantly evolving to including new features and capabilities

Does it allow you to have a mix of AMD and Intel CPUs in a cluster?

The short answer is No. When you enable EVC, it only allows CPUs from a single CPU vendor to be added to the cluster. vCenter Server will not allow you to add Intel hosts and AMD hosts into an EVC-enabled cluster.

For example, I created a VMware vSphere cluster with one Intel ESXi host and one AMD ESXi host. When attempting to enable generation EVC mode with a cluster with those hosts, you will see the following:

“The host’s CPU hardware does not support the cluster’s current Enhanced vMotion Compatibility mode. The host CPU lacks features required by that mode.”

Note the official VMware by Broadcom FAQ page here that details this limitation: VMware EVC and CPU Compatibility FAQ (broadcom.com).

Understanding EVC Modes and Baselines

It is important to understand the different modes that are enabled with EVC in a cluster. These are based on the capabilities of both Intel and AMD processors.

• Intel EVC Mode: Designed for Intel CPUs. It includes options for different generations, such as Sandy Bridge, Ivy Bridge, Haswell, etc.
• AMD EVC Mode: Designed for AMD processors for different CPU generations like Opteron, Bulldozer, Piledriver, and Zen 2, 3, etc.

Choosing one of the modes means you are choosing a baseline feature set to match the capabilities of the oldest CPU generation in the cluster.

Homework to do ahead of time

There are a few things you will want to do ahead of time before enabling EVC on the cluster. Note the following:

1. Check CPU Compatibility: Check the generation and family of CPU on your ESXi hosts. Make sure the family of CPU and generation is supported in the desired EVC mode.
2. Update Firmware: It is a good idea to upgrade your firmware and BIOS on the ESXi hosts

Enabling EVC in Your VMware Environment

Let’s look at the actual process of enabling EVC in your vSphere cluster. Note the following steps:

Access the vSphere Client: Log in to your vCenter Server using the vSphere Client.

Select the Cluster: Navigate to the cluster where you want to enable EVC. You can click on the cluster you want and navigate to Configure, or you can right-click the cluster as shown below and choose Settings.

Configure EVC Mode: Right-click the vSphere cluster and click Settings.

This will land you on the Configure tab, then select VMware EVC under the Configuration section.

Select EVC Mode: Here we choose EVC mode based on the CPU generation of the host(s) in the cluster. Note that you have the option for Enable EVC for AMD hosts or Enable EVC for Intel hosts. This is another indicator that you cannot enable EVC for clusters containing both Intel and AMD processors.

Enable EVC: Click the option for the type of processor in your ESXi hosts. This step locks in the CPU features for the VMs.

Note the error below. It is due to the fact that the CPU chosen in the CPU mode dropdown contains features that are not supported in the host’s current CPU generation.

If we select a family of processor that is supported, the validation will succeed:

Best practices with Enhanced vMotion compatibility

There are a few best practices to keep in mind when enabling EVC on a cluster:

• Consistent CPU Features: Make sure you have chosen the appropriate EVC mode for current hosts and any future hosts that you might bring into the cluster
• Adding New Hosts: Make sure any new hosts support the EVC mode. Be sure again to update the firmware on a host and BIOS to make sure you have the latest updates installed before adding to EVC cluster
• Older Hosts: Old hosts in the cluster should also support the selected EVC mode. If they don’t, you may have to upgrade CPUs or replace them with newer servers.

These are just a few of the recommended best practices. As always, consult the official documentation from VMware on the latest best practices with EVC as these will likely change over time.

Migrating Virtual Machines with EVC

The process of migrating a virtual machine in an EVC cluster is no different than the process of migrating virtual machines in a non-EVC cluster.

The virtual machines themselves are not aware of the difference in CPUs due to EVC. So, they will simply assume the CPU instruction sets displayed on each host which should be consistent with the host they are currently on.

Wrapping up

Enhanced vMotion compatibility is one of those features that just makes life much easier as an administrator working with multiple ESXi hosts in a vSphere cluster. It also helps to lower costs when considering that you can make better use of the available hardware that may not be exactly the same for use in your vSphere cluster. The vMotion feature is a core technology to the cluster working as expected and giving you all the benefits of vSphere HA, vSphere DRS, and other nice features.

Keep in mind that you can’t mix different CPU vendors, like Intel and AMD processors in the same cluster with EVC enabled. It would be a nice feature in the future if this will some how be possible, but as of now, this is not something we can do, even with EVC.