home lab

How to Enable Proxmox Nested Virtualization

Discover how to enable Proxmox Nested Virtualization to help configure virtualization labs & other learning environments

You may or may not have heard about nested virtualization. However, nested virtualization is a great way to set up virtualization labs and other learning environments. Nested virtualization in Proxmox is fairly easy to configure. With the Proxmox hypervisor, you can run nested hypervisors inside virtual machines. Let’s see what nested virtualization is and how to set this up in Proxmox.

What is nested virtualization?

Nested virtualization allows running a hypervisor, like Proxmox VE (PVE) or others like VMware ESXi or Hyper-V, inside a virtual machine (VM), which itself runs on another hypervisor. With nested virtualization configured, a host hypervisor is hosting a guest hypervisor (as a VM). Then, the “nested” hypervisor is capable of hosting its own VMs. It allows utilizing the hardware acceleration from the host system and virtualization technology, enabling a VM within another VM​​​.

Why do it?

You might wonder why you would want to run nested virtualization in Proxmox. Well, nested virtualization is a great way to learn. There may be other production use cases. However, with nested virtualization, you can set up one physical Proxmox host in the home lab and then run a cluster of Proxmox hosts as VMs.

By configuring nested virtualization, you can treat the VM hypervisor like any other VM. It means you can use things like snapshots to capture the state of the VM. Then, you can roll it back like any other VM in your environment.

Using snapshots on these nested virtualization VMs allows playing around with and learning new skills, without worrying about breaking things.

Checking Processor Compatibility and Current Settings

First, determine if your Proxmox host’s processor supports nested virtualization. For Intel CPU, use the command SSH into your Proxmox host as root and run the following:

cat /sys/module/kvm_intel/parameters/nested
Checking intel processor compatibility with proxmox nested virtualization
Checking intel processor compatibility with proxmox nested virtualization

For AMD CPU, use:

cat /sys/module/kvm_amd/parameters/nested

The output will be either “N” or “0” (not enabled) or “Y” or “1” (enabled)​​​​​​.

Checking amd processor compatibility for proxmox nested virtualization
Checking amd processor compatibility for proxmox nested virtualization

Additionally, check the processor model using:

lscpu | egrep --color -i "Vendor ID|Model name"​​

Intel CPUs will look like the following:

Checking processor type when intel
Checking processor type when intel

AMD CPUs will look like the following:

Checking your processor type when amd
Checking your processor type when amd

Enabling Nested Virtualization on Proxmox Host

You can activate it using specific commands if nested virtualization is not enabled. For Intel CPUs, use the commands:

sudo modprobe -r kvm_intel sudo modprobe kvm_intel nested=1 echo "options kvm-intel nested=Y" > /etc/modprobe.d/kvm-intel.conf 

## Reload the kernel module
modprobe -r kvm_intel modprobe kvm_intel

For AMD Proxmox host, the commands are slightly different:

sudo modprobe -r kvm_amd sudo modprobe kvm_amd nested=1 echo "options kvm-amd nested=1" > /etc/modprobe.d/kvm-amd.conf 

## Reload the kernel module
modprobe -r kvm_amd modprobe kvm_amd

These commands effectively enable the nested virtualization feature for your CPU type​​​​​​​.

What happens nested virtualization isn’t enabled?

What will you see if there is no support for nested virtualization? Note the following error message displayed when trying to install a nested Proxmox hypervisor as a VM on a Proxmox physical host:

Error when nested virtualization is not enabled in proxmox
Error when nested virtualization is not enabled in proxmox

Configuring Nested Virtualization in VMs

After enabling nested virtualization on the Proxmox server, you must set up the virtual machines to utilize this feature when you create VM instances.

Creating a proxmox virtual machine with proxmox installed
Creating a proxmox virtual machine with proxmox installed

Turn off the guest VM, then use the command to set the CPU type of the VM to “host”​​​​:

qm set <VMID> --cpu host 
Setting the nested vm cpu to use the host cpu
Setting the nested vm cpu to use the host cpu

For AMD processors, you’ll need to add additional parameters to the VM configuration file:

args: -cpu host,+svm​​.

Validating the Configuration

Finally, restart the VMs and validate the configuration. Check if VT-X is enabled in the VMs, which is necessary to host guest machines. The following command will help you verify if VT-X is enabled​​​​​​:

egrep --color -i "svm|vmx" /proc/cpuinfo
Verifying the nested cpu instructions on a nested proxmox hypervisor
Verifying the nested cpu instructions on a nested proxmox hypervisor

FAQs on Proxmox Nested Virtualization

How Does Nested Virtualization Differ from Regular VM Setup?

Nested virtualization introduces a unique layer by allowing a virtual machine to function as a host for other VMs. This contrasts with standard virtualization, where VMs are directly managed by the physical host system. It’s an advanced feature primarily used for development and testing environments.

Can I Use Both Intel and AMD CPUs for Nested Virtualization in Proxmox?

Yes, Proxmox supports nested virtualization on both Intel and AMD CPUs. However, the commands to enable this feature differ slightly. For Intel, use the kvm_intel module, and for AMD, use kvm_amd. The key is to adjust the kernel module settings to enable the nested feature.

Is It Possible to Enable Nested Virtualization from the Proxmox Web UI?

While some configurations can be managed through the Proxmox web interface, enabling nested virtualization involves command-line operations. Commands like modprobe and editing files in /etc/modprobe.d/ are essential steps that are performed in the CLI environment.

What Should I Check Before Enabling Nested Virtualization?

Ensure that your Proxmox host’s CPU supports virtualization and that the nested virtualization feature is not already enabled. Use lscpu to check your CPU type and cat /sys/module/kvm_{intel/amd}/parameters/nested to verify if nested virtualization is enabled.

How Do I Know if VT-X/AMD-V is Enabled in My VMs?

After setting up nested virtualization, use the command egrep –color -i “svm|vmx” /proc/cpuinfo within your VM. If the output includes “vmx” or “svm,” it indicates that Intel VT-X or AMD-V, respectively, is enabled, allowing your VM to host other VMs.

Do I Need to Restart My Proxmox Host After Configuring Nested Virtualization?

While it’s not always necessary to restart the entire Proxmox host, reloading the relevant kernel modules is crucial. Commands like modprobe -r kvm_intel and modprobe kvm_intel for Intel (and similar for AMD) are used to refresh the settings without needing a full system reboot.

Can Nested Virtualization Impact Proxmox Performance?

Nested virtualization can create additional overhead, as it adds another layer of virtualization. The impact largely depends on the host system’s resources and the complexity of the nested environments. Proper resource allocation and monitoring are key to maintaining optimal performance.

Is Nested Virtualization Suitable for Production Environments?

While technically feasible, nested virtualization is typically used in testing, development, or educational scenarios. For production environments, direct virtualization on the physical host is preferred for better performance and stability.

Is networking harder in nested environments?

It can be. When you nest hypervisors as VMs, it creates another layer to understand with your network configuration.

Wrapping up

Nested virtualization in Proxmox is a great feature to take advantage of. With nested virtualization, you can run nested Proxmox VE servers, or even other hypervisors. It allows creating labs and other learning environments that can take advantage of a snapshot or even backups from Proxmox Backup Server as an example.

Subscribe to VirtualizationHowto via Email 🔔

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Brandon Lee

Brandon Lee is the Senior Writer, Engineer and owner at Virtualizationhowto.com and has over two decades of experience in Information Technology. Having worked for numerous Fortune 500 companies as well as in various industries, Brandon has extensive experience in various IT segments and is a strong advocate for open source technologies. Brandon holds many industry certifications, loves the outdoors and spending time with family.

Related Articles

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.