How to Enable Proxmox Nested Virtualization
Quick Summary
- Note the following error message displayed when trying to install a nested Proxmox hypervisor as a VM on a Proxmox physical host.
- Nested virtualization allows running a hypervisor, like Proxmox VE (PVE) or others like VMware ESXi or Hyper-V, inside a virtual machine (VM), which itself runs on another hypervisor.
- After enabling nested virtualization on the Proxmox server, you must set up the virtual machines to utilize this feature when you create VM instances.
You may or may not have heard about nested virtualization. However, nested virtualization is a great way to set up virtualization labs and other learning environments. Nested virtualization in Proxmox is fairly easy to configure. With the Proxmox hypervisor, you can run nested hypervisors inside virtual machines. Let’s see what nested virtualization is and how to set this up in Proxmox.
Table of contents
What is nested virtualization?
Nested virtualization allows running a hypervisor, like Proxmox VE (PVE) or others like VMware ESXi or Hyper-V, inside a virtual machine (VM), which itself runs on another hypervisor. With nested virtualization configured, a host hypervisor is hosting a guest hypervisor (as a VM). Then, the “nested” hypervisor is capable of hosting its own VMs. It allows utilizing the hardware acceleration from the host system and virtualization technology, enabling a VM within another VM​​​.
Why do it?
You might wonder why you would want to run nested virtualization in Proxmox. Well, nested virtualization is a great way to learn. There may be other production use cases. However, with nested virtualization, you can set up one physical Proxmox host in the home lab and then run a cluster of Proxmox hosts as VMs.
By configuring nested virtualization, you can treat the VM hypervisor like any other VM. It means you can use things like snapshots to capture the state of the VM. Then, you can roll it back like any other VM in your environment.
Using snapshots on these nested virtualization VMs allows playing around with and learning new skills, without worrying about breaking things.
Checking Processor Compatibility and Current Settings
First, determine if your Proxmox host’s processor supports nested virtualization. For Intel CPU, use the command SSH into your Proxmox host as root and run the following:
cat /sys/module/kvm_intel/parameters/nested
For AMD CPU, use:
cat /sys/module/kvm_amd/parameters/nestedThe output will be either “N” or “0” (not enabled) or “Y” or “1” (enabled)​​​​​​.
Additionally, check the processor model using:
lscpu | egrep --color -i "Vendor ID|Model name"​​Intel CPUs will look like the following:
AMD CPUs will look like the following:
Enabling Nested Virtualization on Proxmox Host
You can activate it using specific commands if nested virtualization is not enabled. For Intel CPUs, use the commands:
sudo modprobe -r kvm_intel sudo modprobe kvm_intel nested=1 echo "options kvm-intel nested=Y" > /etc/modprobe.d/kvm-intel.conf
## Reload the kernel module
modprobe -r kvm_intel modprobe kvm_intelFor AMD Proxmox host, the commands are slightly different:
sudo modprobe -r kvm_amd sudo modprobe kvm_amd nested=1 echo "options kvm-amd nested=1" > /etc/modprobe.d/kvm-amd.conf
## Reload the kernel module
modprobe -r kvm_amd modprobe kvm_amdThese commands effectively enable the nested virtualization feature for your CPU type​​​​​​​.
What happens nested virtualization isn’t enabled?
What will you see if there is no support for nested virtualization? Note the following error message displayed when trying to install a nested Proxmox hypervisor as a VM on a Proxmox physical host:
Configuring Nested Virtualization in VMs
After enabling nested virtualization on the Proxmox server, you must set up the virtual machines to utilize this feature when you create VM instances.
Turn off the guest VM, then use the command to set the CPU type of the VM to “host”​​​​:
qm set <VMID> --cpu host 
For AMD processors, you’ll need to add additional parameters to the VM configuration file:
args: -cpu host,+svm​​.Validating the Configuration
Finally, restart the VMs and validate the configuration. Check if VT-X is enabled in the VMs, which is necessary to host guest machines. The following command will help you verify if VT-X is enabled​​​​​​:
egrep --color -i "svm|vmx" /proc/cpuinfo
FAQs on Proxmox Nested Virtualization
Nested virtualization introduces a unique layer by allowing a virtual machine to function as a host for other VMs. This contrasts with standard virtualization, where VMs are directly managed by the physical host system. It’s an advanced feature primarily used for development and testing environments.
Yes, Proxmox supports nested virtualization on both Intel and AMD CPUs. However, the commands to enable this feature differ slightly. For Intel, use the kvm_intel module, and for AMD, use kvm_amd. The key is to adjust the kernel module settings to enable the nested feature.
While some configurations can be managed through the Proxmox web interface, enabling nested virtualization involves command-line operations. Commands like modprobe and editing files in /etc/modprobe.d/ are essential steps that are performed in the CLI environment.
Ensure that your Proxmox host’s CPU supports virtualization and that the nested virtualization feature is not already enabled. Use lscpu to check your CPU type and cat /sys/module/kvm_{intel/amd}/parameters/nested to verify if nested virtualization is enabled.
After setting up nested virtualization, use the command egrep –color -i “svm|vmx” /proc/cpuinfo within your VM. If the output includes “vmx” or “svm,” it indicates that Intel VT-X or AMD-V, respectively, is enabled, allowing your VM to host other VMs.
While it’s not always necessary to restart the entire Proxmox host, reloading the relevant kernel modules is crucial. Commands like modprobe -r kvm_intel and modprobe kvm_intel for Intel (and similar for AMD) are used to refresh the settings without needing a full system reboot.
Nested virtualization can create additional overhead, as it adds another layer of virtualization. The impact largely depends on the host system’s resources and the complexity of the nested environments. Proper resource allocation and monitoring are key to maintaining optimal performance.
While technically feasible, nested virtualization is typically used in testing, development, or educational scenarios. For production environments, direct virtualization on the physical host is preferred for better performance and stability.
It can be. When you nest hypervisors as VMs, it creates another layer to understand with your network configuration.
Wrapping up
Nested virtualization in Proxmox is a great feature to take advantage of. With nested virtualization, you can run nested Proxmox VE servers, or even other hypervisors. It allows creating labs and other learning environments that can take advantage of a snapshot or even backups from Proxmox Backup Server as an example.
