Nutanix

Guest VM Running in Nested Nutanix CE on VMware vSphere Won’t Boot

I have been playing around quite a bit lately in the home lab with a nested installation of Nutanix Community Edition (CE) as it is a great way to get your feet wet with the Nutanix hypervisor for free and running in a nested configuration in a lab environment. I have already covered here how to get up and running quickly with Nutanix CE in a VMware vSphere 6.7 VM which is a fairly straightforward process. However, after getting the environment configured and everything working as expected, I found the guest VM running in Nested Nutanix CE on VMware vSphere won’t boot. What gives? Let’s take a look at the problem and the workaround that was able to get around this issue in the lab.

Nutanix CE Guest VM Not Booting in VMware vSphere

I noticed when I had created my first VM in the nested Nutanix CE installation and kicked off a boot of the VM, after waiting a few minutes, I didn’t see any network connections being made from the VM. A quick hop onto the console and sure enough there appeared to be an issue. The virtual machine was simply stuck on booting and simply displayed the BIOS screen with no other activity. Thinking this was a boot priority issue possibly, I spent some time trying to play around with the boot order and a few other tricks, however, nothing seemed to change this behavior. After quite a bit of time just tinkering myself, I decided to see if this was a known issue with running a nested installed of Nutanix CE or maybe in particular, running on top of VMware vSphere.

Below is a screen capture of the behavior seen. The boot of the Nutanix CE guest VM will simply stick on this screen forever.

Guest-VM-Running-in-Nested-Nutanix-CE-on-VMware-vSphere-Wont-Boot-1
Guest VM Running in Nested Nutanix CE on VMware vSphere Won’t Boot Workaround

After a bit of Googling, I stumbled onto this thread on one of the Nutanix Forums concerning issues booting nested VMs on Nutanix CE running on top of VMware vSphere. The good thing is there is a workaround if you want to run actual nested VMs inside of Nutanix CE and VMware vSphere.

Guest VM Running in Nested Nutanix CE on VMware vSphere Won’t Boot Workaround

Thankfully, the workaround to allow actually running VMs on top of the Nutanix CE AHV hypervisor only involves editing a couple of files, adding, modifying, and deleting a subset of lines from the configuration files.

First, edit the following file:

/home/install/phx_iso/phoenix/svm_template/kvm/default.xml

Create a backup of the default.xml file before making changes.

Creating-a-backup-of-the-default.xml-file-before-editing

Edit the default.xml file and underneath the features section, add the line:

<pmu state=’off’ />

Editing-the-Default.xml-file-to-add-a-configuration-line

The next file that needs to be updated is located here:

/var/cache/libvirt/qemu/capabilities/<your number>.xml

Again, first, backup your existing file before editing.

Backup-the-qemu-capabilities-XML-file
Creating a backup of the default.xml file before editing

Basically, in the qemu capabilities file, you are simply removing the existing pc-i440fx-rhel7.2.0 file and renaming the pc-i440fx-rhel7.3.0 to 7.2.0. My file looked like the following before making the changes.

Before-editing-the-qemu-capabilities-file-for-Nutanix-CE-nested-in-the-home-lab
Before editing the qemu capabilities file for Nutanix CE nested in the home lab

In the end, it will look like this.

After-updating-the-qemu-capabilities-file-in-Nutanix-CE

After making the changes to the two files needed, initiate a reboot of the Nutanix CVM and host. This is found under the Settings > Reboot menu in Prism.

Reboot-the-Nutanix-CE-CVM-and-host-after-making-the-file-changes
Reboot the Nutanix CE CVM and host after making the file changes

After editing the two files in question, rebooting the Nutanix environment, now the guest VMs running inside of the nested Nutanix CE environment boot correctly! The great thing about the above solution is that it takes effect for all VMs that are provisioned in the Nutanix environment.

Booting-Windows-10-1809-inside-a-guest-VM-running-in-Nutanix-CE-nested-in-VMware-vSphere
Booting Windows 10 1809 inside a guest VM running in Nutanix CE nested in VMware vSphere

Wrapping Up

If you are running into the issue of a Guest VM Running in Nested Nutanix CE on VMware vSphere Won’t Boot, try the simple workaround listed in the post. For me, this worked like a charm, and again, kudos to the Nutanix CE forum members for tossing out this solution. I wanted to create this post to help give even more visibility to this workaround in case you have hit this snag in playing around with your Nutanix CE environment with actually running Guest VMs inside Nutanix on vSphere or another hypervisor. I have only personally tested this workaround with VMware vSphere 6.7 Update 2 so I cannot vouch for this solving this issue across all hypervisor environments. More to come!

If you missed the first part of how to deploy nested Nutanix CE in VMware vSphere, check out that post here:

Subscribe to VirtualizationHowto via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Brandon Lee

Brandon Lee is the Senior Writer, Engineer and owner at Virtualizationhowto.com and has over two decades of experience in Information Technology. Having worked for numerous Fortune 500 companies as well as in various industries, Brandon has extensive experience in various IT segments and is a strong advocate for open source technologies. Brandon holds many industry certifications, loves the outdoors and spending time with family.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.