vSAN

Force Delete vSAN Datastore from ESXi Host

VMware vSAN provides an incredible technology that allows combining local storage from multiple ESXi hosts to form logical storage for both virtual machines and modern technologies such as containers. With VMware vSAN 7.0 Update 2, VMware has continued to push the envelope of what the technology can do with improvements across the board. In a home lab environment or even in production, you may find the need to evacuate an ESXi host and get rid of the vSAN partitions on the local disks. I ran into an issue the other day with a disk failure on a lab host that did not allow getting rid of the disk group, etc. I wanted to walk through how you can *easily* force delete vSAN datastore from ESXi host which allows either reclaiming the disks for traditional VMFS datastores or re-adding the host back to the vSAN cluster. again if you had issues with the disk group in general.

vSAN Disk Group architecture

VMware vSAN architecture has two tiers of storage – a cache tier and a capacity tier. The caching tier is used for read caching and write buffering. The capacity tier is persistent storage, housing your virtual machine objects. This layout offers tremendous performance advantages and still maintains a resilient and robust storage infrastructure for your VMs.

One of the logical components to understand with VMware vSAN architecture is the concept of the disk group. The disk group’s purpose is to manage the logical link between the capacity and cache tiers. Understanding the various components of the disk group helps to understand the way vSAN storage and accesses the data. Consider the following:

  • Every ESXi host in a vSAN-enabled vSphere cluster will contain at least 1 disk group
  • Disk groups have 1 cache device and up to 7 capacity drives.
  • vSAN hosts can have up to 5 disk groups with 7 capacity drives each which equals 35 capacity drives.
  • In either the all-flash or hybrid configurations, the cache device must be a flash disk
  • Hybrid configurations use the cache disk in a (70% read cache and 30% write cache allocation). In all-flash configurations, the cache disk is a dedicated 100% write buffer.
Architecture overview of the vsan disk group with cache drives
Architecture overview of the vsan disk group with cache drives

Below is a look at an ESXi host in a vSAN-enabled vSphere cluster and the Storage Devices installed.

Individual drives making up a vsan esxi host disk group
Individual drives making up a vsan esxi host disk group

To view the disk group configuration, you can navigate to the properties of the vSAN cluster > Configure > vSAN > Disk Management. Then, click on the disk group for the particular vSAN ESXi host you want to look at. The Disk Group properties does a good job of detailing and showing which disks are used for which purpose.

Viewing the disk group configuration in a vsan cluster
Viewing the disk group configuration in a vsan cluster

Delete vSAN Datastore from ESXi Host

You might think that it would be super easy to delete a vSAN datastore from an ESXi host. However, VMware has put safeguards in place that doesn’t make it super straightforward to just simply “right-click, delete” kind of operation. To demonstrate this. I have put a lab host into maintenance mode. I first went in and tried to erase partitions from the disks.

After placing a host into maintenance mode, I went to the host > configure > storage devices > erase partitions.

After placing a vsan host into maintenance mode and trying to erase vsan partitions
After placing a vsan host into maintenance mode and trying to erase vsan partitions

It asks to confirm the partitions erase.

Confirming to erase vsan partitions
Confirming to erase vsan partitions

Even in maintenance mode, the process errors out.

Error trying to erase vsan partitions even in maintenance mode
Error trying to erase vsan partitions even in maintenance mode

I didn’t grab screenshots of this, but trying to delete the disk group for the particular host shows the disk group/host is in use with a similar error to above. Let’s see if we can remove the host from the vSAN cluster if it is in maintenance mode.

Trying to remove the host from inventory does not work
Trying to remove the host from inventory does not work

This operation fails as well.

Removal from vsphere inventory fails
Removal from vsphere inventory fails

After rebooting the host, I was able to Remove from inventory.

You can only remove the host if it is not responding
You can only remove the host if it is not responding

Now, the vSAN ESXi host is removed from the cluster. I now only have the remaining two ESXi hosts in the cluster.

Host is successfully removed
Host is successfully removed

Now that we have the host removed from the vSAN cluster from vCenter, let’s see if we can simply erase the partitions on the vSAN disks. This is logged in directly to the ESXi host via the host UI. Click Storage > Actions > Clear partition table.

Attempting to clear partitions after host is removed from the host ui
Attempting to clear partitions after host is removed from the host ui

You will get the confirmation for erasing the partitions.

Confirm deleting the vsan partitions from the host ui
Confirm deleting the vsan partitions from the host ui

The operation fails to remove the partitions even in the host UI.

Failed to delete the partitions from the host ui
Failed to delete the partitions from the host ui

OK, let’s get serious and get down to the command line. Even though we removed the ESXi host from vCenter, it still thinks it is joined to the vSAN cluster. The first thing I did was use the command:

esxcli vsan cluster leave
esxcli vsan cluster get

This will remove the host from thinking it is part of the vSAN cluster. The get command simply validates it is no longer a part of a vSAN cluster.

Leaving the vsan cluster from the command line
Leaving the vsan cluster from the command line

However, after trying to delete the partitions after simply leaving the vSAN cluster through the esxcli command, erasing the partitions still fails. Let’s remove the storage from the vSAN host. To see which storage is a part of the vSAN allocated storage on the host, you can use:

esxcli vsan storage list

This shows the disks that are allocated to vSAN storage and the disk group they belong to.

Viewing storage allocated to vsan
Viewing storage allocated to vsan

Ok, you can remove vSAN storage by disk, ssd, or UUID. I chose to use the UUID of the disk group. Just use the command:

esxcli vsan storage remove --uuid
Removing storage allocated to vsan using the esxcli command line utility
Removing storage allocated to vsan using the esxcli command line utility

Once the storage is deleted, I tried to erase vSAN partitions. Success! The Force Delete vSAN Datastore from ESXi Host is successful. The partitions delete successfully.

Clearing the partitions on the esxi host is now successful
Clearing the partitions on the esxi host is now successful

Now, when looking at the storage from the host UI, there are no datastores appearing. Before the vsandatastore was showing.

No vsan datastores are now listed on the host
No vsan datastores are now listed on the host

Adding the ESXi host back to the vSAN Cluster

Adding a failed vSAN host back to the cluster after repairs

Wrapping Up

If you run into trouble trying to delete the vSAN datastore from an ESXi host that was part of a vSAN cluster, the ESXCLI command-line tool is your friend in this case. There are many safeguards in place that keep you from simply deleting a host from vSAN or deleting the disk group, or even erasing partitions on a vSAN claimed disk. However, using esxcli vsan storage remove, you can Force Delete vSAN Datastore from ESXi Host and reclaim the host for other purposes.

Subscribe to VirtualizationHowto via Email 🔔

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Brandon Lee

Brandon Lee is the Senior Writer, Engineer and owner at Virtualizationhowto.com and has over two decades of experience in Information Technology. Having worked for numerous Fortune 500 companies as well as in various industries, Brandon has extensive experience in various IT segments and is a strong advocate for open source technologies. Brandon holds many industry certifications, loves the outdoors and spending time with family.

Related Articles

2 Comments

  1. Thanks. Just removed a misconfigured VSAN datastore from a host without removing it from the cluster. Just in maintenance mode.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.