vSAN architecture recommendations

@macolm-r vSAN 2-node is great. I ran this a long while before I went to my 3-node cluster I am running currently. The witness node is best run as a virtual appliance which is the recommended way. Also, one neat thing about the witness node is it is the only supported "nested" version of ESXi. The Witness appliance is just a nested ESXi installation that houses metadata. 

Another side point, VMware, at least before Broadcom, would charge you for the license if you run the witness on bare metal. But if you run the virtual appliance, the license for that is free.

Also, vSAN only works with JBOD. It handles the disk provisioning. So you just need to present the disks to vSAN as JBOD disks and form your disk groups. There are two architectures with vSAN now - OSA original storage architecture and ESA, enhanced storage architecture. I haven't had the chance to really play around with ESA to run my production workloads on as a while back it required 25 GbE network connectivity. 

Now with 8.0 Update 2 they have what they call AF-0 Ready Node Profile that allows 10 Gbe. I would probably just spin up the OSA architecture with the "disk groups" concept. You need at least 1 disk for cache and 1 disk for capacity. You can only have 1 cache device per disk group, but multiple disks in your capacity tier. 

@malcolm-r yeah cool thing with the witness host is it will service your whole environment. Also, good to know is they now allow one witness host to service multiple vSAN clusters as well, which is great!

@malcolm-r No worries on the questions! You can mix drives, but I would probably use your NVMe for cache and then use your SATA SSDs for your capacity tier and keep everything consistent. If you mix drives, you would see weird behaviors and performance differences across your hosts most likely. 

alright, so i'm trying to get this set up and the vSAN config isn't liking my setup. i'm 99% sure it's because with my setup (2 physical hosts) i just have everything in one cluster. i didn't really plan my vcenter architecture. i just kinda tossed everything into one pile.

i'm thinking i need to re-assess my layout.