Creating a new forum topic here. I had this question from a reader/viewer:
The question I have is what would be the next step to sharing this "storage pool" to windows (guessing via SMB).
Does Proxmox have native support for ceph fs , does that allow for a large smb share?
My end goal is to have an "elastic" RAID pool for personal plex media storage.
@wikemanuel welcome to the forums! Ok, so it looks like using CephFS you can share out a directory to clients.
Take note of these official forum responses from Proxmox:
This first post makes it sound like Proxmox recommends simply creating a VM on top of your Ceph storage as normal and create your SMB/Samba share from the VM. This route would probably eliminate some complexities of exposing it directly from Ceph:
https://forum.proxmox.com/threads/cephfs-usage-as-samba-or-something.81373/
This looks to be a walkthrough of creating Ceph storage to expose the storage directly:
https://forum.proxmox.com/threads/shared-storage-for-vms-on-cephfs.109494/
I would probably venture to say that it might be best to take the VM route as you could then just backup your VM with your attached storage using Proxmox Backup server to make sure everything is safe. Otherwise, you could probably rsync the directory from CephFS to a storage location outside of Proxmox.
Let me know if this makes sense?
@brandon-lee thank you for the two links. The first option of creating a VM on top of Ceph and subsequent pool from inside the VM seems easiest. The storage pool I am trying to create spans 3 servers and a total of 168TB of disk storage. I may be out of my knowledge realm when saying this, but, it is my understanding that there is a "drive size limitation" in the VM environment when carving out storage (something like 2TB per volume). The goal is to have a single mountpoint for all 168TB and not several drives. Is there a particular type of VM/share combo I should look at using?
Ok so there are several layers to think about here. First is the layer assigned to Proxmox. It looks like Proxmox has no issues with very large volumes:
According to this post you can assign these very large volumes to a VMs:
https://forum.proxmox.com/threads/maximun-size-of-volume-or-disk-that-can-be-assign-to-proxmox.7300/
For 32-bit CPUs on 2.6 kernels, the maximum LV size is 16TB.
For 64-bit CPUs on 2.6 kernels, the maximum LV size is 8EB. (Yes, that is a very large number.)
For 2.4 based kernels, the maximum LV size is 2TB.
Also, on the operating system side, you would need to consider the configuration there as well. It looks like in Windows 2019, NTFS can support an 8 PB volume with 2048 KB cluster size. Note the documentation here:
https://learn.microsoft.com/en-us/windows-server/storage/file-server/ntfs-overview
Also, here is a pretty cool discussion from 45 drives on really large Ceph storage:
