Hey Brandon,
Really appreciate the two suggestions. 🙂
For the first part: Yes, both systems are Linux based.
I’ll check for the root_squash setting. I wasn’t aware of this.
Permission wise, both mount point and share on each systems were owned by the same uid and gid. Also for some reason, it seemed to check for the name of the user rather than the uid, because once I set on the NAS root as the owner of the share, docker seemed to be happier. But I reverted back as I did not wanted root owning the share.
I had try to find the daemon to give it the proper uid, but did not realize it could be done by modifying docker.service. Still learning Docker, I used to run everything directly on dedicated VMs.
However, I like your second suggestion better as it’s way easier indeed, and it will do what I wanted, which is to have the data on the NAS and not on the VMs directly.
I’ll reset the Docker VM to start clean and will just map the share and bind mount it to containers.
Going that route, does the docker daemon still need to have the same uid as the one owning the share?
@wuruwhi Awesome! Yeah i would probably go the route of the second option as it keeps Docker with what the defaults are for most things and that way you can pick and choose what you want to store on your NFS storage. So far I have not had any issues with my data for critical containers using this approach. I have had to restore files a few times and it has worked as expected 👍
@brandon-lee Great to hear that! 🙂
Does the Docker daemon has to run with the same uid as the one owning the share?
@wuruwhi Great question! To clarify, the Docker daemon does not need to run under the same user that owns the NFS directory. Here's how it will work:
The Docker daemon runs as root user or another user like "docker". This user needs to have permissions to access and manage the data root directory or any bind-mounted directories. But, it does not need to match the uid/gid of the user owning the NFS share.
Your mount of the volume (NFS client) will receive access based on what permissions it has been configured with on the server side.
If the NFS share is mounted with the right user the uid/gid and permissions, your Docker containers and the daemon will be able to access it as expected.
So for instance:
sudo mount -t nfs -o rw,uid=1234,gid=1234 nfs-server:/share /mnt/nfs
Then you mount the storage as a bind mount in your container:
docker run -v /mnt/nfs/specific-dir:/app/data my-container
Hopefully this helps 😀
It certainly does. 🙂
I will try this out during the week if I can.
Hey,
So I did some more playing around over the week-end. I started with a clean minimized install of Ubuntu Server and went on to install the necessary components and setting up fstab for the NFS share and bindfs option as well.
It mounted properly and creating files or directory showed that it worked as expected.
I deployed a couple containers and mapped them to the bindfs directory and it deployed and ran as it should. However after a restart of the host, the bind wouldn’t mount anymore. It threw an error saying the directories weren’t empty.
I found that weird.
So I tried the other avenue (which so far still works) of mounting the NFS share in fstab and then in the compose files, modifying the volumes to the mount point location.
The only downside, is that I have to manually create the directories before deploying a container.
Deploying directly fails for directory creation permissions, however, rerunning compose up creates the directories the second time around. Even though I did not adjust anything.
@wuruwhi Thank you for keeping things posted on progress. This is actually what I do with my Ceph storage I am using. I create the directory prior and then create the bind mount path needed for mounting the directories to the folders in Ceph. So this behavior I am familiar with. I would actually like to get to the bottom of this myself as well but haven't chased this down as of yet. I have a suspicion that Docker changes permissions perhaps on the first go around that allows the creation of the directory when it attempts the 2nd time.