0

Background

For a NFSv4 export/share the by default enabled root_squash option will force NFS to change the client’s root to an anonymous ID. This will, in effect, increase security by preventing ownership of the root account on one system migrating to the other system.

Overlayfs allows for a local, transparent, filesystem to be mounted on top of another filesystem. Unfortunately it seems like it accesses the underlying filesystem using the root user, and not the actual user. At least that is the conclusion I draw from the following experiment.

Why mount an Overlayfs over a NFS share? To allow a less trusted machine the ability to pretend it can write to the shared filesystem.

Test setup

First install the NFS kernel server as appropriate for your distro. Then make sure you export only NFSv4. (Though probably not important for this issue, a good security precausion.)

$ sudo cat /proc/fs/nfsd/versions
-2 -3 +4 +4.1 +4.2

If not, have a look at /etc/nfs.conf and set vers3=n.

Then create en Ext4 filesystem in a sparse file and mount it on the local filesystem. This will be the filesystem underpinning our NFS share.

$ truncate -s 512M 512BM-ext4.img
$ mkfs.ext4 512BM-ext4.img
$ sudo mkdir /mnt/ext4-file
$ sudo mount -o loop,noacl 512BM-ext4.img /mnt/ext4-file

Then share/export this filesystem over the network with NFSv4 to an appropriate machine. In this example I will use localhost, but it could be any machine on the local network. Do this by editing /etc/exports to have a line like the following.

/mnt/ext4-file/ localhost(ro,fsid=123123)

Then restart the NFS server on your machine, the service files could be different on your OS.

$ sudo systemctl restart nfs-server.service nfs-mountd.service
$ sudo exportfs -v
/mnt/ext4-file  localhost(sync,wdelay,hide,no_subtree_check,fsid=123123,sec=sys,ro,secure,root_squash,no_all_squash)

Ensure that root_squash and ro is enabled.

It should now be possible to mount the NFSv4 share on your intended client.

$ sudo mount -t nfs -o ro localhost:/mnt/ext4-file /mnt/nfs-share/
$ findmnt /mnt/nfs-share 
TARGET         SOURCE                   FSTYPE OPTIONS
/mnt/nfs-share localhost:/mnt/ext4-file nfs4   rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255

And then we mount overlayfs2 on top of this share.

$ mkdir -p /tmp/overlay/{work,upper}
$ sudo mkdir /mnt/overlay
$ sudo mount -t overlay overlay -o lowerdir=/mnt/nfs-share/,upperdir=/tmp/overlay/upper/,workdir=/tmp/overlay/work/ /mnt/overlay/

For our experiment, we will create a folder in the filesystem that can only be read and written to by a user and read by a group. I chose the group belonging to my user.

$ sudo mkdir /mnt/ext4-file/rovanion
$ sudo chown rovanion:rovanion /mnt/ext4-file/rovanion
$ sudo chmod 2750 /mnt/ext4-file/rovanion
$ touch /mnt/ext4-file/rovanion/hi-from-ext4
$ ls -la /mnt/nfs-share/rovanion/
totalt 8,0K
drwxr-s--- 2 rovanion rovanion 4,0K sep  6 14:14 .
drwxr-xr-x 4 root     root     4,0K sep  6 14:12 ..
-rw-rw-r-- 1 rovanion rovanion    0 sep  6 14:14 hi-from-ext4
$ touch /mnt/nfs-share/rovanion/hi-from-nfs
touch: cannot touch '/mnt/nfs-share/rovanion/hi-from-nfs': Read-only file system

We can list the contents of /mnt/nfs-share/rovanion, but are unable to touch the filesystem even though we have permission to, because the NFS share is mounted as read only. All is as expected.

Failure

But here comes the problem.

$ ls -la /mnt/overlay/rovanion/
ls: cannot open directory '/mnt/overlay/rovanion/': Permission denied
$ ls -l /mnt/overlay/
total 28K
drwx------ 2 root     root      16K Sep  6 13:04 lost+found
drwxr-s--- 2 rovanion rovanion 4.0K Sep  6 14:14 rovanion
$ whoami
rovanion
$ groups
rovanion sudo

We are denied access to list /mnt/overlay/rovanion even though the permissions system should allow us to do so.

My best guess as to what is happening is that Overlayfs does all access to the underlying filesystem as root which by NFS' root squash gets mapped to nobody who is not allowed access to the folder, since nobody does not belong to the group rovanion and others are not allowed to list the folder.

Question

My question then is: Is it possible to work around this issue? To allow a user access to a folder through Overlayfs to which only a select group has access, without disabling root_squash on the NFS export/share or adding o+rx to the folder.

Rovanion
  • 659
  • 3
  • 7
  • 22

1 Answers1

0

The following section of the Linux Kernel documentation on Overlayfs describes the permission model.

Permission checking in the overlay filesystem follows these principles:

1) permission check SHOULD return the same result before and after copy up
2) task creating the overlay mount MUST NOT gain additional privileges
3) non-mounting task MAY gain additional privileges through the overlay, compared to direct access on underlying lower or upper filesystems

This is achieved by performing two permission checks on each access

a) check if current task is allowed access based on local DAC (owner, group, mode and posix acl), as well as MAC checks
b) check if mounting task would be allowed real operation on lower or upper layer based on underlying filesystem permissions, again including MAC checks

Check (a) ensures consistency (1) since owner, group, mode and posix acls are copied up. On the other hand it can result in server enforced permissions (used by NFS, for example) being ignored (3).

Check (b) ensures that no task gains permissions to underlying layers that the mounting task does not have (2). This also means that it is possible to create setups where the consistency rule (1) does not hold; normally, however, the mounting task will have sufficient privileges to perform all operations.

Emphasis mine. My read is that I have managed to create such a situation. One where the mounting task, running as root, does not have the privilege to list the folder and thus Overlayfs disallows the user access to the folder.

If we were to mount the Overlayfs as a user allowed to access the files, that is not root so that the UID does not get squashed, then perhaps we can list the directory and create a file in it?

$ unshare --mount --map-root-user
# mount -t overlay overlay -o lowerdir=/mnt/nfs-share/,upperdir=/tmp/overlay/upper/,workdir=/tmp/overlay/work/ /mnt/overlay/
# ls -la /mnt/overlay/
totalt 28K
drwxrwxr-x 1 root   root    4,0K sep  6 14:08 .
drwxr-xr-x 7 nobody nogroup 4,0K sep  6 14:09 ..
drwx------ 2 nobody nogroup  16K sep  6 13:04 lost+found
drwxr-s--- 2 root   root    4,0K sep  6 14:14 rovanion
# touch /mnt/overlay/rovanion/hi-from-overlay

And that we can! And the file exists only in the overlay.

$ ls /mnt/ext4-file/rovanion/
hi-from-ext4
$ ls /tmp/overlay/upper/rovanion/
hi-from-overlay

Though this solution comes with its own interesting implications. Now we must have user namespaces enabled on our not 100% trusted machines and we suddenly exist in a space where our UID appears to be 0 but is mapped to our normal user ID for the outside world. Unfortunately unshare --mount without --map-root-user does not seem possible.

Rovanion
  • 659
  • 3
  • 7
  • 22