Re: [RFC PATCH 00/35] overlayfs: stack file operations
On Sat, May 5, 2018 at 6:37 PM, Amir Goldstein wrote: > On Fri, May 4, 2018 at 6:23 PM, Miklos Szeredi wrote: >> On Thu, Apr 12, 2018 at 5:07 PM, Miklos Szeredi wrote: >> >>> Git tree is here: >>> >>> git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs.git >>> overlayfs-rorw >> >> Thanks everyone for your review. >> >> Force pushed new version to the above branch. >> >> Hopefully no comment was missed (I didn't add more reverts yet, those >> can wait). I'll do a mailbomb as well next week and start processing >> the metacopy patches. >> > > Ran -g auto xfstests on overlay over xfs+reflink. > There are no failures and no changes w.r.t skipped tests since v4.17-rc1. > > ACK on w.r.t addressing my review comments. > > ACK on the freeze protection fix. > Not sure if the fact that we take __mnt_want_write() on overlay and not on > upper_mnt for ovl_real_ioctl() is a problem (i.e. can upper_mnt become > readonly from under overlay? and do we care?). I don't think we care. The mount part of the real path doesn't play a very important part because it's private to the overlay and it's never modified. It would be interesting to dig into the places it is used (may not be necessary in all cases), but that's for later. Thanks, Miklos
Re: [RFC PATCH 00/35] overlayfs: stack file operations
On Fri, May 4, 2018 at 6:23 PM, Miklos Szeredi wrote: > On Thu, Apr 12, 2018 at 5:07 PM, Miklos Szeredi wrote: > >> Git tree is here: >> >> git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs.git >> overlayfs-rorw > > Thanks everyone for your review. > > Force pushed new version to the above branch. > > Hopefully no comment was missed (I didn't add more reverts yet, those > can wait). I'll do a mailbomb as well next week and start processing > the metacopy patches. > Ran -g auto xfstests on overlay over xfs+reflink. There are no failures and no changes w.r.t skipped tests since v4.17-rc1. ACK on w.r.t addressing my review comments. ACK on the freeze protection fix. Not sure if the fact that we take __mnt_want_write() on overlay and not on upper_mnt for ovl_real_ioctl() is a problem (i.e. can upper_mnt become readonly from under overlay? and do we care?). Thanks, Amir.
Re: [RFC PATCH 00/35] overlayfs: stack file operations
On Thu, Apr 12, 2018 at 5:07 PM, Miklos Szeredi wrote: > Git tree is here: > > git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs.git > overlayfs-rorw Thanks everyone for your review. Force pushed new version to the above branch. Hopefully no comment was missed (I didn't add more reverts yet, those can wait). I'll do a mailbomb as well next week and start processing the metacopy patches. Thanks, Miklos --- Miklos Szeredi (38): vfs: dedpue: return s64 vfs: dedupe: rationalize args vfs: dedupe: extract helper for a single dedup vfs: add path_open() vfs: optionally don't account file in nr_files vfs: add f_op->pre_mmap() vfs: export vfs_ioctl() to modules vfs: export vfs_dedupe_file_range_one() to modules ovl: copy up times ovl: copy up inode flags Revert "Revert "ovl: get_write_access() in truncate"" ovl: copy up file size as well ovl: deal with overlay files in ovl_d_real() ovl: stack file ops ovl: add helper to return real file ovl: add ovl_read_iter() ovl: add ovl_write_iter() ovl: add ovl_fsync() ovl: add ovl_mmap() ovl: add ovl_fallocate() ovl: add lsattr/chattr support ovl: add ovl_fiemap() ovl: add O_DIRECT support ovl: add reflink/copyfile/dedup support vfs: don't open real ovl: copy-up on MAP_SHARED vfs: simplify dentry_open() Revert "ovl: fix may_write_real() for overlayfs directories" Revert "ovl: don't allow writing ioctl on lower layer" vfs: fix freeze protection in mnt_want_write_file() for overlayfs Revert "ovl: fix relatime for directories" Revert "vfs: update ovl inode before relatime check" Revert "vfs: add flags to d_real()" Revert "vfs: do get_write_access() on upper layer of overlayfs" Partially revert "locks: fix file locking on overlayfs" Revert "fsnotify: support overlayfs" vfs: remove open_flags from d_real() ovl: fix documentation of non-standard behavior --- Documentation/filesystems/Locking | 4 +- Documentation/filesystems/overlayfs.txt | 60 ++-- Documentation/filesystems/vfs.txt | 19 +- fs/btrfs/ctree.h| 4 +- fs/btrfs/ioctl.c| 6 +- fs/file_table.c | 13 +- fs/inode.c | 46 +-- fs/internal.h | 17 +- fs/ioctl.c | 1 + fs/locks.c | 20 +- fs/namei.c | 2 +- fs/namespace.c | 69 + fs/ocfs2/file.c | 10 +- fs/open.c | 74 ++--- fs/overlayfs/Kconfig| 21 ++ fs/overlayfs/Makefile | 4 +- fs/overlayfs/dir.c | 33 ++- fs/overlayfs/file.c | 506 fs/overlayfs/inode.c| 63 +++- fs/overlayfs/overlayfs.h| 21 +- fs/overlayfs/ovl_entry.h| 1 + fs/overlayfs/super.c| 65 ++-- fs/overlayfs/util.c | 11 +- fs/read_write.c | 91 +++--- fs/xattr.c | 9 +- fs/xfs/xfs_file.c | 8 +- include/linux/dcache.h | 15 +- include/linux/fs.h | 30 +- include/linux/fsnotify.h| 14 +- include/uapi/linux/fs.h | 1 - mm/util.c | 5 + 31 files changed, 886 insertions(+), 357 deletions(-) create mode 100644 fs/overlayfs/file.c
Re: [RFC PATCH 00/35] overlayfs: stack file operations
On Wed, Apr 25, 2018 at 4:49 PM, J. R. Okajima wrote: > Miklos Szeredi: >> This patch series reverts the VFS hacks (with the exception of d_path) and > > I totally agree with removing d_real things. > It must be good to the world. > > If I understand correctly, this series affects file_inode() too. > So there may exist more commits to revert such as > fea6d2a6 2017-02-14 vfs: Use upper filesystem inode in bprm_fill_uid() Yep. > > Here is another question. > Does overlayfs support atomic_open? No. Overlayfs doesn't support network filesystems as modifiable (upper) layer, and that's where atomic_open makes sense. I think actually no filesystem that defines atomic_open() can be used as upper layer. So that's currently not a worry. If it turns out to be a requirement to use such filesystem as upper, then I'll worry about atomic_open(). Thanks, Miklos
Re: [RFC PATCH 00/35] overlayfs: stack file operations
Miklos Szeredi: > This patch series reverts the VFS hacks (with the exception of d_path) and I totally agree with removing d_real things. It must be good to the world. If I understand correctly, this series affects file_inode() too. So there may exist more commits to revert such as fea6d2a6 2017-02-14 vfs: Use upper filesystem inode in bprm_fill_uid() Here is another question. Does overlayfs support atomic_open? I remember implementing atomic_open on aufs was rather tricky many years ago, and I am interested in how overlayfs addresses it. J. R. Okajima
[RFC PATCH 00/35] overlayfs: stack file operations
Up till now overlayfs didn't stack regular file operations. Instead, when a file was opened on an overlay, the file from one of the underlying layers would be opened and any file operations performed would directly go to the underlying file on a real filesystem. This works well mostly, but various hacks were added to the VFS to work around issues with this: - d_path() and friends - relatime handling - file locking - fsnotify - writecount handling There are also issues that are unresolved before this patchset: - ioctl's that need write access but can be performed on a O_RDONLY fd - ro/rw inconsistency: file on lower layer opened for read-only will return stale data on read after copy-up and modification - ro/rw inconsistency for mmap: file on lower layer mapped shared will contain stale data after copy-up and modification This patch series reverts the VFS hacks (with the exception of d_path) and fixes the unresoved issues. We need to keep d_path related hacks, because memory maps are still not stacked, yet d_path() should keep working on vma->vm_file->f_path. No regressions were observed after running various test suites (xfstests, ltp, unionmount-testsuite, pjd-fstest). Performance impact of stacking was found to be minimal. Memory use for open overlay files increases by about 256bytes or 12%. Git tree is here: git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs.git overlayfs-rorw Thanks, Miklos --- Miklos Szeredi (35): vfs: clean up dedup vfs: add path_open() vfs: optionally don't account file in nr_files ovl: copy up times ovl: copy up inode flags Revert "Revert "ovl: get_write_access() in truncate"" ovl: copy up file size as well ovl: deal with overlay files in ovl_d_real() ovl: stack file ops ovl: add helper to return real file ovl: readd read_iter ovl: readd write_iter ovl: readd fsync ovl: readd mmap ovl: readd fallocate ovl: readd lsattr/chattr support ovl: readd fiemap ovl: readd O_DIRECT support ovl: readd reflink/copyfile/dedup support vfs: don't open real vfs: add f_op->pre_mmap() ovl: copy-up on MAP_SHARED vfs: simplify dentry_open() Revert "ovl: fix relatime for directories" Revert "vfs: update ovl inode before relatime check" Revert "ovl: fix may_write_real() for overlayfs directories" Revert "ovl: don't allow writing ioctl on lower layer" Revert "vfs: add flags to d_real()" Revert "vfs: do get_write_access() on upper layer of overlayfs" Revert "vfs: make argument of d_real_inode() const" Revert "vfs: add d_real_inode() helper" Partially revert "locks: fix file locking on overlayfs" Revert "fsnotify: support overlayfs" vfs: simplify d_op->d_real() ovl: fix documentation of non-standard behavior Documentation/filesystems/Locking | 4 +- Documentation/filesystems/overlayfs.txt | 64 +++-- Documentation/filesystems/vfs.txt | 25 +- fs/file_table.c | 13 +- fs/inode.c | 46 +--- fs/internal.h | 17 +- fs/ioctl.c | 1 + fs/locks.c | 20 +- fs/namei.c | 2 +- fs/namespace.c | 66 + fs/open.c | 79 +++--- fs/overlayfs/Kconfig| 21 ++ fs/overlayfs/Makefile | 4 +- fs/overlayfs/dir.c | 5 + fs/overlayfs/file.c | 466 fs/overlayfs/inode.c| 52 +++- fs/overlayfs/overlayfs.h| 26 ++ fs/overlayfs/ovl_entry.h| 1 + fs/overlayfs/super.c| 82 +++--- fs/overlayfs/util.c | 20 ++ fs/read_write.c | 90 +++--- fs/xattr.c | 9 +- include/linux/dcache.h | 41 +-- include/linux/fs.h | 32 ++- include/linux/fsnotify.h| 14 +- include/uapi/linux/fs.h | 1 - mm/util.c | 5 + 27 files changed, 831 insertions(+), 375 deletions(-) create mode 100644 fs/overlayfs/file.c -- 2.14.3
