Re: nfs subvolume access?

2021-03-10 Thread Hugo Mills
On Wed, Mar 10, 2021 at 08:46:20AM +0100, Ulli Horlacher wrote: > When I try to access a btrfs filesystem via nfs, I get the error: > > root@tsmsrvi:~# mount tsmsrvj:/data/fex /nfs/tsmsrvj/fex > root@tsmsrvi:~# time find /nfs/tsmsrvj/fex | wc -l > find: File system loop detected; '/nfs/tsmsrvj/fex

Re: nfs subvolume access?

2021-03-10 Thread Ulli Horlacher
On Wed 2021-03-10 (07:59), Hugo Mills wrote: > > On tsmsrvj I have in /etc/exports: > > > > /data/fex tsmsrvi(rw,async,no_subtree_check,no_root_squash) > > > > This is a btrfs subvolume with snapshots: > > > > root@tsmsrvj:~# btrfs subvolume list /data > > ID 257 gen 35 top level 5 path f

Re: nfs subvolume access?

2021-03-10 Thread Ulli Horlacher
On Wed 2021-03-10 (08:46), Ulli Horlacher wrote: > When I try to access a btrfs filesystem via nfs, I get the error: > > root@tsmsrvi:~# mount tsmsrvj:/data/fex /nfs/tsmsrvj/fex > root@tsmsrvi:~# time find /nfs/tsmsrvj/fex | wc -l > find: File system loop detected; '/nfs/tsmsrvj/fex/spool' is part

[PATCH v2 00/15] btrfs: support read-write for subpage metadata

2021-03-10 Thread Qu Wenruo
This patchset can be fetched from the following github repo, along with the full subpage RW support: https://github.com/adam900710/linux/tree/subpage This patchset is for metadata read write support. [FULL RW TEST] Since the data write path is not included in this patchset, we can't really test t

[PATCH v2 01/15] btrfs: add sysfs interface for supported sectorsize

2021-03-10 Thread Qu Wenruo
Add extra sysfs interface features/supported_ro_sectorsize and features/supported_rw_sectorsize to indicate subpage support. Currently for supported_rw_sectorsize all architectures only have their PAGE_SIZE listed. While for supported_ro_sectorsize, for systems with 64K page size, 4K sectorsize i

[PATCH v2 08/15] btrfs: make alloc_extent_buffer() check subpage dirty bitmap

2021-03-10 Thread Qu Wenruo
In alloc_extent_buffer(), we make sure that the newly allocated page is never dirty. This is fine for sector size == PAGE_SIZE case, but for subpage it's possible that one extent buffer in the page is dirty, thus the whole page is marked dirty, and could cause false alert. To support subpage, cal

[PATCH v2 03/15] btrfs: remove unnecessary variable shadowing in btrfs_invalidatepage()

2021-03-10 Thread Qu Wenruo
In btrfs_invalidatepage() we re-declare @tree variable as btrfs_ordered_inode_tree. Since it's only used to do the spinlock, we can grab it from inode directly, and remove the unnecessary declaration completely. Signed-off-by: Qu Wenruo --- fs/btrfs/inode.c | 8 ++-- 1 file changed, 2 inser

[PATCH v2 07/15] btrfs: support subpage metadata csum calculation at write time

2021-03-10 Thread Qu Wenruo
Add a new helper, csum_dirty_subpage_buffers(), to iterate through all dirty extent buffers in one bvec. Also extract the code of calculating csum for one extent buffer into csum_one_extent_buffer(), so that both the existing csum_dirty_buffer() and the new csum_dirty_subpage_buffers() can reuse t

[PATCH v2 06/15] btrfs: allow btree_set_page_dirty() to do more sanity check on subpage metadata

2021-03-10 Thread Qu Wenruo
For btree_set_page_dirty(), we should also check the extent buffer sanity for subpage support. Unlike the regular sector size case, since one page can contain multiple extent buffers, we need to make sure there is at least one dirty extent buffer in the page. So this patch will iterate through th

[PATCH v2 09/15] btrfs: make the page uptodate assert to be subpage compatible

2021-03-10 Thread Qu Wenruo
There are quite some assert check on page uptodate in extent buffer write accessors. They ensure the destination page is already uptodate. This is fine for regular sector size case, but not for subpage case, as for subpage we only mark the page uptodate if the page contains no hole and all its ext

[PATCH v2 10/15] btrfs: make set/clear_extent_buffer_dirty() to be subpage compatible

2021-03-10 Thread Qu Wenruo
For set_extent_buffer_dirty() to support subpage sized metadata, just call btrfs_page_set_dirty() to handle both cases. For clear_extent_buffer_dirty(), it needs to clear the page dirty if and only if all extent buffers in the page range are no longer dirty. Also do the same for page error. This

[PATCH v2 05/15] btrfs: introduce helpers for subpage writeback status

2021-03-10 Thread Qu Wenruo
This patch introduces the following functions to handle btrfs subpage writeback status: - btrfs_subpage_set_writeback() - btrfs_subpage_clear_writeback() - btrfs_subpage_test_writeback() Those helpers can only be called when the range is ensured to be inside the page. - btrfs_page_set_writebac

[PATCH v2 11/15] btrfs: make set_btree_ioerr() accept extent buffer and to be subpage compatible

2021-03-10 Thread Qu Wenruo
Current set_btree_ioerr() only accepts @page parameter and grabs extent buffer from page::private. This works fine for sector size == PAGE_SIZE case, but not for subpage case. Adds an extra parameter, @eb, for callers to pass extent buffer to this function, so that subpage code can reuse this fun

[PATCH v2 02/15] btrfs: use min() to replace open-code in btrfs_invalidatepage()

2021-03-10 Thread Qu Wenruo
In btrfs_invalidatepage() we introduce a temporary variable, new_len, to update ordered->truncated_len. But we can use min() to replace it completely and no need for the variable. Signed-off-by: Qu Wenruo --- fs/btrfs/inode.c | 6 ++ 1 file changed, 2 insertions(+), 4 deletions(-) diff --g

[PATCH v2 13/15] btrfs: introduce write_one_subpage_eb() function

2021-03-10 Thread Qu Wenruo
The new function, write_one_subpage_eb(), as a subroutine for subpage metadata write, will handle the extent buffer bio submission. The major differences between the new write_one_subpage_eb() and write_one_eb() is: - No page locking When entering write_one_subpage_eb() the page is no longer loc

[PATCH v2 12/15] btrfs: introduce end_bio_subpage_eb_writepage() function

2021-03-10 Thread Qu Wenruo
The new function, end_bio_subpage_eb_writepage(), will handle the metadata writeback endio. The major differences involved are: - How to grab extent buffer Now page::private is a pointer to btrfs_subpage, we can no longer grab extent buffer directly. Thus we need to use the bv_offset to loca

[PATCH v2 04/15] btrfs: introduce helpers for subpage dirty status

2021-03-10 Thread Qu Wenruo
This patch introduce the following functions to handle btrfs subpage dirty status: - btrfs_subpage_set_dirty() - btrfs_subpage_clear_dirty() - btrfs_subpage_test_dirty() Those helpers can only be called when the range is ensured to be inside the page. - btrfs_page_set_dirty() - btrfs_page_clea

[PATCH v2 15/15] btrfs: introduce submit_eb_subpage() to submit a subpage metadata page

2021-03-10 Thread Qu Wenruo
The new function, submit_eb_subpage(), will submit all the dirty extent buffers in the page. The major difference between submit_eb_page() and submit_eb_subpage() is: - How to grab extent buffer Now we use find_extent_buffer_nospinlock() other than using page::private. All other different han

[PATCH v2 14/15] btrfs: make lock_extent_buffer_for_io() to be subpage compatible

2021-03-10 Thread Qu Wenruo
For subpage metadata, we don't use page locking at all. So just skip the page locking part for subpage. All the remaining routine can be reused. Signed-off-by: Qu Wenruo --- fs/btrfs/extent_io.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/extent_io.c b/fs

Re: [PATCH] btrfs-progs: output sectorsize related warning message into stdout

2021-03-10 Thread David Sterba
On Wed, Mar 10, 2021 at 08:18:16AM +0800, Qu Wenruo wrote: > > > On 2021/3/9 下午9:33, David Sterba wrote: > > On Tue, Mar 09, 2021 at 03:39:09PM +0800, Qu Wenruo wrote: > >> Since commit 90020a760584 ("btrfs-progs: mkfs: refactor how we handle > >> sectorsize override") we have extra warning messa

Re: Re: nfs subvolume access?

2021-03-10 Thread Graham Cobb
On 10/03/2021 08:09, Ulli Horlacher wrote: > On Wed 2021-03-10 (07:59), Hugo Mills wrote: > >>> On tsmsrvj I have in /etc/exports: >>> >>> /data/fex tsmsrvi(rw,async,no_subtree_check,no_root_squash) >>> >>> This is a btrfs subvolume with snapshots: >>> >>> root@tsmsrvj:~# btrfs subvolume lis

Re: [PATCH] btrfs: add test for cases when a dio write has to fallback to a buffered write

2021-03-10 Thread Filipe Manana
On Sun, Mar 7, 2021 at 3:24 PM Eryu Guan wrote: > > On Sun, Mar 07, 2021 at 03:07:43PM +, Filipe Manana wrote: > > On Sun, Mar 7, 2021 at 2:41 PM Eryu Guan wrote: > > > > > > On Thu, Feb 11, 2021 at 05:01:18PM +, fdman...@kernel.org wrote: > > > > From: Filipe Manana > > > > > > > > Test

no memory is freed after snapshots are deleted

2021-03-10 Thread telsch
Dear devs, after my root partiton was full, i deleted the last monthly snapshots. however, no memory was freed. so far rebalancing helped: btrfs balance start -v -musage=0 / btrfs balance start -v -dusage=0 / i have deleted all snapshots, but no memory is being freed this time.

Re: [PATCH v2 00/10] fsdax,xfs: Add reflink&dedupe support for fsdax

2021-03-10 Thread Neal Gompa
On Thu, Feb 25, 2021 at 7:23 PM Shiyang Ruan wrote: > > This patchset is attempt to add CoW support for fsdax, and take XFS, > which has both reflink and fsdax feature, as an example. > > Changes from V1: > - Factor some helper functions to simplify dax fault code > - Introduce iomap_apply2() fo

Re: [PATCH v2 00/10] fsdax,xfs: Add reflink&dedupe support for fsdax

2021-03-10 Thread Matthew Wilcox
On Wed, Mar 10, 2021 at 07:30:41AM -0500, Neal Gompa wrote: > Forgive my ignorance, but is there a reason why this isn't wired up to > Btrfs at the same time? It seems weird to me that adding a feature btrfs doesn't support DAX. only ext2, ext4, XFS and FUSE have DAX support. If you think about

Re: [PATCH v2 00/10] fsdax,xfs: Add reflink&dedupe support for fsdax

2021-03-10 Thread Neal Gompa
On Wed, Mar 10, 2021 at 8:02 AM Matthew Wilcox wrote: > > On Wed, Mar 10, 2021 at 07:30:41AM -0500, Neal Gompa wrote: > > Forgive my ignorance, but is there a reason why this isn't wired up to > > Btrfs at the same time? It seems weird to me that adding a feature > > btrfs doesn't support DAX. on

Re: [PATCH v2 00/10] fsdax,xfs: Add reflink&dedupe support for fsdax

2021-03-10 Thread Matthew Wilcox
On Wed, Mar 10, 2021 at 08:36:06AM -0500, Neal Gompa wrote: > On Wed, Mar 10, 2021 at 8:02 AM Matthew Wilcox wrote: > > > > On Wed, Mar 10, 2021 at 07:30:41AM -0500, Neal Gompa wrote: > > > Forgive my ignorance, but is there a reason why this isn't wired up to > > > Btrfs at the same time? It seem

Re: no memory is freed after snapshots are deleted

2021-03-10 Thread Graham Cobb
On 10/03/2021 12:07, telsch wrote: > Dear devs, > > after my root partiton was full, i deleted the last monthly snapshots. > however, no memory was freed. > so far rebalancing helped: > > btrfs balance start -v -musage=0 / > btrfs balance start -v -dusage=0 / > > i have deleted all

Re: [PATCH] btrfs-progs: build: Use PKG_CONFIG instead of pkg-config

2021-03-10 Thread David Sterba
On Tue, Mar 09, 2021 at 10:24:40PM +0100, Heiko Becker wrote: > Hard-coding the pkg-config executable might result in build errors > on system and cross environments that have prefixed toolchains. The > PKG_CONFIG variable already holds the proper one and is already used > in a few other places. >

Re: [PATCH v2 00/10] fsdax,xfs: Add reflink&dedupe support for fsdax

2021-03-10 Thread Goldwyn Rodrigues
On 13:02 10/03, Matthew Wilcox wrote: > On Wed, Mar 10, 2021 at 07:30:41AM -0500, Neal Gompa wrote: > > Forgive my ignorance, but is there a reason why this isn't wired up to > > Btrfs at the same time? It seems weird to me that adding a feature > > btrfs doesn't support DAX. only ext2, ext4, XFS

Re: [PATCH v2 00/10] fsdax,xfs: Add reflink&dedupe support for fsdax

2021-03-10 Thread Matthew Wilcox
On Wed, Mar 10, 2021 at 08:21:59AM -0600, Goldwyn Rodrigues wrote: > On 13:02 10/03, Matthew Wilcox wrote: > > On Wed, Mar 10, 2021 at 07:30:41AM -0500, Neal Gompa wrote: > > > Forgive my ignorance, but is there a reason why this isn't wired up to > > > Btrfs at the same time? It seems weird to me

Aw: Re: no memory is freed after snapshots are deleted

2021-03-10 Thread telsch
> Don't forget that, in general, deleting a snapshot does nothing - if the > original files are still there (or any other snapshots of the same files > are still there). In my experience, if you *really* need space urgently > you are best of starting with deleting some big files *and* all the > sna

Re: nfs subvolume access?

2021-03-10 Thread Ulli Horlacher
On Wed 2021-03-10 (09:35), Graham Cobb wrote: > >>> root@tsmsrvj:~# find /data/fex | wc -l > >>> 489887 > > > >>I can't remember if this is why, but I've had to put a distinct > >> fsid field in each separate subvolume being exported: > >> > >> /srv/nfs/home -rw,async,fsid=0x1730,no_subtr

Re: Aw: Re: no memory is freed after snapshots are deleted

2021-03-10 Thread Remi Gauvin
On 2021-03-10 10:49 a.m., telsch wrote: > > Any other ideas to fix this? > We can check that there are, in fact, no unexpected subvolumes. btrfs sub list / In particular, I wonder if you have subvolumes/snapshots hidden behind the mounted subvolume. Also, I don't think it's really the case

Re: [PATCH v2 00/10] fsdax,xfs: Add reflink&dedupe support for fsdax

2021-03-10 Thread Goldwyn Rodrigues
On 14:26 10/03, Matthew Wilcox wrote: > On Wed, Mar 10, 2021 at 08:21:59AM -0600, Goldwyn Rodrigues wrote: > > On 13:02 10/03, Matthew Wilcox wrote: > > > On Wed, Mar 10, 2021 at 07:30:41AM -0500, Neal Gompa wrote: > > > > Forgive my ignorance, but is there a reason why this isn't wired up to > > >

Re: no memory is freed after snapshots are deleted

2021-03-10 Thread Zygo Blaxell
On Wed, Mar 10, 2021 at 01:07:47PM +0100, telsch wrote: > Dear devs, > > after my root partiton was full, i deleted the last monthly snapshots. > however, no memory was freed. > so far rebalancing helped: > > btrfs balance start -v -musage=0 / > btrfs balance start -v -dusage=0 / >

Re: Re: Re: no memory is freed after snapshots are deleted

2021-03-10 Thread telsch
> We can check that there are, in fact, no unexpected subvolumes. > > btrfs sub list / > > In particular, I wonder if you have subvolumes/snapshots hidden behind > the mounted subvolume. > > Also, I don't think it's really the case here with only 16GB reported by > du,, but do you have any large,

Multiple files with the same name in one directory

2021-03-10 Thread Martin Raiber
Hi, I have this in a btrfs directory. Linux kernel 5.10.16, no errors in dmesg, no scrub errors: ls -lh total 19G -rwxr-x--- 1 root root  783 Mar 10 14:56 disk_config.dat -rwxr-x--- 1 root root  783 Mar 10 14:56 disk_config.dat -rwxr-x--- 1 root root  783 Mar 10 14:56 dis

Re: nfs subvolume access?

2021-03-10 Thread Forza
From: Ulli Horlacher -- Sent: 2021-03-10 - 16:55 > On Wed 2021-03-10 (09:35), Graham Cobb wrote: > >> >>> root@tsmsrvj:~# find /data/fex | wc -l >> >>> 489887 >> > >> >>I can't remember if this is why, but I've had to put a distinct >> >> fsid field in each separate subvolume

Re: nfs subvolume access?

2021-03-10 Thread Ulli Horlacher
On Wed 2021-03-10 (18:29), Forza wrote: > Did you try the fsid on the export? Yes: root@tsmsrvj:/etc# grep tsm exports /data/fex tsmsrvi(rw,async,no_subtree_check,no_root_squash,fsid=0x0011) root@tsmsrvj:/etc# exportfs -va exporting fex.rus.uni-stuttgart.de:/data/fex exporting tsmsrvi.ru

Re: misc-next a646ddc2bba2: kernel BUG at fs/btrfs/ctree.c:1210! tree mod log

2021-03-10 Thread Zygo Blaxell
On Fri, Mar 05, 2021 at 12:45:08PM +, Filipe Manana wrote: > On Fri, Mar 5, 2021 at 1:08 AM Zygo Blaxell > wrote: > > > > On Tue, Mar 02, 2021 at 04:24:19PM +, Filipe Manana wrote: > > > On Sat, Feb 27, 2021 at 3:53 PM Zygo Blaxell > > > wrote: > > > > > > > > Hit this twice so far, while

Re: [PATCH] fstest: random read fio test for read policy

2021-03-10 Thread Anand Jain
Hi, How about a review of this test case or suggest any better ideas? If we are ok with this, I will be adding 2 other types of workloads that we need to test read policies. Thanks, Anand On 22/2/21 10:48 pm, Anand Jain wrote: This test case runs fio for raid1/10/1c3/1c4 profiles and all

Re: [PATCH 0/3] btrfs: Convert kmap/memset/kunmap to memzero_user()

2021-03-10 Thread Andrew Morton
On Tue, 9 Mar 2021 13:21:34 -0800 ira.we...@intel.com wrote: > Previously this was submitted to convert to zero_user()[1]. zero_user() is > not > the same as memzero_user() and in fact some zero_user() calls may be better > off > as memzero_user(). Regardless it was incorrect to convert btrfs

Re: [PATCH v2 00/10] fsdax,xfs: Add reflink&dedupe support for fsdax

2021-03-10 Thread Dan Williams
On Wed, Mar 10, 2021 at 6:27 AM Matthew Wilcox wrote: > > On Wed, Mar 10, 2021 at 08:21:59AM -0600, Goldwyn Rodrigues wrote: > > On 13:02 10/03, Matthew Wilcox wrote: > > > On Wed, Mar 10, 2021 at 07:30:41AM -0500, Neal Gompa wrote: > > > > Forgive my ignorance, but is there a reason why this isn'

Re: nfs subvolume access?

2021-03-10 Thread Ulli Horlacher
On Wed 2021-03-10 (08:46), Ulli Horlacher wrote: > When I try to access a btrfs filesystem via nfs, I get the error: > > root@tsmsrvi:~# mount tsmsrvj:/data/fex /nfs/tsmsrvj/fex > root@tsmsrvi:~# time find /nfs/tsmsrvj/fex | wc -l > find: File system loop detected; '/nfs/tsmsrvj/fex/spool' is part