[PATCH] btrfs: use tagged writepage to mitigate livelock of snapshot

2018-10-31 Thread Ethan Lien
Snapshot is expected to be fast. But if there are writers steadily create dirty pages in our subvolume, the snapshot may take a very long time to complete. To fix the problem, we use tagged writepage for snapshot flusher as we do in the generic write_cache_pages(), so we can ommit pages dirtied aft

Re: [PATCH V10] Add support for BTRFS raid5/6 to GRUB

2018-10-31 Thread David Sterba
On Wed, Oct 31, 2018 at 07:48:08PM +0100, Goffredo Baroncelli wrote: > On 31/10/2018 13.06, Daniel Kiper wrote: > [...] > > > > v11 pushed. > > > > Goffredo, thank you for doing the work. > > Great ! Many thanks for your support !! Thank you very much for the work! I've updated wiki with the go

Re: [PATCH V10] Add support for BTRFS raid5/6 to GRUB

2018-10-31 Thread Goffredo Baroncelli
On 31/10/2018 13.06, Daniel Kiper wrote: [...] > > v11 pushed. > > Goffredo, thank you for doing the work. Great ! Many thanks for your support !! > > Nick, you can go ahead and rebase yours patchset. > > Daniel > BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerp

[PATCH v2] btrfs: add zstd compression level support

2018-10-31 Thread Nick Terrell
From: Jennifer Liu Adds zstd compression level support to btrfs. Zstd requires different amounts of memory for each level, so the design had to be modified to allow set_level() to allocate memory. We preallocate one workspace of the maximum size to guarantee forward progress. This feature is expe

[PATCH v4 2/2] btrfs: Add zstd support to grub btrfs

2018-10-31 Thread Nick Terrell
Adds zstd support to the btrfs module. Tested on Ubuntu-18.04 with a btrfs /boot partition with and without zstd compression. A test case was also added to the test suite that fails before the patch, and passes after. Signed-off-by: Nick Terrell --- v1 -> v2: - Fix comments from Daniel Kiper. v

[PATCH v4 0/2] btrfs: Add zstd support to grub btrfs

2018-10-31 Thread Nick Terrell
Hi all, This patch set imports the upstream zstd library, adds zstd support to the btrfs module, and adds a test case. I've also tested the patch set by storing my boot partition in btrfs with and without zstd compression and rebooting. The fist patch imports the files needed to support zstd deco

Re: [PATCH 1/3] btrfs-progs: check: lowmem: Factor out inline extent checking code in its own function

2018-10-31 Thread David Sterba
On Wed, Oct 31, 2018 at 11:43:20AM +0200, Nikolay Borisov wrote: > >> --- a/check/mode-lowmem.c > >> +++ b/check/mode-lowmem.c > >> @@ -1800,6 +1800,87 @@ static int repair_inline_ram_bytes(struct > >> btrfs_root *root, > >>return ret; > >> } > >> > >> + > >> +static int check_file_extent_i

[PATCH v2] Btrfs: fix missing delayed iputs on unmount

2018-10-31 Thread Omar Sandoval
From: Omar Sandoval There's a race between close_ctree() and cleaner_kthread(). close_ctree() sets btrfs_fs_closing(), and the cleaner stops when it sees it set, but this is racy; the cleaner might have already checked the bit and could be cleaning stuff. In particular, if it deletes unused block

Re: [PATCH] Btrfs: fix missing delayed iputs on unmount

2018-10-31 Thread Omar Sandoval
On Wed, Oct 31, 2018 at 06:40:29PM +0200, Nikolay Borisov wrote: > > > On 31.10.18 г. 18:35 ч., Omar Sandoval wrote: > > On Wed, Oct 31, 2018 at 10:43:02AM +0200, Nikolay Borisov wrote: > >> > >> > >> On 31.10.18 г. 2:14 ч., Omar Sandoval wrote: > >>> From: Omar Sandoval > >>> > >>> There's a ra

Re: [PATCH] Btrfs: fix missing delayed iputs on unmount

2018-10-31 Thread Nikolay Borisov
On 31.10.18 г. 18:35 ч., Omar Sandoval wrote: > On Wed, Oct 31, 2018 at 10:43:02AM +0200, Nikolay Borisov wrote: >> >> >> On 31.10.18 г. 2:14 ч., Omar Sandoval wrote: >>> From: Omar Sandoval >>> >>> There's a race between close_ctree() and cleaner_kthread(). >>> close_ctree() sets btrfs_fs_clos

Re: [PATCH] Btrfs: fix missing delayed iputs on unmount

2018-10-31 Thread Omar Sandoval
On Wed, Oct 31, 2018 at 03:41:47PM +0800, Lu Fengqi wrote: > On Tue, Oct 30, 2018 at 05:14:42PM -0700, Omar Sandoval wrote: > >From: Omar Sandoval > > > >There's a race between close_ctree() and cleaner_kthread(). > >close_ctree() sets btrfs_fs_closing(), and the cleaner stops when it > >sees it s

Re: [PATCH] Btrfs: fix missing delayed iputs on unmount

2018-10-31 Thread Omar Sandoval
On Wed, Oct 31, 2018 at 10:43:02AM +0200, Nikolay Borisov wrote: > > > On 31.10.18 г. 2:14 ч., Omar Sandoval wrote: > > From: Omar Sandoval > > > > There's a race between close_ctree() and cleaner_kthread(). > > close_ctree() sets btrfs_fs_closing(), and the cleaner stops when it > > sees it se

Re: [PATCH] fstests: fix fssum to actually ignore file holes when supposed to

2018-10-31 Thread Josef Bacik
On Mon, Oct 29, 2018 at 09:43:40AM +, fdman...@kernel.org wrote: > From: Filipe Manana > > Unless the '-s' option is passed to fssum, it should not detect file holes > and have their existence influence the computed checksum for a file. This > tool was added to test btrfs' send/receive featur

Re: [PATCH] Btrfs: fix missing data checksums after a ranged fsync (msync)

2018-10-31 Thread Josef Bacik
On Mon, Oct 29, 2018 at 09:42:06AM +, fdman...@kernel.org wrote: > From: Filipe Manana > > Recently we got a massive simplification for fsync, where for the fast > path we no longer log new extents while their respective ordered extents > are still running. > > However that simplification in

Re: [RFC][PATCH v3 10/10] btrfs: use common file type conversion

2018-10-31 Thread David Sterba
On Wed, Oct 31, 2018 at 09:22:31AM +, Phillip Potter wrote: > Thanks for this - are you happy for me to move the compile-time tests as per > Amir's suggestion? Yes that's ok.

Re: Understanding "btrfs filesystem usage"

2018-10-31 Thread Ulli Horlacher
On Tue 2018-10-30 (17:10), Ulli Horlacher wrote: > > On Mon 2018-10-29 (17:57), Remi Gauvin wrote: > > > On 2018-10-29 02:11 PM, Ulli Horlacher wrote: > > > > > I want to know how many free space is left and have problems in > > > interpreting the output of: > > > > > > btrfs filesystem usage

Re: [PATCH V10] Add support for BTRFS raid5/6 to GRUB

2018-10-31 Thread Daniel Kiper
On Mon, Oct 22, 2018 at 07:49:40PM +, Nick Terrell wrote: > > > > On Oct 22, 2018, at 4:02 AM, Daniel Kiper wrote: > > > > On Thu, Oct 18, 2018 at 07:55:32PM +0200, Goffredo Baroncelli wrote: > >> > >> Hi All, > >> > >> the aim of this patches set is to provide support for a BTRFS raid5/6 > >>

Re: [PATCH v3] fstests: btrfs/057: Fix false alerts due to orphan files

2018-10-31 Thread Nikolay Borisov
On 31.10.18 г. 11:22 ч., Qu Wenruo wrote: > > > On 2018/10/31 下午4:55, Nikolay Borisov wrote: >> >> >> On 30.10.18 г. 12:18 ч., Qu Wenruo wrote: >>> For any recent kernel, there is a chance that btrfs/057 reports false >>> errors. >>> >>> The false error would look like: >>> btrfs/057 4s ...

Re: [PATCH 1/3] btrfs-progs: check: lowmem: Factor out inline extent checking code in its own function

2018-10-31 Thread misono.tomoh...@fujitsu.com
Hello, fsck-test 006 fails for low-mem mode in current devel branch and bisect points this. On 2018/09/13 21:05, Nikolay Borisov wrote: > Since the inline extent code can be largely self-sufficient, factor > it out from check_file_extent. No functional changes. > > Signed-off-by: Nikolay Boriso

Re: [PATCH 1/3] btrfs-progs: check: lowmem: Factor out inline extent checking code in its own function

2018-10-31 Thread misono.tomoh...@fujitsu.com
Hello, fsck-test 006 fails for low-mem mode in current devel branch and bisect points this. > Since the inline extent code can be largely self-sufficient, factor > it out from check_file_extent. No functional changes. > > Signed-off-by: Nikolay Borisov > --- > check/mode-lowmem.c | 142 ++

Re: [PATCH 1/3] btrfs-progs: check: lowmem: Factor out inline extent checking code in its own function

2018-10-31 Thread Nikolay Borisov
On 31.10.18 г. 11:35 ч., misono.tomoh...@fujitsu.com wrote: > Hello, > > fsck-test 006 fails for low-mem mode in current devel branch and bisect > points this. > >> Since the inline extent code can be largely self-sufficient, factor >> it out from check_file_extent. No functional changes. >>

Re: [RFC][PATCH v3 10/10] btrfs: use common file type conversion

2018-10-31 Thread Phillip Potter
On Tue, Oct 30, 2018 at 08:03:17PM +0100, David Sterba wrote: > On Sat, Oct 27, 2018 at 01:53:48AM +0100, Phillip Potter wrote: > > Deduplicate the btrfs file type conversion implementation - file systems > > that use the same file types as defined by POSIX do not need to define > > their own versi

Re: [PATCH v3] fstests: btrfs/057: Fix false alerts due to orphan files

2018-10-31 Thread Qu Wenruo
On 2018/10/31 下午4:55, Nikolay Borisov wrote: > > > On 30.10.18 г. 12:18 ч., Qu Wenruo wrote: >> For any recent kernel, there is a chance that btrfs/057 reports false >> errors. >> >> The false error would look like: >> btrfs/057 4s ... - output mismatch (see >> /home/adam/xfstests-dev/resul

Re: [PATCH v3] fstests: btrfs/057: Fix false alerts due to orphan files

2018-10-31 Thread Nikolay Borisov
On 30.10.18 г. 12:18 ч., Qu Wenruo wrote: > For any recent kernel, there is a chance that btrfs/057 reports false > errors. > > The false error would look like: > btrfs/057 4s ... - output mismatch (see > /home/adam/xfstests-dev/results//btrfs/057.out.bad) > --- tests/btrfs/057.out 201

Re: [PATCH] Btrfs: fix missing delayed iputs on unmount

2018-10-31 Thread Nikolay Borisov
On 31.10.18 г. 2:14 ч., Omar Sandoval wrote: > From: Omar Sandoval > > There's a race between close_ctree() and cleaner_kthread(). > close_ctree() sets btrfs_fs_closing(), and the cleaner stops when it > sees it set, but this is racy; the cleaner might have already checked > the bit and could

Re: [PATCH] Btrfs: fix missing delayed iputs on unmount

2018-10-31 Thread Nikolay Borisov
On 31.10.18 г. 10:26 ч., Nikolay Borisov wrote: > > > On 31.10.18 г. 9:14 ч., Lu Fengqi wrote: >> On Tue, Oct 30, 2018 at 05:14:42PM -0700, Omar Sandoval wrote: >>> From: Omar Sandoval >>> >>> There's a race between close_ctree() and cleaner_kthread(). >>> close_ctree() sets btrfs_fs_closing(

Re: [PATCH] Btrfs: fix missing delayed iputs on unmount

2018-10-31 Thread Nikolay Borisov
On 31.10.18 г. 9:14 ч., Lu Fengqi wrote: > On Tue, Oct 30, 2018 at 05:14:42PM -0700, Omar Sandoval wrote: >> From: Omar Sandoval >> >> There's a race between close_ctree() and cleaner_kthread(). >> close_ctree() sets btrfs_fs_closing(), and the cleaner stops when it >> sees it set, but this is

Re: [PATCH] Btrfs: fix missing delayed iputs on unmount

2018-10-31 Thread Lu Fengqi
On Tue, Oct 30, 2018 at 05:14:42PM -0700, Omar Sandoval wrote: >From: Omar Sandoval > >There's a race between close_ctree() and cleaner_kthread(). >close_ctree() sets btrfs_fs_closing(), and the cleaner stops when it >sees it set, but this is racy; the cleaner might have already checked >the bit a

Re: Where is my disk space ?

2018-10-31 Thread Barbet Alain
> Also, since you don't have any snapshots, you could also find this > conventionally: > > # du -sh /* Usually yes, but here not. It's just like when you remove a file when a process still use it and write in it, and fcsk will be not happy next time. But I reboot & check with btrfs check withtout

Re: [PATCH] Btrfs: fix missing delayed iputs on unmount

2018-10-31 Thread Lu Fengqi
On Tue, Oct 30, 2018 at 05:14:42PM -0700, Omar Sandoval wrote: >From: Omar Sandoval > >There's a race between close_ctree() and cleaner_kthread(). >close_ctree() sets btrfs_fs_closing(), and the cleaner stops when it >sees it set, but this is racy; the cleaner might have already checked >the bit a