Re: kernel calltraces with btrfs and bonnie++

2019-01-24 Thread Qu Wenruo
On 2019/1/25 下午3:39, Stefan K wrote: > Hello, > > if I run 'bonnie++ -c4' the system is unusable and hangs, I got also some > CallTraces in my syslog. Is that a normal behavior? > > My system is: > uname -a > Linux tani 4.19.0-0.bpo.1-amd64 #1 SMP Debian 4.19.12-1~bpo9+1 (2018-12-30) > x86_

kernel calltraces with btrfs and bonnie++

2019-01-24 Thread Stefan K
Hello, if I run 'bonnie++ -c4' the system is unusable and hangs, I got also some CallTraces in my syslog. Is that a normal behavior? My system is: uname -a Linux tani 4.19.0-0.bpo.1-amd64 #1 SMP Debian 4.19.12-1~bpo9+1 (2018-12-30) x86_64 GNU/Linux btrfs fi sh Label: none uuid: 24be286b-ece

[PATCH v4 12/12] btrfs: Do mandatory tree block check before submitting bio

2019-01-24 Thread Qu Wenruo
There are at least 2 reports about memory bit flip sneaking into on-disk data. Currently we only have a relaxed check triggered at btrfs_mark_buffer_dirty() time, as it's not mandatory and only for CONFIG_BTRFS_FS_CHECK_INTEGRITY enabled build, it doesn't help user to detect such problem. This pa

[PATCH v4 05/12] btrfs: extent_io: Kill the BUG_ON() in extent_write_full_page()

2019-01-24 Thread Qu Wenruo
This one is pretty straightforward, __extent_writepage() can only return <0 or 0. So if we hit error from __extent_writepage(), then return the error. Or return the value from flush_write_bio(). Signed-off-by: Qu Wenruo --- fs/btrfs/extent_io.c | 10 -- 1 file changed, 8 insertions(+),

[PATCH v4 06/12] btrfs: extent_io: Kill the BUG_ON() in btree_write_cache_pages()

2019-01-24 Thread Qu Wenruo
The @ret from the original function can only be 0 or <0. <0 case is from write_one_eb() which can only return 0 or <0. So just return @ret if something went wrong. For anything else, return @flush_ret. Also add a ASSERT() to catch unexpected >0 return value from write_one_eb(). Signed-off-by: Qu

[PATCH v4 08/12] btrfs: extent_io: Kill the BUG_ON() in extent_write_locked_range()

2019-01-24 Thread Qu Wenruo
Signed-off-by: Qu Wenruo --- fs/btrfs/extent_io.c | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index 866972ecf70d..fba5d4a0b194 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -4089,8 +4089,10 @@ int extent_w

[PATCH v4 04/12] btrfs: extent_io: Move the BUG_ON() in flush_write_bio() one level up

2019-01-24 Thread Qu Wenruo
We have a BUG_ON() in flush_write_bio() to handle the return value of submit_one_bio(). Move the BUG_ON() one level up to all its callers. No functional change, just to make later BUG_ON() cleanup more obvious. Signed-off-by: Qu Wenruo --- fs/btrfs/extent_io.c | 48

[PATCH v4 00/12] btrfs: Enhancement to tree block validation

2019-01-24 Thread Qu Wenruo
Patchset can be fetched from github: https://github.com/adam900710/linux/tree/write_time_tree_checker Which is based on v5.0-rc1 tag. This patchset has the following two features: - Tree block validation output enhancement * Output validation failure timing (write time or read time) * Always o

[PATCH v4 10/12] btrfs: extent_io: Kill the BUG_ON() in extent_write_cache_pages()

2019-01-24 Thread Qu Wenruo
Signed-off-by: Qu Wenruo --- fs/btrfs/extent_io.c | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index 52c1b0ce6bfd..0d1f09b41421 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -3980,7 +3980,10 @@ static

[PATCH v4 11/12] btrfs: extent_io: Kill the BUG_ON() in extent_writepages()

2019-01-24 Thread Qu Wenruo
The @ret from extent_write_cache_pages() can only be 0 or <0, so we only need to return @ret if something went wrong, or just return @flush_ret. Signed-off-by: Qu Wenruo --- fs/btrfs/extent_io.c | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/b

[PATCH v4 09/12] btrfs: extent_io: Kill the BUG_ON() in lock_extent_buffer_for_io()

2019-01-24 Thread Qu Wenruo
This function needs some extra check on locked pages and eb. For error handling we need to unlock locked pages and the eb. Also add comment for possible return values of lock_extent_buffer_for_io(). There is a rare >0 return value branch, where all pages get locked while write bio is not flushed

[PATCH v4 03/12] btrfs: disk-io: Show the timing of corrupted tree block explicitly

2019-01-24 Thread Qu Wenruo
Just add one extra line to show when the corruption is detected. Currently only read time detection is possible. Signed-off-by: Qu Wenruo Reviewed-by: Nikolay Borisov --- fs/btrfs/disk-io.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 794d5

[PATCH v4 07/12] btrfs: extent_io: Kill the dead branch in extent_write_cache_pages()

2019-01-24 Thread Qu Wenruo
Since __extent_writepage() will no longer return >0 value, (ret == AOP_WRITEPAGE_ACTIVATE) will never be true. Kill that dead branch. Signed-off-by: Qu Wenruo --- fs/btrfs/extent_io.c | 5 - 1 file changed, 5 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index 5573c

[PATCH v4 01/12] btrfs: Always output error message when key/level verification fails

2019-01-24 Thread Qu Wenruo
We have internal report of strange transaction abort due to EUCLEAN without any error message. Since error message inside verify_level_key() is only enabled for CONFIG_BTRFS_DEBUG, the error message won't output for most distro. This patch will make the error message mandatory, so when problem ha

[PATCH v4 02/12] btrfs: extent_io: Kill the forward declaration of flush_write_bio()

2019-01-24 Thread Qu Wenruo
There is no need to forward declare flush_write_bio(), as it only depends on submit_one_bio(). Both of them are pretty small, just move them to kill the forward declaration. Signed-off-by: Qu Wenruo Reviewed-by: Nikolay Borisov Reviewed-by: Johannes Thumshirn --- fs/btrfs/extent_io.c | 66 +++

Re: [PATCH v2] Btrfs: avoid deadlock with memory reclaim due to allocation of devices

2019-01-24 Thread Anand Jain
On 01/12/2019 01:17 AM, fdman...@kernel.org wrote: From: Filipe Manana In a few places we are allocating a device using the GFP_KERNEL flag when it is not safe to do so, because if reclaim is triggered it can cause a transaction commit while we are holding the device list mutex. This mutex i

Re: [PATCH v2] Btrfs: avoid deadlock with memory reclaim due to allocation of devices

2019-01-24 Thread Anand Jain
On 01/19/2019 02:07 AM, David Sterba wrote: On Mon, Jan 14, 2019 at 04:21:43PM +0800, Anand Jain wrote: On 01/12/2019 01:17 AM, fdman...@kernel.org wrote: From: Filipe Manana In a few places we are allocating a device using the GFP_KERNEL flag when it is not safe to do so, because if rec

Re: Nasty corruption on large array, ideas welcome

2019-01-24 Thread Thiago Ramon
On Fri, Jan 25, 2019 at 12:36 AM Anand Jain wrote: > > > > On 01/24/2019 01:30 AM, Thiago Ramon wrote: > > Back again with more info. > > > > I've done a dump-tree on my array, and from the 71GB of tree data, > > I've only got the following errors: > > parent transid verify failed on 3128844846694

Re: Nasty corruption on large array, ideas welcome

2019-01-24 Thread Anand Jain
On 01/24/2019 01:30 AM, Thiago Ramon wrote: Back again with more info. I've done a dump-tree on my array, and from the 71GB of tree data, I've only got the following errors: parent transid verify failed on 31288448466944 wanted 135681 found 135575 parent transid verify failed on 3128844846694

Re: [PATCH -next] btrfs: qgroup: Fix inconsistent IS_ERR and PTR_ERR

2019-01-24 Thread Qu Wenruo
On 2019/1/25 上午9:50, YueHaibing wrote: > Change the call to PTR_ERR to access the value just tested by IS_ERR. > > Fixes: 2b35a512e9cf ("btrfs: qgroup: Use delayed subtree rescan for balance") > Signed-off-by: YueHaibing > --- > fs/btrfs/qgroup.c | 2 +- > 1 file changed, 1 insertion(+), 1 del

[PATCH -next] btrfs: qgroup: Fix inconsistent IS_ERR and PTR_ERR

2019-01-24 Thread YueHaibing
Change the call to PTR_ERR to access the value just tested by IS_ERR. Fixes: 2b35a512e9cf ("btrfs: qgroup: Use delayed subtree rescan for balance") Signed-off-by: YueHaibing --- fs/btrfs/qgroup.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/qgroup.c b/fs/btrfs/qgr

[PATCH] btrfs: qgroup: Make qgroup async transaction commit more aggressive

2019-01-24 Thread Qu Wenruo
[BUG] Btrfs qgroup will still hit EDQUOT under the following case: #!/bin/bash dev=/dev/test/test mnt=/mnt/btrfs umount $mnt &> /dev/null umount $dev &> /dev/null mkfs.btrfs -f $dev mount $dev $mnt -o nospace_cache btrfs subv create $mnt/subv btrfs quota enable $mnt btrfs qu

Re: [PATCH v2] btrfs: add kernel scrub log messages

2019-01-24 Thread Anand Jain
On 01/25/2019 04:21 AM, David Sterba wrote: On Thu, Jan 03, 2019 at 04:17:40PM +0800, Anand Jain wrote: scrub kernel messages helps debug and audit, add them to the log. Signed-off-by: Anand Jain --- v1->v2: fix btrfs_info, using string directly. Add spacing. fs/btrfs/scrub.c | 5 +

Re: [PATCH v2] btrfs: add kernel scrub log messages

2019-01-24 Thread David Sterba
On Thu, Jan 03, 2019 at 04:17:40PM +0800, Anand Jain wrote: > scrub kernel messages helps debug and audit, add them to the log. > > Signed-off-by: Anand Jain > --- > v1->v2: fix btrfs_info, using string directly. Add spacing. > fs/btrfs/scrub.c | 5 + > 1 file changed, 5 insertions(+) > >

Re: [Patch v5 0/7] btrfs: qgroup: Delay subtree scan to reduce overhead

2019-01-24 Thread David Sterba
On Wed, Jan 23, 2019 at 03:15:11PM +0800, Qu Wenruo wrote: > - A lot of coding style fixes: > * naming change from "file"/"subv" to "subvol" > * {} for any else if branch > * avoid err/ret confusion by introducing "tmp_ret" > * proper errno for non-uptodate extent buffer > * struct member

Re: [PATCH v4 5/7] btrfs: qgroup: Introduce per-root swapped blocks infrastructure

2019-01-24 Thread David Sterba
On Wed, Jan 23, 2019 at 07:07:50AM +0800, Qu Wenruo wrote: > > > On 2019/1/23 上午12:55, David Sterba wrote: > > On Tue, Jan 15, 2019 at 04:16:02PM +0800, Qu Wenruo wrote: > > >> + > >> + block = kmalloc(sizeof(*block), GFP_NOFS); > >> + if (!block) { > >> + ret = -ENOMEM; > > > > This

Re: Incremental receive completes succesfully despite missing files

2019-01-24 Thread Chris Murphy
On Thu, Jan 24, 2019 at 3:40 AM Dennis K wrote: > > The fact is, this thread is the first time I've seen explicitly written > that parents must be the same at receiving and sending ends, or else > btrfs-send/receive will produce a subvolume which differs from the source. The central user error, a

Re: [PATCH] btrfs: no need to check return value of debugfs_create functions

2019-01-24 Thread David Sterba
On Mon, Jan 14, 2019 at 03:21:04PM +0100, Greg Kroah-Hartman wrote: > On Fri, Jan 04, 2019 at 02:44:49PM +0100, David Sterba wrote: > > On Fri, Jan 04, 2019 at 02:25:20PM +0100, Greg Kroah-Hartman wrote: > > > When calling debugfs functions, there is no need to ever check the > > > return value. T

Re: [PATCH v3 0/5] btrfs: Enhancement to tree block validation

2019-01-24 Thread David Sterba
On Thu, Jan 24, 2019 at 11:04:47AM +0800, Qu Wenruo wrote: > > > On 2019/1/24 上午1:16, David Sterba wrote: > > On Fri, Jan 18, 2019 at 10:19:51AM +0800, Qu Wenruo wrote: > >> Patchset can be fetched from github: > >> https://github.com/adam900710/linux/tree/write_time_tree_checker > >> Which is ba

[PATCH] btrfs: don't end the transaction for delayed refs in throttle

2019-01-24 Thread Josef Bacik
Previously callers to btrfs_end_transaction_throttle() would commit the transaction if there wasn't enough delayed refs space. This happens in relocation, and if the fs is relatively empty we'll run out of delayed refs space basically immediately, so we'll just be stuck in this loop of committing

Re: [Patch v5 0/7] btrfs: qgroup: Delay subtree scan to reduce overhead

2019-01-24 Thread Qu Wenruo
On 2019/1/24 上午1:47, David Sterba wrote: > On Wed, Jan 23, 2019 at 03:15:11PM +0800, Qu Wenruo wrote: >> v5: >> - Use Josef's superior qgroup deadlock fix. >> No performance regression now. > > Does this pass fstests on your side? Namely if test btrfs/007 does not > crash, > https://lore.kerne

Re: Incremental receive completes succesfully despite missing files

2019-01-24 Thread Dennis K
On 24/1/19 3:23 am, Hans van Kranenburg wrote: > On 1/23/19 4:32 PM, Nikolay Borisov wrote: >> >> >> On 23.01.19 г. 17:25 ч., Hans van Kranenburg wrote: >>> On 1/23/19 12:25 PM, Andrei Borzenkov wrote: On Wed, Jan 23, 2019 at 1:45 PM Dennis Katsonis wrote: > I think my previous e

Re: [PATCH 6/6] btrfs: merge btrfs_set_lock_blocking_rw with it's caller

2019-01-24 Thread Johannes Thumshirn
Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG N

Re: [PATCH 5/6] btrfs: simplify waiting loop in btrfs_tree_lock

2019-01-24 Thread Johannes Thumshirn
Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG N

Re: [PATCH 4/6] btrfs: open code now trivial btrfs_set_lock_blocking

2019-01-24 Thread Johannes Thumshirn
Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG N

Re: [PATCH 3/6] btrfs: replace btrfs_set_lock_blocking_rw with appropriate helpers

2019-01-24 Thread Johannes Thumshirn
Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG N

Re: [PATCH 2/6] btrfs: split btrfs_clear_lock_blocking_rw to read and write helpers

2019-01-24 Thread Johannes Thumshirn
Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG N

Re: [PATCH 1/6] btrfs: split btrfs_set_lock_blocking_rw to read and write helpers

2019-01-24 Thread Johannes Thumshirn
Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG N