[PATCH 2/2] btrfs: Remove fs_info argument from btrfs_uuid_tree_rem

2018-05-29 Thread Lu Fengqi
This function always takes a transaction handle which contains a reference to the fs_info. Use that and remove the extra argument. Signed-off-by: Lu Fengqi --- fs/btrfs/ctree.h | 3 +-- fs/btrfs/inode.c | 5 ++--- fs/btrfs/ioctl.c | 5 ++--- fs/btrfs/uuid-tree.c | 6 +++--- 4 files c

[PATCH 0/2] UUID tree fs_info cleanup

2018-05-29 Thread Lu Fengqi
Just remove the redundant fs_info argument from uuid_tree functions because these functions have already taken transaction which has a reference to the fs_info. No functional change. Lu Fengqi (2): btrfs: Remove fs_info argument from btrfs_uuid_tree_add btrfs: Remove fs_info argument from btr

[PATCH 1/2] btrfs: Remove fs_info argument from btrfs_uuid_tree_add

2018-05-29 Thread Lu Fengqi
This function always takes a transaction handle which contains a reference to the fs_info. Use that and remove the extra argument. Signed-off-by: Lu Fengqi --- fs/btrfs/ctree.h | 3 +-- fs/btrfs/ioctl.c | 4 ++-- fs/btrfs/transaction.c | 7 +++ fs/btrfs/uuid-tree.c | 4 ++-- fs

Re: [PATCH 0/2] UUID tree fs_info cleanup

2018-05-29 Thread Nikolay Borisov
On 29.05.2018 10:01, Lu Fengqi wrote: > Just remove the redundant fs_info argument from uuid_tree functions > because these functions have already taken transaction which has a > reference to the fs_info. > > No functional change. > > Lu Fengqi (2): > btrfs: Remove fs_info argument from btrf

Re: [LKP] [lkp-robot] [mm] 9092c71bb7: blogbench.write_score -12.3% regression

2018-05-29 Thread Huang, Ying
Hi, Josef, Do you have time to take a look at the regression? kernel test robot writes: > Greeting, > > FYI, we noticed a -12.3% regression of blogbench.write_score and a +9.6% > improvement > of blogbench.read_score due to commit: > > > commit: 9092c71bb724dba2ecba849eae69e5c9d39bd3d2 ("mm: u

[PATCH] btrfs-progs: check: Also compare data between mirrors to detect corruption for NODATASUM extents

2018-05-29 Thread Qu Wenruo
As the lzo corruption reported by James Harvey, for data extents without checksum, neither btrfs check nor kernel scrub could detect anything wrong. However if our profile supports duplication, we still have a chance to detect such corruption, as in that case the mirrors will mismatch with each ot

Re: [PATCH v2 0/6] btrfs_search_slot cleanups

2018-05-29 Thread Liu Bo
On Mon, May 28, 2018 at 06:17:18PM +0200, David Sterba wrote: > On Mon, May 28, 2018 at 04:40:28PM +0200, David Sterba wrote: > > On Fri, May 18, 2018 at 11:00:18AM +0800, Liu Bo wrote: > > > Here are a collection of patches I did for btrfs_search_slot(). > > > > > > v2: more explicit commit log f

Re: [PATCH v3 3/4] btrfs: lzo: Add header length check to avoid slab out of bounds access

2018-05-29 Thread Misono Tomohiro
On 2018/05/23 17:23, Qu Wenruo wrote: > James Harvey reported that some corrupted compressed extent data can > lead to various kernel memory corruption. > > Such corrupted extent data belongs to inode with NODATASUM flags, thus > data csum won't help us detecting such bug. > > If lucky enough, ka

Re: [PATCH v3 3/4] btrfs: lzo: Add header length check to avoid slab out of bounds access

2018-05-29 Thread Qu Wenruo
On 2018年05月29日 16:30, Misono Tomohiro wrote: > On 2018/05/23 17:23, Qu Wenruo wrote: >> James Harvey reported that some corrupted compressed extent data can >> lead to various kernel memory corruption. >> >> Such corrupted extent data belongs to inode with NODATASUM flags, thus >> data csum won't

Re: [PATCH] btrfs-progs: check: Also compare data between mirrors to detect corruption for NODATASUM extents

2018-05-29 Thread Su Yue
On 05/29/2018 03:45 PM, Qu Wenruo wrote: > As the lzo corruption reported by James Harvey, for data extents without > checksum, neither btrfs check nor kernel scrub could detect anything > wrong. > > However if our profile supports duplication, we still have a chance to > detect such corruption,

Re: [PATCH 0/2] UUID tree fs_info cleanup

2018-05-29 Thread David Sterba
On Tue, May 29, 2018 at 03:01:52PM +0800, Lu Fengqi wrote: > Just remove the redundant fs_info argument from uuid_tree functions > because these functions have already taken transaction which has a > reference to the fs_info. Thanks. I've also renamed the function btrfs_uuid_tree_rem to btrfs_uuid

Re: [PATCH] btrfs-progs: check: Also compare data between mirrors to detect corruption for NODATASUM extents

2018-05-29 Thread Lu Fengqi
On Tue, May 29, 2018 at 03:45:27PM +0800, Qu Wenruo wrote: >As the lzo corruption reported by James Harvey, for data extents without >checksum, neither btrfs check nor kernel scrub could detect anything >wrong. > >However if our profile supports duplication, we still have a chance to >detect such c

Re: [PATCH 2nd try v4] btrfs: drop uuid_mutex in btrfs_free_extra_devids()

2018-05-29 Thread David Sterba
On Tue, May 29, 2018 at 01:40:34PM +0800, Anand Jain wrote: > >>> v3->v4: As we traverse through the seed device, fs_device gets > >>> updated with > >>> the child seed fs_devices, so make sure we use the same fs_devices > >>> pointer for the mutex_unlock as used for the mutex_lock. > >> >

Re: csum failed root raveled during balance

2018-05-29 Thread Austin S. Hemmelgarn
On 2018-05-28 13:10, ein wrote: On 05/23/2018 01:03 PM, Austin S. Hemmelgarn wrote: On 2018-05-23 06:09, ein wrote: On 05/23/2018 11:09 AM, Duncan wrote: ein posted on Wed, 23 May 2018 10:03:52 +0200 as excerpted: IMHO the best course of action would be to disable checksumming for you vm fil

Possible deadlock in btrfs_alloc_data_chunk_ondemand

2018-05-29 Thread shankarapailoor
Hi, I am fuzzing 4.17-rc4 with BTRFS using Syzkaller and found a possible deadlock. This crash has happened 3 times in an hour. Stack Trace: https://pastebin.com/1mjA9fzv Kernel Configs: https://pastebin.com/6J1KMczH I don't have a reproducer yet, unfortunately. Regards, Shankara Pailoor P.S

[PATCH v3] Btrfs: remove unused check of skip_locking

2018-05-29 Thread Liu Bo
The check is superfluous since all of callers who set search_for_commit also have skip_locking set. ASSERT() is put in place to ensure skip_locking is set by callers. Reviewed-by: Qu Wenruo Signed-off-by: Liu Bo --- v3: Add assertion and comments to ensure skip_locking is not forgot by callers.

Re: [PATCH v2 1/4] fstests: btrfs: add seed sprout functionality test

2018-05-29 Thread Eryu Guan
On Mon, May 28, 2018 at 05:51:45PM +0800, Anand Jain wrote: > Create a seed device and add the sprout device to it. > > Signed-off-by: Anand Jain This series looks fine to me from fstests' point of view, there're just some really minor common issues. But I'd like some reviews from other btrfs f

Re: csum failed root raveled during balance

2018-05-29 Thread ein
On 05/29/2018 02:12 PM, Austin S. Hemmelgarn wrote: > On 2018-05-28 13:10, ein wrote: >> On 05/23/2018 01:03 PM, Austin S. Hemmelgarn wrote: >>> On 2018-05-23 06:09, ein wrote: On 05/23/2018 11:09 AM, Duncan wrote: > ein posted on Wed, 23 May 2018 10:03:52 +0200 as excerpted: > >>>

Re: [PATCH 2/4] Btrfs: make should_defrag_range() understood compressed extents

2018-05-29 Thread Timofey Titovets
вт, 19 дек. 2017 г. в 13:02, Timofey Titovets : > Both, defrag ioctl and autodefrag - call btrfs_defrag_file() > for file defragmentation. > Kernel default target extent size - 256KiB. > Btrfs progs default - 32MiB. > Both bigger then maximum size of compressed extent - 128KiB. > Tha

Re: csum failed root raveled during balance

2018-05-29 Thread Austin S. Hemmelgarn
On 2018-05-29 10:02, ein wrote: On 05/29/2018 02:12 PM, Austin S. Hemmelgarn wrote: On 2018-05-28 13:10, ein wrote: On 05/23/2018 01:03 PM, Austin S. Hemmelgarn wrote: On 2018-05-23 06:09, ein wrote: On 05/23/2018 11:09 AM, Duncan wrote: ein posted on Wed, 23 May 2018 10:03:52 +0200 as excer

Re: [PATCH v3] Btrfs: remove unused check of skip_locking

2018-05-29 Thread David Sterba
On Tue, May 29, 2018 at 09:27:06PM +0800, Liu Bo wrote: > The check is superfluous since all of callers who set search_for_commit > also have skip_locking set. > > ASSERT() is put in place to ensure skip_locking is set by callers. > > Reviewed-by: Qu Wenruo > Signed-off-by: Liu Bo Added to the

Re: off-by-one uncompressed invalid ram_bytes corruptions

2018-05-29 Thread Steve Leung
Qu Wenruo writes: > On 2018年05月28日 11:47, Steve Leung wrote: >> On 05/26/2018 06:57 PM, Qu Wenruo wrote: >>> >>> >>> On 2018年05月26日 22:06, Steve Leung wrote: On 05/20/2018 07:07 PM, Qu Wenruo wrote: > > > On 2018年05月21日 04:43, Steve Leung wrote: >> On 05/19/2018 07:02 PM, Qu

Re: [PATCH v2] btrfs: balance dirty metadata pages in btrfs_finish_ordered_io

2018-05-29 Thread David Sterba
On Mon, May 28, 2018 at 01:48:20PM +0800, Ethan Lien wrote: > [Problem description and how we fix it] > We should balance dirty metadata pages at the end of > btrfs_finish_ordered_io, since a small, unmergeable random write can > potentially produce dirty metadata which is multiple times larger tha

Re: [PATCH] btrfs: set device state reappearing missing device

2018-05-29 Thread David Sterba
On Tue, May 29, 2018 at 02:50:09PM +0800, Gu Jinxiang wrote: > From: Gu JinXiang > > patch for commit 921518924084 > ("btrfs: handle dynamically reappearing missing device"). The commit is not in any of the development branches and I only found the patch in the mailinglist, so this is for someth

Re: [PATCH 0/3] btrfs: Delay block group auto removal to avoid interfering qgroups

2018-05-29 Thread David Sterba
On Mon, May 28, 2018 at 05:20:06PM +0800, Qu Wenruo wrote: > The patchset can be fetched from github: > https://github.com/adam900710/linux/tree/delayed_bg_removal > > It's based on v4.17-rc5 branch. Please refresh the patches on top of today's misc-next. The patch 2/3 has been there already and

Re: off-by-one uncompressed invalid ram_bytes corruptions

2018-05-29 Thread Hans van Kranenburg
On 05/21/2018 03:07 AM, Qu Wenruo wrote: > > [...] > > And the problem of wrong "root=" output also makes me pretty curious. Yeah, there a bunch of error messages in the kernel with always say root=1, regardless of the actual root the tree block is in. I think I once tried to find out why, but a

Re: [PATCH 1/2] btrfs: kill btrfs_write_inode

2018-05-29 Thread Omar Sandoval
On Mon, May 28, 2018 at 06:57:59PM +0200, David Sterba wrote: > On Tue, May 22, 2018 at 01:47:22PM -0400, Josef Bacik wrote: > > From: Josef Bacik > > > > We don't actually need this. It used to be in place for O_SYNC writes, > > but we've used the normal fsync() path for that for years now. Th

Re: [PATCH] Btrfs: clean up error handling in btrfs_truncate()

2018-05-29 Thread Omar Sandoval
On Tue, May 22, 2018 at 09:59:50AM -0700, Omar Sandoval wrote: > From: Omar Sandoval > > btrfs_truncate() uses two variables for error handling, ret and err (if > this sounds familiar, it's because btrfs_truncate_inode_items() did > something similar). This is error prone, as was made evident by

Re: [PATCH] btrfs-progs: check: Also compare data between mirrors to detect corruption for NODATASUM extents

2018-05-29 Thread james harvey
On Tue, May 29, 2018 at 3:45 AM, Qu Wenruo wrote: > As the lzo corruption reported by James Harvey, for data extents without > checksum, neither btrfs check nor kernel scrub could detect anything > wrong. > > However if our profile supports duplication, we still have a chance to > detect such corr

[PATCH 1/2] btrfs: returns -ENOMEM if path allocation failed in btrfs_cross_ref_exist()

2018-05-29 Thread Su Yue
Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 44093f92a532..99df6199dffc 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -3311,7 +3311,7 @@ int btrfs_cr

Re: [PATCH 1/2] btrfs: returns -ENOMEM if path allocation failed in btrfs_cross_ref_exist()

2018-05-29 Thread Su Yue
Sorry, the [PATCH 2/2] is abandoned. So the subject should be [PATCH]. On 05/30/2018 10:07 AM, Su Yue wrote: > Signed-off-by: Su Yue > --- > fs/btrfs/extent-tree.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c > index 4

[PATCH 2/2] btrfs: drop useless member qgroup_reserved of btrfs_pending_snapshot

2018-05-29 Thread Gu Jinxiang
From: Gu JinXiang Since there is no more use of qgroup_reserved member in struct btrfs_pending_snapshot, remove it. Signed-off-by: Gu JinXiang --- fs/btrfs/transaction.h | 1 - 1 file changed, 1 deletion(-) diff --git a/fs/btrfs/transaction.h b/fs/btrfs/transaction.h index d8c0826bc2c7..94439

[PATCH 1/2] btrfs: drop unused parameter qgroup_reserved

2018-05-29 Thread Gu Jinxiang
From: Gu JinXiang Since commit 7775c8184ec0 ("btrfs: remove unused parameter from btrfs_subvolume_release_metadata") parameter qgroup_reserved is not used by caller of function btrfs_subvolume_reserve_metadata. So remove it. Signed-off-by: Gu JinXiang --- fs/btrfs/ctree.h | 3 +-- fs/btr

[PATCH REBASED 0/2] btrfs: Delay block group auto removal to avoid interfering qgroups

2018-05-29 Thread Qu Wenruo
The patchset can be fetched from github: https://github.com/adam900710/linux/tree/delayed_bg_removal It's based on David's misc-next branch, HEAD is: commit c77a75861d82a497bd1a26054588cd9af59eb5db (david/misc-next) Author: Ethan Lien Date: Mon May 28 13:48:20 2018 +0800 btrfs: balance dir

[PATCH REBASED 1/2] btrfs: Use btrfs_mark_bg_unused() to replace open code

2018-05-29 Thread Qu Wenruo
Introduce a small helper, btrfs_mark_bg_unused(), to accquire needed locks and add a block group to unused_bgs list. No functional modification, and only 3 callers are involved. Signed-off-by: Qu Wenruo --- fs/btrfs/ctree.h | 1 + fs/btrfs/extent-tree.c | 36 +

[PATCH REBASED 2/2] btrfs: Delayed empty block group auto removal to next transaction

2018-05-29 Thread Qu Wenruo
Under certain KVM load and LTP tests, we are possible to hit the following calltrace if quota is enabled: -- BTRFS critical (device vda2): unable to find logical 8820195328 length 4096 BTRFS critical (device vda2): unable to find logical 8820195328 length 4096 [ cut here ]--

[PATCH 2/2] btrfs-progs: ins: dump-tree: Print tree name for tree block backref

2018-05-29 Thread Misono Tomohiro
Print tree name instead of number to make output more readable. Signed-off-by: Misono Tomohiro --- print-tree.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/print-tree.c b/print-tree.c index 90173c2b..8d551d09 100644 --- a/print-tree.c +++ b/print-tree.c @@ -470,8 +47

[PATCH 0/2] btrfs-progs: ins: dump-tree: Small updates for extent tree output

2018-05-29 Thread Misono Tomohiro
Two small updates for extent tree to make output more human readable. Misono Tomohiro (2): btrfs-progs: ins: dump-tree: Use %lld for extent data backref offset btrfs-progs: ins: dump-tree: Print tree name for tree block backref print-tree.c | 9 + 1 file changed, 5 insertions(+), 4 d

[PATCH 1/2] btrfs-progs: ins: dump-tree: Use %lld for extent data backref offset

2018-05-29 Thread Misono Tomohiro
Offset of extent data backref in extent item can be negative. So, let's use %lld insteand of %llu to output readable value. Signed-off-by: Misono Tomohiro --- print-tree.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/print-tree.c b/print-tree.c index a1a7954a..90173c2b

[PATCH v4 3/4] btrfs: lzo: Add header length check to avoid slab out of bounds access

2018-05-29 Thread Qu Wenruo
James Harvey reported that some corrupted compressed extent data can lead to various kernel memory corruption. Such corrupted extent data belongs to inode with NODATASUM flags, thus data csum won't help us detecting such bug. If lucky enough, kasan could catch it like: ===

Re: [PATCH v4 3/4] btrfs: lzo: Add header length check to avoid slab out of bounds access

2018-05-29 Thread Misono Tomohiro
Reviewed-by: Misono Tomohiro Thanks, Tomohiro Misono On 2018/05/30 13:58, Qu Wenruo wrote: > James Harvey reported that some corrupted compressed extent data can > lead to various kernel memory corruption. > > Such corrupted extent data belongs to inode with NODATASUM flags, thus > data csum wo

Re: [PATCH] btrfs: set device state reappearing missing device

2018-05-29 Thread Anand Jain
On 05/29/2018 02:50 PM, Gu Jinxiang wrote: From: Gu JinXiang patch for commit 921518924084 ("btrfs: handle dynamically reappearing missing device"). --- v4: :: . No need to set in_fs_metadata flag as its already set on missing device --- > Since BTRFS_DEV_STATE_IN_FS_METADATA will be clear

Re: [PATCH 1/2] btrfs: returns -ENOMEM if path allocation failed in btrfs_cross_ref_exist()

2018-05-29 Thread Nikolay Borisov
On 30.05.2018 05:07, Su Yue wrote: > Signed-off-by: Su Yue Generally we discourage commit log messages. I understand this patch is very obvious but put a sentence saying that btrfs_alloc_path can fail due to enomem hence returning enomem is the right ting to do here. > --- > fs/btrfs/extent-t

Re: [PATCH 2/2] btrfs-progs: ins: dump-tree: Print tree name for tree block backref

2018-05-29 Thread Nikolay Borisov
On 30.05.2018 07:55, Misono Tomohiro wrote: > Print tree name instead of number to make output more readable. > > Signed-off-by: Misono Tomohiro > --- > print-tree.c | 5 +++-- > 1 file changed, 3 insertions(+), 2 deletions(-) > > diff --git a/print-tree.c b/print-tree.c > index 90173c2b..8d

[PATCH v2] btrfs: returns -ENOMEM if path allocation failed in btrfs_cross_ref_exist()

2018-05-29 Thread Su Yue
btrfs_alloc_path() may fail due to no enough memory, so let the function return -ENOMEM instead of -ENOENT is better. Signed-off-by: Su Yue --- Changelog: v2: Add commit message to describe the patch. Thanks Nikolay. fs/btrfs/extent-tree.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(

Re: [PATCH v2] btrfs: returns -ENOMEM if path allocation failed in btrfs_cross_ref_exist()

2018-05-29 Thread Nikolay Borisov
On 30.05.2018 09:49, Su Yue wrote: > btrfs_alloc_path() may fail due to no enough memory, > so let the function return -ENOMEM instead of -ENOENT is better. > > Signed-off-by: Su Yue Reviewed-by: Nikolay Borisov > --- > Changelog: > v2: > Add commit message to describe the patch. Thanks Ni

Re: [PATCH REBASED 1/2] btrfs: Use btrfs_mark_bg_unused() to replace open code

2018-05-29 Thread Nikolay Borisov
On 30.05.2018 07:40, Qu Wenruo wrote: > Introduce a small helper, btrfs_mark_bg_unused(), to accquire needed > locks and add a block group to unused_bgs list. > > No functional modification, and only 3 callers are involved. > > Signed-off-by: Qu Wenruo Reviewed-by: Nikolay Borisov > --- >

[PATCH 0/6] fix device_list_mutex usage and cleanups in device_list_add()

2018-05-29 Thread Anand Jain
Patch 1/6 is a preparatory patch to add the required device_list_mutex in device_list_add() in the patch 2/6. Patch 3/6 updates the comments on device_list_mutex. Patch 4/6 is just a cleanup patch no functional changes. Patch 5/6 adds the required device_list_mutex in btrfs_free_stale_device().

[PATCH 2/6] btrfs: fix device_list_add() missing device_list_mutex()

2018-05-29 Thread Anand Jain
When the device is added or if the device pointer is retrieved for writing, make sure that device_list_mutex is held. Also make sure it is held when we check fs_devices::opened and fs_devices::total_devices is updated. Signed-off-by: Anand Jain --- fs/btrfs/volumes.c | 16 1 fil

[PATCH 1/6] btrfs: do btrfs_free_stale_devices() outside of device_list_add()

2018-05-29 Thread Anand Jain
btrfs_free_stale_devices() looks for device path reused for another fsid, and deletes the older fs_devices::device entry. In preparation to handle proper locks in the device_list_add(), move the btrfs_free_stale_devices() outside of the device_list_add() as these two functions has two distinct pur

[PATCH 3/6] btrfs: update device_list_mutex protected items

2018-05-29 Thread Anand Jain
Include items of fs_uuids::fs_devices to be protected by device_list_mutex, like (but not limited to) fs_devices::opened, fs_devices::total_devices and fs_devices::num_devices. This is been done in the code already. Signed-off-by: Anand Jain --- fs/btrfs/volumes.c | 3 ++- 1 file changed, 2 inse

[PATCH 5/6] btrfs: fix btrfs_free_stale_devices() with needed locks

2018-05-29 Thread Anand Jain
btrfs_free_stale_devices() finds the stale (not opened) matching device path in the fs_uuid list. We are already under uuid_mutex so when we check for each fs_devices, hold the respective device_list_mutex. Signed-off-by: Anand Jain --- fs/btrfs/volumes.c | 25 +++-- 1 file c

[PATCH 6/6] btrfs: rename btrfs_free_stale_devices drop btrfs prefix

2018-05-29 Thread Anand Jain
btrfs_free_stale_devices() is a static function drop the btrfs prefix. Signed-off-by: Anand Jain --- fs/btrfs/volumes.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index a6a23b0185e5..afb3504f4097 100644 --- a/fs/btrfs/volumes.c

[PATCH 4/6] btrfs: btrfs_free_stale_devices() rename local variables

2018-05-29 Thread Anand Jain
Over the years we named %fs_devices and %devices to represent, the struct btrfs_fs_devices and the struct btrfs_device. So follow the same norm here too. No functional changes. Signed-off-by: Anand Jain --- fs/btrfs/volumes.c | 35 ++- 1 file changed, 18 insertion

Re: [PATCH 2/2] btrfs-progs: ins: dump-tree: Print tree name for tree block backref

2018-05-29 Thread Misono Tomohiro
On 2018/05/30 15:28, Nikolay Borisov wrote: > > > On 30.05.2018 07:55, Misono Tomohiro wrote: >> Print tree name instead of number to make output more readable. >> >> Signed-off-by: Misono Tomohiro >> --- >> print-tree.c | 5 +++-- >> 1 file changed, 3 insertions(+), 2 deletions(-) >> >> diff -