Re: [PATCH] Btrfs: implement unlocked buffered write

2018-05-22 Thread Christoph Hellwig
On Tue, May 22, 2018 at 02:31:36PM -0400, Chris Mason wrote: > > And what protects two writes from interleaving their results now? > > page locks...ish, we at least won't have results interleaved in a single > page. For btrfs it'll actually be multiple pages since we try to do more > than one at

[PATCH v4 3/3] btrfs: balance: add kernel log for end or paused

2018-05-22 Thread Anand Jain
Add a kernel log when the balance ends, either for cancel or completed or if it is paused. Signed-off-by: Anand Jain --- v3->v4: nothing. v2->v3: nothing. v1->v2: Moved from 2/3 to 3/3 fs/btrfs/volumes.c | 7 +++ 1 file changed, 7 insertions(+) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/vol

[PATCH v4 1/3] btrfs: add helper function describe_block_group()

2018-05-22 Thread Anand Jain
Improve on describe_relocation() add a common helper function to describe the block groups. Signed-off-by: Anand Jain --- v3->v4: Just pass full flag name in the define DESCRIBE_FLAG(flag,..), so that it can be used at couple of more places. Rename describe_block_groups() to btrf

[PATCH v4 2/3] btrfs: balance: add args info during start and resume

2018-05-22 Thread Anand Jain
Balance arg info is an important information to be reviewed for the system audit. So this patch adds them to the kernel log. Example: ->btrfs bal start --full-balance -f -mprofiles=raid1,convert=single,soft -dlimit=10..20,usage=50 /btrfs kernel: BTRFS info (device sdc): balance: start -f -d us

Re: csum failed root raveled during balance

2018-05-22 Thread Nikolay Borisov
On 22.05.2018 23:05, ein wrote: > Hello devs, > > I tested BTRFS in production for about a month: > > 21:08:17 up 34 days, 2:21, 3 users, load average: 0.06, 0.02, 0.00 > > Without power blackout, hardware failure, SSD's SMART is flawless etc. > The tests ended with: > > root@node0:~# dmes

[PATCH v4 0/3] btrfs: balance: improve kernel logs

2018-05-22 Thread Anand Jain
v3->v4: Pls ref to individual patches. Based on misc-next. v2->v3: Inspried by describe_relocation(), improves it, makes it a helper function and use it to log the balance operations. Kernel logs are very important for the forensic investigations of the issues, these patchs make balance log

Re: [PATCH v2] btrfs: handle failures of set_extent_bits in add_excluded_extent

2018-05-22 Thread Nikolay Borisov
On 23.05.2018 06:37, Gu Jinxiang wrote: > set_extent_bits may fail, return the result in add_excluded_extent. > > Signed-off-by: Gu Jinxiang Reviewed-by: Nikolay Borisov > Changelog: > v2-v1: > 1.remove goto to make the function run linearly. > 2.change commit description not pointing out

Re: [PATCH v3 1/3] btrfs: add helper function describe_block_group()

2018-05-22 Thread Anand Jain
On 05/22/2018 08:26 PM, David Sterba wrote: On Mon, May 21, 2018 at 02:37:43PM +0800, Anand Jain wrote: Improve on describe_relocation() add a common helper function to describe the block groups. Signed-off-by: Anand Jain --- v3: Born. fs/btrfs/relocation.c | 30 +++---

Re: [PATCH v3 2/3] btrfs: balance: add args info during start and resume

2018-05-22 Thread Anand Jain
On 05/22/2018 08:35 PM, David Sterba wrote: On Mon, May 21, 2018 at 02:37:44PM +0800, Anand Jain wrote: Balance arg info is an important information to be reviewed for the system audit. So this patch adds them to the kernel log. Example: -> btrfs bal start -dprofiles='raid1|single',convert=r

Re: [PATCH 2/2] Btrfs: fix dedupe vs chattr NODATASUM race

2018-05-22 Thread Nikolay Borisov
On 23.05.2018 01:02, Omar Sandoval wrote: > From: Omar Sandoval > > In btrfs_extent_same(), we must check the NODATASUM flag while the > inodes are locked. Otherwise, it's possible that btrfs_ioctl_setflags() > will change the flags after we check. This was correct until a recent > change. > >

Re: [PATCH 1/2] Btrfs: fix clone vs chattr NODATASUM race

2018-05-22 Thread Nikolay Borisov
On 23.05.2018 01:02, Omar Sandoval wrote: > From: Omar Sandoval > > In btrfs_clone_files(), we must check the NODATASUM flag while the > inodes are locked. Otherwise, it's possible that btrfs_ioctl_setflags() > will change the flags after we check and we can end up with a party > checksummed fi

[PATCH v2] btrfs: handle failures of set_extent_bits in add_excluded_extent

2018-05-22 Thread Gu Jinxiang
set_extent_bits may fail, return the result in add_excluded_extent. Signed-off-by: Gu Jinxiang Changelog: v2-v1: 1.remove goto to make the function run linearly. 2.change commit description not pointing out the failure detail, since set_extent_bits's failure type may be added. --- fs/btrfs/

[PATCH v3] btrfs: drop uuid_mutex in btrfs_free_extra_devids()

2018-05-22 Thread Anand Jain
btrfs_free_extra_devids() is called only in the mount context which traverses through the fs_devices::devices and frees the orphan devices devices in the given %fs_devices if any. As the search for the orphan device is limited to fs_devices::devices so we don't need the global uuid_mutex. There ca

Re: [PATCH v2 0/6] btrfs_search_slot cleanups

2018-05-22 Thread Su Yue
On 05/22/2018 08:35 PM, Nikolay Borisov wrote: > > > On 22.05.2018 15:02, David Sterba wrote: >> On Tue, May 22, 2018 at 07:05:14PM +0800, Su Yue wrote: >>> Hi Liu and David, >>> During my local xfstests on kdave/for-next, btrfs/139 failed and >>> btrfs BUG_ON due to qgroup rescan. >>>

Re: [PATCH] Btrfs: fix memory and mount leak in btrfs_ioctl_rm_dev_v2()

2018-05-22 Thread Su Yue
On 05/23/2018 06:44 AM, Omar Sandoval wrote: > From: Omar Sandoval > > If we have invalid flags set, when we error out we must drop our writer > counter and free the buffer we allocated for the arguments. This bug is > trivially reproduced with the following program: > > #include >

Re: [PATCH v2 3/4] btrfs: lzo: Add header length check to avoid slab out of bounds access

2018-05-22 Thread Qu Wenruo
On 2018年05月22日 23:06, David Sterba wrote: > On Mon, May 21, 2018 at 01:19:26PM +0800, Qu Wenruo wrote: >> James Harvey reported that some corrupted compressed extent data can >> lead to various kernel memory corruption. >> >> Such corrupted extent data belongs to inode with NODATASUM flags, thus

[PATCH v3 1/5] xfstests: create swap group

2018-05-22 Thread Omar Sandoval
From: Omar Sandoval I'm going to add a bunch of tests for swap files, so create a group for them and add the existing tests. Signed-off-by: Omar Sandoval --- tests/generic/group | 4 ++-- tests/xfs/group | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/generic/g

[PATCH v3 5/5] generic: test invalid swap file activation

2018-05-22 Thread Omar Sandoval
From: Omar Sandoval Swap files cannot have holes, and they must at least two pages. swapon(8) and mkswap(8) have stricter restrictions, so add versions of those commands without any restrictions. Signed-off-by: Omar Sandoval --- .gitignore| 2 ++ src/Makefile | 2 +- src

[PATCH v3 0/5] xfstests: generic swap file tests

2018-05-22 Thread Omar Sandoval
From: Omar Sandoval Changes since v2: - Remove file before formatting in _format_swapfile - Comment on chattr in _format_swapfile - Remove unnecessary label - Fix typo in commit message of patch 3 - Redirect chattr +C output in patch 5 to fix XFS failures Omar Sandoval (5): xfstests: create s

[PATCH v3 3/5] generic: add test for dedupe on an active swapfile

2018-05-22 Thread Omar Sandoval
From: Omar Sandoval Similar to the reflink test generic/356, but makes sure we can't dedupe an active swapfile. Signed-off-by: Omar Sandoval --- tests/generic/492 | 76 +++ tests/generic/492.out | 7 tests/generic/group | 1 + 3 files change

[PATCH v3 4/5] generic: add test for truncate/fpunch of an active swapfile

2018-05-22 Thread Omar Sandoval
From: Omar Sandoval These should not be allowed. Signed-off-by: Omar Sandoval --- tests/generic/493 | 73 +++ tests/generic/493.out | 8 + tests/generic/group | 1 + 3 files changed, 82 insertions(+) create mode 100755 tests/generic/493 cre

[PATCH v3 2/5] generic: enable swapfile tests on Btrfs

2018-05-22 Thread Omar Sandoval
From: Omar Sandoval Commit 8c96cfbfe530 ("generic/35[67]: disable swapfile tests on Btrfs") disabled the swapfile tests on Btrfs because it did not support swapfiles at the time. Now that we're adding support, we want these tests to run, but they don't. _require_scratch_swapfile always fails for

Re: [PATCH v2 2/5] generic: enable swapfile tests on Btrfs

2018-05-22 Thread Omar Sandoval
On Tue, May 22, 2018 at 02:41:45PM +0800, Eryu Guan wrote: > On Wed, May 16, 2018 at 01:38:46PM -0700, Omar Sandoval wrote: > > From: Omar Sandoval > > > > Commit 8c96cfbfe530 ("generic/35[67]: disable swapfile tests on Btrfs") > > disabled the swapfile tests on Btrfs because it did not support >

[PATCH] Btrfs: fix memory and mount leak in btrfs_ioctl_rm_dev_v2()

2018-05-22 Thread Omar Sandoval
From: Omar Sandoval If we have invalid flags set, when we error out we must drop our writer counter and free the buffer we allocated for the arguments. This bug is trivially reproduced with the following program: #include #include #include #include #in

[PATCH 2/2] Btrfs: fix dedupe vs chattr NODATASUM race

2018-05-22 Thread Omar Sandoval
From: Omar Sandoval In btrfs_extent_same(), we must check the NODATASUM flag while the inodes are locked. Otherwise, it's possible that btrfs_ioctl_setflags() will change the flags after we check. This was correct until a recent change. Fixes: 0a38effca37c ("Btrfs: split btrfs_extent_same") Sign

[PATCH 1/2] Btrfs: fix clone vs chattr NODATASUM race

2018-05-22 Thread Omar Sandoval
From: Omar Sandoval In btrfs_clone_files(), we must check the NODATASUM flag while the inodes are locked. Otherwise, it's possible that btrfs_ioctl_setflags() will change the flags after we check and we can end up with a party checksummed file. Fixes: 0e7b824c4ef9 ("Btrfs: don't make a file part

[PATCH 0/2] Btrfs: fix partly checksummed file races

2018-05-22 Thread Omar Sandoval
From: Omar Sandoval Clone and dedupe both have checks to make sure that we're not mixing checksummed and not checksummed extents in a single file. However, both checks are racy; we need to have the inodes locked or else the flags can change after we check. The clone check has always been wrong. T

csum failed root raveled during balance

2018-05-22 Thread ein
Hello devs, I tested BTRFS in production for about a month: 21:08:17 up 34 days, 2:21, 3 users, load average: 0.06, 0.02, 0.00 Without power blackout, hardware failure, SSD's SMART is flawless etc. The tests ended with: root@node0:~# dmesg | grep BTRFS | grep warn 185:980:[2927472.393557] BT

Re: [PATCH 00/13] convert block layer to bioset_init()/mempool_init()

2018-05-22 Thread Mike Snitzer
On Tue, May 22 2018 at 2:41am -0400, Christoph Hellwig wrote: > On Mon, May 21, 2018 at 07:38:55PM -0400, Kent Overstreet wrote: > > On Mon, May 21, 2018 at 02:24:32PM -0400, Mike Snitzer wrote: > > > Every single data structure change in this series should be reviewed for > > > unforeseen align

Re: [RFC PATCH v2 00/12] get rid of GFP_ZONE_TABLE/BAD

2018-05-22 Thread Michal Hocko
On Mon 21-05-18 23:20:21, Huaisheng Ye wrote: > From: Huaisheng Ye > > Replace GFP_ZONE_TABLE and GFP_ZONE_BAD with encoded zone number. > > Delete ___GFP_DMA, ___GFP_HIGHMEM and ___GFP_DMA32 from GFP bitmasks, > the bottom three bits of GFP mask is reserved for storing encoded > zone number. >

Re: [PATCH] Btrfs: implement unlocked buffered write

2018-05-22 Thread Chris Mason
On 22 May 2018, at 14:08, Christoph Hellwig wrote: On Wed, May 16, 2018 at 11:52:37AM +0800, robbieko wrote: From: Robbie Ko This idea is from direct io. By this patch, we can make the buffered write parallel, and improve the performance and latency. But because we can not update isize witho

Re: [PATCH] Btrfs: implement unlocked buffered write

2018-05-22 Thread Christoph Hellwig
On Wed, May 16, 2018 at 11:52:37AM +0800, robbieko wrote: > From: Robbie Ko > > This idea is from direct io. By this patch, we can make the buffered > write parallel, and improve the performance and latency. But because we > can not update isize without i_mutex, the unlocked buffered write just >

Re: [PATCH 1/2] btrfs: kill btrfs_write_inode

2018-05-22 Thread Omar Sandoval
On Tue, May 22, 2018 at 01:47:22PM -0400, Josef Bacik wrote: > From: Josef Bacik > > We don't actually need this. It used to be in place for O_SYNC writes, > but we've used the normal fsync() path for that for years now. Specifically, I believe 148f948ba877 ("vfs: Introduce new helpers for sync

Re: [PATCH v2] Btrfs: fix error handling in btrfs_truncate()

2018-05-22 Thread Omar Sandoval
On Tue, May 22, 2018 at 10:41:11AM -0700, Omar Sandoval wrote: > On Tue, May 22, 2018 at 10:37:14AM -0700, Omar Sandoval wrote: > > On Tue, May 22, 2018 at 07:17:48PM +0200, David Sterba wrote: > > > On Tue, May 22, 2018 at 09:47:58AM -0700, Omar Sandoval wrote: > > > > From: Omar Sandoval > > > >

[PATCH 1/2] btrfs: kill btrfs_write_inode

2018-05-22 Thread Josef Bacik
From: Josef Bacik We don't actually need this. It used to be in place for O_SYNC writes, but we've used the normal fsync() path for that for years now. The other case we hit this is through sync(), which will commit the transaction anyway. All this does is make us commit the transaction a bunc

[PATCH 2/2] btrfs: always wait on ordered extents at fsync time

2018-05-22 Thread Josef Bacik
From: Josef Bacik There's a priority inversion that exists currently with btrfs fsync. In some cases we will collect outstanding ordered extents onto a list and only wait on them at the very last second. However this "very last second" falls inside of a transaction handle, so if we are in a low

Re: [PATCH v2] Btrfs: fix error handling in btrfs_truncate()

2018-05-22 Thread Omar Sandoval
On Tue, May 22, 2018 at 10:37:14AM -0700, Omar Sandoval wrote: > On Tue, May 22, 2018 at 07:17:48PM +0200, David Sterba wrote: > > On Tue, May 22, 2018 at 09:47:58AM -0700, Omar Sandoval wrote: > > > From: Omar Sandoval > > > > > > Jun Wu at Facebook reported that an internal service was seeing a

Re: [PATCH v2] Btrfs: fix error handling in btrfs_truncate()

2018-05-22 Thread Omar Sandoval
On Tue, May 22, 2018 at 07:17:48PM +0200, David Sterba wrote: > On Tue, May 22, 2018 at 09:47:58AM -0700, Omar Sandoval wrote: > > From: Omar Sandoval > > > > Jun Wu at Facebook reported that an internal service was seeing a return > > value of 1 from ftruncate() on Btrfs in some cases. > > Do y

Re: [PATCH] Btrfs: implement unlocked buffered write

2018-05-22 Thread David Sterba
On Wed, May 16, 2018 at 11:52:37AM +0800, robbieko wrote: > From: Robbie Ko > > This idea is from direct io. By this patch, we can make the buffered > write parallel, and improve the performance and latency. But because we > can not update isize without i_mutex, the unlocked buffered write just >

Re: [PATCH] Btrfs: implement unlocked buffered write

2018-05-22 Thread Omar Sandoval
On Wed, May 16, 2018 at 11:52:37AM +0800, robbieko wrote: > From: Robbie Ko > > This idea is from direct io. By this patch, we can make the buffered > write parallel, and improve the performance and latency. But because we > can not update isize without i_mutex, the unlocked buffered write just >

Re: [PATCH v2] Btrfs: fix error handling in btrfs_truncate()

2018-05-22 Thread David Sterba
On Tue, May 22, 2018 at 09:47:58AM -0700, Omar Sandoval wrote: > From: Omar Sandoval > > Jun Wu at Facebook reported that an internal service was seeing a return > value of 1 from ftruncate() on Btrfs in some cases. Do you have a reproducer? To estimate how likely is to hit the problem in practi

Re: [PATCH v2] btrfs: Skip some btrfs_cross_ref_exist() check in nocow path

2018-05-22 Thread David Sterba
On Thu, May 17, 2018 at 02:58:29PM +0800, Ethan Lien wrote: > In nocow path, we check if the extent is snapshotted in > btrfs_cross_ref_exist(). We can do the similar check earlier and avoid > unnecessary search into extent tree. > > A fio test on a Intel D-1531, 16GB RAM, SSD RAID-5 machine as fo

[PATCH] Btrfs: clean up error handling in btrfs_truncate()

2018-05-22 Thread Omar Sandoval
From: Omar Sandoval btrfs_truncate() uses two variables for error handling, ret and err (if this sounds familiar, it's because btrfs_truncate_inode_items() did something similar). This is error prone, as was made evident by "Btrfs: fix error handling in btrfs_truncate()". We only have err because

[PATCH v2] Btrfs: fix error handling in btrfs_truncate()

2018-05-22 Thread Omar Sandoval
From: Omar Sandoval Jun Wu at Facebook reported that an internal service was seeing a return value of 1 from ftruncate() on Btrfs in some cases. This is coming from the NEED_TRUNCATE_BLOCK return value from btrfs_truncate_inode_items(). btrfs_truncate() uses two variables for error handling, ret

Re: [PATCH] btrfs: balance dirty metadata pages in btrfs_finish_ordered_io

2018-05-22 Thread David Sterba
On Fri, Apr 27, 2018 at 03:05:24PM +0800, Ethan Lien wrote: > We should balance dirty metadata pages at the end of > btrfs_finish_ordered_io, since a small, unmergeable random write can > potentially produce dirty metadata which is multiple times larger than > the data itself. For example, a small,

Re: [PATCH 2/2] btrfs: propagate failures of __exclude_logged_extent to upper caller

2018-05-22 Thread David Sterba
On Tue, May 22, 2018 at 05:46:51PM +0800, Gu Jinxiang wrote: > Function btrfs_exclude_logged_extents may call __exclude_logged_extent > which may fail. > Propagate the failures of __exclude_logged_extent to upper caller. > > Signed-off-by: Gu Jinxiang Reviewed-by: David Sterba The whole group

Re: [PATCH 1/2] btrfs: handle failures of set_extent_bits in add_excluded_extent

2018-05-22 Thread David Sterba
On Tue, May 22, 2018 at 05:46:50PM +0800, Gu Jinxiang wrote: > set_extent_bits may return 0/-EEXIST, so return the result in > add_excluded_extent. This is misleading, set_extent_bits can return anything that gets propagated from the callees, which is 0 and -EEXIST for now but will be also -ENOMEM

Re: [PATCH 0/4] Cleanup alloc_reserved_tree_block signature

2018-05-22 Thread David Sterba
On Mon, May 21, 2018 at 12:27:19PM +0300, Nikolay Borisov wrote: > At the moment alloc_reserved_tree_block takes 8 friggin arguments. The irony > is that all of those are really derived from 3 structures. This series ends > up > reducing the arguments to 3. As a result some code, private to > a

Re: [PATCH 4/4] btrfs: Streamline shared ref check in alloc_reserved_tree_block

2018-05-22 Thread David Sterba
On Mon, May 21, 2018 at 12:27:23PM +0300, Nikolay Borisov wrote: > Instead of setting "parent" to ref->parent only when dealing with > a shared ref and subsequently performing another check to see > if (parent > 0), check the "node->type" directly and act accordingly. > This makes the code more str

Re: [PATCH 3/4] btrfs: Pass btrfs_delayed_extent_op to alloc_reserved_tree_block

2018-05-22 Thread David Sterba
On Mon, May 21, 2018 at 12:27:22PM +0300, Nikolay Borisov wrote: > Instead of taking only specific member of this structure, which results > in 2 extra arguments, just take the delayed_extent_op struct and > reference the arguments inside the functions. No functional changes. > > Signed-off-by: Ni

Re: [PATCH 2/4] btrfs: Simplify alloc_reserved_tree_block interface

2018-05-22 Thread David Sterba
On Mon, May 21, 2018 at 12:27:21PM +0300, Nikolay Borisov wrote: > This function currently takes 7 parameters, most of which are > proxies for values from btrfs_delayed_ref_node struct which is not > passed. This patch simplifies the interface of the function by > simply passing said delayed ref no

Re: [PATCH 1/4] btrfs: Remove fs_info argument from alloc_reserved_tree_block

2018-05-22 Thread David Sterba
On Mon, May 21, 2018 at 12:27:20PM +0300, Nikolay Borisov wrote: > This function already takes a transaction handle which contains a > reference to the fs_info. So use this and remove the extra argument. > > Signed-off-by: Nikolay Borisov Reviewed-by: David Sterba -- To unsubscribe from this lis

Re: [PATCH v2 3/4] btrfs: lzo: Add header length check to avoid slab out of bounds access

2018-05-22 Thread David Sterba
On Mon, May 21, 2018 at 01:19:26PM +0800, Qu Wenruo wrote: > James Harvey reported that some corrupted compressed extent data can > lead to various kernel memory corruption. > > Such corrupted extent data belongs to inode with NODATASUM flags, thus > data csum won't help us detecting such bug. >

Re: [PATCH 3/4] btrfs: lzo: Add header length check to avoid slab out of bounds access

2018-05-22 Thread David Sterba
On Thu, May 17, 2018 at 11:14:46AM +0300, Nikolay Borisov wrote: > srclen comes from the async_extent struct, which in turns is > initialized in compress_file_range with the value of "total_compressed", > and the value there is actually initialized by > btrfs_compress_pages->lzo_compress_pages (th

Re: [PATCH v2 2/4] btrfs: lzo: Add comment about the how btrfs records its lzo compressed data

2018-05-22 Thread David Sterba
On Mon, May 21, 2018 at 01:19:25PM +0800, Qu Wenruo wrote: > Although it's not that complex, but such comment could still save > several minutes for newer reader/reviewer. > > Signed-off-by: Qu Wenruo > --- > fs/btrfs/lzo.c | 23 +++ > 1 file changed, 23 insertions(+) > > di

Re: [PATCH] Btrfs: fix error handling in btrfs_truncate()

2018-05-22 Thread David Sterba
On Tue, May 22, 2018 at 03:11:04PM +0300, Nikolay Borisov wrote: > > > On 22.05.2018 14:53, David Sterba wrote: > > On Fri, May 18, 2018 at 02:43:02PM -0700, Omar Sandoval wrote: > >> From: Omar Sandoval > >> > >> Jun Wu at Facebook reported that an internal service was seeing a return > >> valu

Re: [PATCH v3 2/3] btrfs: balance: add args info during start and resume

2018-05-22 Thread David Sterba
On Mon, May 21, 2018 at 02:37:44PM +0800, Anand Jain wrote: > Balance arg info is an important information to be reviewed for the > system audit. So this patch adds them to the kernel log. > > Example: > > -> btrfs bal start -dprofiles='raid1|single',convert=raid5 > -mprofiles='raid1|single',con

Re: [PATCH v2 0/6] btrfs_search_slot cleanups

2018-05-22 Thread Nikolay Borisov
On 22.05.2018 15:02, David Sterba wrote: > On Tue, May 22, 2018 at 07:05:14PM +0800, Su Yue wrote: >> Hi Liu and David, >> During my local xfstests on kdave/for-next, btrfs/139 failed and >> btrfs BUG_ON due to qgroup rescan. >> The bisect result is commit 560215eb3f32("Merge branch >>

Re: [PATCH v3 1/3] btrfs: add helper function describe_block_group()

2018-05-22 Thread David Sterba
On Mon, May 21, 2018 at 02:37:43PM +0800, Anand Jain wrote: > Improve on describe_relocation() add a common helper function to describe > the block groups. > > Signed-off-by: Anand Jain > --- > v3: Born. > > fs/btrfs/relocation.c | 30 +++--- > fs/btrfs/volumes.c| 44

Re: [PATCH v2 0/6] btrfs_search_slot cleanups

2018-05-22 Thread David Sterba
On Tue, May 22, 2018 at 07:05:14PM +0800, Su Yue wrote: > Hi Liu and David, > During my local xfstests on kdave/for-next, btrfs/139 failed and > btrfs BUG_ON due to qgroup rescan. > The bisect result is commit 560215eb3f32("Merge branch > 'ext/liubo/search-cleanups-wip' into for-next-ne

Re: [PATCH v2] btrfs: Use btrfs_mark_bg_unused() to replace open code

2018-05-22 Thread David Sterba
On Tue, May 22, 2018 at 04:43:47PM +0800, Qu Wenruo wrote: > Introduce a small helper, btrfs_mark_bg_unused(), to accquire needed > locks and add a block group to unused_bgs list. The helper is nice but hides that there's a reference taken on the 'bg'. This would be good to add at least to the fun

Re: [PATCH] Btrfs: fix error handling in btrfs_truncate()

2018-05-22 Thread Nikolay Borisov
On 22.05.2018 14:53, David Sterba wrote: > On Fri, May 18, 2018 at 02:43:02PM -0700, Omar Sandoval wrote: >> From: Omar Sandoval >> >> Jun Wu at Facebook reported that an internal service was seeing a return >> value of 1 from ftruncate() on Btrfs when compression is enabled. This >> is coming f

Re: [PATCH] Btrfs: fix error handling in btrfs_truncate()

2018-05-22 Thread David Sterba
On Fri, May 18, 2018 at 02:43:02PM -0700, Omar Sandoval wrote: > From: Omar Sandoval > > Jun Wu at Facebook reported that an internal service was seeing a return > value of 1 from ftruncate() on Btrfs when compression is enabled. This > is coming from the NEED_TRUNCATE_BLOCK return value from > b

RE: [External] Re: [RFC PATCH v2 10/12] mm/zsmalloc: update usage of address zone modifiers

2018-05-22 Thread Huaisheng HS1 Ye
From: owner-linux...@kvack.org On Behalf Of Matthew Wilcox > > On Mon, May 21, 2018 at 11:20:31PM +0800, Huaisheng Ye wrote: > > @@ -343,7 +343,7 @@ static void destroy_cache(struct zs_pool *pool) > > static unsigned long cache_alloc_handle(struct zs_pool *pool, gfp_t gfp) > > { > > return (

Re: [RFC PATCH v2 10/12] mm/zsmalloc: update usage of address zone modifiers

2018-05-22 Thread Matthew Wilcox
On Mon, May 21, 2018 at 11:20:31PM +0800, Huaisheng Ye wrote: > @@ -343,7 +343,7 @@ static void destroy_cache(struct zs_pool *pool) > static unsigned long cache_alloc_handle(struct zs_pool *pool, gfp_t gfp) > { > return (unsigned long)kmem_cache_alloc(pool->handle_cachep, > -

Re: [PATCH] btrfs: Fix to use original error code of btrfs_read_fs_root_no_name()

2018-05-22 Thread David Sterba
On Mon, May 21, 2018 at 01:57:27PM +0900, Misono Tomohiro wrote: > btrfs_read_fs_root_no_name() may return ERR_PTR(-ENOENT) or > ERR_PTR(-ENOMEM) and therefore search_ioctl() and > btrfs_search_path_in_tree() should use PTR_ERR() instead of -ENOENT, > which all other callers of btrfs_read_fs_root_n

Re: [PATCH] btrfs: remove unnecessary if judge

2018-05-22 Thread David Sterba
On Mon, May 21, 2018 at 05:32:14PM +0800, Gu Jinxiang wrote: > Since add_excluded_extent always returns 0, > no need to judge ret. The whole calltree starting in this function needs to be examined, if there's a function that can retrun value (like set_extent_bits) or if there's a BUG_ON that needs

Re: [PATCH v2 0/6] btrfs_search_slot cleanups

2018-05-22 Thread Su Yue
Hi Liu and David, During my local xfstests on kdave/for-next, btrfs/139 failed and btrfs BUG_ON due to qgroup rescan. The bisect result is commit 560215eb3f32("Merge branch 'ext/liubo/search-cleanups-wip' into for-next-next-v4.18-20180521") which seems merged this patchset.

Re: [PATCH] Btrfs: allow empty subvol= again

2018-05-22 Thread David Sterba
> > This is silly but IMO it's a regression. If you agree, this should > > probably go to 4.17 + stable. I'll submit an xfstests test shortly. > > Update, we fixed the userspace bug now so I'd be fine if we dropped this > patch. I guess there's still a possibility that there are other users > that

Re: [PATCH 2/2] btrfs: propagate failures of __exclude_logged_extent to upper caller

2018-05-22 Thread Nikolay Borisov
On 22.05.2018 12:46, Gu Jinxiang wrote: > Function btrfs_exclude_logged_extents may call __exclude_logged_extent > which may fail. > Propagate the failures of __exclude_logged_extent to upper caller. > > Signed-off-by: Gu Jinxiang Reviewed-by: Nikolay Borisov > --- > fs/btrfs/extent-tree.c

Re: [PATCH 1/2] btrfs: handle failures of set_extent_bits in add_excluded_extent

2018-05-22 Thread Nikolay Borisov
On 22.05.2018 12:46, Gu Jinxiang wrote: > set_extent_bits may return 0/-EEXIST, so return the result in > add_excluded_extent. > > Signed-off-by: Gu Jinxiang Reviewed-by: Nikolay Borisov > --- > fs/btrfs/extent-tree.c | 11 --- > 1 file changed, 8 insertions(+), 3 deletions(-) > >

Re: [PATCH v2] btrfs: drop uuid_mutex in btrfs_free_extra_devids()

2018-05-22 Thread David Sterba
On Tue, May 22, 2018 at 05:29:29PM +0800, Anand Jain wrote: > btrfs_free_extra_devids() frees the orphan fsid::devid with in > btrfs_fs_devices::devices, so we dont need uuid_mutex, instead hold the > btrfs_fs_devices::device_list_mutex. This still lacks the reason why, the text merely repeats wha

Re: [RFC PATCH v2 00/12] get rid of GFP_ZONE_TABLE/BAD

2018-05-22 Thread Huaisheng HS1 Ye
From: owner-linux...@kvack.org On Behalf Of Christoph Hellwig > This seems to be missing patch 1 and generally be in somewhat odd format. > Can you try to resend it with git-send-email and against current Linus' > tree? > Sure, I will rebase them to current mainline ASAP. > Also I'd suggest you d

RE: [External] Re: [RFC PATCH v2 02/12] arch/x86/kernel/amd_gart_64: update usage of address zone modifiers

2018-05-22 Thread Huaisheng HS1 Ye
From: owner-linux...@kvack.org On Behalf Of Christoph Hellwig > > This code doesn't exist in current mainline. What kernel version > is your patch against? > > On Mon, May 21, 2018 at 11:20:23PM +0800, Huaisheng Ye wrote: > > From: Huaisheng Ye > > > > Use __GFP_ZONE_MASK to replace (__GFP_DMA

Re: [PATCH 01/12] block: convert bounce, q->bio_split to bioset_init()/mempool_init()

2018-05-22 Thread Christoph Hellwig
Looks good, Reviewed-by: Christoph Hellwig -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 08/12] target: convert to bioset_init()/mempool_init()

2018-05-22 Thread Christoph Hellwig
Looks good, Reviewed-by: Christoph Hellwig -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 09/12] fs: convert block_dev.c to bioset_init()

2018-05-22 Thread Christoph Hellwig
Looks good, Reviewed-by: Christoph Hellwig -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 11/12] xfs: convert to bioset_init()/mempool_init()

2018-05-22 Thread Christoph Hellwig
This creates a minor conflict with my iomap series, but that should be easy to merge if needed. Otherwise looks good: Reviewed-by: Christoph Hellwig -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info

Re: [PATCH 12/12] block: Drop bioset_create()

2018-05-22 Thread Christoph Hellwig
Looks good, Reviewed-by: Christoph Hellwig -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 04/12] lightnvm: convert to bioset_init()/mempool_init()

2018-05-22 Thread Javier Gonzalez
> On 21 May 2018, at 00.25, Kent Overstreet wrote: > > Signed-off-by: Kent Overstreet > --- > drivers/lightnvm/pblk-core.c | 30 ++--- > drivers/lightnvm/pblk-init.c | 72 > drivers/lightnvm/pblk-read.c | 4 +- > drivers/lightnvm/pblk-recovery.c

Re: [RFC PATCH 1/8] btrfs: use iocb for __btrfs_buffered_write

2018-05-22 Thread David Sterba
On Tue, May 22, 2018 at 03:40:18PM +0900, Misono Tomohiro wrote: > > @@ -1815,7 +1816,6 @@ static ssize_t __btrfs_direct_write(struct kiocb > > *iocb, struct iov_iter *from) > > { > > struct file *file = iocb->ki_filp; > > struct inode *inode = file_inode(file); > > - loff_t pos = iocb-

[PATCH 2/2] btrfs: propagate failures of __exclude_logged_extent to upper caller

2018-05-22 Thread Gu Jinxiang
Function btrfs_exclude_logged_extents may call __exclude_logged_extent which may fail. Propagate the failures of __exclude_logged_extent to upper caller. Signed-off-by: Gu Jinxiang --- fs/btrfs/extent-tree.c | 9 +++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/ex

[PATCH 1/2] btrfs: handle failures of set_extent_bits in add_excluded_extent

2018-05-22 Thread Gu Jinxiang
set_extent_bits may return 0/-EEXIST, so return the result in add_excluded_extent. Signed-off-by: Gu Jinxiang --- fs/btrfs/extent-tree.c | 11 --- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 75cfb80d2551..2e85e99b5e6

Re: [RFC PATCH v2 00/12] get rid of GFP_ZONE_TABLE/BAD

2018-05-22 Thread Christoph Hellwig
This seems to be missing patch 1 and generally be in somewhat odd format. Can you try to resend it with git-send-email and against current Linus' tree? Also I'd suggest you do cleanups like adding and using __GFP_ZONE_MASK at the beginning of the series before doing any real changes. -- To unsubsc

Re: [RFC PATCH v2 02/12] arch/x86/kernel/amd_gart_64: update usage of address zone modifiers

2018-05-22 Thread Christoph Hellwig
This code doesn't exist in current mainline. What kernel version is your patch against? On Mon, May 21, 2018 at 11:20:23PM +0800, Huaisheng Ye wrote: > From: Huaisheng Ye > > Use __GFP_ZONE_MASK to replace (__GFP_DMA | __GFP_HIGHMEM | __GFP_DMA32). > > ___GFP_DMA, ___GFP_HIGHMEM and ___GFP_DMA

[PATCH v2] btrfs: drop uuid_mutex in close_fs_devices()

2018-05-22 Thread Anand Jain
close_fs_devices() closes devices of a given fsid, and it is limited to all the devices of a fsid, so we don't have to hold the global uuid_mutex, instead we need the device_list_mutex as the device state is being changed. Signed-off-by: Anand Jain --- v1->v2: Add comment why we don't need uuid_m

[PATCH v2] btrfs: drop uuid_mutex in btrfs_free_extra_devids()

2018-05-22 Thread Anand Jain
btrfs_free_extra_devids() frees the orphan fsid::devid with in btrfs_fs_devices::devices, so we dont need uuid_mutex, instead hold the btrfs_fs_devices::device_list_mutex. Signed-off-by: Anand Jain --- v1->v2: replace uuid_mutex with device_list_mutex instead of delete. change log updated

Re: [PATCH 0/15] Review uuid_mutex usage

2018-05-22 Thread Anand Jain
On 05/16/2018 12:33 AM, David Sterba wrote: On Thu, Apr 12, 2018 at 10:29:23AM +0800, Anand Jain wrote: uuid_mutex lock is not a per-fs lock but a global lock. The main aim of this patch-set is to critically review the usage of this lock, and delete the unnecessary once. By doing this we impro

Re: [PATCH 12/15] btrfs: drop uuid_mutex in close_fs_devices()

2018-05-22 Thread Anand Jain
On 05/16/2018 12:30 AM, David Sterba wrote: On Thu, Apr 12, 2018 at 10:29:35AM +0800, Anand Jain wrote: close_fs_devices() closes devices of a given fsid, and it is limited to all the devices of a fsid, so we don't have to hold the global uuid_mutex, instead we need the device_list_mutex as th

Re: [PATCH 10/15] btrfs: drop uuid_mutex in btrfs_free_extra_devids()

2018-05-22 Thread Anand Jain
On 05/16/2018 12:26 AM, David Sterba wrote: On Thu, Apr 12, 2018 at 10:29:33AM +0800, Anand Jain wrote: btrfs_free_extra_devids() frees the orphan fsid::devid but its search is limited to btrfs_fs_devices::devices, so we dont need uuid_mutex. From that it's not clear why there's no locking

Re: [PATCH v2] btrfs: Use btrfs_mark_bg_unused() to replace open code

2018-05-22 Thread Nikolay Borisov
On 22.05.2018 11:43, Qu Wenruo wrote: > Introduce a small helper, btrfs_mark_bg_unused(), to accquire needed > locks and add a block group to unused_bgs list. > > No functional modification, and only 3 callers are involved.> > Signed-off-by: Qu Wenruo Reviewed-by: Nikolay Borisov > --- > Thi

[PATCH v2] btrfs: Use btrfs_mark_bg_unused() to replace open code

2018-05-22 Thread Qu Wenruo
Introduce a small helper, btrfs_mark_bg_unused(), to accquire needed locks and add a block group to unused_bgs list. No functional modification, and only 3 callers are involved. Signed-off-by: Qu Wenruo --- This patch should provide the basis for later block group auto-removal to get more info (

Re: [PATCH] btrfs: handle failures of set_extent_bits in add_excluded_extent

2018-05-22 Thread Nikolay Borisov
On 22.05.2018 06:51, Gu Jinxiang wrote: > set_extent_bits may return 0/-EEXIST, so return the result in > add_excluded_extent. And handle the failures in upper callers. > > Caller of add_excluded_extent and failure process currently: > exclude_super_stripes > <- btrfs_make_block_group //h

Re: [PATCH] btrfs: Use btrfs_add_unused_bgs() to replace open code

2018-05-22 Thread Qu Wenruo
On 2018年05月22日 15:49, Nikolay Borisov wrote: > > > On 22.05.2018 10:45, Qu Wenruo wrote: >> >> >> On 2018年05月22日 15:37, Nikolay Borisov wrote: >>> >>> >>> On 22.05.2018 10:29, Qu Wenruo wrote: Introduce a small helper, btrfs_add_unused_bgs(), to accquire needed >>> >>> This function name s

Re: [PATCH] btrfs: Use btrfs_add_unused_bgs() to replace open code

2018-05-22 Thread Nikolay Borisov
On 22.05.2018 10:45, Qu Wenruo wrote: > > > On 2018年05月22日 15:37, Nikolay Borisov wrote: >> >> >> On 22.05.2018 10:29, Qu Wenruo wrote: >>> Introduce a small helper, btrfs_add_unused_bgs(), to accquire needed >> >> This function name sounds a bit awkard, mainly because you use the >> plural for

Re: [PATCH] btrfs: Use btrfs_add_unused_bgs() to replace open code

2018-05-22 Thread Qu Wenruo
On 2018年05月22日 15:37, Nikolay Borisov wrote: > > > On 22.05.2018 10:29, Qu Wenruo wrote: >> Introduce a small helper, btrfs_add_unused_bgs(), to accquire needed > > This function name sounds a bit awkard, mainly because you use the > plural form. How about btrfs_mark_bg_unused() ? The name see

Re: [PATCH] btrfs: Use btrfs_add_unused_bgs() to replace open code

2018-05-22 Thread Nikolay Borisov
On 22.05.2018 10:29, Qu Wenruo wrote: > Introduce a small helper, btrfs_add_unused_bgs(), to accquire needed This function name sounds a bit awkard, mainly because you use the plural form. How about btrfs_mark_bg_unused() ? The name seems more unambiguous. > locks and add a block group to unuse

[PATCH] btrfs: Use btrfs_add_unused_bgs() to replace open code

2018-05-22 Thread Qu Wenruo
Introduce a small helper, btrfs_add_unused_bgs(), to accquire needed locks and add a block group to unused_bgs list. No functional modification, and only 3 callers are involved. Signed-off-by: Qu Wenruo --- This patch should provide the basis for later block group auto-removal to get more info (