[PATCH 5/9] btrfs-progs: Fix Wimplicit-fallthrough warning

2018-11-15 Thread Qu Wenruo
Although most fallthrough case is pretty obvious, we still need to teach the dumb compiler that it's an explicit fallthrough. Also reformat the code to use common indent. Signed-off-by: Qu Wenruo --- utils.c | 30 ++ 1 file changed, 22 insertions(+), 8 deletions(-)

[PATCH 2/9] btrfs-progs: fix gcc8 default build warning caused by '-Wformat-truncation'

2018-11-15 Thread Qu Wenruo
From: Su Yanjun When using gcc8 compiles utils.c, it complains as below: utils.c:852:45: warning: '%s' directive output may be truncated writing up to 4095 bytes into a region of size 4084 [-Wformat-truncation=] snprintf(path, sizeof(path), "/dev/mapper/%s", name);

[PATCH 8/9] btrfs-progs: Fix Wtype-limits warning

2018-11-15 Thread Qu Wenruo
The only hit is the following code: tlv_len = le16_to_cpu(tlv_hdr->tlv_len); if (tlv_type == 0 || tlv_type > BTRFS_SEND_A_MAX || tlv_len > BTRFS_SEND_BUF_SIZE) { error("invalid tlv in cmd tlv_type = %hu, tlv_len = %hu",

[PATCH 1/9] btrfs-progs: Makefile.extrawarn: Import cc-disable-warning

2018-11-15 Thread Qu Wenruo
We imported cc-option but forgot to import cc-disable-warning. Fixes: b556a992c3ad ("btrfs-progs: build: allow to build with various compiler warnings") Signed-off-by: Qu Wenruo --- Makefile.extrawarn | 6 ++ 1 file changed, 6 insertions(+) diff --git a/Makefile.extrawarn b/Makefile.extraw

[PATCH 0/9] btrfs-progs: Make W=1 clean (no "again")

2018-11-15 Thread Qu Wenruo
This patchset can be fetched from github: https://github.com/adam900710/btrfs-progs/tree/warning_fixes Which is based on v4.19 tag. This patchset will make "make W=1" reports no warning. This patch will first introduce fix to Makefile.extrawarn to make "cc-disable-warning" works, then disable sig

[PATCH 7/9] btrfs-progs: Fix Wmaybe-uninitialized warning

2018-11-15 Thread Qu Wenruo
The only location is the following code: int level = path->lowest_level + 1; BUG_ON(path->lowest_level + 1 >= BTRFS_MAX_LEVEL); while(level < BTRFS_MAX_LEVEL) { slot = path->slots[level] + 1; ... } path->slots[level] = slot;

[PATCH 6/9] btrfs-progs: Fix Wsuggest-attribute=format warning

2018-11-15 Thread Qu Wenruo
Add __attribute__ ((format (printf, 4, 0))) to fix the vprintf calling function. Signed-off-by: Qu Wenruo --- string-table.c | 1 + 1 file changed, 1 insertion(+) diff --git a/string-table.c b/string-table.c index 95833768960d..455285702d51 100644 --- a/string-table.c +++ b/string-table.c @@ -4

[PATCH 3/9] btrfs-progs: Makefile.extrawarn: Don't warn on sign compare

2018-11-15 Thread Qu Wenruo
Under most case, we are just using 'int' for 'unsigned int', and doesn't care about the sign. The Wsign-compare is causing tons of false alerts. Suppressing it would make W=1 less noisy. Signed-off-by: Qu Wenruo --- Makefile.extrawarn | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff

[PATCH 4/9] btrfs-progs: Fix Wempty-body warning

2018-11-15 Thread Qu Wenruo
messages.h:49:24: warning: suggest braces around empty body in an 'if' statement [-Wempty-body] PRINT_TRACE_ON_ERROR;\ Just extra braces would solve the problem. Signed-off-by: Qu Wenruo --- messages.h | 15 ++- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a

[PATCH 9/9] btrfs-progs: Cleanup warning reported by -Wmissing-prototypes except free space tree

2018-11-15 Thread Qu Wenruo
The following missing prototypes will be fixed: 1) btrfs.c::handle_special_globals() 2) check/mode-lowmem.c::repair_ternary_lowmem() 3) extent-tree.c::btrfs_search_overlap_extent() Above 3 can be fixed by making them static 4) utils.c::btrfs_check_nodesize() Fixed by moving it to fsfea

Re: [PATCH] btrfs-progs: fix gcc8 default build warning caused by '-Wformat-truncation'

2018-11-15 Thread Qu Wenruo
On 2018/10/26 上午9:58, Su Yanjun wrote: > When using gcc8 compiles utils.c, it complains as below: > > utils.c:852:45: warning: '%s' directive output may be truncated writing > up to 4095 bytes into a region of size 4084 [-Wformat-truncation=] >snprintf(path, sizeof(path), "/dev/mapper/%s", n

Re: [PATCH] btrfs-progs: Cleanup warning reported by -Wmissing-prototypes except free space tree

2018-11-15 Thread Qu Wenruo
On 2018/11/16 下午2:39, Nikolay Borisov wrote: > > > On 16.11.18 г. 8:13 ч., Qu Wenruo wrote: >> The following missing prototypes will be fixed: >> 1) btrfs.c::handle_special_globals() >> 2) check/mode-lowmem.c::repair_ternary_lowmem() >> 3) extent-tree.c::btrfs_search_overlap_extent() >>Abo

Re: [PATCH] btrfs-progs: Cleanup warning reported by -Wmissing-prototypes except free space tree

2018-11-15 Thread Nikolay Borisov
On 16.11.18 г. 8:13 ч., Qu Wenruo wrote: > The following missing prototypes will be fixed: > 1) btrfs.c::handle_special_globals() > 2) check/mode-lowmem.c::repair_ternary_lowmem() > 3) extent-tree.c::btrfs_search_overlap_extent() >Above 3 can be fixed by making them static > > 4) utils.c::b

Re: [PATCH 5/9] btrfs: replace cancel is successful if scrub cancel is successful

2018-11-15 Thread Anand Jain
On 11/15/2018 11:27 PM, David Sterba wrote: On Sun, Nov 11, 2018 at 10:22:20PM +0800, Anand Jain wrote: In btrfs_dev_replace_cancel() we should check if the btrfs_scrub_cancel() is successful. If the btrfs_scrub_cancel() fails, return BTRFS_IOCTL_DEV_REPLACE_RESULT_NOT_STARTED so that user ca

Re: [PATCH 4/9] btrfs: fix UAF due to race between replace start and cancel

2018-11-15 Thread Anand Jain
On 11/15/2018 11:25 PM, David Sterba wrote: On Thu, Nov 15, 2018 at 03:00:21PM +0100, David Sterba wrote: On Wed, Nov 14, 2018 at 09:28:34AM +0800, Anand Jain wrote: mutex_unlock(&dev_replace->lock_finishing_cancel_unmount); return result; There's a compiler warning: fs/bt

[PATCH] btrfs-progs: Cleanup warning reported by -Wmissing-prototypes except free space tree

2018-11-15 Thread Qu Wenruo
The following missing prototypes will be fixed: 1) btrfs.c::handle_special_globals() 2) check/mode-lowmem.c::repair_ternary_lowmem() 3) extent-tree.c::btrfs_search_overlap_extent() Above 3 can be fixed by making them static 4) utils.c::btrfs_check_nodesize() Fixed by moving it to fsfeatures.

Re: [PATCH V10 18/19] block: kill QUEUE_FLAG_NO_SG_MERGE

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:53:05PM +0800, Ming Lei wrote: > Since bdced438acd83ad83a6c ("block: setup bi_phys_segments after splitting"), > physical segment number is mainly figured out in blk_queue_split() for > fast path, and the flag of BIO_SEG_VALID is set there too. > > Now only blk_recount_s

Re: [PATCH V10 17/19] block: don't use bio->bi_vcnt to figure out segment number

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:53:04PM +0800, Ming Lei wrote: > It is wrong to use bio->bi_vcnt to figure out how many segments > there are in the bio even though CLONED flag isn't set on this bio, > because this bio may be splitted or advanced. > > So always use bio_segments() in blk_recount_segments

Re: [PATCH V10 16/19] block: document usage of bio iterator helpers

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:53:03PM +0800, Ming Lei wrote: > Now multi-page bvec is supported, some helpers may return page by > page, meantime some may return segment by segment, this patch > documents the usage. > > Cc: Dave Chinner > Cc: Kent Overstreet > Cc: Mike Snitzer > Cc: dm-de...@redha

Re: [PATCH V10 15/19] block: always define BIO_MAX_PAGES as 256

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:53:02PM +0800, Ming Lei wrote: > Now multi-page bvec can cover CONFIG_THP_SWAP, so we don't need to > increase BIO_MAX_PAGES for it. You mentioned to it in the cover letter, but this needs more explanation in the commit message. Why did CONFIG_THP_SWAP require > 256? Why

Re: [PATCH V10 14/19] block: enable multipage bvecs

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:53:01PM +0800, Ming Lei wrote: > This patch pulls the trigger for multi-page bvecs. > > Now any request queue which supports queue cluster will see multi-page > bvecs. > > Cc: Dave Chinner > Cc: Kent Overstreet > Cc: Mike Snitzer > Cc: dm-de...@redhat.com > Cc: Alexa

Re: [PATCH V10 13/19] iomap & xfs: only account for new added page

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:53:00PM +0800, Ming Lei wrote: > After multi-page is enabled, one new page may be merged to a segment > even though it is a new added page. > > This patch deals with this issue by post-check in case of merge, and > only a freshly new added page need to be dealt with for

Re: [PATCH v2] btrfs: add zstd compression level support

2018-11-15 Thread Nick Terrell
> On Nov 13, 2018, at 5:29 AM, Timofey Titovets wrote: > > вт, 13 нояб. 2018 г. в 04:52, Nick Terrell : >> >> >> >>> On Nov 12, 2018, at 4:33 PM, David Sterba wrote: >>> >>> On Wed, Oct 31, 2018 at 11:11:08AM -0700, Nick Terrell wrote: From: Jennifer Liu Adds zstd compress

Re: [PATCH V10 12/19] block: allow bio_for_each_segment_all() to iterate over multi-page bvec

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:52:59PM +0800, Ming Lei wrote: > This patch introduces one extra iterator variable to > bio_for_each_segment_all(), > then we can allow bio_for_each_segment_all() to iterate over multi-page bvec. > > Given it is just one mechannical & simple change on all > bio_for_eac

Re: [PATCH V10 11/19] bcache: avoid to use bio_for_each_segment_all() in bch_bio_alloc_pages()

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:52:58PM +0800, Ming Lei wrote: > bch_bio_alloc_pages() is always called on one new bio, so it is safe > to access the bvec table directly. Given it is the only kind of this > case, open code the bvec table access since bio_for_each_segment_all() > will be changed to suppo

Re: [PATCH V10 10/19] block: loop: pass multi-page bvec to iov_iter

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:52:57PM +0800, Ming Lei wrote: > iov_iter is implemented with bvec itererator, so it is safe to pass > multipage bvec to it, and this way is much more efficient than > passing one page in each bvec. > > Cc: Dave Chinner > Cc: Kent Overstreet > Cc: Mike Snitzer > Cc: d

Re: [PATCH V10 09/19] block: introduce bio_bvecs()

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:52:56PM +0800, Ming Lei wrote: > There are still cases in which we need to use bio_bvecs() for get the > number of multi-page segment, so introduce it. > > Cc: Dave Chinner > Cc: Kent Overstreet > Cc: Mike Snitzer > Cc: dm-de...@redhat.com > Cc: Alexander Viro > Cc:

Re: [PATCH V10 08/19] btrfs: move bio_pages_all() to btrfs

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:52:55PM +0800, Ming Lei wrote: > BTRFS is the only user of this helper, so move this helper into > BTRFS, and implement it via bio_for_each_segment_all(), since > bio->bi_vcnt may not equal to number of pages after multipage bvec > is enabled. Shouldn't you also get rid

Re: [PATCH V10 07/19] btrfs: use bvec_last_segment to get bio's last page

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:52:54PM +0800, Ming Lei wrote: > Preparing for supporting multi-page bvec. > > Cc: Dave Chinner > Cc: Kent Overstreet > Cc: Mike Snitzer > Cc: dm-de...@redhat.com > Cc: Alexander Viro > Cc: linux-fsde...@vger.kernel.org > Cc: Shaohua Li > Cc: linux-r...@vger.kernel.

Re: [PATCH V10 06/19] fs/buffer.c: use bvec iterator to truncate the bio

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:52:53PM +0800, Ming Lei wrote: > Once multi-page bvec is enabled, the last bvec may include more than one > page, this patch use bvec_last_segment() to truncate the bio. > > Cc: Dave Chinner > Cc: Kent Overstreet > Cc: Mike Snitzer > Cc: dm-de...@redhat.com > Cc: Alex

Re: [PATCH V10 05/19] block: introduce bvec_last_segment()

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:52:52PM +0800, Ming Lei wrote: > BTRFS and guard_bio_eod() need to get the last singlepage segment > from one multipage bvec, so introduce this helper to make them happy. > > Cc: Dave Chinner > Cc: Kent Overstreet > Cc: Mike Snitzer > Cc: dm-de...@redhat.com > Cc: Ale

[PATCH v6 2/2] btrfs: Add zstd support to grub btrfs

2018-11-15 Thread Nick Terrell
- Adds zstd support to the btrfs module. - Adds a test case for btrfs zstd support. - Changes top_srcdir to srcdir in the btrfs module's lzo include following comments from Daniel Kiper about the zstd include. Tested on Ubuntu-18.04 with a btrfs /boot partition with and without zstd compression.

[PATCH v6 0/2] btrfs: Add zstd support to grub btrfs

2018-11-15 Thread Nick Terrell
Hi all, This patch set imports the upstream zstd library, adds zstd support to the btrfs module, and adds a test case. I've also tested the patch set by storing my boot partition in btrfs with and without zstd compression and rebooting. Best, Nick Terrell Changelog: v1 -> v2: - Switch to upstre

Re: [PATCH V10 04/19] block: use bio_for_each_bvec() to map sg

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:52:51PM +0800, Ming Lei wrote: > It is more efficient to use bio_for_each_bvec() to map sg, meantime > we have to consider splitting multipage bvec as done in > blk_bio_segment_split(). > > Cc: Dave Chinner > Cc: Kent Overstreet > Cc: Mike Snitzer > Cc: dm-de...@redh

Re: [PATCH V10 03/19] block: use bio_for_each_bvec() to compute multi-page bvec count

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:05:10PM -0500, Mike Snitzer wrote: > On Thu, Nov 15 2018 at 3:20pm -0500, > Omar Sandoval wrote: > > > On Thu, Nov 15, 2018 at 04:52:50PM +0800, Ming Lei wrote: > > > First it is more efficient to use bio_for_each_bvec() in both > > > blk_bio_segment_split() and __blk_

Re: [PATCH V10 03/19] block: use bio_for_each_bvec() to compute multi-page bvec count

2018-11-15 Thread Mike Snitzer
On Thu, Nov 15 2018 at 3:20pm -0500, Omar Sandoval wrote: > On Thu, Nov 15, 2018 at 04:52:50PM +0800, Ming Lei wrote: > > First it is more efficient to use bio_for_each_bvec() in both > > blk_bio_segment_split() and __blk_recalc_rq_segments() to compute how > > many multi-page bvecs there are in

Re: [PATCH V10 03/19] block: use bio_for_each_bvec() to compute multi-page bvec count

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:52:50PM +0800, Ming Lei wrote: > First it is more efficient to use bio_for_each_bvec() in both > blk_bio_segment_split() and __blk_recalc_rq_segments() to compute how > many multi-page bvecs there are in the bio. > > Secondly once bio_for_each_bvec() is used, the bvec ma

Re: BTRFS on production: NVR 16+ IP Cameras

2018-11-15 Thread Roman Mamedov
On Thu, 15 Nov 2018 11:39:58 -0700 Juan Alberto Cirez wrote: > Is BTRFS mature enough to be deployed on a production system to underpin > the storage layer of a 16+ ipcameras-based NVR (or VMS if you prefer)? What are you looking to gain from using Btrfs on an NVR system? It doesn't sound like

Re: BTRFS on production: NVR 16+ IP Cameras

2018-11-15 Thread Nikolay Borisov
On 15.11.18 г. 20:39 ч., Juan Alberto Cirez wrote: > Is BTRFS mature enough to be deployed on a production system to underpin > the storage layer of a 16+ ipcameras-based NVR (or VMS if you prefer)? > > Based on our limited experience with BTRFS (1+ year) under the above > scenario the answer s

BTRFS on production: NVR 16+ IP Cameras

2018-11-15 Thread Juan Alberto Cirez
Is BTRFS mature enough to be deployed on a production system to underpin the storage layer of a 16+ ipcameras-based NVR (or VMS if you prefer)? Based on our limited experience with BTRFS (1+ year) under the above scenario the answer seems to be no; but I wanted you ask the community at large f

Re: [PATCH V10 02/19] block: introduce bio_for_each_bvec()

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:52:49PM +0800, Ming Lei wrote: > This helper is used for iterating over multi-page bvec for bio > split & merge code. > > Cc: Dave Chinner > Cc: Kent Overstreet > Cc: Mike Snitzer > Cc: dm-de...@redhat.com > Cc: Alexander Viro > Cc: linux-fsde...@vger.kernel.org > Cc

Re: [PATCH V10 01/19] block: introduce multi-page page bvec helpers

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:52:48PM +0800, Ming Lei wrote: > This patch introduces helpers of 'mp_bvec_iter_*' for multipage > bvec support. > > The introduced helpers treate one bvec as real multi-page segment, > which may include more than one pages. > > The existed helpers of bvec_iter_* are in

general protection fault in btrfs_scan_one_device

2018-11-15 Thread syzbot
Hello, syzbot found the following crash on: HEAD commit:442b8cea2477 Add linux-next specific files for 20181109 git tree: linux-next console output: https://syzkaller.appspot.com/x/log.txt?x=173046a340 kernel config: https://syzkaller.appspot.com/x/.config?x=2f72bdb11df9fbe8 dashb

Re: [PATCH] Btrfs: send, fix infinite loop due to directory rename dependencies

2018-11-15 Thread David Sterba
On Wed, Nov 14, 2018 at 06:32:37PM +, fdman...@kernel.org wrote: > From: Robbie Ko > > When doing an incremental send, due to the need of delaying directory move > (rename) operations we can end up in infinite loop at > apply_children_dir_moves(). > > An example scenario that triggers this p

Re: [PATCH 0/9 v2] fix replace-start and replace-cancel racing

2018-11-15 Thread David Sterba
On Sun, Nov 11, 2018 at 10:22:15PM +0800, Anand Jain wrote: > v1->v2: > 2/9: Drop writeback required > 3/9: Drop writeback required > 7/9: Use the condition within the WARN_ON() > 6/9: Use the condition within the ASSERT() > > Replace-start and replace-cancel threads can race to create a m

Re: [PATCH 7/9] btrfs: quiten warn if the replace is canceled at finish

2018-11-15 Thread David Sterba
On Sun, Nov 11, 2018 at 10:22:22PM +0800, Anand Jain wrote: > When we successfully cancel the replace its scrub returns -ECANCELED, > which then passed to btrfs_dev_replace_finishing(), it cleans up based > on the scrub returned status and propagates the same -ECANCELED back > the parent function.

Re: [PATCH 8/9] btrfs: user requsted replace cancel is not an error

2018-11-15 Thread David Sterba
On Sun, Nov 11, 2018 at 10:22:23PM +0800, Anand Jain wrote: > As of now only user requested replace cancel can cancel the replace-scrub > so no need to log error for it. This has probably some user visible effect or threre are steps to reproduce the message even if it's not expected (ie. the case

Re: [PATCH 5/9] btrfs: replace cancel is successful if scrub cancel is successful

2018-11-15 Thread David Sterba
On Sun, Nov 11, 2018 at 10:22:20PM +0800, Anand Jain wrote: > In btrfs_dev_replace_cancel() we should check if the > btrfs_scrub_cancel() is successful. If the btrfs_scrub_cancel() fails, return > BTRFS_IOCTL_DEV_REPLACE_RESULT_NOT_STARTED so that user can try to > cancel the replace again. I've u

Re: [PATCH 4/9] btrfs: fix UAF due to race between replace start and cancel

2018-11-15 Thread David Sterba
On Thu, Nov 15, 2018 at 03:00:21PM +0100, David Sterba wrote: > On Wed, Nov 14, 2018 at 09:28:34AM +0800, Anand Jain wrote: > > >> mutex_unlock(&dev_replace->lock_finishing_cancel_unmount); > > >> return result; > > > > > > There's a compiler warning: > > > > > > fs/btrfs/dev-re

Re: [PATCH 4/9] btrfs: fix UAF due to race between replace start and cancel

2018-11-15 Thread David Sterba
On Wed, Nov 14, 2018 at 09:28:34AM +0800, Anand Jain wrote: > >>mutex_unlock(&dev_replace->lock_finishing_cancel_unmount); > >>return result; > > > > There's a compiler warning: > > > > fs/btrfs/dev-replace.c: In function ‘btrfs_dev_replace_cancel’: > > fs/btrfs/dev-replace.c:865:9: warni

Re: [PATCH V10 12/19] block: allow bio_for_each_segment_all() to iterate over multi-page bvec

2018-11-15 Thread David Sterba
On Thu, Nov 15, 2018 at 04:52:59PM +0800, Ming Lei wrote: > diff --git a/block/blk-zoned.c b/block/blk-zoned.c > index 13ba2011a306..789b09ae402a 100644 > --- a/block/blk-zoned.c > +++ b/block/blk-zoned.c > @@ -123,6 +123,7 @@ static int blk_report_zones(struct gendisk *disk, > sector_t sector, >

Re: [PATCH v2 0/6] btrfs: qgroup: Delay subtree scan to reduce overhead

2018-11-15 Thread David Sterba
On Thu, Nov 15, 2018 at 01:23:25PM +0800, Qu Wenruo wrote: > > > On 2018/11/15 上午3:05, David Sterba wrote: > > On Tue, Nov 13, 2018 at 05:58:14PM +, Filipe Manana wrote: > >> That's the infinite loop issue fixed by one of the patches submitted > >> for 4.20-rc2: > >> > >> https://git.kernel.o

Re: [PATCH 0/6] btrfs: fix compiler warning with make W=1

2018-11-15 Thread David Sterba
On Thu, Nov 15, 2018 at 09:30:42AM +0800, Qu Wenruo wrote: > BTW, could we just make "-Wunused-but-set-variable" default for btrfs > module instead of following the global setting? We'd need to fix all the warnings first, there are more left. We could make it on by default for the debugging build

[PATCH V10 19/19] block: kill BLK_MQ_F_SG_MERGE

2018-11-15 Thread Ming Lei
QUEUE_FLAG_NO_SG_MERGE has been killed, so kill BLK_MQ_F_SG_MERGE too. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-de...@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org Cc: Shaohua Li Cc: linux-r...@vger.kernel.org Cc: linux-er...@lists.ozlabs.org Cc: David St

[PATCH V10 18/19] block: kill QUEUE_FLAG_NO_SG_MERGE

2018-11-15 Thread Ming Lei
Since bdced438acd83ad83a6c ("block: setup bi_phys_segments after splitting"), physical segment number is mainly figured out in blk_queue_split() for fast path, and the flag of BIO_SEG_VALID is set there too. Now only blk_recount_segments() and blk_recalc_rq_segments() use this flag. Basically blk

[PATCH V10 15/19] block: always define BIO_MAX_PAGES as 256

2018-11-15 Thread Ming Lei
Now multi-page bvec can cover CONFIG_THP_SWAP, so we don't need to increase BIO_MAX_PAGES for it. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-de...@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org Cc: Shaohua Li Cc: linux-r...@vger.kernel.org Cc: linux-er...@li

[PATCH V10 17/19] block: don't use bio->bi_vcnt to figure out segment number

2018-11-15 Thread Ming Lei
It is wrong to use bio->bi_vcnt to figure out how many segments there are in the bio even though CLONED flag isn't set on this bio, because this bio may be splitted or advanced. So always use bio_segments() in blk_recount_segments(), and it shouldn't cause any performance loss now because the phys

[PATCH V10 16/19] block: document usage of bio iterator helpers

2018-11-15 Thread Ming Lei
Now multi-page bvec is supported, some helpers may return page by page, meantime some may return segment by segment, this patch documents the usage. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-de...@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org Cc: Shaohua Li

[PATCH V10 14/19] block: enable multipage bvecs

2018-11-15 Thread Ming Lei
This patch pulls the trigger for multi-page bvecs. Now any request queue which supports queue cluster will see multi-page bvecs. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-de...@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org Cc: Shaohua Li Cc: linux-r...@vg

[PATCH V10 12/19] block: allow bio_for_each_segment_all() to iterate over multi-page bvec

2018-11-15 Thread Ming Lei
This patch introduces one extra iterator variable to bio_for_each_segment_all(), then we can allow bio_for_each_segment_all() to iterate over multi-page bvec. Given it is just one mechannical & simple change on all bio_for_each_segment_all() users, this patch does tree-wide change in one single p

[PATCH V10 13/19] iomap & xfs: only account for new added page

2018-11-15 Thread Ming Lei
After multi-page is enabled, one new page may be merged to a segment even though it is a new added page. This patch deals with this issue by post-check in case of merge, and only a freshly new added page need to be dealt with for iomap & xfs. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitze

[PATCH V10 10/19] block: loop: pass multi-page bvec to iov_iter

2018-11-15 Thread Ming Lei
iov_iter is implemented with bvec itererator, so it is safe to pass multipage bvec to it, and this way is much more efficient than passing one page in each bvec. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-de...@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org C

[PATCH V10 11/19] bcache: avoid to use bio_for_each_segment_all() in bch_bio_alloc_pages()

2018-11-15 Thread Ming Lei
bch_bio_alloc_pages() is always called on one new bio, so it is safe to access the bvec table directly. Given it is the only kind of this case, open code the bvec table access since bio_for_each_segment_all() will be changed to support for iterating over multipage bvec. Cc: Dave Chinner Cc: Kent

[PATCH V10 09/19] block: introduce bio_bvecs()

2018-11-15 Thread Ming Lei
There are still cases in which we need to use bio_bvecs() for get the number of multi-page segment, so introduce it. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-de...@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org Cc: Shaohua Li Cc: linux-r...@vger.kernel.org

[PATCH V10 08/19] btrfs: move bio_pages_all() to btrfs

2018-11-15 Thread Ming Lei
BTRFS is the only user of this helper, so move this helper into BTRFS, and implement it via bio_for_each_segment_all(), since bio->bi_vcnt may not equal to number of pages after multipage bvec is enabled. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-de...@redhat.com Cc: Alexande

[PATCH V10 07/19] btrfs: use bvec_last_segment to get bio's last page

2018-11-15 Thread Ming Lei
Preparing for supporting multi-page bvec. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-de...@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org Cc: Shaohua Li Cc: linux-r...@vger.kernel.org Cc: linux-er...@lists.ozlabs.org Cc: David Sterba Cc: linux-btrfs@vger.ke

[PATCH V10 06/19] fs/buffer.c: use bvec iterator to truncate the bio

2018-11-15 Thread Ming Lei
Once multi-page bvec is enabled, the last bvec may include more than one page, this patch use bvec_last_segment() to truncate the bio. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-de...@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org Cc: Shaohua Li Cc: linux-r.

[PATCH V10 05/19] block: introduce bvec_last_segment()

2018-11-15 Thread Ming Lei
BTRFS and guard_bio_eod() need to get the last singlepage segment from one multipage bvec, so introduce this helper to make them happy. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-de...@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org Cc: Shaohua Li Cc: linux-r

[PATCH V10 03/19] block: use bio_for_each_bvec() to compute multi-page bvec count

2018-11-15 Thread Ming Lei
First it is more efficient to use bio_for_each_bvec() in both blk_bio_segment_split() and __blk_recalc_rq_segments() to compute how many multi-page bvecs there are in the bio. Secondly once bio_for_each_bvec() is used, the bvec may need to be splitted because its length can be very longer than max

[PATCH V10 02/19] block: introduce bio_for_each_bvec()

2018-11-15 Thread Ming Lei
This helper is used for iterating over multi-page bvec for bio split & merge code. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-de...@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org Cc: Shaohua Li Cc: linux-r...@vger.kernel.org Cc: linux-er...@lists.ozlabs.org

[PATCH V10 04/19] block: use bio_for_each_bvec() to map sg

2018-11-15 Thread Ming Lei
It is more efficient to use bio_for_each_bvec() to map sg, meantime we have to consider splitting multipage bvec as done in blk_bio_segment_split(). Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-de...@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org Cc: Shaohua Li

[PATCH V10 01/19] block: introduce multi-page page bvec helpers

2018-11-15 Thread Ming Lei
This patch introduces helpers of 'mp_bvec_iter_*' for multipage bvec support. The introduced helpers treate one bvec as real multi-page segment, which may include more than one pages. The existed helpers of bvec_iter_* are interfaces for supporting current bvec iterator which is thought as single

[PATCH V10 00/19] block: support multi-page bvec

2018-11-15 Thread Ming Lei
Hi, This patchset brings multi-page bvec into block layer: 1) what is multi-page bvec? Multipage bvecs means that one 'struct bio_bvec' can hold multiple pages which are physically contiguous instead of one single page used in linux kernel for long time. 2) why is multi-page bvec introduced? K

Re: [PATCH 5/6] btrfs: remove unused variable tree in end_compressed_bio_write()

2018-11-15 Thread Johannes Thumshirn
On 14/11/2018 22:04, David Sterba wrote: > The fix has been folded to the original patch already, same the other > one removing 'tree'. If you used linux-next to run the build, then it's > probably lagging behind the development branches that are synced to > kernel.org after some testing. The devel