Re: [PATCH] cgroup writeback: use online cgroup when switching from dying bdi_writebacks

2019-07-19 Thread Dennis Zhou
the no internal process constraint. You end up with IO being attributed to cgroups you may not necessarily expect and how IO competes then I'm not really sure. Below is what I sent them. This punts to root instead which isn't necessarily better. Thanks, Dennis [1] https://lore.kernel.org

Re: [PATCH AUTOSEL 5.2 129/249] blk-iolatency: only account submitted bios

2019-07-15 Thread Dennis Zhou
On Mon, Jul 15, 2019 at 09:44:54AM -0400, Sasha Levin wrote: > From: Dennis Zhou > > [ Upstream commit a3fb01ba5af066521f3f3421839e501bb2c71805 ] > > As is, iolatency recognizes done_bio and cleanup as ending paths. If a > request is marked REQ_NOWAIT and fails to get a r

[PATCH v2] blk-iolatency: fix STS_AGAIN handling

2019-07-05 Thread Dennis Zhou
favor of cleaning up always on the rq_qos_done_bio() path. Fixes: a3fb01ba5af0 ("blk-iolatency: only account submitted bios") Debugged-by: Tejun Heo Debugged-by: Josef Bacik Signed-off-by: Dennis Zhou --- block/blk-iolatency.c | 51 --- 1 f

[PATCH] blk-iolatency: fix STS_AGAIN handling

2019-07-05 Thread Dennis Zhou
favor of cleaning up always on the rq_qos_done_bio() path. Fixes: a3fb01ba5af0 ("blk-iolatency: only account submitted bios") Debugged-by: Tejun Heo Debugged-by: Josef Bacik Signed-off-by: Dennis Zhou --- block/blk-iolatency.c | 29 +++-- 1 file changed, 3 in

Re: [PATCH] blk-iolatency: only account submitted bios

2019-06-15 Thread Dennis Zhou
Hi Jens, On Thu, May 23, 2019 at 04:10:18PM -0400, Dennis Zhou wrote: > As is, iolatency recognizes done_bio and cleanup as ending paths. If a > request is marked REQ_NOWAIT and fails to get a request, the bio is > cleaned up via rq_qos_cleanup() and ended in bio_wouldblock_error(

[PATCH] blk-iolatency: only account submitted bios

2019-05-23 Thread Dennis Zhou
were actually submitted. Signed-off-by: Dennis Zhou Cc: Josef Bacik --- block/blk-iolatency.c | 4 1 file changed, 4 insertions(+) diff --git a/block/blk-iolatency.c b/block/blk-iolatency.c index 507212d75ee2..58bac44ba78a 100644 --- a/block/blk-iolatency.c +++ b/block/blk-iolatency.c

[PATCH v2] block: fix fio jobs for 027 and add cgroup support

2019-05-01 Thread Dennis Zhou
reference on the queue. However, blkgs were being cleaned up after the last reference to the request_queue was put back. This creates a blktest cgroup for the fio jobs to utilize and should be useful for catching this scenario in the future. Signed-off-by: Dennis Zhou --- tests/bloc

Re: [PATCH] block: fix fio jobs for 027 and add cgroup support

2019-04-04 Thread Dennis Zhou
On Thu, Apr 04, 2019 at 11:02:24AM -0700, Omar Sandoval wrote: > On Fri, Mar 29, 2019 at 03:14:32PM -0700, Dennis Zhou wrote: > > Previously, the test was broken as "$fio_jobs" was considered as a > > string instead of additional parameters. This is fixed here. > >

[PATCH] block: fix fio jobs for 027 and add cgroup support

2019-03-29 Thread Dennis Zhou
reference on the queue. However, blkgs were being cleaned up after the last reference to the request_queue was put back. This creates a blktest cgroup for the fio jobs to utilize and should be useful for catching this scenario in the future. Signed-off-by: Dennis Zhou --- tests/bloc

[LSF/MM TOPIC] A little TLC for Discard

2019-02-20 Thread Dennis Zhou
Hi everyone, Ric proposed a similar topic already [1], but I'd like to just add a little more to that discussion with a broader proposal for discussing discard. Proposal: Discard is a topic that seems to need a little bit of TLC from the kernel side. The two main approaches with discard are eithe

Re: [PATCH] blkcg: add rcu lock to bio_clone_blkg_association()

2018-12-21 Thread Dennis Zhou
On Fri, Dec 21, 2018 at 08:26:02AM -0700, Jens Axboe wrote: > On 12/21/18 7:54 AM, Dennis Zhou wrote: > > I cleaned up blkg_tryget_closest() to require rcu_read_lock() earlier. > > However, this was a subtle case too which clearly was too subtle for me. > > The idea was

[PATCH] blkcg: add rcu lock to bio_clone_blkg_association()

2018-12-21 Thread Dennis Zhou
d the blkg->parent pointers are unused. This adds the appropriate read lock in bio_clone_blkg_association(). Fixes: 80fd3c272c1a ("blkcg: clean up blkg_tryget_closest()") Reported-by: syzbot+a36a3ba92bea3b315...@syzkaller.appspotmail.com Signed-off-by: Dennis Zhou --- block/bio.c | 4

[PATCH blktests 2/2] blktests: add Ming Lei's scsi-stress-remove

2018-12-20 Thread Dennis Zhou
re.kernel.org/lkml/20181211230308.66276-1-den...@kernel.org/ Signed-off-by: Dennis Zhou Cc: Ming Lei --- v2: - Used helpers in common/scsi_debug per Omar's suggestion. tests/block/027 | 73 + tests/block/027.out | 2 ++ 2 files chan

[PATCH blktests 1/2] blktests: split out cgroup2 controller and file check

2018-12-20 Thread Dennis Zhou
This is a prep patch for a new test that will race blkg association and request_queue cleanup. As blkg association is a underlying cgroup io controller feature, we need the ability to check if the controller is available. Signed-off-by: Dennis Zhou --- v2: - fixed minor typo. common/cgroup

[PATCH blktests v2 0/2] Add scsi-stress-remove to blktests

2018-12-20 Thread Dennis Zhou
This patchset is ontop of osandov#josef 98db2e11c97f. diffstats below: Dennis Zhou (2): blktests: split out cgroup2 controller and file check blktests: add Ming Lei's scsi-stress-remove common/cgroup | 18 +++ tests/block/027 | 73

Re: [PATCH blktests v2 2/2] blktests: add Ming Lei's scsi-stress-remove

2018-12-19 Thread Dennis Zhou
On Wed, Dec 19, 2018 at 02:49:42PM -0800, Omar Sandoval wrote: > On Thu, Dec 13, 2018 at 01:28:44PM -0500, Dennis Zhou wrote: > > This test exposed a race condition with shutting down a request_queue > > and the new blkg association. The issue ended up being that while the > &g

[PATCH] blkcg: clean up blkg_tryget_closest()

2018-12-19 Thread Dennis Zhou
The implementation of blkg_tryget_closest() wasn't super obvious and became a point of suspicion when debugging [1]. So let's clean it up so it's obviously not the problem. [1] https://lore.kernel.org/linux-block/a7e97e4b-0dd8-3a54-23b7-a0f27b17f...@kernel.dk/ Signed-off-

[PATCH] blkcg: remove unused __blkg_release_rcu()

2018-12-19 Thread Dennis Zhou
An earlier commit 7fcf2b033b84 ("blkcg: change blkg reference counting to use percpu_ref") moved around the release call from blkg_put() to be a part of the percpu_ref cleanup. Remove the additional unused code which should have been removed earlier. Signed-off-by: Dennis Zhou ---

Re: [PATCH] block: save irq state in blkg_lookup_create()

2018-12-19 Thread Dennis Zhou
xd9/0x1a8 > [ 104.324427] do_idle+0x162/0x2b2 > [ 104.324891] ? arch_cpu_idle_exit+0x28/0x28 > [ 104.325467] ? mark_held_locks+0x28/0x7f > [ 104.326031] ? _raw_spin_unlock_irqrestore+0x44/0x6b > [ 104.326719] cpu_startup_entry+0x1d/0x1f > [ 104.327261] start_secondary+0x2

Re: dm: don't reuse bio for flushes

2018-12-19 Thread Dennis Zhou
On Wed, Dec 19, 2018 at 09:14:25AM -0700, Jens Axboe wrote: > On 12/19/18 9:13 AM, Mike Snitzer wrote: > > On Wed, Dec 19 2018 at 11:11am -0500, > > Mike Snitzer wrote: > > > >> On Wed, Dec 19 2018 at 10:50am -0500, > >> Jens Axboe wrote: > >> > >>> DM currently has a statically allocated bio th

Re: [PATCH v3] block: fix blk-iolatency accounting underflow

2018-12-17 Thread Dennis Zhou
On Mon, Dec 17, 2018 at 11:42:28AM -0800, Liu Bo wrote: > On Mon, Dec 17, 2018 at 8:04 AM Dennis Zhou wrote: > > > > The blk-iolatency controller measures the time from rq_qos_throttle() to > > rq_qos_done_bio() and attributes this time to the first bio that needs > >

[PATCH v3] block: fix blk-iolatency accounting underflow

2018-12-17 Thread Dennis Zhou
https://lore.kernel.org/lkml/20181205171039.73066-1-den...@kernel.org/ Fixes: 5cdf2e3fea5e ("blkcg: associate blkg when associating a device") Signed-off-by: Dennis Zhou Cc: Josef Bacik --- block/blk-iolatency.c | 2 +- block/blk-rq-qos.h| 5 + include/linux/blk_types.h | 1 +

[PATCH blktests v2 2/2] blktests: add Ming Lei's scsi-stress-remove

2018-12-13 Thread Dennis Zhou
elp prevent this from happening again, integrate Ming's test into blktests so that it can more easily be ran. Signed-off-by: Dennis Zhou Cc: Ming Lei --- v2: - Change scheduler retrieving logic based on Ming's comment tests/block/022 | 90 +

Re: [PATCH blktests 2/2] blktests: add Ming Lei's scsi-stress-remove

2018-12-13 Thread Dennis Zhou
On Thu, Dec 13, 2018 at 09:24:09AM +0800, Ming Lei wrote: > On Wed, Dec 12, 2018 at 06:09:59PM -0500, Dennis Zhou wrote: > > This test exposed a race condition with shutting down a request_queue > > and the new blkg association. The issue ended up being that while the > > re

Re: [PATCH] blkcg: handle dying request_queue when associating a blkg

2018-12-13 Thread Dennis Zhou
On Wed, Dec 12, 2018 at 03:54:52PM -0800, Bart Van Assche wrote: > On Tue, 2018-12-11 at 23:06 -0500, Dennis Zhou wrote: > > Hi Bart, > > > > On Tue, Dec 11, 2018 at 03:16:13PM -0800, Bart Van Assche wrote: > > > On Tue, 2018-12-11 at 18:03 -0500, Dennis Zhou wrote:

[PATCH blktests 1/2] blktests: split out cgroup2 controller and file check

2018-12-12 Thread Dennis Zhou
This is a prep patch for a new test that will race blkg association and request_queue cleanup. As blkg association is a underlying cgroup io controller feature, we need the ability to check if the controller is available. Signed-off-by: Dennis Zhou --- common/cgroup | 18 +- 1

[PATCH blktests 2/2] blktests: add Ming Lei's scsi-stress-remove

2018-12-12 Thread Dennis Zhou
elp prevent this from happening again, integrate Ming's test into blktests so that it can more easily be ran. Signed-off-by: Dennis Zhou Cc: Ming Lei --- tests/block/022 | 96 + tests/block/022.out | 2 + 2 files changed, 98 insertions(+) cre

[PATCH blktests 0/2] Add scsi-stress-remove to blktests

2018-12-12 Thread Dennis Zhou
lore.kernel.org/lkml/20181211230308.66276-1-den...@kernel.org/ This patchset is ontop of osandov#josef ad08c1fe0d9f. diffstats below: Dennis Zhou (2): blktests: split out cgroup2 controller and file check blktests: add Ming Lei's scsi-stress-remove common/cgroup | 18 ++---

Re: [PATCH] blkcg: handle dying request_queue when associating a blkg

2018-12-11 Thread Dennis Zhou
Hi Bart, On Tue, Dec 11, 2018 at 03:16:13PM -0800, Bart Van Assche wrote: > On Tue, 2018-12-11 at 18:03 -0500, Dennis Zhou wrote: > > diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c > > index 6bd0619a7d6e..c30661ddc873 100644 > > --- a/block/blk-cgroup.c > &g

[PATCH] blkcg: handle dying request_queue when associating a blkg

2018-12-11 Thread Dennis Zhou
ead request_queue. [1] https://lore.kernel.org/lkml/20180911184137.35897-1-dennissz...@gmail.com/ [2] https://lore.kernel.org/lkml/20181126211946.77067-1-den...@kernel.org/ Fixes: 5cdf2e3fea5e ("blkcg: associate blkg when associating a device") Reported-and-tested-by: Ming Lei Signed-off-by

[PATCH v2] block: fix iolat timestamp and restore accounting semantics

2018-12-11 Thread Dennis Zhou
https://lore.kernel.org/lkml/20181205171039.73066-1-den...@kernel.org/ Fixes: 5cdf2e3fea5e ("blkcg: associate blkg when associating a device") Signed-off-by: Dennis Zhou Cc: Josef Bacik --- v2: - Switched to reusing BIO_QUEUE_ENTERED rather than adding a second timestamp to the bio stru

Re: block: kernel panic in __bio_associate_blkg+0x1e

2018-12-10 Thread Dennis Zhou
On Tue, Dec 11, 2018 at 11:22:18AM +0800, Ming Lei wrote: > On Tue, Dec 11, 2018 at 11:09 AM Dennis Zhou wrote: > > > > Hi Ming, > > > > On Tue, Dec 11, 2018 at 10:36:07AM +0800, Ming Lei wrote: > > > Hi Jens and Dennis, > > > > > > Just foun

Re: [PATCH] block: fix iolat timestamp and restore accounting semantics

2018-12-10 Thread Dennis Zhou
Hi Josef, On Mon, Dec 10, 2018 at 01:25:08PM -0500, Josef Bacik wrote: > On Mon, Dec 10, 2018 at 11:35:10AM -0500, Dennis Zhou wrote: > > The blk-iolatency controller measures the time from rq_qos_throttle() to > > rq_qos_done_bio() and attributes this time to the first bio th

Re: block: kernel panic in __bio_associate_blkg+0x1e

2018-12-10 Thread Dennis Zhou
Hi Ming, On Tue, Dec 11, 2018 at 10:36:07AM +0800, Ming Lei wrote: > Hi Jens and Dennis, > > Just found the following issue when testing for-4.21/block when > running stress io & device > remove on scsi_debug, and it should be caused by recent blkcg changes. > > [ 37.144330] sd 8:0:0:15: [sds]

[PATCH] block: fix iolat timestamp and restore accounting semantics

2018-12-10 Thread Dennis Zhou
-iolatency in rq_qos_done_bio(). [1] https://lore.kernel.org/lkml/20181205171039.73066-1-den...@kernel.org/ Signed-off-by: Dennis Zhou Cc: Josef Bacik --- block/blk-iolatency.c | 17 ++--- include/linux/blk_types.h | 12 2 files changed, 18 insertions(+), 11 deletions

[PATCH] blkcg: put back rcu lock in blkcg_bio_issue_check()

2018-12-06 Thread Dennis Zhou
I was a little overzealous in removing the rcu_read_lock() call from blkcg_bio_issue_check() and it broke blk-throttle. Put it back. Fixes: e35403a034bf ("blkcg: associate blkg when associating a device") Signed-off-by: Dennis Zhou --- include/linux/blk-cgroup.h | 3 +++ 1 file

[PATCH 03/14] blkcg: convert blkg_lookup_create() to find closest blkg

2018-12-05 Thread Dennis Zhou
if it fails to create, return the closest blkg found or the q->root_blkg. blkg_try_get_closest() is introduced and used during association so a bio is always attached to a blkg. Signed-off-by: Dennis Zhou Acked-by: Tejun Heo Reviewed-by: Josef Bacik --- block/bio.c

[PATCH 01/14] blkcg: fix ref count issue with bio_blkcg() using task_css

2018-12-05 Thread Dennis Zhou
complexity. I have created a private version documenting the inconsistency and noting not to use it. Signed-off-by: Dennis Zhou Acked-by: Tejun Heo Reviewed-by: Josef Bacik --- block/bfq-cgroup.c | 4 +- block/bfq-iosched.c| 2 +- block/bio.c| 10 +++- block/blk

[PATCH 06/14] blkcg: associate blkg when associating a device

2018-12-05 Thread Dennis Zhou
tion into the bio_set_dev macro(). This should cover the majority of cases where the device is set/changed keeping the two pointers consistent. Fallback code is added to blkcg_bio_issue_check() to catch any missing paths. Signed-off-by: Dennis Zhou Reviewed-by: Josef Bacik --- bl

[PATCH 09/14] blkcg: associate writeback bios with a blkg

2018-12-05 Thread Dennis Zhou
One of the goals of this series is to remove a separate reference to the css of the bio. This can and should be accessed via bio_blkcg(). In this patch, wbc_init_bio() now requires a bio to have a device associated with it. Signed-off-by: Dennis Zhou Reviewed-by: Josef Bacik Acked-by: Tejun Heo

[PATCH 05/14] dm: set the static flush bio device on demand

2018-12-05 Thread Dennis Zhou
to bio_set_dev() because the call is before the bdev is associated with a gendisk (bd_disk is %NULL). To get around this, set the device on the static bio every time and use that to clone to the other bios. Signed-off-by: Dennis Zhou Acked-by: Mike Snitzer Cc: Alasdair Kergon --- block/bio.c

[PATCH 02/14] blkcg: update blkg_lookup_create() to do locking

2018-12-05 Thread Dennis Zhou
cking and implement this pattern. The old blkg_lookup_create() is renamed to __blkg_lookup_create(). If a call site wants to do its own error handling or already owns the queue lock, they can use __blkg_lookup_create(). This will be used in upcoming patches. Signed-off-by: Dennis Zhou Review

[PATCH 08/14] blkcg: associate a blkg for pages being evicted by swap

2018-12-05 Thread Dennis Zhou
-off-by: Dennis Zhou Reviewed-by: Josef Bacik Acked-by: Tejun Heo --- block/bio.c | 62 +++-- include/linux/bio.h | 6 ++--- mm/page_io.c| 2 +- 3 files changed, 42 insertions(+), 28 deletions(-) diff --git a/block/bio.c b/block/bio.c

[PATCH 07/14] blkcg: consolidate bio_issue_init() to be a part of core

2018-12-05 Thread Dennis Zhou
bio_issue_init among other things initializes the timestamp for an IO. Rather than have this logic handled by policies, this consolidates it to be on the init paths (normal, clone, bounce clone). Signed-off-by: Dennis Zhou Acked-by: Tejun Heo Reviewed-by: Liu Bo Reviewed-by: Josef Bacik

[PATCH 14/14] blkcg: rename blkg_try_get() to blkg_tryget()

2018-12-05 Thread Dennis Zhou
blkg reference counting now uses percpu_ref rather than atomic_t. Let's make this consistent with css_tryget. This renames blkg_try_get to blkg_tryget and now returns a bool rather than the blkg or %NULL. Signed-off-by: Dennis Zhou Reviewed-by: Josef Bacik Acked-by: Tejun Heo --- block/

[PATCH 11/14] blkcg: remove additional reference to the css

2018-12-05 Thread Dennis Zhou
. The association continues to rely on the tryget logic walking up the blkg tree. This streamlines the three ways that association can happen: normal, swap, and writeback. Signed-off-by: Dennis Zhou Acked-by: Tejun Heo Reviewed-by: Josef Bacik --- block/bio.c

[PATCH v6 00/14] block: always associate blkg and refcount cleanup

2018-12-05 Thread Dennis Zhou
0011-blkcg-remove-additional-reference-to-the-css.patch 0012-blkcg-remove-bio_disassociate_task.patch 0013-blkcg-change-blkg-reference-counting-to-use-percpu_r.patch 0014-blkcg-rename-blkg_try_get-to-blkg_tryget.patch This patchset is on top of linux-block#for-4.21/block 154989e45fd8. diffstats b

[PATCH 04/14] blkcg: introduce common blkg association logic

2018-12-05 Thread Dennis Zhou
css will be reused and association will be redone as the request_queue may have changed. Signed-off-by: Dennis Zhou Reviewed-by: Josef Bacik Acked-by: Tejun Heo --- block/bio.c | 62 --- block/blk-iolatency.c | 10 ++- block/blk-throttle.c

[PATCH 10/14] blkcg: remove bio->bi_css and instead use bio->bi_blkg

2018-12-05 Thread Dennis Zhou
-by: Dennis Zhou Reviewed-by: Josef Bacik Acked-by: Tejun Heo --- block/bio.c| 59 +- block/bounce.c | 2 +- drivers/block/loop.c | 5 ++-- drivers/md/raid0.c | 2 +- include/linux/bio.h| 11 +++ include/li

[PATCH 12/14] blkcg: remove bio_disassociate_task()

2018-12-05 Thread Dennis Zhou
Now that a bio only holds a blkg reference, so clean up is simply putting back that reference. Remove bio_disassociate_task() as it just calls bio_disassociate_blkg() and call the latter directly. Signed-off-by: Dennis Zhou Acked-by: Tejun Heo Reviewed-by: Josef Bacik --- block/bio.c

[PATCH 13/14] blkcg: change blkg reference counting to use percpu_ref

2018-12-05 Thread Dennis Zhou
Every bio is now associated with a blkg putting blkg_get, blkg_try_get, and blkg_put on the hot path. Switch over the refcnt in blkg to use percpu_ref. Signed-off-by: Dennis Zhou Acked-by: Tejun Heo Reviewed-by: Josef Bacik --- block/blk-cgroup.c | 41

Re: [PATCH 05/14] dm: set the static flush bio device on demand

2018-12-04 Thread Dennis Zhou
On Tue, Dec 04, 2018 at 03:28:07PM -0500, Mike Snitzer wrote: > On Tue, Dec 04 2018 at 1:35pm -0500, > Dennis Zhou wrote: > > > The next patch changes the macro bio_set_dev() to associate a bio with a > > blkg based on the device set. However, dm creates a static bio to be

[PATCH 02/14] blkcg: update blkg_lookup_create() to do locking

2018-12-04 Thread Dennis Zhou
cking and implement this pattern. The old blkg_lookup_create() is renamed to __blkg_lookup_create(). If a call site wants to do its own error handling or already owns the queue lock, they can use __blkg_lookup_create(). This will be used in upcoming patches. Signed-off-by: Dennis Zhou Review

[PATCH 03/14] blkcg: convert blkg_lookup_create() to find closest blkg

2018-12-04 Thread Dennis Zhou
if it fails to create, return the closest blkg found or the q->root_blkg. blkg_try_get_closest() is introduced and used during association so a bio is always attached to a blkg. Signed-off-by: Dennis Zhou Acked-by: Tejun Heo Reviewed-by: Josef Bacik --- block/bio.c

[PATCH 04/14] blkcg: introduce common blkg association logic

2018-12-04 Thread Dennis Zhou
css will be reused and association will be redone as the request_queue may have changed. Signed-off-by: Dennis Zhou Reviewed-by: Josef Bacik Acked-by: Tejun Heo --- block/bio.c | 62 --- block/blk-iolatency.c | 10 ++- block/blk-throttle.c

[PATCH 05/14] dm: set the static flush bio device on demand

2018-12-04 Thread Dennis Zhou
to bio_set_dev() because the call is before the bdev is associated with a gendisk (bd_disk is %NULL). To get around this, set the device on the static bio every time and use that to clone to the other bios. Signed-off-by: Dennis Zhou Cc: Alasdair Kergon Cc: Mike Snitzer --- block/bio.c

[PATCH 01/14] blkcg: fix ref count issue with bio_blkcg() using task_css

2018-12-04 Thread Dennis Zhou
complexity. I have created a private version documenting the inconsistency and noting not to use it. Signed-off-by: Dennis Zhou Acked-by: Tejun Heo Reviewed-by: Josef Bacik --- block/bfq-cgroup.c | 4 +- block/bfq-iosched.c| 2 +- block/bio.c| 10 +++- block/blk

[PATCH 07/14] blkcg: consolidate bio_issue_init() to be a part of core

2018-12-04 Thread Dennis Zhou
bio_issue_init among other things initializes the timestamp for an IO. Rather than have this logic handled by policies, this consolidates it to be on the init paths (normal, clone, bounce clone). Signed-off-by: Dennis Zhou Acked-by: Tejun Heo Reviewed-by: Liu Bo --- block/bio.c

[PATCH 06/14] blkcg: associate blkg when associating a device

2018-12-04 Thread Dennis Zhou
tion into the bio_set_dev macro(). This should cover the majority of cases where the device is set/changed keeping the two pointers consistent. Fallback code is added to blkcg_bio_issue_check() to catch any missing paths. Signed-off-by: Dennis Zhou --- block/bio.c| 1 +

[PATCH 10/14] blkcg: remove bio->bi_css and instead use bio->bi_blkg

2018-12-04 Thread Dennis Zhou
-by: Dennis Zhou Reviewed-by: Josef Bacik Acked-by: Tejun Heo --- block/bio.c| 59 +- block/bounce.c | 2 +- drivers/block/loop.c | 5 ++-- drivers/md/raid0.c | 2 +- include/linux/bio.h| 11 +++ include/li

[PATCH 09/14] blkcg: associate writeback bios with a blkg

2018-12-04 Thread Dennis Zhou
One of the goals of this series is to remove a separate reference to the css of the bio. This can and should be accessed via bio_blkcg(). In this patch, wbc_init_bio() now requires a bio to have a device associated with it. Signed-off-by: Dennis Zhou Reviewed-by: Josef Bacik Acked-by: Tejun Heo

[PATCH 12/14] blkcg: remove bio_disassociate_task()

2018-12-04 Thread Dennis Zhou
Now that a bio only holds a blkg reference, so clean up is simply putting back that reference. Remove bio_disassociate_task() as it just calls bio_disassociate_blkg() and call the latter directly. Signed-off-by: Dennis Zhou Acked-by: Tejun Heo --- block/bio.c | 11 +-- include

[PATCH 11/14] blkcg: remove additional reference to the css

2018-12-04 Thread Dennis Zhou
. The association continues to rely on the tryget logic walking up the blkg tree. This streamlines the three ways that association can happen: normal, swap, and writeback. Signed-off-by: Dennis Zhou Acked-by: Tejun Heo --- block/bio.c| 66 -- include/

[PATCH 13/14] blkcg: change blkg reference counting to use percpu_ref

2018-12-04 Thread Dennis Zhou
Every bio is now associated with a blkg putting blkg_get, blkg_try_get, and blkg_put on the hot path. Switch over the refcnt in blkg to use percpu_ref. Signed-off-by: Dennis Zhou Acked-by: Tejun Heo --- block/blk-cgroup.c | 41 -- include/linux/blk

[PATCH 14/14] blkcg: rename blkg_try_get() to blkg_tryget()

2018-12-04 Thread Dennis Zhou
blkg reference counting now uses percpu_ref rather than atomic_t. Let's make this consistent with css_tryget. This renames blkg_try_get to blkg_tryget and now returns a bool rather than the blkg or %NULL. Signed-off-by: Dennis Zhou Reviewed-by: Josef Bacik Acked-by: Tejun Heo --- block/

[PATCH 08/14] blkcg: associate a blkg for pages being evicted by swap

2018-12-04 Thread Dennis Zhou
-off-by: Dennis Zhou Reviewed-by: Josef Bacik Acked-by: Tejun Heo --- block/bio.c | 62 +++-- include/linux/bio.h | 6 ++--- mm/page_io.c| 2 +- 3 files changed, 42 insertions(+), 28 deletions(-) diff --git a/block/bio.c b/block/bio.c

[PATCH v5 00/14] block: always associate blkg and refcount cleanup

2018-12-04 Thread Dennis Zhou
ryget.patch This patchset is on top of linu-block#for-4.21/block 154989e45fd8. diffstats below: Dennis Zhou (14): blkcg: fix ref count issue with bio_blkcg() using task_css blkcg: update blkg_lookup_create() to do locking blkcg: convert blkg_lookup_create() to find closest blkg blkcg: intro

Re: [PATCH 05/13] blkcg: associate blkg when associating a device

2018-12-03 Thread Dennis Zhou
On Fri, Nov 30, 2018 at 01:54:26AM -0800, Christoph Hellwig wrote: > > diff --git a/include/linux/bio.h b/include/linux/bio.h > > index 62715a5a4f32..8bc9d9b29fd3 100644 > > --- a/include/linux/bio.h > > +++ b/include/linux/bio.h > > @@ -486,6 +486,12 @@ extern unsigned int bvec_nr_vecs(unsigned sh

Re: [PATCH 04/13] blkcg: introduce common blkg association logic

2018-12-03 Thread Dennis Zhou
ave taken this out in v5. It seems that this should be a one-off case which should work with the patch below. Thanks, Dennis --- >From 3ee13402af369ee8618549b63593d68ffca574ca Mon Sep 17 00:00:00 2001 From: Dennis Zhou Date: Mon, 3 Dec 2018 10:56:34 -0800 Subject: [PATCH 05/14] dm: set flush bio device on demand The nex

Re: [PATCH 05/13] blkcg: associate blkg when associating a device

2018-11-29 Thread Dennis Zhou
On Thu, Nov 29, 2018 at 07:53:33AM -0800, Tejun Heo wrote: > On Mon, Nov 26, 2018 at 04:19:38PM -0500, Dennis Zhou wrote: > > diff --git a/include/linux/bio.h b/include/linux/bio.h > > index 62715a5a4f32..8bc9d9b29fd3 100644 > > --- a/include/linux/bio.h > > +++ b/includ

Re: [PATCH 04/13] blkcg: introduce common blkg association logic

2018-11-29 Thread Dennis Zhou
On Thu, Nov 29, 2018 at 07:49:17AM -0800, Tejun Heo wrote: > On Mon, Nov 26, 2018 at 04:19:37PM -0500, Dennis Zhou wrote: > > There are 3 ways blkg association can happen: association with the > > current css, with the page css (swap), or from the wbc css (writeback). > > &

Re: [PATCH 00/13 v4] block: always associate blkg and refcount cleanup

2018-11-27 Thread Dennis Zhou
On Tue, Nov 27, 2018 at 04:10:01PM -0500, Josef Bacik wrote: > On Mon, Nov 26, 2018 at 04:19:33PM -0500, Dennis Zhou wrote: > > Hi everyone, > > > > This is respin of v3 [1] with fixes for the errors reported in [2] and > > [3]. v3 was reverted in [4]. > >

[PATCH 01/13] blkcg: fix ref count issue with bio_blkcg() using task_css

2018-11-26 Thread Dennis Zhou
complexity. I have created a private version documenting the inconsistency and noting not to use it. Signed-off-by: Dennis Zhou Acked-by: Tejun Heo --- block/bfq-cgroup.c | 4 +- block/bfq-iosched.c| 2 +- block/bio.c| 10 +++- block/blk-iolatency.c | 2

[PATCH 03/13] blkcg: convert blkg_lookup_create() to find closest blkg

2018-11-26 Thread Dennis Zhou
if it fails to create, return the closest blkg found or the q->root_blkg. blkg_try_get_closest() is introduced and used during association so a bio is always attached to a blkg. Signed-off-by: Dennis Zhou Acked-by: Tejun Heo --- block/bio.c| 17 ++--- block/blk-cg

[PATCH 04/13] blkcg: introduce common blkg association logic

2018-11-26 Thread Dennis Zhou
css will be reused and association will be redone as the request_queue may have changed. Signed-off-by: Dennis Zhou --- block/bio.c | 81 ++- block/blk-iolatency.c | 10 ++ block/blk-throttle.c | 6 ++-- include/linux/bio.h | 5 ++- 4 files

[PATCH 02/13] blkcg: update blkg_lookup_create() to do locking

2018-11-26 Thread Dennis Zhou
cking and implement this pattern. The old blkg_lookup_create() is renamed to __blkg_lookup_create(). If a call site wants to do its own error handling or already owns the queue lock, they can use __blkg_lookup_create(). This will be used in upcoming patches. Signed-off-by: Dennis Zhou Review

[PATCH 06/13] blkcg: consolidate bio_issue_init() to be a part of core

2018-11-26 Thread Dennis Zhou
bio_issue_init among other things initializes the timestamp for an IO. Rather than have this logic handled by policies, this consolidates it to be on the init paths (normal, clone, bounce clone). Signed-off-by: Dennis Zhou Acked-by: Tejun Heo Reviewed-by: Liu Bo --- block/bio.c

[PATCH 11/13] blkcg: remove bio_disassociate_task()

2018-11-26 Thread Dennis Zhou
Now that a bio only holds a blkg reference, so clean up is simply putting back that reference. Remove bio_disassociate_task() as it just calls bio_disassociate_blkg() and call the latter directly. Signed-off-by: Dennis Zhou --- block/bio.c | 11 +-- include/linux/bio.h | 2

[PATCH 07/13] blkcg: associate a blkg for pages being evicted by swap

2018-11-26 Thread Dennis Zhou
-off-by: Dennis Zhou Reviewed-by: Josef Bacik Acked-by: Tejun Heo --- block/bio.c | 62 +++-- include/linux/bio.h | 6 ++--- mm/page_io.c| 2 +- 3 files changed, 42 insertions(+), 28 deletions(-) diff --git a/block/bio.c b/block/bio.c

[PATCH 13/13] blkcg: rename blkg_try_get() to blkg_tryget()

2018-11-26 Thread Dennis Zhou
blkg reference counting now uses percpu_ref rather than atomic_t. Let's make this consistent with css_tryget. This renames blkg_try_get to blkg_tryget and now returns a bool rather than the blkg or %NULL. Signed-off-by: Dennis Zhou Reviewed-by: Josef Bacik Acked-by: Tejun Heo --- block/

[PATCH 12/13] blkcg: change blkg reference counting to use percpu_ref

2018-11-26 Thread Dennis Zhou
Every bio is now associated with a blkg putting blkg_get, blkg_try_get, and blkg_put on the hot path. Switch over the refcnt in blkg to use percpu_ref. Signed-off-by: Dennis Zhou Acked-by: Tejun Heo --- block/blk-cgroup.c | 41 -- include/linux/blk

[PATCH 05/13] blkcg: associate blkg when associating a device

2018-11-26 Thread Dennis Zhou
st to have made it to generic_make_request_checks(). Signed-off-by: Dennis Zhou --- block/bio.c| 1 + block/blk-iolatency.c | 4 +--- block/blk-throttle.c | 1 - fs/buffer.c| 2 +- fs/ext4/page-io.c | 2 +- include/linux/bio.h| 7 +

[PATCH 09/13] blkcg: remove bio->bi_css and instead use bio->bi_blkg

2018-11-26 Thread Dennis Zhou
-by: Dennis Zhou Reviewed-by: Josef Bacik Acked-by: Tejun Heo --- block/bio.c| 56 +++--- block/bounce.c | 2 +- drivers/block/loop.c | 5 ++-- drivers/md/raid0.c | 2 +- include/linux/bio.h| 11 +++- incl

[PATCH 08/13] blkcg: associate writeback bios with a blkg

2018-11-26 Thread Dennis Zhou
One of the goals of this series is to remove a separate reference to the css of the bio. This can and should be accessed via bio_blkcg. In this patch, wbc_init_bio() now requires a bio to have a device associated with it. Signed-off-by: Dennis Zhou Reviewed-by: Josef Bacik Acked-by: Tejun Heo

[PATCH 10/13] blkcg: remove additional reference to the css

2018-11-26 Thread Dennis Zhou
. The association continues to rely on the tryget logic walking up the blkg tree. This streamlines the three ways that association can happen: normal, swap, and writeback. Signed-off-by: Dennis Zhou Acked-by: Tejun Heo --- block/bio.c| 60 +- include/

[PATCH 00/13 v4] block: always associate blkg and refcount cleanup

2018-11-26 Thread Dennis Zhou
0013-blkcg-rename-blkg_try_get-to-blkg_tryget.patch This patchset is on top of linux-block#for-4.21/block 5f0ed774ed29. diffstats below: Dennis Zhou (13): blkcg: fix ref count issue with bio_blkcg() using task_css blkcg: update blkg_lookup_create() to do locking blkcg: convert blkg_looku

[PATCH] blkcg: revert blkcg cleanups series

2018-11-01 Thread Dennis Zhou
, b3b9f24f5fcc, e2b0989954ae, f0fcb3ec89f3, c839e7a03f92, bdc2491708c4, 74b7c02a9bc1, 5bf9a1f3b4ef, a7b39b4e961c, 07b05bcc3213, 49f4c2dc2b50, 27e6fa996c53 Signed-off-by: Dennis Zhou --- Documentation/admin-guide/cgroup-v2.rst | 8 +- block/bfq-cgroup.c | 4 +- block/bfq

[PATCH 0/2] block: fixes for always associate blkg

2018-10-20 Thread Dennis Zhou
4.20/block bbc152825afc. diffstats below: Dennis Zhou (2): blkcg: fix edge case for blk_get_rl() under memory pressure blkcg: reassociate bios when make_request() is called recursively block/bio.c| 20 block/blk-core.c | 1 + include/linux/bio.h

[PATCH 2/2] blkcg: reassociate bios when make_request() is called recursively

2018-10-20 Thread Dennis Zhou
: a7b39b4e961c ("blkcg: always associate a bio with a blkg") Reported-by: Valdis Kletnieks Signed-off-by: Dennis Zhou Tested-by: Valdis Kletnieks --- block/bio.c | 20 block/blk-core.c| 1 + include/linux/bio.h | 3 +++ 3 files changed, 24 insertions(+) di

[PATCH 1/2] blkcg: fix edge case for blk_get_rl() under memory pressure

2018-10-20 Thread Dennis Zhou
: e2b0989954ae ("blkcg: cleanup and make blk_get_rl use blkg_lookup_create") Signed-off-by: Dennis Zhou --- include/linux/blk-cgroup.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/blk-cgroup.h b/include/linux/blk-cgroup.h index b7fd08013de2..1e76ceebe

Re: [BUG] ext4/block null pointer crashes in linux-next

2018-10-19 Thread Dennis Zhou
On Fri, Oct 19, 2018 at 10:47:19PM -0400, valdis.kletni...@vt.edu wrote: > On Fri, 19 Oct 2018 18:21:00 -0400, Dennis Zhou said: > > > Do you by chance run any encryption or anything on top of your hard > > drive or ssd? > > ext4 on an LVM LV that's part of a

Re: [BUG] ext4/block null pointer crashes in linux-next

2018-10-19 Thread Dennis Zhou
On Fri, Oct 19, 2018 at 11:52:36AM -0400, valdis.kletni...@vt.edu wrote: > On Wed, 17 Oct 2018 17:20:29 -0400, Dennis Zhou said: > > > > I apologize, but I'm having a hard time reproducing this myself. I am > > not able to hit this issue in my qemu instance with linu

Re: [BUG] ext4/block null pointer crashes in linux-next

2018-10-17 Thread Dennis Zhou
On Wed, Oct 17, 2018 at 11:47:35AM -0400, valdis.kletni...@vt.edu wrote: > On Tue, 16 Oct 2018 14:25:13 -0400, Dennis Zhou said: > > > > > grep execve /root/rpm-exec-strace > > > > execve("/usr/bin/rpm", ["rpm", "-Uvh", "--force&

Re: [BUG] ext4/block null pointer crashes in linux-next

2018-10-16 Thread Dennis Zhou
On Tue, Oct 16, 2018 at 12:02:03PM -0400, Dennis Zhou wrote: > Hi Vladis, > > On Mon, Oct 15, 2018 at 07:28:48PM -0400, valdis.kletni...@vt.edu wrote: > > So I finally had a chance to find a replicator and finish bisecting this > > and: > > > > [/us

Re: [BUG] ext4/block null pointer crashes in linux-next

2018-10-16 Thread Dennis Zhou
mit > commit e2b0989954ae7c80609f77e7ce203bea6d2c54e1 > Author: Dennis Zhou (Facebook) > Date: Tue Sep 11 14:41:35 2018 -0400 > > blkcg: cleanup and make blk_get_rl use blkg_lookup_create > > I was able to do a bit of sleuthing with strace, and I tracked it down to one > of >