On 10/23/2017 04:51 PM, Christoph Hellwig wrote:
> This helper allows reinserting a bio into a new queue without much
> overhead, but requires all queue limits to be the same for the upper
> and lower queues, and it does not provide any recursion preventions.
>
> Signed-off-by: Christoph Hellwig
On 10/23/2017 04:51 PM, Christoph Hellwig wrote:
> This helpers allows to bounce steal the uncompleted bios from a request so
> that they can be reissued on another path.
>
> Signed-off-by: Christoph Hellwig
> Reviewed-by: Sagi Grimberg
> ---
> block/blk-core.c | 20
>
On 10/23/2017 04:51 PM, Christoph Hellwig wrote:
> The hidden gendisks introduced in the next patch need to keep the dev
> field in their struct device empty so that udev won't try to create
> block device nodes for them. To support that rewrite disk_devt to
> look at the major and first_minor fie
On 10/23/2017 04:51 PM, Christoph Hellwig wrote:
> With this flag a driver can create a gendisk that can be used for I/O
> submission inside the kernel, but which is not registered as user
> facing block device. This will be useful for the NVMe multipath
> implementation.
>
> Signed-off-by: Chris
On 10/23/2017 04:51 PM, Christoph Hellwig wrote:
> That we we can also poll non blk-mq queues. Mostly needed for
> the NVMe multipath code, but could also be useful elsewhere.
>
> Signed-off-by: Christoph Hellwig
> ---
> block/blk-core.c | 11 +++
> block/blk-mq.c
On 10/23/2017 04:51 PM, Christoph Hellwig wrote:
> For kref_get_unless_zero to protect against lookup vs free races we need
> to use it in all places where we aren't guaranteed to already hold a
> reference. There is no such guarantee in nvme_find_get_ns, so switch to
> kref_get_unless_zero in thi
On 10/23/2017 04:51 PM, Christoph Hellwig wrote:
> Now that we are protected against lookup vs free races for the namespace
> by using kref_get_unless_zero we don't need the hack of NULLing out the
> disk private data during removal.
>
> Signed-off-by: Christoph Hellwig
> Reviewed-by: Sagi Grimbe
On 10/23/2017 04:51 PM, Christoph Hellwig wrote:
> Instead of allocating a separate struct device for the character device
> handle embedd it into struct nvme_ctrl and use it for the main controller
> refcounting. This removes double refcounting and gets us an automatic
> reference for the charact
On 10/23/2017 04:51 PM, Christoph Hellwig wrote:
> Use the core chrdev code to set up the link between the character device
> and the nvme controller. This allows us to get rid of the global list
> of all controllers, and also ensures that we have both a reference to
> the controller and the trans
On 10/23/2017 04:51 PM, Christoph Hellwig wrote:
> This is a much more sensible check than just the admin queue.
>
> Signed-off-by: Christoph Hellwig
> Reviewed-by: Sagi Grimberg
> Reviewed-by: Johannes Thumshirn
> ---
> drivers/nvme/host/core.c | 2 +-
> 1 file changed, 1 insertion(+), 1 dele
On 10/23/2017 04:51 PM, Christoph Hellwig wrote:
> This allows us to manage the various uniqueue namespace identifiers
> together instead needing various variables and arguments.
>
> Signed-off-by: Christoph Hellwig
> Reviewed-by: Keith Busch
> Reviewed-by: Sagi Grimberg
> ---
> drivers/nvme/h
On 10/23/2017 04:51 PM, Christoph Hellwig wrote:
> This adds a new nvme_subsystem structure so that we can track multiple
> controllers that belong to a single subsystem. For now we only use it
> to store the NQN, and to check that we don't have duplicate NQNs unless
> the involved subsystems supp
On 10/23/2017 04:51 PM, Christoph Hellwig wrote:
> Introduce a new struct nvme_ns_head that holds information about an actual
> namespace, unlike struct nvme_ns, which only holds the per-controller
> namespace information. For private namespaces there is a 1:1 relation of
> the two, but for shared
[...]
>> However, you have completely ignored mine, Linus and Bartlomiej's
>> comments about that we want the blkmq port being a separate patch(es)
>> and then make the CMDQ patches on top. This worries me, because it
>> seems like our messages don't reach you.
>
> Rubbish! I gave a very good rea
On 10/23/2017 04:51 PM, Christoph Hellwig wrote:
> This patch adds native multipath support to the nvme driver. For each
> namespace we create only single block device node, which can be used
> to access that namespace through any of the controllers that refer to it.
> The gendisk for each control
On 10/23/2017 04:51 PM, Christoph Hellwig wrote:
> We do this by adding a helper that returns the ns_head for a device that
> can belong to either the per-controller or per-subsystem block device
> nodes, and otherwise reuse all the existing code.
>
> Signed-off-by: Christoph Hellwig
> Reviewed-b
On 10/23/2017 5:51 PM, Christoph Hellwig wrote:
This helpers allows to bounce steal the uncompleted bios from a request so
that they can be reissued on another path.
Signed-off-by: Christoph Hellwig
Reviewed-by: Sagi Grimberg
---
block/blk-core.c | 20
include/l
Define and use a blk-mq queue. Discards and flushes are processed
synchronously, but reads and writes asynchronously. In order to support
slow DMA unmapping, DMA unmapping is not done until after the next request
is started. That means the request is not completed until then. If there is
no next re
From: Venkat Gopalakrishnan
This patch adds CMDQ support for command-queue compatible
hosts.
Command queue is added in eMMC-5.1 specification. This
enables the controller to process upto 32 requests at
a time.
Adrian Hunter contributed renaming to cqhci, recovery, suspend
and resume, cqhci_off,
Add CQHCI initialization and implement CQHCI operations for Intel GLK.
Signed-off-by: Adrian Hunter
---
drivers/mmc/host/Kconfig | 1 +
drivers/mmc/host/sdhci-pci-core.c | 155 +-
2 files changed, 155 insertions(+), 1 deletion(-)
diff --git a/drive
Add CQE support to the block driver, including:
- optionally using DCMD for flush requests
- "manually" issuing discard requests
- issuing read / write requests to the CQE
- supporting block-layer timeouts
- handling recovery
- supporting re-tuning
CQE offers 25% - 50% bett
Until mmc has blk-mq support fully implemented and tested, add a
parameter use_blk_mq, default to false unless config option MMC_MQ_DEFAULT
is selected.
Signed-off-by: Adrian Hunter
---
drivers/mmc/Kconfig | 11 +++
drivers/mmc/core/core.c | 7 +++
drivers/mmc/core/core.h |
Hi
Here is V12 of the hardware command queue patches without the software
command queue patches, now using blk-mq and now with blk-mq support for
non-CQE I/O.
HW CMDQ offers 25% - 50% better random multi-threaded I/O. I see a slight
2% drop in sequential read speed but no change to sequential wr
From: Tang Junhui
bucket_in_use is updated in gc thread which triggered by invalidating or
writing sectors_to_gc dirty data, It's a long interval. Therefore, when we
use it to compare with the threshold, it is often not timely, which leads
to inaccurate judgment and often results in bucket deplet
On 24/10/17 10:39, Ulf Hansson wrote:
> [...]
>
>>> However, you have completely ignored mine, Linus and Bartlomiej's
>>> comments about that we want the blkmq port being a separate patch(es)
>>> and then make the CMDQ patches on top. This worries me, because it
>>> seems like our messages don't r
Changes from v2
- Combine 2 serises, fixing false positives and enhance performance
- Add Christoph Hellwig's patch simplifying submit_bio_wait() code
- Add 2 more 'init with lockdep map' macros for completionm
- Rename init_completion_with_map() to init_completion_map()
Changes from v1
- Fix kcon
Make whether to allow recording full stack, in cross-release feature,
switchable at boot time via a kernel parameter, 'crossrelease_fullstack'.
In case of a splat with no stack trace, one could just reboot and set
the kernel parameter to get the full data without having to recompile
the kernel.
Ch
Now the performance regression was fixed, re-enable
CONFIG_LOCKDEP_CROSSRELEASE and CONFIG_LOCKDEP_COMPLETIONS.
Signed-off-by: Byungchul Park
---
lib/Kconfig.debug | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 90ea784..fe8fceb
Sometimes, we want to initialize completions with sparate lockdep maps
to assign lock classes as desired. For example, the workqueue code
needs to directly manage lockdep maps, since only the code is aware of
how to classify lockdep maps properly.
Provide additional macros initializing completions
Johan Hovold reported a performance regression by crossrelease like:
> Boot time (from "Linux version" to login prompt) had in fact doubled
> since 4.13 where it took 17 seconds (with my current config) compared to
> the 35 seconds I now see with 4.14-rc4.
>
> I quick bisect pointed to lockdep and
Darrick and Dave Chinner posted the following warning:
> ==
> WARNING: possible circular locking dependency detected
> 4.14.0-rc1-fixes #1 Tainted: GW
> --
> loop0/31693 is trying to acq
Trailing white space is not accepted in kernel coding style. Remove
them.
Signed-off-by: Byungchul Park
---
include/linux/genhd.h | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/include/linux/genhd.h b/include/linux/genhd.h
index ea652bf..6d85a75 100644
--- a/include/linu
The workqueue added manual acquisitions to catch deadlock cases.
Now crossrelease was introduced, some of those are redundant, since
wait_for_completion() already includes the acquisition for itself.
Removed it.
Signed-off-by: Byungchul Park
---
include/linux/workqueue.h | 4 ++--
kernel/workqu
From: Christoph Hellwig
Simplify the code by getting rid of the submit_bio_ret structure.
Signed-off-by: Christoph Hellwig
---
block/bio.c | 19 +--
1 file changed, 5 insertions(+), 14 deletions(-)
diff --git a/block/bio.c b/block/bio.c
index 101c2a9..5e901bf 100644
--- a/bloc
On Mon, Oct 23, 2017 at 06:12:29PM +0200, Roman Penyaev wrote:
> Hi Ming,
>
> On Fri, Oct 20, 2017 at 3:39 PM, Ming Lei wrote:
> > On Wed, Oct 18, 2017 at 12:22:06PM +0200, Roman Pen wrote:
> >> Hi all,
> >>
> >> the patch below fixes queue stalling when shared hctx marked for restart
> >> (BLK_M
* Byungchul Park wrote:
> Johan Hovold reported a performance regression by crossrelease like:
Pplease add Reported-by and Analyzed-by tags - you didn't even Cc: Johan!
Thanks,
Ingo
better changelog:
>
Subject: locking/lockdep: Remove the BROKEN flag from
CONFIG_LOCKDEP_CROSSRELEASE and CONFIG_LOCKDEP_COMPLETIONS
From: Byungchul Park
Date: Tue, 24 Oct 2017 18:38:04 +0900
Now that the performance regression is fixed, re-enable
CONFIG_LOCKDEP_CROSSRELEASE=y
Cannot pick up this series yet, but I have enhanced the changelog to:
=>
Subject: locking/lockdep: Introduce CONFIG_CROSSRELEASE_STACK_TRACE and make it
not unwind by default
From: Byungchul Park
Date: Tue, 24 Oct 2017 18:38:03 +0900
Johan Hovold reported a heavy performance regres
* Byungchul Park wrote:
> Trailing white space is not accepted in kernel coding style. Remove
> them.
>
> Signed-off-by: Byungchul Park
> ---
> include/linux/genhd.h | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/include/linux/genhd.h b/include/linux/genhd.h
> i
* Byungchul Park wrote:
> Make whether to allow recording full stack, in cross-release feature,
> switchable at boot time via a kernel parameter, 'crossrelease_fullstack'.
> In case of a splat with no stack trace, one could just reboot and set
> the kernel parameter to get the full data without
* Byungchul Park wrote:
> Darrick and Dave Chinner posted the following warning:
>
> > ==
> > WARNING: possible circular locking dependency detected
> > 4.14.0-rc1-fixes #1 Tainted: GW
> > --
On 2017/10/24 下午4:57, tang.jun...@zte.com.cn wrote:
> From: Tang Junhui
>
> bucket_in_use is updated in gc thread which triggered by invalidating or
> writing sectors_to_gc dirty data, It's a long interval. Therefore, when we
> use it to compare with the threshold, it is often not timely, which l
On 10/24/2017 12:04 AM, Bart Van Assche wrote:
> Avoid that submitting an SG_IO ioctl triggers a kernel oops that
> is preceded by:
>
> usercopy: kernel memory overwrite attempt detected to (null) () (6
> bytes)
> kernel BUG at mm/usercopy.c:72!
>
> Additionally, make it easier to diagnose crash
Hi Jens,
I just noticed a testing leftover that made it in one of this window's
patches. Can you pick this up (and eventually squash it)?
Thanks,
Javier
Javier González (1):
lightnvm: pblk: remove testing leftover
drivers/lightnvm/pblk.h | 5 -
1 file changed, 5 deletions(-)
--
2.7.4
fixes: 8bd400204bd5 ("lightnvm: pblk: cleanup unused and static functions")
Signed-off-by: Javier González
---
drivers/lightnvm/pblk.h | 5 -
1 file changed, 5 deletions(-)
diff --git a/drivers/lightnvm/pblk.h b/drivers/lightnvm/pblk.h
index 6b64288de6f7..90961033a79f 100644
--- a/drivers/li
On 10/24/2017 07:56 AM, Javier González wrote:
> fixes: 8bd400204bd5 ("lightnvm: pblk: cleanup unused and static functions")
> Signed-off-by: Javier González
Added, thanks.
--
Jens Axboe
On Mon, Oct 23, 2017 at 06:43:16PM +0800, Ming Lei wrote:
> On Fri, Oct 20, 2017 at 04:45:23PM +0200, Christoph Hellwig wrote:
> > We need to look for an active PM request until the next softbarrier
> > instead of looking for the first non-PM request. Otherwise any cause
> > of request reordering
In preparation for unconditionally passing the struct timer_list pointer to
all timer callbacks, switch to using the new timer_setup() and from_timer()
to pass the timer pointer explicitly.
Cc: Matias Bjorling
Cc: linux-block@vger.kernel.org
Signed-off-by: Kees Cook
---
drivers/lightnvm/pblk-co
> +static int bsg_transport_complete_rq(struct request *rq, struct sg_io_v4
> *hdr)
> +{
> + struct bsg_job *job = blk_mq_rq_to_pdu(rq);
> + int ret = 0;
> +
> + /*
> + * The assignments below don't make much sense, but are kept for
> + * bug by bug backwards compatibility:
>
On 10/23/2017 01:17 AM, Martin K. Petersen wrote:
>
> Christoph,
>
>>> Yes, I expected the bsg bits to go through Jens' tree.
>>
>> Ok, then I misremembered it, and we'll have to delay the remaining
>> patches until the next merge window, as they depend on the previous
>> ones.
>
> I don't mind
> On 23 Oct 2017, at 16.51, Christoph Hellwig wrote:
>
> This helper allows reinserting a bio into a new queue without much
> overhead, but requires all queue limits to be the same for the upper
> and lower queues, and it does not provide any recursion preventions.
>
> Signed-off-by: Christoph H
Hi---
On 10/24/2017 01:57 AM, tang.jun...@zte.com.cn wrote:
> From: Tang Junhui
>
> bucket_in_use is updated in gc thread which triggered by invalidating or
> writing sectors_to_gc dirty data, It's a long interval. Therefore, when we
> use it to compare with the threshold, it is often not timely
From: Josef Bacik
If you do not set sk_sndtimeo you will get -ERESTARTSYS if there is a
pending signal when you enter sendmsg, which we handle properly.
However if you set a timeout for your commands we'll set sk_sndtimeo to
that timeout, which means that sendmsg will start returning -EINTR
inste
On Wed, Mar 22, 2017 at 11:29:25PM +0400, Dmitry Monakhov wrote:
> It is reasonable drop page cache on discard, otherwise that pages may
> be written by writeback second later, so thin provision devices will
> not be happy. This seems to be a security leak in case of secure discard
> case.
>
> A
On Mon, Oct 23 2017 at 10:51am -0400,
Christoph Hellwig wrote:
> With this flag a driver can create a gendisk that can be used for I/O
> submission inside the kernel, but which is not registered as user
> facing block device. This will be useful for the NVMe multipath
> implementation.
Having t
On Tue, Oct 24, 2017 at 12:08:58PM +0200, Ingo Molnar wrote:
> This is really unnecessarily complex.
I mis-understood your suggestion. I will change it.
> The proper logic is to introduce the crossrelease_fullstack boot parameter,
> and to
> also have a Kconfig option that enables it:
>
>
On Tue, Oct 24, 2017 at 12:15:51PM +0200, Ingo Molnar wrote:
> > @@ -1409,9 +1403,12 @@ struct gendisk *alloc_disk_node(int minors, int
> > node_id)
> > disk_to_dev(disk)->type = &disk_type;
> > device_initialize(disk_to_dev(disk));
> > }
> > +
> > + lockdep_init_map(
On 2017/10/25 上午2:50, Michael Lyle wrote:
> Hi---
>
> On 10/24/2017 01:57 AM, tang.jun...@zte.com.cn wrote:
>> From: Tang Junhui
>>
>> bucket_in_use is updated in gc thread which triggered by invalidating or
>> writing sectors_to_gc dirty data, It's a long interval. Therefore, when we
>> use it t
If an IO scheduler is selected via elevator= and it doesn't match
the driver in question wrt blk-mq support, then we fail to boot.
The elevator= parameter is deprecated and only supported for
non-mq devices. Augment the elevator lookup API so that we
pass in if we're looking for an mq capable sche
On 10/24/2017 03:33 PM, Omar Sandoval wrote:
> On Wed, Mar 22, 2017 at 11:29:25PM +0400, Dmitry Monakhov wrote:
>> It is reasonable drop page cache on discard, otherwise that pages may
>> be written by writeback second later, so thin provision devices will
>> not be happy. This seems to be a secur
On 10/24/2017 01:57 PM, Josef Bacik wrote:
> From: Josef Bacik
>
> If you do not set sk_sndtimeo you will get -ERESTARTSYS if there is a
> pending signal when you enter sendmsg, which we handle properly.
> However if you set a timeout for your commands we'll set sk_sndtimeo to
> that timeout, whi
On Tue, Oct 24, 2017 at 12:05:16PM +0200, Ingo Molnar wrote:
>
> * Byungchul Park wrote:
>
> > Johan Hovold reported a performance regression by crossrelease like:
>
> Pplease add Reported-by and Analyzed-by tags - you didn't even Cc: Johan!
Excuse me but, I am sure, whom is the issue analyzed
From: Tang Junhui
Thanks to Mike and Coly's comments.
>> + if(ca->set->avail_nbuckets > 0) {
>> + ca->set->avail_nbuckets--;
>> + bch_update_bucket_in_use(ca->set);
>> + }
>> +
>
> I am not sure this needs an atomic
On Tue, Oct 24, 2017 at 6:21 PM, wrote:
>>> static void bch_btree_gc_finish(struct cache_set *c)
>> available should be removed and this function should return 0.
> I have changed this function to a void type, so nothing need return.
Sorry, I misread that. :)
>>> + stats.in_use
From: Tang Junhui
bucket_in_use is updated in gc thread which triggered by invalidating or
writing sectors_to_gc dirty data, It's a long interval. Therefore, when we
use it to compare with the threshold, it is often not timely, which leads
to inaccurate judgment and often results in bucket deplet
Hi Junhui---
I have applied the patch [with a small whitespace edit] to my tree. I
will try to get some test mileage on it.
Reviewed-by: Michael Lyle
Thanks--
Mike
On 10/24/2017 07:26 PM, tang.jun...@zte.com.cn wrote:
> From: Tang Junhui
>
> bucket_in_use is updated in gc thread which trig
There are two things I didn't apply as Ingo suggested, since I didn't
understand his intention exactly:
1. Adding 'Analyzed-by' tag at the 2nd patch
2. Using a inline function instead #define at the 7th patch
Let me know if the above should still be applied.
Changes from v3
- Exclude a pat
Now that the performance regression is fixed, re-enable
CONFIG_LOCKDEP_CROSSRELEASE=y and CONFIG_LOCKDEP_COMPLETIONS=y.
Signed-off-by: Byungchul Park
---
lib/Kconfig.debug | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 3db9167.
Sometimes, we want to initialize completions with sparate lockdep maps
to assign lock classes as desired. For example, the workqueue code
needs to directly manage lockdep maps, since only the code is aware of
how to classify lockdep maps properly.
Provide additional macros initializing completions
Johan Hovold reported a heavy performance regression caused by lockdep
cross-release:
> Boot time (from "Linux version" to login prompt) had in fact doubled
> since 4.13 where it took 17 seconds (with my current config) compared to
> the 35 seconds I now see with 4.14-rc4.
>
> I quick bisect
From: Christoph Hellwig
Simplify the code by getting rid of the submit_bio_ret structure.
Signed-off-by: Christoph Hellwig
---
block/bio.c | 19 +--
1 file changed, 5 insertions(+), 14 deletions(-)
diff --git a/block/bio.c b/block/bio.c
index 5f5472e..99d0ca5 100644
--- a/bloc
The workqueue added manual acquisitions to catch deadlock cases.
Now crossrelease was introduced, some of those are redundant, since
wait_for_completion() already includes the acquisition for itself.
Removed it.
Signed-off-by: Byungchul Park
---
include/linux/workqueue.h | 4 ++--
kernel/workqu
The boot parameter, crossrelease_fullstack, was introduced to control
whether to enable unwind in cross-release or not. Add a Kconfig doing
the same thing.
Suggested-by: Ingo Molnar
Signed-off-by: Byungchul Park
---
kernel/locking/lockdep.c | 4
lib/Kconfig.debug| 15 +
Darrick posted the following warning and Dave Chinner analyzed it:
> ==
> WARNING: possible circular locking dependency detected
> 4.14.0-rc1-fixes #1 Tainted: GW
> --
> loop0/31693 is t
* Byungchul Park wrote:
> On Tue, Oct 24, 2017 at 12:05:16PM +0200, Ingo Molnar wrote:
> >
> > * Byungchul Park wrote:
> >
> > > Johan Hovold reported a performance regression by crossrelease like:
> >
> > Pplease add Reported-by and Analyzed-by tags - you didn't even Cc: Johan!
>
> Excuse
* Byungchul Park wrote:
> > Isn't lockdep_map a zero size structure that is always defined? If yes then
> > there's no need for an #ifdef.
>
> No, a zero size structure for lockdep_map is not provided yet.
> There are two options I can do:
>
> 1. Add a zero size structure for lockdep_map and
* Byungchul Park wrote:
> Darrick posted the following warning and Dave Chinner analyzed it:
>
> > ==
> > WARNING: possible circular locking dependency detected
> > 4.14.0-rc1-fixes #1 Tainted: GW
> > --
On Mon, Oct 16, 2017 at 3:59 PM, Ilya Dryomov wrote:
> Hi Christoph, Martin,
>
> blkdev_issue_zeroout() now checks for any error. This required a minor
> refactor, so I dropped the stable tag, Jens can add it back if needed.
>
> v2 -> v3:
> - another code flow change in blkdev_issue_zeroout() sug
On Thu, Oct 19, 2017 at 2:14 AM, Al Viro wrote:
> On Wed, Oct 18, 2017 at 02:38:38PM +0200, Ilya Dryomov wrote:
>> Check for CAP_SYS_ADMIN before calling into the driver, similar to
>> blkdev_flushbuf(). This is safer and can spare a check in the driver.
>>
>> (Currently BLKROSET is overridden by
On Wed, Oct 25, 2017 at 08:01:24AM +0200, Ingo Molnar wrote:
> Beyond the #ifdef reduction I mentioned in the other thread, there's four
> other
> things I noticed that need to be fixed in this patch:
>
> - Please write out 'minor' instead of the 'm' abbreviation that is
> meaningless.
>'
* Byungchul Park wrote:
> The workqueue added manual acquisitions to catch deadlock cases.
> Now crossrelease was introduced, some of those are redundant, since
> wait_for_completion() already includes the acquisition for itself.
> Removed it.
>
> Signed-off-by: Byungchul Park
> ---
> include
81 matches
Mail list logo
