Re: [PATCH V5 00/14] blk-mq-sched: improve sequential I/O performance(part 1)

2017-10-09 Thread Ming Lei
On Mon, Oct 09, 2017 at 11:04:39PM +0800, Ming Lei wrote: > Hi John, > > On Mon, Oct 09, 2017 at 01:09:22PM +0100, John Garry wrote: > > On 30/09/2017 11:27, Ming Lei wrote: > > > Hi Jens, > > > > > > In Red Hat internal storage test wrt. blk-mq scheduler, we > > > found that I/O performance is m

[PATCH] scsi/aic7xxx: Convert timers to use timer_setup()

2017-10-09 Thread Kees Cook
stat_timer only ever assigns the same function and data, so consolidate to using timer_setup(), adjust callback, drop everything else used to pass things around, and remove needless typedefs. reset_timer is unused; remove it. Cc: Hannes Reinecke Cc: "James E.J. Bottomley" Cc: "Martin K. Peterse

Re: [PATCH v7 0/9] Hello Jens,

2017-10-09 Thread Bart Van Assche
On Mon, 2017-10-09 at 16:13 -0700, Bart Van Assche wrote: > [ ... ] Just like for v6, the title of this series should have been: "block, scsi, md: Improve suspend and resume"

[PATCH v7 4/9] block: Make q_usage_counter also track legacy requests

2017-10-09 Thread Bart Van Assche
From: Ming Lei This patch makes it possible to pause request allocation for the legacy block layer by calling blk_mq_freeze_queue() and blk_mq_unfreeze_queue(). Signed-off-by: Ming Lei [ bvanassche: Combined two patches into one, edited a comment and made sure REQ_NOWAIT is handled properly i

[PATCH v7 2/9] md: Introduce md_stop_all_writes()

2017-10-09 Thread Bart Van Assche
Introduce md_stop_all_writes() because the next patch will add a second caller for this function. This patch does not change any functionality. Signed-off-by: Bart Van Assche Cc: Shaohua Li Cc: linux-r...@vger.kernel.org Cc: Ming Lei Cc: Christoph Hellwig Cc: Hannes Reinecke Cc: Johannes Thum

[PATCH v7 5/9] block: Introduce blk_get_request_flags()

2017-10-09 Thread Bart Van Assche
A side effect of this patch is that the GFP mask that is passed to several allocation functions in the legacy block layer is changed from GFP_KERNEL into __GFP_DIRECT_RECLAIM. Signed-off-by: Bart Van Assche Cc: Christoph Hellwig Cc: Ming Lei Cc: Hannes Reinecke Cc: Johannes Thumshirn --- blo

[PATCH v7 9/9] block, scsi: Make SCSI quiesce and resume work reliably

2017-10-09 Thread Bart Van Assche
The contexts from which a SCSI device can be quiesced or resumed are: * Writing into /sys/class/scsi_device/*/device/state. * SCSI parallel (SPI) domain validation. * The SCSI device power management methods. See also scsi_bus_pm_ops. It is essential during suspend and resume that neither the file

[PATCH v7 1/9] md: Rename md_notifier into md_reboot_notifier

2017-10-09 Thread Bart Van Assche
This avoids confusion with the pm notifier that will be added through a later patch. Signed-off-by: Bart Van Assche Cc: Shaohua Li Cc: linux-r...@vger.kernel.org Cc: Ming Lei Cc: Christoph Hellwig Cc: Hannes Reinecke Cc: Johannes Thumshirn --- drivers/md/md.c | 6 +++--- 1 file changed, 3 i

[PATCH v7 7/9] ide, scsi: Tell the block layer at request allocation time about preempt requests

2017-10-09 Thread Bart Van Assche
Convert blk_get_request(q, op, __GFP_RECLAIM) into blk_get_request_flags(q, op, BLK_MQ_PREEMPT). This patch does not change any functionality. Signed-off-by: Bart Van Assche Cc: Martin K. Petersen Acked-by: David S. Miller [ for IDE ] Cc: Ming Lei Cc: Christoph Hellwig Cc: Hannes Reinecke Cc

[PATCH v7 3/9] md: Neither resync nor reshape while the system is frozen

2017-10-09 Thread Bart Van Assche
Some people use the md driver on laptops and use the suspend and resume functionality. Since it is essential that submitting of new I/O requests stops before a hibernation image is created, interrupt the md resync and reshape actions if the system is being frozen. Note: the resync and reshape will

[PATCH v7 0/9] Hello Jens,

2017-10-09 Thread Bart Van Assche
It is known that during the resume following a hibernate, especially when using an md RAID1 array created on top of SCSI devices, sometimes the system hangs instead of coming up properly. This patch series fixes this problem. This patch series is an alternative for Ming Lei's "block/scsi: safe SCSI

[PATCH v7 6/9] block: Introduce BLK_MQ_REQ_PREEMPT

2017-10-09 Thread Bart Van Assche
Set RQF_PREEMPT if BLK_MQ_REQ_PREEMPT is passed to blk_get_request_flags(). Signed-off-by: Bart Van Assche Cc: Christoph Hellwig Cc: Ming Lei Cc: Hannes Reinecke Cc: Johannes Thumshirn --- block/blk-core.c | 4 +++- block/blk-mq.c | 2 ++ include/linux/blk-mq.h | 1 + 3 files c

[PATCH v7 8/9] block: Add the QUEUE_FLAG_PREEMPT_ONLY request queue flag

2017-10-09 Thread Bart Van Assche
This flag will be used in the next patch to let the block layer core know whether or not a SCSI request queue has been quiesced. A quiesced SCSI queue namely only processes RQF_PREEMPT requests. Signed-off-by: Bart Van Assche Cc: Ming Lei Cc: Christoph Hellwig Cc: Hannes Reinecke Cc: Johannes

Re: [PATCH] scsi: libiscsi: fix shifting of DID_REQUEUE host byte

2017-10-09 Thread Lee Duncan
On 10/09/2017 04:33 AM, Johannes Thumshirn wrote: > The SCSI host byte should be shifted left by 16 in order to have > scsi_decide_disposition() do the right thing (.i.e. requeue the command). > > Signed-off-by: Johannes Thumshirn > Fixes: 661134ad3765 ("[SCSI] libiscsi, bnx2i: make bound ep ch

[PATCH] scsi: aic7xxx: explain a mysterious build failure message

2017-10-09 Thread Adam Borowski
"*** Install db development libraries" doesn't ring a bell that it's talking about libdb (Berkeley DB). Because of BDB's relicensing to Affero, incompatible with many users including Linux, most if not all distributions can be expected to keep version 5.3 for the foreseable future, thus we can poi

Re: [PATCH linux-firmware 1/1] qed: Add firmware 8.30.16.0

2017-10-09 Thread Ben Hutchings
On Wed, 2017-09-13 at 03:46 -0700, Rahul Verma wrote: > The new qed firmware contains fixes to firmware and added > support for new features, > -Add UFP support. > -DCQCN support for unlimited number of QP > -Add IP type to GFT filter profile. > -Added new TCP function counters. > -Support flow ID

Re: [PATCH V5 00/14] blk-mq-sched: improve sequential I/O performance(part 1)

2017-10-09 Thread Ming Lei
Hi John, On Mon, Oct 09, 2017 at 01:09:22PM +0100, John Garry wrote: > On 30/09/2017 11:27, Ming Lei wrote: > > Hi Jens, > > > > In Red Hat internal storage test wrt. blk-mq scheduler, we > > found that I/O performance is much bad with mq-deadline, especially > > about sequential I/O on some mult

Re: [PATCH] scsi: libiscsi: fix shifting of DID_REQUEUE host byte

2017-10-09 Thread Johannes Thumshirn
On Mon, Oct 09, 2017 at 02:35:06PM +0200, Steffen Maier wrote: > Use wrapper functions to advertize their use in an attempt to avoid wrong > shifting in the future? After a second thought and a bit of coccinelle magic I converted all drivers under drivers/scsi to use set_host_byte(), msg and drive

Re: [PATCH 10/13] timer: Remove expires and data arguments from DEFINE_TIMER

2017-10-09 Thread Ralf Baechle
On Wed, Oct 04, 2017 at 04:27:04PM -0700, Kees Cook wrote: > Subject: [PATCH 10/13] timer: Remove expires and data arguments from > DEFINE_TIMER > > Drop the arguments from the macro and adjust all callers with the > following script: > > perl -pi -e 's/DEFINE_TIMER\((.*), 0, 0\);/DEFINE_TIME

[PATCH 3.16 051/192] scsi: virtio_scsi: let host do exception handling

2017-10-09 Thread Ben Hutchings
3.16.49-rc1 review patch. If anyone has any objections, please let me know. -- From: Paolo Bonzini commit e72c9a2a67a6400c8ef3d01d4c461dbbbfa0e1f0 upstream. virtio_scsi tries to do exception handling after the default 30 seconds timeout expires. However, it's better to let th

Re: [PATCH 09/13] timer: Remove users of expire and data arguments to DEFINE_TIMER

2017-10-09 Thread Ralf Baechle
On Wed, Oct 04, 2017 at 04:27:03PM -0700, Kees Cook wrote: > Subject: [PATCH 09/13] timer: Remove users of expire and data arguments to > DEFINE_TIMER > > The expire and data arguments of DEFINE_TIMER are only used in two places > and are ignored by the code (malta-display.c only uses mod_timer(

Re: [PATCH] scsi: libiscsi: fix shifting of DID_REQUEUE host byte

2017-10-09 Thread Johannes Thumshirn
On Mon, Oct 09, 2017 at 02:35:06PM +0200, Steffen Maier wrote: > Use wrapper functions to advertize their use in an attempt to avoid wrong > shifting in the future? Not sure, converting all the users would be a lot of churn for relatively low benefit: linux (master)$ git grep "result = DID_" driv

Re: [PATCH] scsi: libiscsi: fix shifting of DID_REQUEUE host byte

2017-10-09 Thread Steffen Maier
Use wrapper functions to advertize their use in an attempt to avoid wrong shifting in the future? On 10/09/2017 01:33 PM, Johannes Thumshirn wrote: The SCSI host byte should be shifted left by 16 in order to have scsi_decide_disposition() do the right thing (.i.e. requeue the command). Signed-

Re: [PATCH V5 00/14] blk-mq-sched: improve sequential I/O performance(part 1)

2017-10-09 Thread John Garry
On 30/09/2017 11:27, Ming Lei wrote: Hi Jens, In Red Hat internal storage test wrt. blk-mq scheduler, we found that I/O performance is much bad with mq-deadline, especially about sequential I/O on some multi-queue SCSI devcies(lpfc, qla2xxx, SRP...) Turns out one big issue causes the performanc

[PATCH] scsi: libiscsi: fix shifting of DID_REQUEUE host byte

2017-10-09 Thread Johannes Thumshirn
The SCSI host byte should be shifted left by 16 in order to have scsi_decide_disposition() do the right thing (.i.e. requeue the command). Signed-off-by: Johannes Thumshirn Fixes: 661134ad3765 ("[SCSI] libiscsi, bnx2i: make bound ep check common") Cc: Lee Duncan Cc: Hannes Reinecke Cc: Bart Van

[PATCH V6 1/5] blk-mq-sched: fix scheduler bad performance

2017-10-09 Thread Ming Lei
When hw queue is busy, we shouldn't take requests from scheduler queue any more, otherwise it is difficult to do IO merge. This patch fixes the awful IO performance on some SCSI devices(lpfc, qla2xxx, ...) when mq-deadline/kyber is used by not taking requests if hw queue is busy. Reviewed-by: Bar

[PATCH V6 0/5] blk-mq-sched: improve sequential I/O performance

2017-10-09 Thread Ming Lei
Hi Jens, In Red Hat internal storage test wrt. blk-mq scheduler, we found that I/O performance is much bad with mq-deadline, especially about sequential I/O on some multi-queue SCSI devcies(lpfc, qla2xxx, SRP...) Turns out one big issue causes the performance regression: requests are still dequeu

[PATCH V6 2/5] blk-mq-sched: move actual dispatching into one helper

2017-10-09 Thread Ming Lei
So that it becomes easy to support to dispatch from sw queue in the following patch. No functional change. Reviewed-by: Bart Van Assche Reviewed-by: Omar Sandoval Suggested-by: Christoph Hellwig # for simplifying dispatch logic Signed-off-by: Ming Lei --- block/blk-mq-sched.c | 43 ++

[PATCH V6 5/5] blk-mq-sched: don't dequeue request until all in ->dispatch are flushed

2017-10-09 Thread Ming Lei
During dispatching, we moved all requests from hctx->dispatch to one temporary list, then dispatch them one by one from this list. Unfortunately during this period, run queue from other contexts may think the queue is idle, then start to dequeue from sw/scheduler queue and still try to dispatch bec

[PATCH V6 3/5] sbitmap: introduce __sbitmap_for_each_set()

2017-10-09 Thread Ming Lei
We need to iterate ctx starting from any ctx in round robin way, so introduce this helper. Cc: Omar Sandoval Signed-off-by: Ming Lei --- include/linux/sbitmap.h | 64 - 1 file changed, 47 insertions(+), 17 deletions(-) diff --git a/include/linux/

[PATCH V6 4/5] blk-mq-sched: improve dispatching from sw queue

2017-10-09 Thread Ming Lei
SCSI devices use host-wide tagset, and the shared driver tag space is often quite big. Meantime there is also queue depth for each lun( .cmd_per_lun), which is often small, for example, on both lpfc and qla2xxx, .cmd_per_lun is just 3. So lots of requests may stay in sw queue, and we always flush

Re: [PATCH V5 7/7] blk-mq-sched: don't dequeue request until all in ->dispatch are flushed

2017-10-09 Thread Ming Lei
On Tue, Oct 03, 2017 at 02:11:28AM -0700, Christoph Hellwig wrote: > This looks good in general: > > Reviewed-by: Christoph Hellwig > > Minor nitpicks below: > > > const bool has_sched_dispatch = e && e->type->ops.mq.dispatch_request; > > This is now only tested once, so you can remove the

Re: [PATCH V5 6/7] blk-mq-sched: improve dispatching from sw queue

2017-10-09 Thread Ming Lei
On Tue, Oct 03, 2017 at 02:05:27AM -0700, Christoph Hellwig wrote: > On Sat, Sep 30, 2017 at 06:27:19PM +0800, Ming Lei wrote: > > SCSI devices use host-wide tagset, and the shared > > driver tag space is often quite big. Meantime > > there is also queue depth for each lun(.cmd_per_lun), > > which

Re: [PATCH V5 5/7] blk-mq-sched: move actual dispatching into one helper

2017-10-09 Thread Ming Lei
On Mon, Oct 02, 2017 at 07:19:56AM -0700, Christoph Hellwig wrote: > Can you move this to the beginning of your series, just after > the other edits to blk_mq_sched_dispatch_requests? OK. > > > +static void blk_mq_do_dispatch_sched(struct request_queue *q, > > +st