Re: [PATCH blktests 0/3] Add SCSI generic test group

2017-05-18 Thread Johannes Thumshirn
On 05/19/2017 12:46 AM, Omar Sandoval wrote: > Looking at this some more, it seems like the syzkaller reproducer always > bangs on /dev/sg0. How hard would it be to adapt it to run on the sg > device for every test device instead? Can't be too hard I guess ;-). Maybe I can even clean it up a bit

Re: [PATCH v4 05/27] btrfs: btrfs_wait_tree_block_writeback can be void return

2017-05-18 Thread Liu Bo
On Tue, May 09, 2017 at 11:49:08AM -0400, Jeff Layton wrote: > Nothing checks its return value. Reviewed-by: Liu Bo -liubo > > Signed-off-by: Jeff Layton > --- > fs/btrfs/disk-io.c | 6 +++--- > fs/btrfs/disk-io.h | 2 +- > 2 files changed, 4 insertions(+), 4 deletions(-) > > diff --git a/fs

[PATCH] block: partitions: mac: Mark mac_fix_string() as __maybe_unused

2017-05-18 Thread Matthias Kaehlcke
The function is only used when CONFIG_PPC_PMAC=y. Adding the attribute fixes the following warning when building with clang: block/partitions/mac.c:22:20: error: unused function 'mac_fix_string' [-Werror,-Wunused-function] Signed-off-by: Matthias Kaehlcke --- block/partitions/mac.c | 2 +-

Re: BUG: hot removal during writes on ext4 formatted nvme device

2017-05-18 Thread Jon Derrick
Hi Ming, Dmitry, Ming, > Also the following patch fixes one issue in remove path. > > http://marc.info/?l=linux-block&m=149498450028434&w=2 > > So could you test v4.12-rc1(d3cfb2a0 is merged) with the above patch? Thanks for the suggestion but it still resulted in the same BUG. Dmitry,

Re: [PATCH blktests 0/3] Add SCSI generic test group

2017-05-18 Thread Omar Sandoval
On Thu, May 18, 2017 at 03:29:45PM +0200, Johannes Thumshirn wrote: > On 05/18/2017 03:19 PM, Christoph Hellwig wrote: > > All SG_IO test should also apply to block device nodes that support > > the ioctl.. > > > > But these are not necessarily SG_IO tests, are they? > > The test included is doe

Re: [PATCH blktests 2/3] tests/sg: add SCSI generic test grouop

2017-05-18 Thread Omar Sandoval
On Thu, May 18, 2017 at 02:06:20PM -0700, Omar Sandoval wrote: > On Thu, May 18, 2017 at 02:13:07PM +0200, Johannes Thumshirn wrote: > > Add a test group for tests of the SCSI generic driver and and > > functions common to the SCSI generic driver and it's test cases. > > > > Signed-off-by: Johanne

Re: [PATCH blktests 3/3] sg/001: add regression test for syzcaller generated GPF

2017-05-18 Thread Omar Sandoval
On Thu, May 18, 2017 at 02:13:08PM +0200, Johannes Thumshirn wrote: > Add a regression test for commit 48ae8484e9fc ("scsi: sg: don't return > bogus Sg_requests"). This is a general protection fault triggered by > syzcaller. > > Signed-off-by: Johannes Thumshirn > --- > tests/sg/001 | 48 +++

Re: [PATCH blktests 2/3] tests/sg: add SCSI generic test grouop

2017-05-18 Thread Omar Sandoval
On Thu, May 18, 2017 at 02:13:07PM +0200, Johannes Thumshirn wrote: > Add a test group for tests of the SCSI generic driver and and > functions common to the SCSI generic driver and it's test cases. > > Signed-off-by: Johannes Thumshirn > --- > common/sg | 22 ++ > tests

Re: [PATCH 2/2] nvme: avoid to hang in remove disk

2017-05-18 Thread Keith Busch
On Thu, May 18, 2017 at 11:35:43PM +0800, Ming Lei wrote: > On Thu, May 18, 2017 at 03:49:31PM +0200, Christoph Hellwig wrote: > > On Wed, May 17, 2017 at 09:27:29AM +0800, Ming Lei wrote: > > > If some writeback requests are submitted just before queue is killed, > > > and these requests may not b

Re: [PATCH 2/2] nvme: avoid to hang in remove disk

2017-05-18 Thread Ming Lei
On Thu, May 18, 2017 at 03:49:31PM +0200, Christoph Hellwig wrote: > On Wed, May 17, 2017 at 09:27:29AM +0800, Ming Lei wrote: > > If some writeback requests are submitted just before queue is killed, > > and these requests may not be canceled in nvme_dev_disable() because > > they are not started

Re: [PATCH 1/2] nvme: fix race between removing and reseting failure

2017-05-18 Thread Ming Lei
On Thu, May 18, 2017 at 03:47:34PM +0200, Christoph Hellwig wrote: > On Wed, May 17, 2017 at 09:27:28AM +0800, Ming Lei wrote: > > When one NVMe PCI device is being resetted and found reset failue, > > nvme_remove_dead_ctrl() is called to handle the failure: blk-mq hw queues > > are put into stoppe

Re: dedicated error codes for the block layer

2017-05-18 Thread David Sterba
On Thu, May 18, 2017 at 03:17:57PM +0200, Christoph Hellwig wrote: > This series introduces a new blk_status_t error code type for the block > layer so that we can have tigher control and explicit semantics for > block layer errors. > > All but the last three patches are cleanups that lead to the

Re: [PATCH 12/15] block: merge blk_types.h into bio.h

2017-05-18 Thread Bart Van Assche
On Thu, 2017-05-18 at 15:18 +0200, Christoph Hellwig wrote: > We've cleaned up our headers sufficiently that we don't need this split > anymore. Hello Christoph, Request-based drivers need the structure definitions from and the type definitions from but do not need the definition of struct bio.

Re: BUG: hot removal during writes on ext4 formatted nvme device

2017-05-18 Thread Dmitry Monakhov
Jon Derrick writes: > Hello, > > I've encountered a BUG that I've experienced during hot removal on an > ext4-formatted nvme device undergoing writes. I have been able to verify > that 4.5, 4.6, 4.10.12, 4.11, and 4.12-rc1 show similar issues (the v4.6 > trace below shows issues with block that h

Re: [PATCH 1/2] nvme: fix race between removing and reseting failure

2017-05-18 Thread Keith Busch
On Wed, May 17, 2017 at 09:27:28AM +0800, Ming Lei wrote: > When one NVMe PCI device is being resetted and found reset failue, > nvme_remove_dead_ctrl() is called to handle the failure: blk-mq hw queues > are put into stopped first, then schedule .remove_work to release the driver. > > Unfortunate

Re: [PATCH 2/2] nvme: avoid to hang in remove disk

2017-05-18 Thread Christoph Hellwig
On Wed, May 17, 2017 at 09:27:29AM +0800, Ming Lei wrote: > If some writeback requests are submitted just before queue is killed, > and these requests may not be canceled in nvme_dev_disable() because > they are not started yet, it is still possible for blk-mq to hold > these requests in .requeue l

Re: [PATCH 1/2] nvme: fix race between removing and reseting failure

2017-05-18 Thread Christoph Hellwig
On Wed, May 17, 2017 at 09:27:28AM +0800, Ming Lei wrote: > When one NVMe PCI device is being resetted and found reset failue, > nvme_remove_dead_ctrl() is called to handle the failure: blk-mq hw queues > are put into stopped first, then schedule .remove_work to release the driver. > > Unfortunate

Re: [PATCH 2/5] mmc: core: Allocate per-request data using the block layer core

2017-05-18 Thread Linus Walleij
On Thu, May 18, 2017 at 2:42 PM, Adrian Hunter wrote: > On 18/05/17 11:21, Linus Walleij wrote: >> It's better to do what everyone else does and let the core do this >> allocation of extra data (tag) instead. > > I agree it is much nicer, but the extra bounce buffer allocations still seem > gratu

Re: [PATCH blktests 0/3] Add SCSI generic test group

2017-05-18 Thread Johannes Thumshirn
On 05/18/2017 03:19 PM, Christoph Hellwig wrote: > All SG_IO test should also apply to block device nodes that support > the ioctl.. > But these are not necessarily SG_IO tests, are they? The test included is doesn't hit the SG_IO path in the sg driver, but the sg_read path. Of cause we can mak

[PATCH 02/15] scsi/osd: don't save block errors into req_results

2017-05-18 Thread Christoph Hellwig
We will only have sense data if the command exectured and got a SCSI result, so this is pointless. Signed-off-by: Christoph Hellwig --- drivers/scsi/osd/osd_initiator.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/scsi/osd/osd_initiator.c b/drivers/scsi/osd/osd_ini

dedicated error codes for the block layer

2017-05-18 Thread Christoph Hellwig
This series introduces a new blk_status_t error code type for the block layer so that we can have tigher control and explicit semantics for block layer errors. All but the last three patches are cleanups that lead to the new type. The series it mostly limited to the block layer and drivers, and t

[PATCH 03/15] gfs2: remove the unused sd_log_error field

2017-05-18 Thread Christoph Hellwig
Signed-off-by: Christoph Hellwig --- fs/gfs2/incore.h | 1 - fs/gfs2/lops.c | 4 +--- 2 files changed, 1 insertion(+), 4 deletions(-) diff --git a/fs/gfs2/incore.h b/fs/gfs2/incore.h index b7cf65d13561..aa3d44527fa2 100644 --- a/fs/gfs2/incore.h +++ b/fs/gfs2/incore.h @@ -815,7 +815,6 @@ struc

[PATCH 04/15] dm: fix REQ_RAHEAD handling

2017-05-18 Thread Christoph Hellwig
A few (but not all) dm targets use a special EWOULDBLOCK error code for failing REQ_RAHEAD requests that fail due to a lack of available resources. But no one else knows about this magic code, and lower level drivers also don't generate it when failing read-ahead requests for similar reasons. So r

[PATCH 01/15] nvme-lightnvm: use blk_execute_rq in nvme_nvm_submit_user_cmd

2017-05-18 Thread Christoph Hellwig
Instead of reinventing it poorly. Signed-off-by: Christoph Hellwig Reviewed-by: Javier González --- drivers/nvme/host/lightnvm.c | 12 +--- 1 file changed, 1 insertion(+), 11 deletions(-) diff --git a/drivers/nvme/host/lightnvm.c b/drivers/nvme/host/lightnvm.c index f5df78ed1e10..f3885

[PATCH 05/15] fs: remove the unused error argument to dio_end_io()

2017-05-18 Thread Christoph Hellwig
Signed-off-by: Christoph Hellwig --- fs/btrfs/inode.c | 6 +++--- fs/direct-io.c | 3 +-- include/linux/fs.h | 2 +- 3 files changed, 5 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 17cbe9306faf..758b2666885e 100644 --- a/fs/btrfs/inode.c +++ b/fs/btr

[PATCH 07/15] block_dev: propagate bio_iov_iter_get_pages error in __blkdev_direct_IO

2017-05-18 Thread Christoph Hellwig
Once we move the block layer to its own status code we'll still want to propagate the bio_iov_iter_get_pages, so restructure __blkdev_direct_IO to take ret into account when returning the errno. --- fs/block_dev.c | 7 --- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/fs/block_

[PATCH 08/15] dm mpath: merge do_end_io_bio into multipath_end_io_bio

2017-05-18 Thread Christoph Hellwig
This simplifies the code and especially the error passing a bit and will help with the next patch. Signed-off-by: Christoph Hellwig --- drivers/md/dm-mpath.c | 42 -- 1 file changed, 16 insertions(+), 26 deletions(-) diff --git a/drivers/md/dm-mpath.c b/d

[PATCH 10/15] dm: change ->end_io calling convention

2017-05-18 Thread Christoph Hellwig
Turn the error paramter into a pointer so that target drivers can change the value, and make sure only DM_ENDIO_* values are returned from the methods. Signed-off-by: Christoph Hellwig --- drivers/md/dm-cache-target.c | 4 ++-- drivers/md/dm-flakey.c| 8 drivers/md/dm-log-wri

[PATCH 12/15] block: merge blk_types.h into bio.h

2017-05-18 Thread Christoph Hellwig
We've cleaned up our headers sufficiently that we don't need this split anymore. Signed-off-by: Christoph Hellwig --- block/blk-wbt.c| 2 +- drivers/target/target_core_pscsi.c | 2 +- include/linux/bio.h| 307 +++- include/l

[PATCH 11/15] fs-writeback: move wbc_to_write_flags out of line

2017-05-18 Thread Christoph Hellwig
This way writeback.h doesn't need to pull in blk_types.h. Signed-off-by: Christoph Hellwig --- fs/fs-writeback.c | 13 + include/linux/writeback.h | 11 +-- 2 files changed, 14 insertions(+), 10 deletions(-) diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c index 63

Re: [PATCH blktests 0/3] Add SCSI generic test group

2017-05-18 Thread Christoph Hellwig
All SG_IO test should also apply to block device nodes that support the ioctl..

[PATCH 14/15] blk-mq: switch ->queue_rq return value to blk_status_t

2017-05-18 Thread Christoph Hellwig
Use the same values for use for request completion errors as the return value from ->queue_rq. BLK_STS_RESOURCE is special cased to cause a requeue, and all the others are completed as-is. Signed-off-by: Christoph Hellwig --- block/blk-mq.c| 37 --

[PATCH 09/15] dm: don't return errnos from ->map

2017-05-18 Thread Christoph Hellwig
Instead use the special DM_MAPIO_KILL return value to return -EIO just like we do for the request based path. Note that dm-log-writes returned -ENOMEM in a few places, which now becomes -EIO instead. No consumer treats -ENOMEM special so this shouldn't be an issue (and it should use a mempool to

[PATCH 06/15] fs: simplify dio_bio_complete

2017-05-18 Thread Christoph Hellwig
Only read bio->bi_error once in the common path. Signed-off-by: Christoph Hellwig --- fs/direct-io.c | 6 ++ 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/fs/direct-io.c b/fs/direct-io.c index 04247a6c3f73..bb711e4b86c2 100644 --- a/fs/direct-io.c +++ b/fs/direct-io.c @@ -477

Re: [PATCH 2/5] mmc: core: Allocate per-request data using the block layer core

2017-05-18 Thread Adrian Hunter
On 18/05/17 11:21, Linus Walleij wrote: > On Tue, May 16, 2017 at 1:54 PM, Adrian Hunter > wrote: >> On 10/05/17 11:24, Linus Walleij wrote: >>> The mmc_queue_req is a per-request state container the MMC core uses >>> to carry bounce buffers, pointers to asynchronous requests and so on. >>> Curre

Re: [PATCH 2/6 v2] mmc: core: Allocate per-request data using the block layer core

2017-05-18 Thread Linus Walleij
On Thu, May 18, 2017 at 11:32 AM, Christoph Hellwig wrote: > Btw, you can also remove the struct request backpointer in > struct mmc_queue_req now - blk_mq_rq_from_pdu will do it for you > without the need for a pointer. Thanks I made a patch for this in the front of my next clean-up series. Yo

[PATCH blktests 2/3] tests/sg: add SCSI generic test grouop

2017-05-18 Thread Johannes Thumshirn
Add a test group for tests of the SCSI generic driver and and functions common to the SCSI generic driver and it's test cases. Signed-off-by: Johannes Thumshirn --- common/sg | 22 ++ tests/sg/group | 40 2 files changed, 62 inser

[PATCH blktests 0/3] Add SCSI generic test group

2017-05-18 Thread Johannes Thumshirn
Add a test group for the SCSI generic driver and one syzcaller reproducer for this group. The reprodcuer is distributed as a C program, so the makefile is amended to build C files to be used in the test. I didn't get the TIMEOUT to work (not even with block/001) so I decided to just require the '

[PATCH blktests 3/3] sg/001: add regression test for syzcaller generated GPF

2017-05-18 Thread Johannes Thumshirn
Add a regression test for commit 48ae8484e9fc ("scsi: sg: don't return bogus Sg_requests"). This is a general protection fault triggered by syzcaller. Signed-off-by: Johannes Thumshirn --- tests/sg/001 | 48 tests/sg/001.out | 2 ++ 2 files c

[PATCH blktests 1/3] Add ability to build test-cases

2017-05-18 Thread Johannes Thumshirn
Add the ability to build test cases from C files. This is handy for things like syzcaller reproducers and all other kinds of test binaries. Signed-off-by: Johannes Thumshirn --- Makefile | 26 +++- src/.gitignore | 1 + src/Makefile | 14 ++ src/sg-001.c | 430 +

Re: Outstanding MQ questions from MMC

2017-05-18 Thread Christoph Hellwig
On Wed, Mar 29, 2017 at 05:09:37AM +0200, Linus Walleij wrote: > 2. When MMC cards are ejected a serious error condition occurs. So for this > reason we spool out the queue with > > req->rq_flags |= RQF_QUIET; > blk_end_request_all(req, -EIO); > > This will shut up a huge amount of console errors

Re: Outstanding MQ questions from MMC

2017-05-18 Thread Linus Walleij
On Tue, Apr 18, 2017 at 5:31 PM, Alex Lemberg wrote: > There is an additional functionality, which is require the host lock > to be held for several write commands - the FFU. > In case of FFU, the FW can be download/write in several iterations > of Write command (CMD25). This sequence should not

Re: [PATCH 4/6 v2] mmc: block: move single ioctl() commands to block requests

2017-05-18 Thread Christoph Hellwig
On Thu, May 18, 2017 at 11:29:34AM +0200, Linus Walleij wrote: > We are storing the ioctl() in/out argument as a pointer in > the per-request struct mmc_blk_request container. Btw, for the main ioctl data (not the little reponse field) it might make sense to use blk_rq_map_user, which will do a ge

Re: [PATCH 2/6 v2] mmc: core: Allocate per-request data using the block layer core

2017-05-18 Thread Christoph Hellwig
Btw, you can also remove the struct request backpointer in struct mmc_queue_req now - blk_mq_rq_from_pdu will do it for you without the need for a pointer.

[PATCH 4/6 v2] mmc: block: move single ioctl() commands to block requests

2017-05-18 Thread Linus Walleij
This wraps single ioctl() commands into block requests using the custom block layer request types REQ_OP_DRV_IN and REQ_OP_DRV_OUT. By doing this we are loosening the grip on the big host lock, since two calls to mmc_get_card()/mmc_put_card() are removed. We are storing the ioctl() in/out argumen

[PATCH 6/6 v2] mmc: queue: delete mmc_req_is_special()

2017-05-18 Thread Linus Walleij
commit cdf8a6fb48882651049e468e6b16956fb83db86c "mmc: block: Introduce queue semantics" deleted the last user of mmc_req_is_special() and it was a horrible hack to classify requests as "special" or "not special" to begin with, so delete the helper. Signed-off-by: Linus Walleij --- ChangeLog v1->v

[PATCH 5/6 v2] mmc: block: move multi-ioctl() to use block layer

2017-05-18 Thread Linus Walleij
This switches also the multiple-command ioctl() call to issue all ioctl()s through the block layer instead of going directly to the device. We extend the passed argument with an argument count and loop over all passed commands in the ioctl() issue function called from the block layer. By doing th

[PATCH 3/6 v2] mmc: block: Tag is_rpmb as bool

2017-05-18 Thread Linus Walleij
The variable is_rpmb is clearly a bool and even assigned true and false, yet declared as an int. Signed-off-by: Linus Walleij --- ChangeLog v1->v2: - No changes, just resending --- drivers/mmc/core/block.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/mmc/core/block

[PATCH 2/6 v2] mmc: core: Allocate per-request data using the block layer core

2017-05-18 Thread Linus Walleij
The mmc_queue_req is a per-request state container the MMC core uses to carry bounce buffers, pointers to asynchronous requests and so on. Currently allocated as a static array of objects, then as a request comes in, a mmc_queue_req is assigned to it, and used during the lifetime of the request. T

[PATCH 1/6 v2] mmc: core: Delete bounce buffer Kconfig option

2017-05-18 Thread Linus Walleij
This option is activated by all multiplatform configs and what not so we almost always have it turned on, and the memory it saves is negligible, even more so moving forward. The actual bounce buffer only gets allocated only when used, the only thing the ifdefs are saving is a little bit of code. I

Re: [PATCH 5/5] mmc: block: move multi-ioctl() to use block layer

2017-05-18 Thread Linus Walleij
On Fri, May 12, 2017 at 11:09 PM, Avri Altman wrote: >> + req = blk_get_request(mq->queue, >> + idata[0]->ic.write_flag ? REQ_OP_DRV_OUT : REQ_OP_DRV_IN, >> + __GFP_RECLAIM); > > It is possible, e.g. as in RPMB access, that some commands are read and some > are write.

Re: [PATCH 1/2] block: Introduce blkdev_issue_flush_no_wait()

2017-05-18 Thread Anand Jain
On 05/16/2017 07:56 PM, Christoph Hellwig wrote: On Tue, May 16, 2017 at 05:39:13PM +0800, Anand Jain wrote: blkdev_issue_flush() is a blocking function and returns only after the flush bio is completed, so a module handling more than one device can't issue flush for all the devices unless it

Re: [RFC PATCH 0/2] Introduce blkdev_issue_flush_no_wait()

2017-05-18 Thread Anand Jain
On 05/18/2017 01:14 AM, David Sterba wrote: On Tue, May 16, 2017 at 02:07:23PM +, Bart Van Assche wrote: On Tue, 2017-05-16 at 17:39 +0800, Anand Jain wrote: BTRFS wanted a block device flush function which does not wait for its completion, so that the flush for the next device can be cal

Re: [RFC PATCH 0/2] Introduce blkdev_issue_flush_no_wait()

2017-05-18 Thread Anand Jain
On 05/16/2017 10:07 PM, Bart Van Assche wrote: On Tue, 2017-05-16 at 17:39 +0800, Anand Jain wrote: BTRFS wanted a block device flush function which does not wait for its completion, so that the flush for the next device can be called in the same thread. Here is a RFC patch to provide the fun

Re: [PATCH 2/5] mmc: core: Allocate per-request data using the block layer core

2017-05-18 Thread Linus Walleij
On Tue, May 16, 2017 at 1:54 PM, Adrian Hunter wrote: > On 10/05/17 11:24, Linus Walleij wrote: >> The mmc_queue_req is a per-request state container the MMC core uses >> to carry bounce buffers, pointers to asynchronous requests and so on. >> Currently allocated as a static array of objects, then

Re: [PATCH 2/5] mmc: core: Allocate per-request data using the block layer core

2017-05-18 Thread Linus Walleij
On Tue, May 16, 2017 at 11:02 AM, Ulf Hansson wrote: > On 10 May 2017 at 10:24, Linus Walleij wrote: >> @@ -1662,7 +1655,7 @@ static void mmc_blk_rw_try_restart(struct mmc_queue >> *mq, struct request *req, >> if (mmc_card_removed(mq->card)) { >> req->rq_flags |= RQF_QUI

Re: [PATCH 1/5] mmc: core: Delete bounce buffer Kconfig option

2017-05-18 Thread Linus Walleij
On Mon, May 15, 2017 at 4:04 PM, Bartlomiej Zolnierkiewicz wrote: > [ I added Daniel, Marc & Steven to cc: ] >> The option CONFIG_MMC_BLOCK_BOUNCE is default y so the >> majority of platforms in the kernel already have it on, and >> it then gets turned off at runtime since most of these have >>

blktests and /dev/sg syzkaller reproducers

2017-05-18 Thread Johannes Thumshirn
Hi Omar, I was thinking of adding some of the syszcaller reproducers I've collected, which have been crashing the SCSI generic driver. This would require blktests to have a build infrastructure (not a big deal I know). So my question is, are you comfortable with adding these tests (in a new grou

Re: races between blk-cgroup operations and I/O scheds in blk-mq (?)

2017-05-18 Thread Paolo Valente
> Il giorno 17 mag 2017, alle ore 21:12, Tejun Heo ha scritto: > > Hello, > > On Mon, May 15, 2017 at 09:49:13PM +0200, Paolo Valente wrote: >> So, unless you tell me that there are other races I haven't seen, or, >> even worse, that I'm just talking nonsense, I have thought of a simple >> solu