On Thu, Oct 01, 2015 at 12:16:04PM +0200, Jens Axboe wrote: > On 10/01/2015 11:00 AM, Michael S. Tsirkin wrote: > >On Thu, Oct 01, 2015 at 03:10:14AM +0200, Thomas D. wrote: > >>Hi, > >> > >>I have a virtual machine which fails to boot linux-4.1.8 while mounting > >>file systems: > >> > >>>* Mounting local filesystem ... > >>>------------[ cut here ]------------ > >>>kernel BUG at drivers/block/virtio_blk.c:172! > >>>invalid opcode: 000 [#1] SMP > >>>Modules linked in: pcspkr psmouse dm_log_userspace virtio_net e1000 fuse > >>>nfs lockd grace sunrpc fscache dm_snapshot dm_bufio dm_mirror > >>>dm_region_hash dm_log usbhid usb_storage sr_mod cdrom > >>>CPU: 7 PIDL 2254 Comm: dmcrypt_write Not tainted 4.1.8-gentoo #1 > >>>Hardware name: Red Hat KVM, BIOS seabios-1.7.5-8.el7 04/01/2014 > >>>task: ffff88061fb70000 ti: ffff88061ff30000 task.ti: ffff88061ff30000 > >>>RIP: 0010:[<ffffffffb4557b30>] [<ffffffffb4557b30>] > >>>virtio_queue_rq+0x210/0x2b0 > >>>RSP: 0018:ffff88061ff33ba8 EFLAGS: 00010202 > >>>RAX: 00000000000000b1 RBX: ffff88061fb2fc00 RCX: ffff88061ff33c30 > >>>RDX: 0000000000000008 RSI: ffff88061ff33c50 RDI: ffff88061fb2fc00 > >>>RBP: ffff88061ff33bf8 R08: ffff88061eef3540 R09: ffff88061ff33c30 > >>>R10: 0000000000000000 R11: 00000000000000af R12: 0000000000000000 > >>>R13: ffff88061eef3540 R14: ffff88061eef3540 R15: ffff880622c7ca80 > >>>FS: 0000000000000000(0000) GS:ffff88063fdc0000(0000) > >>>knlGS:0000000000000000 > >>>CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > >>>CR2: 0000000001ffe468 CR3: 00000000bb343000 CR4: 00000000001406e0 > >>>Stack: > >>> ffff880622d4c478 0000000000000000 ffff88061ff33bd8 ffff88061fb2f > >>> 0000000000000001 ffff88061fb2fc00 ffff88061ff33c30 0000000000000 > >>> ffff88061eef3540 0000000000000000 ffff88061ff33c98 ffffffffb43eb > >>> > >>>Call Trace: > >>> [<ffffffffb43eb500>] __blk_mq_run_hw_queue+0x1d0/0x370 > >>> [<ffffffffb43eb315>] blk_mq_run_hw_queue+0x95/0xb0 > >>> [<ffffffffb43ec804>] blk_mq_flush_plug_list+0x129/0x140 > >>> [<ffffffffb43e33d8>] blk_finish_plug+0x18/0x50 > >>> [<ffffffffb45e3bea>] dmcrypt_write+0x1da/0x1f0 > >>> [<ffffffffb4108c90>] ? wake_up_state+0x20/0x20 > >>> [<ffffffffb45e3a10>] ? crypt_iv_lmk_dtr+0x60/0x60 > >>> [<ffffffffb40fb789>] kthread_create_on_node+0x180/0x180 > >>> [<ffffffffb4705e92>] ret_from_fork+0x42/0x70 > >>> [<ffffffffb40fb6c0>] ? kthread_create_on_node+0x180/0x180 > >>>Code: 00 0000 41 c7 85 78 01 00 00 08 00 00 00 49 c7 85 80 01 00 00 00 00 > >>>00 00 41 89 85 7c 01 00 00 e9 93 fe ff ff 66 0f 1f 44 00 00 <0f> 0b 66 0f > >>>1f 44 00 00 49 8b 87 b0 00 00 00 41 83 e6 ef 4a 8b > >>>RIP [<ffffffffb4557b30>] virtio_queue_rq+0x210/0x2b0 > >>> RSP: <ffff88061ff33ba8> > >>>---[ end trace 8078357c459d5fc0 ]--- > > > > > >So this BUG_ON is from 1cf7e9c68fe84248174e998922b39e508375e7c1. > > commit 1cf7e9c68fe84248174e998922b39e508375e7c1 > > Author: Jens Axboe <ax...@kernel.dk> > > Date: Fri Nov 1 10:52:52 2013 -0600 > > > > virtio_blk: blk-mq support > > > > > > BUG_ON(req->nr_phys_segments + 2 > vblk->sg_elems); > > > > > >On probe, we do > > /* We can handle whatever the host told us to handle. */ > > blk_queue_max_segments(q, vblk->sg_elems-2); > > > > > >To debug this, > >maybe you can print out sg_elems at init time and when this fails, > >to make sure some kind of memory corruption > >does not change sg_elems after initialization? > > > > > >Jens, how may we get more segments than blk_queue_max_segments? > >Is driver expected to validate and drop such requests? > > The answer is that this should not happen. If the driver informs of a limit > on the number of segments, that should never be exceeded. If it does, then > it's a bug in either the SG mapping, or in the building of the request - > either the request gets built too large for some reason, or the mapping > doesn't always coalesce segments even though it should. > > The problem is that we get notified out-of-band, when we attempt to push the > request to the driver. At this point, much of the context could be lost, > like it is in your case. > > Looking at the specific virtio_blk case, it does seem that it is > checking the segment count before mapping. Does the below fix the > problem, or does the BUG_ON() still trigger?
Jens, I have no idea whether this is the right thing to do, so please merge this patch directly if it makes sense. > > diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c > index 6ca35495a5be..1501701b0202 100644 > --- a/drivers/block/virtio_blk.c > +++ b/drivers/block/virtio_blk.c > @@ -169,8 +169,6 @@ static int virtio_queue_rq(struct blk_mq_hw_ctx *hctx, > int err; > bool notify = false; > > - BUG_ON(req->nr_phys_segments + 2 > vblk->sg_elems); > - > vbr->req = req; > if (req->cmd_flags & REQ_FLUSH) { > vbr->out_hdr.type = cpu_to_virtio32(vblk->vdev, > VIRTIO_BLK_T_FLUSH); > @@ -203,6 +201,7 @@ static int virtio_queue_rq(struct blk_mq_hw_ctx *hctx, > > num = blk_rq_map_sg(hctx->queue, vbr->req, vbr->sg); > if (num) { > + BUG_ON(num + 2 > vblk->sg_elems); > if (rq_data_dir(vbr->req) == WRITE) > vbr->out_hdr.type |= cpu_to_virtio32(vblk->vdev, > VIRTIO_BLK_T_OUT); > else > > -- > Jens Axboe _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization