On Mon, 2017-09-18 at 18:39 -0400, Keith Busch wrote: > The nvme driver's use of blk_mq_reinit_tagset only happens during > controller initialisation, but I'm seeing lost commands well after that > during normal and stable running. > > The timing is pretty narrow to hit, but I'm pretty sure this is what's > happening. For nvme, this occurs when nvme_timeout() runs concurrently > with nvme_handle_cqe() for the same struct request. For scsi-mq, > the same situation may arise if scsi_mq_done() runs concurrently with > scsi_times_out().
Hello Keith, Are you sure that scenario can happen? The blk-mq core calls test_and_set_bit() for the REQ_ATOM_COMPLETE flag before any completion or timeout handler is called. See also blk_mark_rq_complete(). This avoids that the .complete() and .timeout() functions run concurrently. In case this wouldn't be clear, a block driver must not call blk_mq_end_request() after the timeout handler has finished because that would trigger a use-after-free of a request structure. I noticed that your patch includes changes for blk_mq_start_request(). No timeout or completion handler should be running while blk_mq_start_request() is being executed. If these changes make a difference in your tests then I think that means that there is something wrong with the NVMe driver. Bart.
