Re: [PATCH 7.1] virtio-scsi: fix race in virtio_scsi_dataplane_start()

2022-08-05 Thread Michael S. Tsirkin
On Thu, Aug 04, 2022 at 02:56:59PM -0400, Stefan Hajnoczi wrote:
> On Wed, Aug 03, 2022 at 12:28:24PM -0400, Stefan Hajnoczi wrote:
> > As soon as virtio_scsi_data_plane_start() attaches host notifiers the
> > IOThread may start virtqueue processing. There is a race between
> > IOThread virtqueue processing and virtio_scsi_data_plane_start() because
> > it only assigns s->dataplane_started after attaching host notifiers.
> > 
> > When a virtqueue handler function in the IOThread calls
> > virtio_scsi_defer_to_dataplane() it may see !s->dataplane_started and
> > attempt to start dataplane even though we're already in the IOThread:
> > 
> >   #0  0x7f67b360857c __pthread_kill_implementation (libc.so.6 + 0xa257c)
> >   #1  0x7f67b35bbd56 raise (libc.so.6 + 0x55d56)
> >   #2  0x7f67b358e833 abort (libc.so.6 + 0x28833)
> >   #3  0x7f67b358e75b __assert_fail_base.cold (libc.so.6 + 0x2875b)
> >   #4  0x7f67b35b4cd6 __assert_fail (libc.so.6 + 0x4ecd6)
> >   #5  0x55ca87fd411b memory_region_transaction_commit (qemu-kvm + 
> > 0x67511b)
> >   #6  0x55ca87e17811 virtio_pci_ioeventfd_assign (qemu-kvm + 0x4b8811)
> >   #7  0x55ca87e14836 virtio_bus_set_host_notifier (qemu-kvm + 0x4b5836)
> >   #8  0x55ca87f8e14e virtio_scsi_set_host_notifier (qemu-kvm + 0x62f14e)
> >   #9  0x55ca87f8dd62 virtio_scsi_dataplane_start (qemu-kvm + 0x62ed62)
> >   #10 0x55ca87e14610 virtio_bus_start_ioeventfd (qemu-kvm + 0x4b5610)
> >   #11 0x55ca87f8c29a virtio_scsi_handle_ctrl (qemu-kvm + 0x62d29a)
> >   #12 0x55ca87fa5902 virtio_queue_host_notifier_read (qemu-kvm + 
> > 0x646902)
> >   #13 0x55ca882c099e aio_dispatch_handler (qemu-kvm + 0x96199e)
> >   #14 0x55ca882c1761 aio_poll (qemu-kvm + 0x962761)
> >   #15 0x55ca880e1052 iothread_run (qemu-kvm + 0x782052)
> >   #16 0x55ca882c562a qemu_thread_start (qemu-kvm + 0x96662a)
> > 
> > This patch assigns s->dataplane_started before attaching host notifiers
> > so that virtqueue handler functions that run in the IOThread before
> > virtio_scsi_data_plane_start() returns correctly identify that dataplane
> > does not need to be started.
> > 
> > Note that s->dataplane_started does not need the AioContext lock because
> > it is set before attaching host notifiers and cleared after detaching
> > host notifiers. In other words, the IOThread always sees the value true
> > and the main loop thread does not modify it while the IOThread is
> > active.
> > 
> > Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=2099541
> > Reported-by: Qing Wang 
> 
> Qing Wang has confirmed that this solves the assertion failures.
> 
> Paolo/Michael: Can this still be merged for QEMU 7.1?
> 
> Stefan

As a bugfix, for sure.

Can I have your ack?

-- 
MST




Re: [PATCH 7.1] virtio-scsi: fix race in virtio_scsi_dataplane_start()

2022-08-04 Thread Stefan Hajnoczi
On Wed, Aug 03, 2022 at 12:28:24PM -0400, Stefan Hajnoczi wrote:
> As soon as virtio_scsi_data_plane_start() attaches host notifiers the
> IOThread may start virtqueue processing. There is a race between
> IOThread virtqueue processing and virtio_scsi_data_plane_start() because
> it only assigns s->dataplane_started after attaching host notifiers.
> 
> When a virtqueue handler function in the IOThread calls
> virtio_scsi_defer_to_dataplane() it may see !s->dataplane_started and
> attempt to start dataplane even though we're already in the IOThread:
> 
>   #0  0x7f67b360857c __pthread_kill_implementation (libc.so.6 + 0xa257c)
>   #1  0x7f67b35bbd56 raise (libc.so.6 + 0x55d56)
>   #2  0x7f67b358e833 abort (libc.so.6 + 0x28833)
>   #3  0x7f67b358e75b __assert_fail_base.cold (libc.so.6 + 0x2875b)
>   #4  0x7f67b35b4cd6 __assert_fail (libc.so.6 + 0x4ecd6)
>   #5  0x55ca87fd411b memory_region_transaction_commit (qemu-kvm + 
> 0x67511b)
>   #6  0x55ca87e17811 virtio_pci_ioeventfd_assign (qemu-kvm + 0x4b8811)
>   #7  0x55ca87e14836 virtio_bus_set_host_notifier (qemu-kvm + 0x4b5836)
>   #8  0x55ca87f8e14e virtio_scsi_set_host_notifier (qemu-kvm + 0x62f14e)
>   #9  0x55ca87f8dd62 virtio_scsi_dataplane_start (qemu-kvm + 0x62ed62)
>   #10 0x55ca87e14610 virtio_bus_start_ioeventfd (qemu-kvm + 0x4b5610)
>   #11 0x55ca87f8c29a virtio_scsi_handle_ctrl (qemu-kvm + 0x62d29a)
>   #12 0x55ca87fa5902 virtio_queue_host_notifier_read (qemu-kvm + 0x646902)
>   #13 0x55ca882c099e aio_dispatch_handler (qemu-kvm + 0x96199e)
>   #14 0x55ca882c1761 aio_poll (qemu-kvm + 0x962761)
>   #15 0x55ca880e1052 iothread_run (qemu-kvm + 0x782052)
>   #16 0x55ca882c562a qemu_thread_start (qemu-kvm + 0x96662a)
> 
> This patch assigns s->dataplane_started before attaching host notifiers
> so that virtqueue handler functions that run in the IOThread before
> virtio_scsi_data_plane_start() returns correctly identify that dataplane
> does not need to be started.
> 
> Note that s->dataplane_started does not need the AioContext lock because
> it is set before attaching host notifiers and cleared after detaching
> host notifiers. In other words, the IOThread always sees the value true
> and the main loop thread does not modify it while the IOThread is
> active.
> 
> Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=2099541
> Reported-by: Qing Wang 

Qing Wang has confirmed that this solves the assertion failures.

Paolo/Michael: Can this still be merged for QEMU 7.1?

Stefan


signature.asc
Description: PGP signature