Re: Reminder: 2 open syzbot bugs in vhost subsystem
On 2019/7/2 下午1:17, Eric Biggers wrote: [This email was generated by a script. Let me know if you have any suggestions to make it better, or if you want it re-generated with the latest status.] Of the currently open syzbot reports against the upstream kernel, I've manually marked 2 of them as possibly being bugs in the vhost subsystem. I've listed these reports below, sorted by an algorithm that tries to list first the reports most likely to be still valid, important, and actionable. Of these 2 bugs, 1 was seen in mainline in the last week. If you believe a bug is no longer valid, please close the syzbot report by sending a '#syz fix', '#syz dup', or '#syz invalid' command in reply to the original thread, as explained at https://goo.gl/tpsmEJ#status If you believe I misattributed a bug to the vhost subsystem, please let me know, and if possible forward the report to the correct people or mailing list. Here are the bugs: Title: memory leak in vhost_net_ioctl Last occurred: 0 days ago Reported: 26 days ago Branches: Mainline Dashboard link: https://syzkaller.appspot.com/bug?id=12ba349d7e26ccfe95317bc376e812ebbae2ee0f Original thread: https://lkml.kernel.org/lkml/188da1058a9c2...@google.com/T/#u This bug has a C reproducer. The original thread for this bug has received 4 replies; the last was 17 days ago. If you fix this bug, please add the following tag to the commit: Reported-by: syzbot+0789f0c7e45efd7bb...@syzkaller.appspotmail.com If you send any email or patch for this bug, please consider replying to the original thread. For the git send-email command to use, or tips on how to reply if the thread isn't in your mailbox, see the "Reply instructions" at https://lkml.kernel.org/r/188da1058a9c2...@google.com Cc Hillf who should had a fix for this. Hillf, would you please post a formal patch for this? (for -net) Title: INFO: task hung in vhost_init_device_iotlb Last occurred: 125 days ago Reported: 153 days ago Branches: Mainline and others Dashboard link: https://syzkaller.appspot.com/bug?id=cb1ea8daf03a5942c2ab314679148cf6e128ef58 Original thread: https://lkml.kernel.org/lkml/7e86fd0580955...@google.com/T/#u Unfortunately, this bug does not have a reproducer. The original thread for this bug received 2 replies; the last was 152 days ago. If you fix this bug, please add the following tag to the commit: Reported-by: syzbot+40e28a8bd59d10ed0...@syzkaller.appspotmail.com If you send any email or patch for this bug, please consider replying to the original thread. For the git send-email command to use, or tips on how to reply if the thread isn't in your mailbox, see the "Reply instructions" at https://lkml.kernel.org/r/7e86fd0580955...@google.com Can syzbot still reproduce this issue? Thanks ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH v2 0/3] vsock/virtio: several fixes in the .probe() and .remove()
On Mon, Jul 01, 2019 at 04:11:13PM +0100, Stefan Hajnoczi wrote: > On Fri, Jun 28, 2019 at 02:36:56PM +0200, Stefano Garzarella wrote: > > During the review of "[PATCH] vsock/virtio: Initialize core virtio vsock > > before registering the driver", Stefan pointed out some possible issues > > in the .probe() and .remove() callbacks of the virtio-vsock driver. > > > > This series tries to solve these issues: > > - Patch 1 adds RCU critical sections to avoid use-after-free of > > 'the_virtio_vsock' pointer. > > - Patch 2 stops workers before to call vdev->config->reset(vdev) to > > be sure that no one is accessing the device. > > - Patch 3 moves the works flush at the end of the .remove() to avoid > > use-after-free of 'vsock' object. > > > > v2: > > - Patch 1: use RCU to protect 'the_virtio_vsock' pointer > > - Patch 2: no changes > > - Patch 3: flush works only at the end of .remove() > > - Removed patch 4 because virtqueue_detach_unused_buf() returns all the > > buffers > > allocated. > > > > v1: https://patchwork.kernel.org/cover/10964733/ > > This looks good to me. Thanks for the review! > > Did you run any stress tests? For example an SMP guest constantly > connecting and sending packets together with a script that > hotplug/unplugs vhost-vsock-pci from the host side. Yes, I started an SMP guest (-smp 4 -monitor tcp:127.0.0.1:1234,server,nowait) and I run these scripts to stress the .probe()/.remove() path: - guest while true; do cat /dev/urandom | nc-vsock -l 4321 > /dev/null & cat /dev/urandom | nc-vsock -l 5321 > /dev/null & cat /dev/urandom | nc-vsock -l 6321 > /dev/null & cat /dev/urandom | nc-vsock -l 7321 > /dev/null & wait done - host while true; do cat /dev/urandom | nc-vsock 3 4321 > /dev/null & cat /dev/urandom | nc-vsock 3 5321 > /dev/null & cat /dev/urandom | nc-vsock 3 6321 > /dev/null & cat /dev/urandom | nc-vsock 3 7321 > /dev/null & sleep 2 echo "device_del v1" | nc 127.0.0.1 1234 sleep 1 echo "device_add vhost-vsock-pci,id=v1,guest-cid=3" | nc 127.0.0.1 1234 sleep 1 done Do you think is enough or is better to have a test more accurate? Thanks, Stefano ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH] scsi: virtio_scsi: Use struct_size() helper
On Wed, Jun 19, 2019 at 02:28:33PM -0500, Gustavo A. R. Silva wrote:
> One of the more common cases of allocation size calculations is finding
> the size of a structure that has a zero-sized array at the end, along
> with memory for some number of elements for that array. For example:
>
> struct virtio_scsi {
> ...
> struct virtio_scsi_vq req_vqs[];
> };
>
> Make use of the struct_size() helper instead of an open-coded version
> in order to avoid any potential type mistakes.
>
> So, replace the following form:
>
> sizeof(*vscsi) + sizeof(vscsi->req_vqs[0]) * num_queues
>
> with:
>
> struct_size(vscsi, req_vqs, num_queues)
>
> This code was detected with the help of Coccinelle.
>
> Signed-off-by: Gustavo A. R. Silva
> ---
> drivers/scsi/virtio_scsi.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
Reviewed-by: Stefan Hajnoczi
signature.asc
Description: PGP signature
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH v2 0/3] vsock/virtio: several fixes in the .probe() and .remove()
On Fri, Jun 28, 2019 at 02:36:56PM +0200, Stefano Garzarella wrote: > During the review of "[PATCH] vsock/virtio: Initialize core virtio vsock > before registering the driver", Stefan pointed out some possible issues > in the .probe() and .remove() callbacks of the virtio-vsock driver. > > This series tries to solve these issues: > - Patch 1 adds RCU critical sections to avoid use-after-free of > 'the_virtio_vsock' pointer. > - Patch 2 stops workers before to call vdev->config->reset(vdev) to > be sure that no one is accessing the device. > - Patch 3 moves the works flush at the end of the .remove() to avoid > use-after-free of 'vsock' object. > > v2: > - Patch 1: use RCU to protect 'the_virtio_vsock' pointer > - Patch 2: no changes > - Patch 3: flush works only at the end of .remove() > - Removed patch 4 because virtqueue_detach_unused_buf() returns all the > buffers > allocated. > > v1: https://patchwork.kernel.org/cover/10964733/ This looks good to me. Did you run any stress tests? For example an SMP guest constantly connecting and sending packets together with a script that hotplug/unplugs vhost-vsock-pci from the host side. Stefan signature.asc Description: PGP signature ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH v2 1/3] vsock/virtio: use RCU to avoid use-after-free on the_virtio_vsock
On Fri, Jun 28, 2019 at 02:36:57PM +0200, Stefano Garzarella wrote: > Some callbacks used by the upper layers can run while we are in the > .remove(). A potential use-after-free can happen, because we free > the_virtio_vsock without knowing if the callbacks are over or not. > > To solve this issue we move the assignment of the_virtio_vsock at the > end of .probe(), when we finished all the initialization, and at the > beginning of .remove(), before to release resources. > For the same reason, we do the same also for the vdev->priv. > > We use RCU to be sure that all callbacks that use the_virtio_vsock > ended before freeing it. This is not required for callbacks that > use vdev->priv, because after the vdev->config->del_vqs() we are sure > that they are ended and will no longer be invoked. My question is answered in Patch 3. signature.asc Description: PGP signature ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH v2 3/3] vsock/virtio: fix flush of works during the .remove()
On Fri, Jun 28, 2019 at 02:36:59PM +0200, Stefano Garzarella wrote: > This patch moves the flush of works after vdev->config->del_vqs(vdev), > because we need to be sure that no workers run before to free the > 'vsock' object. > > Since we stopped the workers using the [tx|rx|event]_run flags, > we are sure no one is accessing the device while we are calling > vdev->config->reset(vdev), so we can safely move the workers' flush. > > Before the vdev->config->del_vqs(vdev), workers can be scheduled > by VQ callbacks, so we must flush them after del_vqs(), to avoid > use-after-free of 'vsock' object. Nevermind, I looked back at Patch 2 and saw the send_pkt and loopback work functions were also updated. Thanks! Stefan signature.asc Description: PGP signature ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH v2 3/3] vsock/virtio: fix flush of works during the .remove()
On Fri, Jun 28, 2019 at 02:36:59PM +0200, Stefano Garzarella wrote: > This patch moves the flush of works after vdev->config->del_vqs(vdev), > because we need to be sure that no workers run before to free the > 'vsock' object. > > Since we stopped the workers using the [tx|rx|event]_run flags, > we are sure no one is accessing the device while we are calling > vdev->config->reset(vdev), so we can safely move the workers' flush. What about send_pkt and loopback work? How were they stopped safely? For example, if send_pkt work executes then we're in trouble since it accesses the tx virtqueue which is deleted by ->del_vqs(). signature.asc Description: PGP signature ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH v2 1/3] vsock/virtio: use RCU to avoid use-after-free on the_virtio_vsock
On Fri, Jun 28, 2019 at 02:36:57PM +0200, Stefano Garzarella wrote: > Some callbacks used by the upper layers can run while we are in the > .remove(). A potential use-after-free can happen, because we free > the_virtio_vsock without knowing if the callbacks are over or not. > > To solve this issue we move the assignment of the_virtio_vsock at the > end of .probe(), when we finished all the initialization, and at the > beginning of .remove(), before to release resources. > For the same reason, we do the same also for the vdev->priv. > > We use RCU to be sure that all callbacks that use the_virtio_vsock > ended before freeing it. This is not required for callbacks that > use vdev->priv, because after the vdev->config->del_vqs() we are sure > that they are ended and will no longer be invoked. ->del_vqs() is only called at the very end, did you forget to move it earlier? In particular, the virtqueue handler callbacks schedule a workqueue. The work functions use container_of() to get vsock. We need to be sure that the work item isn't freed along with vsock while the work item is still pending. How do we know that the virtqueue handler is never called in such a way that it sees vsock != NULL (there is no explicit memory barrier on the read side) and then schedules a work item after flush_work() has run? Stefan signature.asc Description: PGP signature ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [RFC PATCH] virtio_ring: Use DMA API if guest memory is encrypted
On Thu, Jun 27, 2019 at 10:58:40PM -0300, Thiago Jung Bauermann wrote: > > Michael S. Tsirkin writes: > > > On Mon, Jun 03, 2019 at 10:13:59PM -0300, Thiago Jung Bauermann wrote: > >> > >> > >> Michael S. Tsirkin writes: > >> > >> > On Wed, Apr 17, 2019 at 06:42:00PM -0300, Thiago Jung Bauermann wrote: > >> >> I rephrased it in terms of address translation. What do you think of > >> >> this version? The flag name is slightly different too: > >> >> > >> >> > >> >> VIRTIO_F_ACCESS_PLATFORM_NO_TRANSLATION This feature has the same > >> >> meaning as VIRTIO_F_ACCESS_PLATFORM both when set and when not set, > >> >> with the exception that address translation is guaranteed to be > >> >> unnecessary when accessing memory addresses supplied to the device > >> >> by the driver. Which is to say, the device will always use physical > >> >> addresses matching addresses used by the driver (typically meaning > >> >> physical addresses used by the CPU) and not translated further. This > >> >> flag should be set by the guest if offered, but to allow for > >> >> backward-compatibility device implementations allow for it to be > >> >> left unset by the guest. It is an error to set both this flag and > >> >> VIRTIO_F_ACCESS_PLATFORM. > >> > > >> > > >> > > >> > > >> > OK so VIRTIO_F_ACCESS_PLATFORM is designed to allow unpriveledged > >> > drivers. This is why devices fail when it's not negotiated. > >> > >> Just to clarify, what do you mean by unprivileged drivers? Is it drivers > >> implemented in guest userspace such as with VFIO? Or unprivileged in > >> some other sense such as needing to use bounce buffers for some reason? > > > > I had drivers in guest userspace in mind. > > Great. Thanks for clarifying. > > I don't think this flag would work for guest userspace drivers. Should I > add a note about that in the flag definition? I think you need to clarify access protection rules. Is it only translation that is bypassed or is any platform-specific protection mechanism bypassed too? > >> > This confuses me. > >> > If driver is unpriveledged then what happens with this flag? > >> > It can supply any address it wants. Will that corrupt kernel > >> > memory? > >> > >> Not needing address translation doesn't necessarily mean that there's no > >> IOMMU. On powerpc we don't use VIRTIO_F_ACCESS_PLATFORM but there's > >> always an IOMMU present. And we also support VFIO drivers. The VFIO API > >> for pseries (sPAPR section in Documentation/vfio.txt) has extra ioctls > >> to program the IOMMU. > >> > >> For our use case, we don't need address translation because we set up an > >> identity mapping in the IOMMU so that the device can use guest physical > >> addresses. OK so I think I am beginning to see it in a different light. Right now the specific platform creates an identity mapping. That in turn means DMA API can be fast - it does not need to do anything. What you are looking for is a way to tell host it's an identity mapping - just as an optimization. Is that right? So this is what I would call this option: VIRTIO_F_ACCESS_PLATFORM_IDENTITY_ADDRESS and the explanation should state that all device addresses are translated by the platform to identical addresses. In fact this option then becomes more, not less restrictive than VIRTIO_F_ACCESS_PLATFORM - it's a promise by guest to only create identity mappings, and only before driver_ok is set. This option then would always be negotiated together with VIRTIO_F_ACCESS_PLATFORM. Host then must verify that 1. full 1:1 mappings are created before driver_ok or can we make sure this happens before features_ok? that would be ideal as we could require that features_ok fails 2. mappings are not modified between driver_ok and reset i guess attempts to change them will fail - possibly by causing a guest crash or some other kind of platform-specific error So far so good, but now a question: how are we handling guest address width limitations? Is VIRTIO_F_ACCESS_PLATFORM_IDENTITY_ADDRESS subject to guest address width limitations? I am guessing we can make them so ... This needs to be documented. > > > > And can it access any guest physical address? > > Sorry, I was mistaken. We do support VFIO in guests but not for virtio > devices, only for regular PCI devices. In which case they will use > address translation. Not sure how this answers the question. > >> If the guest kernel is concerned that an unprivileged driver could > >> jeopardize its integrity it should not negotiate this feature flag. > > > > Unfortunately flag negotiation is done through config space > > and so can be overwritten by the driver. > > Ok, so the guest kernel has to forbid VFIO access on devices where this > flag is advertised. That's possible in theory but in practice we did not yet teach VFIO not to attach to legacy devices without VIRTIO_F_ACCESS_PLATFORM. So all security relies on host denying driver_ok without VIRTIO_F_ACCESS
Re: [PATCH v4 0/5] iommu/amd: Convert the AMD iommu driver to the dma-iommu api
Hi, On Sun, Jun 23, 2019 at 11:19:45PM -0700, Christoph Hellwig wrote: > Joerg, any chance you could review this? Toms patches to convert the > AMD and Intel IOMMU drivers to the dma-iommu code are going to make my > life in DMA land significantly easier, so I have a vested interest > in this series moving forward :) I really appreciate Toms work on this. Tom, please rebase and resubmit this series after the next merge window and I will do more performance testing on it. If all goes well I and no other issues show up I can apply it for v5.4. Regards, Joerg ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
