Re: [Qemu-devel] [RFC PATCH 0/2] virtio-mmio: add irqfd support for vhost-net based on virtio-mmio
On 10/27/2014 12:23 PM, Li Liu wrote: On 2014/10/27 17:37, Peter Maydell wrote: On 25 October 2014 09:24, john.liuli john.li...@huawei.com wrote: To get the interrupt reason to support such VIRTIO_NET_F_STATUS features I add a new register offset VIRTIO_MMIO_ISRMEM which will help to establish a shared memory region between qemu and virtio-mmio device. Then the interrupt reason can be accessed by guest driver through this region. At the same time, the virtio-mmio dirver check this region to see irqfd is supported or not during the irq handler registration, and different handler will be assigned. If you want to add a new register you should probably propose an update to the virtio spec. However, it seems to me it would be better to get generic PCI/PCIe working on the ARM virt board instead; then we can let virtio-mmio quietly fade away. This has been on the todo list for ages (and there have been RFC patches posted for plain PCI), it's just nobody's had time to work on it. thanks -- PMM So you mean virtio-mmio will be replaced by PCI/PCIe on ARM at last? If so, let this patch go with the wind:). Thx. Hi, As a fix of current situation where ISR is only partially updated when vhost-irqfd handles standard IRQ and waiting for PCI emuluation, wouldn't it make sense to store ISR content on vhost driver side and introduce ioctls to read/write it. When using vhost BE, virtio QEMU device would use those ioctl to read/update the ISR content. On top of that we would update the ISR in vhost before triggering the irqfd. If I do not miss anything this would at least make things functional with irqfd. As a second step, we could try to introduce in-kernel emulation of ISR/ACK to fix the performance issue related to going to user-side each time ISR/ACK accesses are done. Do you think it is worth investigating this direction? Thank you in advance Best Regards Eric Li. . ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [Qemu-devel] [RFC PATCH 0/2] virtio-mmio: add irqfd support for vhost-net based on virtio-mmio
On Mon, Oct 27, 2014 at 12:58 PM, Peter Maydell peter.mayd...@linaro.org wrote: On 27 October 2014 11:23, Li Liu john.li...@huawei.com wrote: So you mean virtio-mmio will be replaced by PCI/PCIe on ARM at last? That is the plan, yes. I can't make any promises on timescales at the moment, though... Linaro has scheduled resources to work on this (Ard, cc'ed) and we expect to be able to deliver this within a reasonable time frame. -Christoffer ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [Qemu-devel] [RFC PATCH 0/2] virtio-mmio: add irqfd support for vhost-net based on virtio-mmio
On 2014/11/5 16:43, Eric Auger wrote: On 10/27/2014 12:23 PM, Li Liu wrote: On 2014/10/27 17:37, Peter Maydell wrote: On 25 October 2014 09:24, john.liuli john.li...@huawei.com wrote: To get the interrupt reason to support such VIRTIO_NET_F_STATUS features I add a new register offset VIRTIO_MMIO_ISRMEM which will help to establish a shared memory region between qemu and virtio-mmio device. Then the interrupt reason can be accessed by guest driver through this region. At the same time, the virtio-mmio dirver check this region to see irqfd is supported or not during the irq handler registration, and different handler will be assigned. If you want to add a new register you should probably propose an update to the virtio spec. However, it seems to me it would be better to get generic PCI/PCIe working on the ARM virt board instead; then we can let virtio-mmio quietly fade away. This has been on the todo list for ages (and there have been RFC patches posted for plain PCI), it's just nobody's had time to work on it. thanks -- PMM So you mean virtio-mmio will be replaced by PCI/PCIe on ARM at last? If so, let this patch go with the wind:). Thx. Hi, As a fix of current situation where ISR is only partially updated when vhost-irqfd handles standard IRQ and waiting for PCI emuluation, wouldn't it make sense to store ISR content on vhost driver side and introduce ioctls to read/write it. When using vhost BE, virtio QEMU device would use those ioctl to read/update the ISR content. On top of that we would update the ISR in vhost before triggering the irqfd. If I do not miss anything this would at least make things functional with irqfd. As a second step, we could try to introduce in-kernel emulation of ISR/ACK to fix the performance issue related to going to user-side each time ISR/ACK accesses are done. Do you think it is worth investigating this direction? Hi, About this problem I have a talk with Li Liu. As MST said, we could use multiple GSI to support vhost-net with irqfd. And we have figured out a way to solve this problem. The method is as same as virtio-pci which is to assign multiple irqs for virtio-mmio. Also it can support multiqueue virtio-net on arm. Would you have a look at this method? Thank you very much. - virtio-mmio: support for multiple irqs http://www.spinics.net/lists/kernel/msg1858860.html Thanks, Shannon Thank you in advance Best Regards Eric Li. . ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [Qemu-devel] [RFC PATCH 0/2] virtio-mmio: add irqfd support for vhost-net based on virtio-mmio
On 2014/11/6 9:59, Shannon Zhao wrote: On 2014/11/5 16:43, Eric Auger wrote: On 10/27/2014 12:23 PM, Li Liu wrote: On 2014/10/27 17:37, Peter Maydell wrote: On 25 October 2014 09:24, john.liuli john.li...@huawei.com wrote: To get the interrupt reason to support such VIRTIO_NET_F_STATUS features I add a new register offset VIRTIO_MMIO_ISRMEM which will help to establish a shared memory region between qemu and virtio-mmio device. Then the interrupt reason can be accessed by guest driver through this region. At the same time, the virtio-mmio dirver check this region to see irqfd is supported or not during the irq handler registration, and different handler will be assigned. If you want to add a new register you should probably propose an update to the virtio spec. However, it seems to me it would be better to get generic PCI/PCIe working on the ARM virt board instead; then we can let virtio-mmio quietly fade away. This has been on the todo list for ages (and there have been RFC patches posted for plain PCI), it's just nobody's had time to work on it. thanks -- PMM So you mean virtio-mmio will be replaced by PCI/PCIe on ARM at last? If so, let this patch go with the wind:). Thx. Hi, As a fix of current situation where ISR is only partially updated when vhost-irqfd handles standard IRQ and waiting for PCI emuluation, wouldn't it make sense to store ISR content on vhost driver side and introduce ioctls to read/write it. When using vhost BE, virtio QEMU device would use those ioctl to read/update the ISR content. On top of that we would update the ISR in vhost before triggering the irqfd. If I do not miss anything this would at least make things functional with irqfd. As a second step, we could try to introduce in-kernel emulation of ISR/ACK to fix the performance issue related to going to user-side each time ISR/ACK accesses are done. Do you think it is worth investigating this direction? Hi, About this problem I have a talk with Li Liu. As MST said, we could use multiple GSI to support vhost-net with irqfd. And we have figured out a way to solve this problem. The method is as same as virtio-pci which is to assign multiple irqs for virtio-mmio. Also it can support multiqueue virtio-net on arm. Would you have a look at this method? Thank you very much. - virtio-mmio: support for multiple irqs http://www.spinics.net/lists/kernel/msg1858860.html Thanks, Shannon Yeah, I think multiple GSI is more compatible with MSI-X. And even virtio-mmio will fade away at last. It still make senses for ARM32 which can't support PCI/PCIe. BTW, this patch is handed over to Shannon and please refer to new patch at http://www.spinics.net/lists/kernel/msg1858860.html. Li. Thank you in advance Best Regards Eric Li. . . ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [Qemu-devel] [RFC PATCH 0/2] virtio-mmio: add irqfd support for vhost-net based on virtio-mmio
On 25 October 2014 09:24, john.liuli john.li...@huawei.com wrote: To get the interrupt reason to support such VIRTIO_NET_F_STATUS features I add a new register offset VIRTIO_MMIO_ISRMEM which will help to establish a shared memory region between qemu and virtio-mmio device. Then the interrupt reason can be accessed by guest driver through this region. At the same time, the virtio-mmio dirver check this region to see irqfd is supported or not during the irq handler registration, and different handler will be assigned. If you want to add a new register you should probably propose an update to the virtio spec. However, it seems to me it would be better to get generic PCI/PCIe working on the ARM virt board instead; then we can let virtio-mmio quietly fade away. This has been on the todo list for ages (and there have been RFC patches posted for plain PCI), it's just nobody's had time to work on it. thanks -- PMM ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [RFC PATCH 0/2] virtio-mmio: add irqfd support for vhost-net based on virtio-mmio
On Mon, Oct 27, 2014 at 05:19:23PM +0800, Li Liu wrote: On 2014/10/26 19:52, Michael S. Tsirkin wrote: On Sat, Oct 25, 2014 at 04:24:52PM +0800, john.liuli wrote: From: Li Liu john.li...@huawei.com This set of patches try to implemet irqfd support of vhost-net based on virtio-mmio. I had posted a mail to talking about the status of vhost-net on kvm-arm refer to http://www.spinics.net/lists/kvm-arm/msg10804.html. Some dependent patches are listed in the mail too. Basically the vhost-net brings great performance improvements, almost 50%+. It's easy to implement irqfd support with PCI MSI-X. But till now arm32 do not provide equivalent mechanism to let a device allocate multiple interrupts. And even the aarch64 provid LPI but also not available in a short time. As Gauguey Remy said Vhost does not emulate a complete virtio adapter but only manage virtqueue operations. Vhost module don't update the ISR register, so if with only one irq then it's no way to get the interrupt reason even we can inject the irq correctly. Well guests don't read ISR in MSI-X mode so why does it help to set the ISR bit? Yeah, vhost don't need to set ISR under MSI-X mode. But for ARM without MSI-X kind mechanism guest can't get the interrupt reason through the only one irq hanlder (with one gsi resource). So I build a shared memory region to provide the interrupt reason instead of ISR regiser by qemu without bothering vhost. Then even there's only one irq with only one irq hanlder, it still can distinguish why irq occur. Li. OK so this might allow sharing IRQs between config and VQs which might be a handy optimization even for PCI (at the moment reading ISR causes an exit). Need to see how all this would work on MP systems though. To get the interrupt reason to support such VIRTIO_NET_F_STATUS features I add a new register offset VIRTIO_MMIO_ISRMEM which will help to establish a shared memory region between qemu and virtio-mmio device. Then the interrupt reason can be accessed by guest driver through this region. At the same time, the virtio-mmio dirver check this region to see irqfd is supported or not during the irq handler registration, and different handler will be assigned. I want to know it's the right direction? Does it comply with the virtio-mmio spec.? Or anyone have more good ideas to emulate mis-x based on virtio-mmio? I hope to get feedback and guidance. Thx for any help. Li Liu (2): Add a new register offset let interrupt reason available Assign a new irq handler while irqfd enabled drivers/virtio/virtio_mmio.c | 55 +++--- include/linux/virtio_mmio.h |3 +++ 2 files changed, 55 insertions(+), 3 deletions(-) -- 1.7.9.5 . ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [RFC PATCH 0/2] virtio-mmio: add irqfd support for vhost-net based on virtio-mmio
On 2014/10/26 19:52, Michael S. Tsirkin wrote: On Sat, Oct 25, 2014 at 04:24:52PM +0800, john.liuli wrote: From: Li Liu john.li...@huawei.com This set of patches try to implemet irqfd support of vhost-net based on virtio-mmio. I had posted a mail to talking about the status of vhost-net on kvm-arm refer to http://www.spinics.net/lists/kvm-arm/msg10804.html. Some dependent patches are listed in the mail too. Basically the vhost-net brings great performance improvements, almost 50%+. It's easy to implement irqfd support with PCI MSI-X. But till now arm32 do not provide equivalent mechanism to let a device allocate multiple interrupts. And even the aarch64 provid LPI but also not available in a short time. As Gauguey Remy said Vhost does not emulate a complete virtio adapter but only manage virtqueue operations. Vhost module don't update the ISR register, so if with only one irq then it's no way to get the interrupt reason even we can inject the irq correctly. Well guests don't read ISR in MSI-X mode so why does it help to set the ISR bit? Yeah, vhost don't need to set ISR under MSI-X mode. But for ARM without MSI-X kind mechanism guest can't get the interrupt reason through the only one irq hanlder (with one gsi resource). So I build a shared memory region to provide the interrupt reason instead of ISR regiser by qemu without bothering vhost. Then even there's only one irq with only one irq hanlder, it still can distinguish why irq occur. Li. To get the interrupt reason to support such VIRTIO_NET_F_STATUS features I add a new register offset VIRTIO_MMIO_ISRMEM which will help to establish a shared memory region between qemu and virtio-mmio device. Then the interrupt reason can be accessed by guest driver through this region. At the same time, the virtio-mmio dirver check this region to see irqfd is supported or not during the irq handler registration, and different handler will be assigned. I want to know it's the right direction? Does it comply with the virtio-mmio spec.? Or anyone have more good ideas to emulate mis-x based on virtio-mmio? I hope to get feedback and guidance. Thx for any help. Li Liu (2): Add a new register offset let interrupt reason available Assign a new irq handler while irqfd enabled drivers/virtio/virtio_mmio.c | 55 +++--- include/linux/virtio_mmio.h |3 +++ 2 files changed, 55 insertions(+), 3 deletions(-) -- 1.7.9.5 . ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[RFC PATCH 0/2] virtio-mmio: add irqfd support for vhost-net based on virtio-mmio
From: Li Liu john.li...@huawei.com This set of patches try to implemet irqfd support of vhost-net based on virtio-mmio. I had posted a mail to talking about the status of vhost-net on kvm-arm refer to http://www.spinics.net/lists/kvm-arm/msg10804.html. Some dependent patches are listed in the mail too. Basically the vhost-net brings great performance improvements, almost 50%+. It's easy to implement irqfd support with PCI MSI-X. But till now arm32 do not provide equivalent mechanism to let a device allocate multiple interrupts. And even the aarch64 provid LPI but also not available in a short time. As Gauguey Remy said Vhost does not emulate a complete virtio adapter but only manage virtqueue operations. Vhost module don't update the ISR register, so if with only one irq then it's no way to get the interrupt reason even we can inject the irq correctly. To get the interrupt reason to support such VIRTIO_NET_F_STATUS features I add a new register offset VIRTIO_MMIO_ISRMEM which will help to establish a shared memory region between qemu and virtio-mmio device. Then the interrupt reason can be accessed by guest driver through this region. At the same time, the virtio-mmio dirver check this region to see irqfd is supported or not during the irq handler registration, and different handler will be assigned. I want to know it's the right direction? Does it comply with the virtio-mmio spec.? Or anyone have more good ideas to emulate mis-x based on virtio-mmio? I hope to get feedback and guidance. Thx for any help. Li Liu (2): Add a new register offset let interrupt reason available Assign a new irq handler while irqfd enabled drivers/virtio/virtio_mmio.c | 55 +++--- include/linux/virtio_mmio.h |3 +++ 2 files changed, 55 insertions(+), 3 deletions(-) -- 1.7.9.5 ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [Qemu-devel] [RFC PATCH 0/2] virtio-mmio: add irqfd support for vhost-net based on virtio-mmio
On 2014/10/27 17:37, Peter Maydell wrote: On 25 October 2014 09:24, john.liuli john.li...@huawei.com wrote: To get the interrupt reason to support such VIRTIO_NET_F_STATUS features I add a new register offset VIRTIO_MMIO_ISRMEM which will help to establish a shared memory region between qemu and virtio-mmio device. Then the interrupt reason can be accessed by guest driver through this region. At the same time, the virtio-mmio dirver check this region to see irqfd is supported or not during the irq handler registration, and different handler will be assigned. If you want to add a new register you should probably propose an update to the virtio spec. However, it seems to me it would be better to get generic PCI/PCIe working on the ARM virt board instead; then we can let virtio-mmio quietly fade away. This has been on the todo list for ages (and there have been RFC patches posted for plain PCI), it's just nobody's had time to work on it. thanks -- PMM So you mean virtio-mmio will be replaced by PCI/PCIe on ARM at last? If so, let this patch go with the wind:). Thx. Li. . ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [RFC PATCH 0/2] virtio-mmio: add irqfd support for vhost-net based on virtio-mmio
On Sat, Oct 25, 2014 at 04:24:52PM +0800, john.liuli wrote: From: Li Liu john.li...@huawei.com This set of patches try to implemet irqfd support of vhost-net based on virtio-mmio. I had posted a mail to talking about the status of vhost-net on kvm-arm refer to http://www.spinics.net/lists/kvm-arm/msg10804.html. Some dependent patches are listed in the mail too. Basically the vhost-net brings great performance improvements, almost 50%+. It's easy to implement irqfd support with PCI MSI-X. But till now arm32 do not provide equivalent mechanism to let a device allocate multiple interrupts. And even the aarch64 provid LPI but also not available in a short time. As Gauguey Remy said Vhost does not emulate a complete virtio adapter but only manage virtqueue operations. Vhost module don't update the ISR register, so if with only one irq then it's no way to get the interrupt reason even we can inject the irq correctly. Well guests don't read ISR in MSI-X mode so why does it help to set the ISR bit? To get the interrupt reason to support such VIRTIO_NET_F_STATUS features I add a new register offset VIRTIO_MMIO_ISRMEM which will help to establish a shared memory region between qemu and virtio-mmio device. Then the interrupt reason can be accessed by guest driver through this region. At the same time, the virtio-mmio dirver check this region to see irqfd is supported or not during the irq handler registration, and different handler will be assigned. I want to know it's the right direction? Does it comply with the virtio-mmio spec.? Or anyone have more good ideas to emulate mis-x based on virtio-mmio? I hope to get feedback and guidance. Thx for any help. Li Liu (2): Add a new register offset let interrupt reason available Assign a new irq handler while irqfd enabled drivers/virtio/virtio_mmio.c | 55 +++--- include/linux/virtio_mmio.h |3 +++ 2 files changed, 55 insertions(+), 3 deletions(-) -- 1.7.9.5 ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization