Re: [PATCH 4/4] vl: Prioritize realizations of devices

Hi, On 8/25/21 6:23 AM, Jason Wang wrote: > On Tue, Aug 24, 2021 at 11:50 PM Peter Xu wrote: >> On Tue, Aug 24, 2021 at 10:52:24AM +0800, Jason Wang wrote: >>> It looks to me this doesn't solve the issue of using virtio-mmio with vhost? >> No IOMMU supported for any of the MMIO devices, right? O

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On Fri, 3 Sep 2021 12:03:06 -0400 Peter Xu wrote: > On Fri, Sep 03, 2021 at 03:00:05PM +0200, Igor Mammedov wrote: > > PS: > > Another, albeit machine depended approach to resolve IOMMU ordering problem > > can be adding to a specific machine pre_plug hook, an IOMMU handling. > > Which is called

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On Fri, Sep 03, 2021 at 03:00:05PM +0200, Igor Mammedov wrote: > PS: > Another, albeit machine depended approach to resolve IOMMU ordering problem > can be adding to a specific machine pre_plug hook, an IOMMU handling. > Which is called during IOMMU realize time and check if existing buses > witho

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On Thu, Sep 02, 2021 at 04:57:04PM +0200, Markus Armbruster wrote: > Peter Xu writes: > > > Hi, Dan, > > > > On Thu, Sep 02, 2021 at 02:53:00PM +0100, Daniel P. Berrangé wrote: > >> The correct ordering of devices/backends is generally pretty obvious > >> for libvirt to determine. Most of the pro

Re: [PATCH 4/4] vl: Prioritize realizations of devices

Peter Xu writes: > Hi, Dan, > > On Thu, Sep 02, 2021 at 02:53:00PM +0100, Daniel P. Berrangé wrote: >> The correct ordering of devices/backends is generally pretty obvious >> for libvirt to determine. Most of the problems we've had related to >> ordering are on the QEMU side, because the ARGV giv

Re: [PATCH 4/4] vl: Prioritize realizations of devices

Peter Xu writes: > On Thu, Sep 02, 2021 at 10:26:16AM +0200, Igor Mammedov wrote: >> On Mon, 30 Aug 2021 15:02:53 -0400 >> Peter Xu wrote: >> >> > On Thu, Aug 26, 2021 at 09:43:59AM -0400, Peter Xu wrote: >> > > > > A simple state machine can track "has IOMMU" state. It has three >> > > > > s

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On Thu, 2 Sep 2021 09:45:43 -0400 Peter Xu wrote: > On Thu, Sep 02, 2021 at 10:26:16AM +0200, Igor Mammedov wrote: > > On Mon, 30 Aug 2021 15:02:53 -0400 > > Peter Xu wrote: > > > > > On Thu, Aug 26, 2021 at 09:43:59AM -0400, Peter Xu wrote: > > > > > > A simple state machine can track "has

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On Thu, Sep 02, 2021 at 10:21:19AM -0400, Peter Xu wrote: > Hi, Dan, > > On Thu, Sep 02, 2021 at 02:53:00PM +0100, Daniel P. Berrangé wrote: > > The correct ordering of devices/backends is generally pretty obvious > > for libvirt to determine. Most of the problems we've had related to > > ordering

Re: [PATCH 4/4] vl: Prioritize realizations of devices

Hi, Dan, On Thu, Sep 02, 2021 at 02:53:00PM +0100, Daniel P. Berrangé wrote: > The correct ordering of devices/backends is generally pretty obvious > for libvirt to determine. Most of the problems we've had related to > ordering are on the QEMU side, because the ARGV given to QEMU made > correct s

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On Thu, Sep 02, 2021 at 09:45:43AM -0400, Peter Xu wrote: > On Thu, Sep 02, 2021 at 10:26:16AM +0200, Igor Mammedov wrote: > > On Mon, 30 Aug 2021 15:02:53 -0400 > > Peter Xu wrote: > > > > > On Thu, Aug 26, 2021 at 09:43:59AM -0400, Peter Xu wrote: > > > > > > A simple state machine can track "h

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On Thu, Sep 02, 2021 at 10:26:16AM +0200, Igor Mammedov wrote: > On Mon, 30 Aug 2021 15:02:53 -0400 > Peter Xu wrote: > > > On Thu, Aug 26, 2021 at 09:43:59AM -0400, Peter Xu wrote: > > > > > A simple state machine can track "has IOMMU" state. It has three > > > > > states > > > > > "no so far"

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On Mon, 30 Aug 2021 15:02:53 -0400 Peter Xu wrote: > On Thu, Aug 26, 2021 at 09:43:59AM -0400, Peter Xu wrote: > > > > A simple state machine can track "has IOMMU" state. It has three states > > > > "no so far", "yes", and "no", and two events "add IOMMU" and "add device > > > > that needs to kn

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On Thu, 26 Aug 2021 09:43:59 -0400 Peter Xu wrote: > On Thu, Aug 26, 2021 at 01:36:29PM +0200, Igor Mammedov wrote: > > On Thu, 26 Aug 2021 10:01:01 +0200 > > Markus Armbruster wrote: > > > > > Peter Xu writes: > > > > > > > On Wed, Aug 25, 2021 at 05:50:23PM -0400, Peter Xu wrote: >

Re: [PATCH 4/4] vl: Prioritize realizations of devices

Peter Xu writes: > On Thu, Aug 26, 2021 at 09:43:59AM -0400, Peter Xu wrote: >> > > A simple state machine can track "has IOMMU" state. It has three states >> > > "no so far", "yes", and "no", and two events "add IOMMU" and "add device >> > > that needs to know". State diagram: >> > > >> > >

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On Thu, Aug 26, 2021 at 09:43:59AM -0400, Peter Xu wrote: > > > A simple state machine can track "has IOMMU" state. It has three states > > > "no so far", "yes", and "no", and two events "add IOMMU" and "add device > > > that needs to know". State diagram: > > > > > > n

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On Thu, Aug 26, 2021 at 01:36:29PM +0200, Igor Mammedov wrote: > On Thu, 26 Aug 2021 10:01:01 +0200 > Markus Armbruster wrote: > > > Peter Xu writes: > > > > > On Wed, Aug 25, 2021 at 05:50:23PM -0400, Peter Xu wrote: > > >> On Wed, Aug 25, 2021 at 02:28:55PM +0200, Markus Armbruster wrote:

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On Thu, 26 Aug 2021 10:01:01 +0200 Markus Armbruster wrote: > Peter Xu writes: > > > On Wed, Aug 25, 2021 at 05:50:23PM -0400, Peter Xu wrote: > >> On Wed, Aug 25, 2021 at 02:28:55PM +0200, Markus Armbruster wrote: > >> > Having thought about this a bit more... ... > > Any further thoughts

Re: [PATCH 4/4] vl: Prioritize realizations of devices

Peter Xu writes: > On Wed, Aug 25, 2021 at 05:50:23PM -0400, Peter Xu wrote: >> On Wed, Aug 25, 2021 at 02:28:55PM +0200, Markus Armbruster wrote: >> > Having thought about this a bit more... >> > >> > Constraints on realize order are nothing new. For instance, when a >> > device plugs into a b

Re: [PATCH 4/4] vl: Prioritize realizations of devices

Peter Xu writes: > On Wed, Aug 25, 2021 at 02:28:55PM +0200, Markus Armbruster wrote: >> Markus Armbruster writes: >> >> > Peter Xu writes: >> > >> >> On Mon, Aug 23, 2021 at 05:56:23PM -0400, Eduardo Habkost wrote: >> >>> I don't have any other example, but I assume address assignment >> >>>

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On Wed, Aug 25, 2021 at 05:50:23PM -0400, Peter Xu wrote: > On Wed, Aug 25, 2021 at 02:28:55PM +0200, Markus Armbruster wrote: > > Markus Armbruster writes: > > > > > Peter Xu writes: > > > > > >> On Mon, Aug 23, 2021 at 05:56:23PM -0400, Eduardo Habkost wrote: > > >>> I don't have any other exa

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On Wed, Aug 25, 2021 at 02:28:55PM +0200, Markus Armbruster wrote: > Markus Armbruster writes: > > > Peter Xu writes: > > > >> On Mon, Aug 23, 2021 at 05:56:23PM -0400, Eduardo Habkost wrote: > >>> I don't have any other example, but I assume address assignment > >>> based on ordering is a commo

Re: [PATCH 4/4] vl: Prioritize realizations of devices

Markus Armbruster writes: > Peter Xu writes: > >> On Mon, Aug 23, 2021 at 05:56:23PM -0400, Eduardo Habkost wrote: >>> I don't have any other example, but I assume address assignment >>> based on ordering is a common pattern in device code. >>> >>> I would take a very close and careful look at

Re: [PATCH 4/4] vl: Prioritize realizations of devices

Peter Xu writes: > On Mon, Aug 23, 2021 at 05:56:23PM -0400, Eduardo Habkost wrote: >> I don't have any other example, but I assume address assignment >> based on ordering is a common pattern in device code. >> >> I would take a very close and careful look at the devices with >> non-default vmsd

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On 24.08.21 21:52, Peter Xu wrote: On Tue, Aug 24, 2021 at 06:24:27PM +0200, David Hildenbrand wrote: Not so much; here's the list of priorities and the devices using it: |+-| | priority | devices | |+-

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On Tue, Aug 24, 2021 at 11:50 PM Peter Xu wrote: > > On Tue, Aug 24, 2021 at 10:52:24AM +0800, Jason Wang wrote: > > It looks to me this doesn't solve the issue of using virtio-mmio with vhost? > > No IOMMU supported for any of the MMIO devices, right? Or am I wrong? > Thanks, I guess virtio I

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On Tue, Aug 24, 2021 at 06:24:27PM +0200, David Hildenbrand wrote: > > > Not so much; here's the list of priorities and the devices using it: > > > > > > |+-| > > > | priority | devices | > > > |+-| > > >

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On 24.08.21 00:05, Michael S. Tsirkin wrote: On Mon, Aug 23, 2021 at 03:18:51PM -0400, Peter Xu wrote: On Mon, Aug 23, 2021 at 02:49:12PM -0400, Eduardo Habkost wrote: On Wed, Aug 18, 2021 at 03:43:18PM -0400, Peter Xu wrote: QEMU creates -device objects in order as specified by the user's cmd

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On Tue, Aug 24, 2021 at 10:52:24AM +0800, Jason Wang wrote: > It looks to me this doesn't solve the issue of using virtio-mmio with vhost? No IOMMU supported for any of the MMIO devices, right? Or am I wrong? Thanks, -- Peter Xu

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On Tue, Aug 24, 2021 at 6:37 AM Peter Xu wrote: > > On Mon, Aug 23, 2021 at 06:05:07PM -0400, Michael S. Tsirkin wrote: > > On Mon, Aug 23, 2021 at 03:18:51PM -0400, Peter Xu wrote: > > > On Mon, Aug 23, 2021 at 02:49:12PM -0400, Eduardo Habkost wrote: > > > > On Wed, Aug 18, 2021 at 03:43:18PM -0

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On Tue, Aug 24, 2021 at 3:18 AM Peter Xu wrote: > > On Mon, Aug 23, 2021 at 02:49:12PM -0400, Eduardo Habkost wrote: > > On Wed, Aug 18, 2021 at 03:43:18PM -0400, Peter Xu wrote: > > > QEMU creates -device objects in order as specified by the user's cmdline. > > > However that ordering may not be

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On Mon, Aug 23, 2021 at 05:56:23PM -0400, Eduardo Habkost wrote: > I don't have any other example, but I assume address assignment > based on ordering is a common pattern in device code. > > I would take a very close and careful look at the devices with > non-default vmsd priority. If you can pro

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On Mon, Aug 23, 2021 at 05:54:44PM -0400, Michael S. Tsirkin wrote: > > I can use a custom sort to replace qsort() to guarantee that. > You don't have to do that. Simply use the device position on the command > line for comparisons when priority is the same. Indeed. :) Thanks, -- Peter Xu

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On Mon, Aug 23, 2021 at 06:05:07PM -0400, Michael S. Tsirkin wrote: > On Mon, Aug 23, 2021 at 03:18:51PM -0400, Peter Xu wrote: > > On Mon, Aug 23, 2021 at 02:49:12PM -0400, Eduardo Habkost wrote: > > > On Wed, Aug 18, 2021 at 03:43:18PM -0400, Peter Xu wrote: > > > > QEMU creates -device objects i

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On Mon, Aug 23, 2021 at 03:18:51PM -0400, Peter Xu wrote: > On Mon, Aug 23, 2021 at 02:49:12PM -0400, Eduardo Habkost wrote: > > On Wed, Aug 18, 2021 at 03:43:18PM -0400, Peter Xu wrote: > > > QEMU creates -device objects in order as specified by the user's cmdline. > > > However that ordering may

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On Mon, Aug 23, 2021 at 05:31:46PM -0400, Peter Xu wrote: > On Mon, Aug 23, 2021 at 05:07:03PM -0400, Eduardo Habkost wrote: > > To give just one example: > > > > $ (echo 'info pci';echo quit;) | qemu-system-x86_64 -device virtio-net-pci > > -device e1000e -monitor stdio | tail -n 20 > > Bus 0

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On Mon, Aug 23, 2021 at 05:31:46PM -0400, Peter Xu wrote: > On Mon, Aug 23, 2021 at 05:07:03PM -0400, Eduardo Habkost wrote: > > To give just one example: > > > > $ (echo 'info pci';echo quit;) | qemu-system-x86_64 -device virtio-net-pci > > -device e1000e -monitor stdio | tail -n 20 > > Bus 0

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On Mon, Aug 23, 2021 at 05:07:03PM -0400, Eduardo Habkost wrote: > To give just one example: > > $ (echo 'info pci';echo quit;) | qemu-system-x86_64 -device virtio-net-pci > -device e1000e -monitor stdio | tail -n 20 > Bus 0, device 4, function 0: > Ethernet controller: PCI device 1af4:1

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On Mon, Aug 23, 2021 at 03:18:51PM -0400, Peter Xu wrote: > On Mon, Aug 23, 2021 at 02:49:12PM -0400, Eduardo Habkost wrote: > > On Wed, Aug 18, 2021 at 03:43:18PM -0400, Peter Xu wrote: > > > QEMU creates -device objects in order as specified by the user's cmdline. > > > However that ordering may

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On Mon, Aug 23, 2021 at 02:49:12PM -0400, Eduardo Habkost wrote: > On Wed, Aug 18, 2021 at 03:43:18PM -0400, Peter Xu wrote: > > QEMU creates -device objects in order as specified by the user's cmdline. > > However that ordering may not be the ideal order. For example, some > > platform > > devic

Re: [PATCH 4/4] vl: Prioritize realizations of devices

On Wed, Aug 18, 2021 at 03:43:18PM -0400, Peter Xu wrote: > QEMU creates -device objects in order as specified by the user's cmdline. > However that ordering may not be the ideal order. For example, some platform > devices (vIOMMUs) may want to be created earlier than most of the rest > devices (e

[PATCH 4/4] vl: Prioritize realizations of devices

QEMU creates -device objects in order as specified by the user's cmdline. However that ordering may not be the ideal order. For example, some platform devices (vIOMMUs) may want to be created earlier than most of the rest devices (e.g., vfio-pci, virtio). This patch orders the QemuOptsList of '-d