Re: [Qemu-devel] [RFC 00/15] qmp: Report supported device types on 'query-machines'
Marcel Apfelbaumwrites: > On 11/24/2016 05:41 PM, Markus Armbruster wrote: >> Marcel Apfelbaum writes: >> >>> On 11/24/2016 03:34 PM, Markus Armbruster wrote: Eduardo Habkost writes: > On Wed, Nov 23, 2016 at 06:43:16PM +0200, Marcel Apfelbaum wrote: >> On 11/22/2016 03:11 AM, Eduardo Habkost wrote: >>> The Problem > >>> >>> [...] >>> Our decision to have hybrid PCI/PCIe devices and buses breeds considerable complexity. I wish we had avoided them, but I believe it's too late to change now. >> This still does not solve the problem that some devices makes >> sense only on a specific arch. >>> >>> Hi Markus, >>> Examples? >>> >>> One quick example would be that we don't want to see >>> Intel's IOH 3420 PCIe Root Port in an ARM machine, >>> or a pxb on a Q35 machine (in this case we want pxb-pcie) >> >> Such a device would be weird. But would it be wrong? > > Define wrong :) I do: > Wrong enough for >> QEMU to reject it? > > QEMU accepts them and they even function correctly as far as I know. > >> Unless QEMU rejects it, there's no reason not to >> list it as pluggable. >> > > This is the gray area I can't argue. I do think that Eduardo's > work may present an opportunity to change QEMU's mantra: > "everything goes as long as it works" to "here is what this configuration > supports". I guess you argument is that in reality, the devices you quoted are always part of specific chipsets, so "we dont want to see" them elsewhere. If I understand you correctly, there are two cases you don't want to see elsewhere: (1) The real PCI device only ever exists as function of another device. (2) The real PCI device only ever exists on certain boards. We accept these devices elsewhere due to the way we model them. Because we conflate PCI functions and devices, we can't model (1) correctly. I think the appropriate solution would be modelling functions separate from devices, then provide the functions in question only as part of the devices where you want to see them, by making them not user-pluggable. Because we model a board's chipset as a set of independent devices instead of a composite device, we can't model (2) correctly. I think the appropriate solution to (2) is modelling composite chipset devices, then provide the devices in question only as part of the chipset devices where you want to see them, by making them not user-pluggable. Adding a bunch of special types to QOM so that introspection claims "you can't plug that thing" (even though you actually could) would be an inappropriate solution. As long as you can plug them, QEMU should be honest about it, even if we think you shouldn't plug them. "Can't" is mechanism. "Shouldn't" is policy. Baking policy into introspection by making it lie doesn't strike me as a good idea. [...]
Re: [Qemu-devel] [RFC 00/15] qmp: Report supported device types on 'query-machines'
On 11/24/2016 05:41 PM, Markus Armbruster wrote: Marcel Apfelbaumwrites: On 11/24/2016 03:34 PM, Markus Armbruster wrote: Eduardo Habkost writes: On Wed, Nov 23, 2016 at 06:43:16PM +0200, Marcel Apfelbaum wrote: On 11/22/2016 03:11 AM, Eduardo Habkost wrote: The Problem [...] Our decision to have hybrid PCI/PCIe devices and buses breeds considerable complexity. I wish we had avoided them, but I believe it's too late to change now. This still does not solve the problem that some devices makes sense only on a specific arch. Hi Markus, Examples? One quick example would be that we don't want to see Intel's IOH 3420 PCIe Root Port in an ARM machine, or a pxb on a Q35 machine (in this case we want pxb-pcie) Such a device would be weird. But would it be wrong? Define wrong :) Wrong enough for QEMU to reject it? QEMU accepts them and they even function correctly as far as I know. Unless QEMU rejects it, there's no reason not to list it as pluggable. This is the gray area I can't argue. I do think that Eduardo's work may present an opportunity to change QEMU's mantra: "everything goes as long as it works" to "here is what this configuration supports". Thanks, Marcel I do believe there are other examples, I'll try to think of more. Thanks, Marcel [...]
Re: [Qemu-devel] [RFC 00/15] qmp: Report supported device types on 'query-machines'
Marcel Apfelbaumwrites: > On 11/24/2016 03:34 PM, Markus Armbruster wrote: >> Eduardo Habkost writes: >> >>> On Wed, Nov 23, 2016 at 06:43:16PM +0200, Marcel Apfelbaum wrote: On 11/22/2016 03:11 AM, Eduardo Habkost wrote: > The Problem >>> > > [...] > >> Our decision to have hybrid PCI/PCIe devices and buses breeds >> considerable complexity. I wish we had avoided them, but I believe it's >> too late to change now. >> This still does not solve the problem that some devices makes sense only on a specific arch. >> > > Hi Markus, > >> Examples? >> > > One quick example would be that we don't want to see > Intel's IOH 3420 PCIe Root Port in an ARM machine, > or a pxb on a Q35 machine (in this case we want pxb-pcie) Such a device would be weird. But would it be wrong? Wrong enough for QEMU to reject it? Unless QEMU rejects it, there's no reason not to list it as pluggable. > I do believe there are other examples, I'll try to think of more. > > Thanks, > Marcel > > [...]
Re: [Qemu-devel] [RFC 00/15] qmp: Report supported device types on 'query-machines'
Eduardo Habkostwrites: > On Thu, Nov 24, 2016 at 02:34:05PM +0100, Markus Armbruster wrote: >> Eduardo Habkost writes: >> >> > On Wed, Nov 23, 2016 at 06:43:16PM +0200, Marcel Apfelbaum wrote: >> >> On 11/22/2016 03:11 AM, Eduardo Habkost wrote: >> >> > The Problem >> >> > === >> >> > >> >> > Currently management software has no way to find out which device >> >> > types can be plugged in a machine, unless the machine is already >> >> > initialized. >> >> I doubt it can find the precise list of device types even for an >> initialized machine. > > My plan is to make it possible. See: > [RFC 06/15] qdev: Add device_type field to BusClass > > I will change how it looks like on v2 (mostly due to the PCI/PCIe > issues), but basically it makes all bus objects able to tell > which device types they accept. Sounds like a step forward. >> >> Hi Eduardo, >> >> Thank you for this interesting series. I think this is a problem >> >> worth addressing. >> >> >> >> > Even after the machine is initialized, there's no way to map >> >> > existing bus types to supported device types unless management >> >> > software hardcodes the mapping between bus types and device >> >> > types. >> >> > >> >> >> >> Here I am a little lost. >> >> >> >> We are going for machine => supported devices or >> >> bus-type => supported devices? >> > >> > On this series, we go for machine-type => supported-devices. >> > >> > A bus-type => supported-devices map wouldn't work because >> > different PCIe bus instances might accept different types of >> > devices (so supported-devices depend on the specific bus >> > instance, not only on the bus-type). >> > >> > v2 will probably be more detailed. I plan to change it to: >> > >> > query-machine(machine-type) => list of BusInfo >> > >> > BusInfo would contain: >> > * bus-type >> > * bus-path >> > * accepted-device-types (list of type/interface names) >> >> Let me take a step back and explore what questions a management >> application may want to ask. I think the basic questions are: >> >> * Which devices models could be plugged now? >> >> "Now" means in the current state of the machine. Depending on "now", >> this could be hot- or cold-plug. > > Yes. > >> >> * Into which sockets could device model T be plugged now? >> >> Mind, I wrote "socket", not bus. Buses provide sockets, but they are >> not the only provider of sockets. >> >> We need a way to name sockets. (QOM path to device or bus, socket >> address on device or bus) could work. > > Naming "sockets" was not on my model, but I think we can do that. > > I will probably send RFC v2 without any socket abstraction, so > other conceptual changes I am introducing can be reviewed > (especially the PCI/PCIe mess I am diving into, right now). But I > will try to document how the design can evolve to handle sockets. Makes sense. >> Are we on the same page so far? > > Yes. > >> >> Related: same questions with "newly created machine of machine type M" >> instead of "now". This isn't basic, because you could always create the >> machine, then ask the basic question. But not having to create the >> machine is convenient, in particular as long as a single QEMU process >> can only ever create one machine. >> >> Related: same, with creation options. >> >> Your series provides a solution for a special non-basic case: newly >> created machine of machine type M with no optional parts, i.e. with >> -nodefaults and nothing else. Or maybe with the default set of optional >> parts, i.e. without -nodefaults, not sure. >> >> I'm not demanding you provide a solution for the general case first. It >> would be nice, but it could be too complex for a first step. I do >> recommend you design your solution for the special case with the general >> case in mind. Say, by trying to ensure the special case query returns >> something that will do or can be evolved to do for the general case, >> too. > > Yep. I will keep that in mind. > >> >> >> > Example: floppy support on q35 vs i440fx >> >> > >> >> > >> >> > There's no way for libvirt to find out that there's no floppy >> >> > controller on pc-q35-* machine-types by default. >> >> > >> >> >> >> Again "by default". So do we want to query the init state of a machine? >> >> What devices are there? Or what devices *can be* there? >> > >> > "by default" means what's present when using "-machine " >> > with no extra -device arguments. >> >> Not just -device. Some legacy options can also create buses. For >> instance, -device if=scsi,... creates a SCSI bus. > > Right. I found that out when debugging the -nodefaults issues. > >> >> > We want to know what _buses_ are always there. Which in turn lets >> > management know which _device_ types _can_ be plugged. >> > >> >> >> >> > With this series, pc-i440fx-* will report "floppy" as a supported >> >> > device type, but pc-q35-* will not. >> >> >
Re: [Qemu-devel] [RFC 00/15] qmp: Report supported device types on 'query-machines'
On 11/24/2016 03:34 PM, Markus Armbruster wrote: Eduardo Habkostwrites: On Wed, Nov 23, 2016 at 06:43:16PM +0200, Marcel Apfelbaum wrote: On 11/22/2016 03:11 AM, Eduardo Habkost wrote: The Problem [...] Our decision to have hybrid PCI/PCIe devices and buses breeds considerable complexity. I wish we had avoided them, but I believe it's too late to change now. This still does not solve the problem that some devices makes sense only on a specific arch. Hi Markus, Examples? One quick example would be that we don't want to see Intel's IOH 3420 PCIe Root Port in an ARM machine, or a pxb on a Q35 machine (in this case we want pxb-pcie) I do believe there are other examples, I'll try to think of more. Thanks, Marcel [...]
Re: [Qemu-devel] [RFC 00/15] qmp: Report supported device types on 'query-machines'
On Thu, Nov 24, 2016 at 02:34:05PM +0100, Markus Armbruster wrote: > Eduardo Habkostwrites: > > > On Wed, Nov 23, 2016 at 06:43:16PM +0200, Marcel Apfelbaum wrote: > >> On 11/22/2016 03:11 AM, Eduardo Habkost wrote: > >> > The Problem > >> > === > >> > > >> > Currently management software has no way to find out which device > >> > types can be plugged in a machine, unless the machine is already > >> > initialized. > > I doubt it can find the precise list of device types even for an > initialized machine. My plan is to make it possible. See: [RFC 06/15] qdev: Add device_type field to BusClass I will change how it looks like on v2 (mostly due to the PCI/PCIe issues), but basically it makes all bus objects able to tell which device types they accept. > > >> Hi Eduardo, > >> Thank you for this interesting series. I think this is a problem > >> worth addressing. > >> > >> > Even after the machine is initialized, there's no way to map > >> > existing bus types to supported device types unless management > >> > software hardcodes the mapping between bus types and device > >> > types. > >> > > >> > >> Here I am a little lost. > >> > >> We are going for machine => supported devices or > >> bus-type => supported devices? > > > > On this series, we go for machine-type => supported-devices. > > > > A bus-type => supported-devices map wouldn't work because > > different PCIe bus instances might accept different types of > > devices (so supported-devices depend on the specific bus > > instance, not only on the bus-type). > > > > v2 will probably be more detailed. I plan to change it to: > > > > query-machine(machine-type) => list of BusInfo > > > > BusInfo would contain: > > * bus-type > > * bus-path > > * accepted-device-types (list of type/interface names) > > Let me take a step back and explore what questions a management > application may want to ask. I think the basic questions are: > > * Which devices models could be plugged now? > > "Now" means in the current state of the machine. Depending on "now", > this could be hot- or cold-plug. Yes. > > * Into which sockets could device model T be plugged now? > > Mind, I wrote "socket", not bus. Buses provide sockets, but they are > not the only provider of sockets. > > We need a way to name sockets. (QOM path to device or bus, socket > address on device or bus) could work. Naming "sockets" was not on my model, but I think we can do that. I will probably send RFC v2 without any socket abstraction, so other conceptual changes I am introducing can be reviewed (especially the PCI/PCIe mess I am diving into, right now). But I will try to document how the design can evolve to handle sockets. > > Are we on the same page so far? Yes. > > Related: same questions with "newly created machine of machine type M" > instead of "now". This isn't basic, because you could always create the > machine, then ask the basic question. But not having to create the > machine is convenient, in particular as long as a single QEMU process > can only ever create one machine. > > Related: same, with creation options. > > Your series provides a solution for a special non-basic case: newly > created machine of machine type M with no optional parts, i.e. with > -nodefaults and nothing else. Or maybe with the default set of optional > parts, i.e. without -nodefaults, not sure. > > I'm not demanding you provide a solution for the general case first. It > would be nice, but it could be too complex for a first step. I do > recommend you design your solution for the special case with the general > case in mind. Say, by trying to ensure the special case query returns > something that will do or can be evolved to do for the general case, > too. Yep. I will keep that in mind. > > >> > Example: floppy support on q35 vs i440fx > >> > > >> > > >> > There's no way for libvirt to find out that there's no floppy > >> > controller on pc-q35-* machine-types by default. > >> > > >> > >> Again "by default". So do we want to query the init state of a machine? > >> What devices are there? Or what devices *can be* there? > > > > "by default" means what's present when using "-machine " > > with no extra -device arguments. > > Not just -device. Some legacy options can also create buses. For > instance, -device if=scsi,... creates a SCSI bus. Right. I found that out when debugging the -nodefaults issues. > > > We want to know what _buses_ are always there. Which in turn lets > > management know which _device_ types _can_ be plugged. > > > >> > >> > With this series, pc-i440fx-* will report "floppy" as a supported > >> > device type, but pc-q35-* will not. > >> > > >> > Example: Legacy PCI vs vs PCIe devices > >> > -- > >> > > >> > Some devices require a PCIe bus to be available, others work on > >> > both legacy PCI and PCIe, while others work only on
Re: [Qemu-devel] [RFC 00/15] qmp: Report supported device types on 'query-machines'
Eduardo Habkostwrites: > On Wed, Nov 23, 2016 at 06:43:16PM +0200, Marcel Apfelbaum wrote: >> On 11/22/2016 03:11 AM, Eduardo Habkost wrote: >> > The Problem >> > === >> > >> > Currently management software has no way to find out which device >> > types can be plugged in a machine, unless the machine is already >> > initialized. I doubt it can find the precise list of device types even for an initialized machine. >> Hi Eduardo, >> Thank you for this interesting series. I think this is a problem >> worth addressing. >> >> > Even after the machine is initialized, there's no way to map >> > existing bus types to supported device types unless management >> > software hardcodes the mapping between bus types and device >> > types. >> > >> >> Here I am a little lost. >> >> We are going for machine => supported devices or >> bus-type => supported devices? > > On this series, we go for machine-type => supported-devices. > > A bus-type => supported-devices map wouldn't work because > different PCIe bus instances might accept different types of > devices (so supported-devices depend on the specific bus > instance, not only on the bus-type). > > v2 will probably be more detailed. I plan to change it to: > > query-machine(machine-type) => list of BusInfo > > BusInfo would contain: > * bus-type > * bus-path > * accepted-device-types (list of type/interface names) Let me take a step back and explore what questions a management application may want to ask. I think the basic questions are: * Which devices models could be plugged now? "Now" means in the current state of the machine. Depending on "now", this could be hot- or cold-plug. * Into which sockets could device model T be plugged now? Mind, I wrote "socket", not bus. Buses provide sockets, but they are not the only provider of sockets. We need a way to name sockets. (QOM path to device or bus, socket address on device or bus) could work. Are we on the same page so far? Related: same questions with "newly created machine of machine type M" instead of "now". This isn't basic, because you could always create the machine, then ask the basic question. But not having to create the machine is convenient, in particular as long as a single QEMU process can only ever create one machine. Related: same, with creation options. Your series provides a solution for a special non-basic case: newly created machine of machine type M with no optional parts, i.e. with -nodefaults and nothing else. Or maybe with the default set of optional parts, i.e. without -nodefaults, not sure. I'm not demanding you provide a solution for the general case first. It would be nice, but it could be too complex for a first step. I do recommend you design your solution for the special case with the general case in mind. Say, by trying to ensure the special case query returns something that will do or can be evolved to do for the general case, too. >> > Example: floppy support on q35 vs i440fx >> > >> > >> > There's no way for libvirt to find out that there's no floppy >> > controller on pc-q35-* machine-types by default. >> > >> >> Again "by default". So do we want to query the init state of a machine? >> What devices are there? Or what devices *can be* there? > > "by default" means what's present when using "-machine " > with no extra -device arguments. Not just -device. Some legacy options can also create buses. For instance, -device if=scsi,... creates a SCSI bus. > We want to know what _buses_ are always there. Which in turn lets > management know which _device_ types _can_ be plugged. > >> >> > With this series, pc-i440fx-* will report "floppy" as a supported >> > device type, but pc-q35-* will not. >> > >> > Example: Legacy PCI vs vs PCIe devices >> > -- >> > >> > Some devices require a PCIe bus to be available, others work on >> > both legacy PCI and PCIe, while others work only on a legacy PCI >> > bus. >> > >> > Currently management software has no way to know which devices >> > can be added to a given machine, unless it hardcodes machine-type >> > names and device-types names. >> > >> >> Again it seems a double problem, machine => devices vs pci/pcie bus => >> devices. >> The bus => devices match is not related to a machine type. > > A bus-type => device-type match would not depend on the > machine-type, but it would not be useful: different bus instances > can accept different device-types (and the way the bus topology > is configured depend on the machine-type). > > >> >> > The Proposed Interface >> > == >> > >> > This series adds a new field to the output of 'query-machines': >> > 'supported-device-types'. It will contain a list of QOM type >> > names, that can be used to find the list of device types that can >> > be plugged in the machine by default. >> >> What do you mean "by default"? Without bridges
Re: [Qemu-devel] [RFC 00/15] qmp: Report supported device types on 'query-machines'
On 11/23/2016 07:35 PM, Eduardo Habkost wrote: On Wed, Nov 23, 2016 at 06:43:16PM +0200, Marcel Apfelbaum wrote: On 11/22/2016 03:11 AM, Eduardo Habkost wrote: The Problem === Currently management software has no way to find out which device types can be plugged in a machine, unless the machine is already initialized. Hi Eduardo, Thank you for this interesting series. I think this is a problem worth addressing. [...] PCI vs PCIe --- Machines with PCIe buses will report INTERFACE_PCIE_DEVICE on supported-device-types. Machines with legacy PCI buses will report TYPE_PCI_DEVICE on supported-device-types. The problem with the current approach is that PCIe devices are TYPE_PCI_DEVICE subclasses. The allows PCI device classes to indicate they are PCIe-capable, but there's no obvious way to indicate that a device is PCIe-only. This needs to be addressed in a future version of this series. Suggestions are welcome. As we talked offline, I personally like an interface IPCIType with a field having 3 possible values {pci/pcie/hybrid} To understand how hybrid works we need some rules, like "pci on pci bus/pcie on pcie bus" "pcie on a non-root pcie bus/pcie otherwise I don't think we'll have a lot of rules, simple boolean fields for the interface should be enough. What you propose makes sense, the only difference is that the boolean fields would be just interface names that can be used as the "implements" argument on qom-list-types. e.g.: * Hybrid PCI device-types would implement both "legacy-pci-device" and "pcie-device" interfaces. * PCIe-only device-types would implement only the "pcie-device" interface. * Legacy-PCI-only device-types would implement only the "legacy-pci-device" interface. Then, the bus instances would have a 'accepted-device-types' field: * A legacy PCI bus would accept only "legacy-pci-device" devices. * A PCIe-only bus would accept only "pcie-device" devices. * A PCIe bus that accepts legacy PCI devices (the root bus?) would accept both "legacy-pci-device" and "pcie-device" devices. Then, query-machines would return the list of bus instances that machine init creates, containing the bus ID, bus type, and accepted-device-types. Does that make sense? Sure, the described approach solves the problem. This still does not solve the problem that some devices makes sense only on a specific arch. Right now, we can simply compile out arch-specific devices that can never be plugged in a QEMU binary. But if we still want them compiled in for some reason, we can categorize them on a different type/interface name, and the corresponding machine-type would tell management that their buses accept only the arch-specific type/interface name. Adding an Interface for each arch could work, yes. Incomplete bus lists on some machines - With this series, not all machines classes are changed to add the full list of device types on the 'supported-device-types'. To allow the code to be updated gradually, qmp-machine-info.py has a STRICT_ARCHES variable, that will make the test code require a complete device type list only on some architectures. Out of scope: Configurable buses There's no way to map machine options like "usb=on|off" to device-types or buses. I plan to propose a new interface that allows machine options to be mapped to buses/device-types later. Out of scope: Deciding where to plug devices Once management software discovers which devices can be plugged to a machine, it still has to discover or define where devices can/should/will be plugged. This is out of the scope of this series. That's a pitty :( I was hoping this series will solve this issue. But if it prepares the grounds for it is also a good step . The bus ID will be in the scope of v2. Thanks, Marcel Thanks, Marcel [...]
Re: [Qemu-devel] [RFC 00/15] qmp: Report supported device types on 'query-machines'
On Wed, Nov 23, 2016 at 06:43:16PM +0200, Marcel Apfelbaum wrote: > On 11/22/2016 03:11 AM, Eduardo Habkost wrote: > > The Problem > > === > > > > Currently management software has no way to find out which device > > types can be plugged in a machine, unless the machine is already > > initialized. > > > > Hi Eduardo, > Thank you for this interesting series. I think this is a problem > worth addressing. > > > Even after the machine is initialized, there's no way to map > > existing bus types to supported device types unless management > > software hardcodes the mapping between bus types and device > > types. > > > > Here I am a little lost. > > We are going for machine => supported devices or > bus-type => supported devices? On this series, we go for machine-type => supported-devices. A bus-type => supported-devices map wouldn't work because different PCIe bus instances might accept different types of devices (so supported-devices depend on the specific bus instance, not only on the bus-type). v2 will probably be more detailed. I plan to change it to: query-machine(machine-type) => list of BusInfo BusInfo would contain: * bus-type * bus-path * accepted-device-types (list of type/interface names) > > > Example: floppy support on q35 vs i440fx > > > > > > There's no way for libvirt to find out that there's no floppy > > controller on pc-q35-* machine-types by default. > > > > Again "by default". So do we want to query the init state of a machine? > What devices are there? Or what devices *can be* there? "by default" means what's present when using "-machine " with no extra -device arguments. We want to know what _buses_ are always there. Which in turn lets management know which _device_ types _can_ be plugged. > > > With this series, pc-i440fx-* will report "floppy" as a supported > > device type, but pc-q35-* will not. > > > > Example: Legacy PCI vs vs PCIe devices > > -- > > > > Some devices require a PCIe bus to be available, others work on > > both legacy PCI and PCIe, while others work only on a legacy PCI > > bus. > > > > Currently management software has no way to know which devices > > can be added to a given machine, unless it hardcodes machine-type > > names and device-types names. > > > > Again it seems a double problem, machine => devices vs pci/pcie bus => > devices. > The bus => devices match is not related to a machine type. A bus-type => device-type match would not depend on the machine-type, but it would not be useful: different bus instances can accept different device-types (and the way the bus topology is configured depend on the machine-type). > > > The Proposed Interface > > == > > > > This series adds a new field to the output of 'query-machines': > > 'supported-device-types'. It will contain a list of QOM type > > names, that can be used to find the list of device types that can > > be plugged in the machine by default. > > What do you mean "by default"? Without bridges or part of the machine itself? I mean "when you just run -machine with no extra -device arguments". > > The type names reported on > > the new field can then be used as the 'implements' argument on > > the 'qom-list-types' command, to find out which device types can > > be plugged on the machine. > > > > Example output > > -- > > > > (QEMU) query-machines > > { > > "return": [ > > [...] > > { > > "supported-device-types": [ > > "sys-bus-device" > > > I don't know how "sys-bus-device" can help us... :) Yes, I added comments about it below. :) > > > ], > > "cpu-max": 1, > > "hotpluggable-cpus": false, > > "name": "none" > > }, > > [...] > > { > > "supported-device-types": [ > > "sys-bus-device" > > ], > > "cpu-max": 1, > > "hotpluggable-cpus": false, > > "name": "xenpv" > > }, > > [...] > > { > > "supported-device-types": [ > > "sys-bus-device", > > "floppy", > > "i2c-slave", > > "pci-device", > > "isa-device", > > "ide-device" > > Is don't know is this high level classification is useful, > here is an example: > >pvi-device is supported => then we look for all pci devices? > But what if some pci devices make sense on a machine type, > but not on another? If not all pci devices are supported, then the machine must not return "pci-device" as supported. We need to define a new type/interface name that would be implemented only by the supported devices. e.g. "legacy-pci-device". > > > > > ], > > "name": "pc-i440fx-2.8", > > "alias": "pc", > > "is-default": true, > >
Re: [Qemu-devel] [RFC 00/15] qmp: Report supported device types on 'query-machines'
On 11/22/2016 03:11 AM, Eduardo Habkost wrote: The Problem === Currently management software has no way to find out which device types can be plugged in a machine, unless the machine is already initialized. Hi Eduardo, Thank you for this interesting series. I think this is a problem worth addressing. Even after the machine is initialized, there's no way to map existing bus types to supported device types unless management software hardcodes the mapping between bus types and device types. Here I am a little lost. We are going for machine => supported devices or bus-type => supported devices? Example: floppy support on q35 vs i440fx There's no way for libvirt to find out that there's no floppy controller on pc-q35-* machine-types by default. Again "by default". So do we want to query the init state of a machine? What devices are there? Or what devices *can be* there? With this series, pc-i440fx-* will report "floppy" as a supported device type, but pc-q35-* will not. Example: Legacy PCI vs vs PCIe devices -- Some devices require a PCIe bus to be available, others work on both legacy PCI and PCIe, while others work only on a legacy PCI bus. Currently management software has no way to know which devices can be added to a given machine, unless it hardcodes machine-type names and device-types names. Again it seems a double problem, machine => devices vs pci/pcie bus => devices. The bus => devices match is not related to a machine type. The Proposed Interface == This series adds a new field to the output of 'query-machines': 'supported-device-types'. It will contain a list of QOM type names, that can be used to find the list of device types that can be plugged in the machine by default. What do you mean "by default"? Without bridges or part of the machine itself? The type names reported on the new field can then be used as the 'implements' argument on the 'qom-list-types' command, to find out which device types can be plugged on the machine. Example output -- (QEMU) query-machines { "return": [ [...] { "supported-device-types": [ "sys-bus-device" I don't know how "sys-bus-device" can help us... :) ], "cpu-max": 1, "hotpluggable-cpus": false, "name": "none" }, [...] { "supported-device-types": [ "sys-bus-device" ], "cpu-max": 1, "hotpluggable-cpus": false, "name": "xenpv" }, [...] { "supported-device-types": [ "sys-bus-device", "floppy", "i2c-slave", "pci-device", "isa-device", "ide-device" Is don't know is this high level classification is useful, here is an example: pvi-device is supported => then we look for all pci devices? But what if some pci devices make sense on a machine type, but not on another? ], "name": "pc-i440fx-2.8", "alias": "pc", "is-default": true, "cpu-max": 255, "hotpluggable-cpus": true }, [...] { "supported-device-types": [ "sys-bus-device", "floppy", "isa-device", "ide-device" ], "cpu-max": 1, "hotpluggable-cpus": true, "name": "isapc" }, [...] { "supported-device-types": [ "sys-bus-device", "floppy", "i2c-slave", "pci-device", "isa-device", "ide-device" ], "cpu-max": 128, "hotpluggable-cpus": true, "name": "xenfv" }, [...] { "alias": "q35", "supported-device-types": [ "sys-bus-device", "i2c-slave", "PCIE-device", "isa-device", "ide-device" ], "cpu-max": 288, "hotpluggable-cpus": true, "name": "pc-q35-2.8" }, [...] ] } Considered alternatives === Indirect mapping (machine => bus => device) --- This RFC implements a mechanism to implement ax machine-type => supported-device-types mapping. An alternative solution I considered was to expose an indirect mapping: machine-type => default-bus-types followed by bus-type => supported-device-types. As I previously stated, I don't know if it helps. bus-type can support different devices on different archs. But exposing only the resulting supported device-types list imposes less restrictions on how the
Re: [Qemu-devel] [RFC 00/15] qmp: Report supported device types on 'query-machines'
(CCing people from the spapr PCI-express thread) On Mon, Nov 21, 2016 at 11:11:58PM -0200, Eduardo Habkost wrote: > The Problem > === > > Currently management software has no way to find out which device > types can be plugged in a machine, unless the machine is already > initialized. > > Even after the machine is initialized, there's no way to map > existing bus types to supported device types unless management > software hardcodes the mapping between bus types and device > types. > > Example: floppy support on q35 vs i440fx > > > There's no way for libvirt to find out that there's no floppy > controller on pc-q35-* machine-types by default. > > With this series, pc-i440fx-* will report "floppy" as a supported > device type, but pc-q35-* will not. > > Example: Legacy PCI vs vs PCIe devices > -- > > Some devices require a PCIe bus to be available, others work on > both legacy PCI and PCIe, while others work only on a legacy PCI > bus. > > Currently management software has no way to know which devices > can be added to a given machine, unless it hardcodes machine-type > names and device-types names. Another example: spapr and PCIe root bus --- See the thread at: Subject: [RFC PATCH qemu] spapr_pci: Create PCI-express root bus by default If we mak new spapr machine-type versions create a PCIe root bus, management software will need a way to find out: 1) The type of the default bus for the machine type; 2) The ID of the default bus for the machine type. Otherwise, management software will have to hardcode it based on machine-type version. The proposed interface should solve this problem. There are other comment below, at "Limitations" section: > > The Proposed Interface > == > > This series adds a new field to the output of 'query-machines': > 'supported-device-types'. It will contain a list of QOM type > names, that can be used to find the list of device types that can > be plugged in the machine by default. The type names reported on > the new field can then be used as the 'implements' argument on > the 'qom-list-types' command, to find out which device types can > be plugged on the machine. > > Example output > -- > > (QEMU) query-machines > { > "return": [ > [...] > { > "supported-device-types": [ > "sys-bus-device" > ], > "cpu-max": 1, > "hotpluggable-cpus": false, > "name": "none" > }, > [...] > { > "supported-device-types": [ > "sys-bus-device" > ], > "cpu-max": 1, > "hotpluggable-cpus": false, > "name": "xenpv" > }, > [...] > { > "supported-device-types": [ > "sys-bus-device", > "floppy", > "i2c-slave", > "pci-device", > "isa-device", > "ide-device" > ], > "name": "pc-i440fx-2.8", > "alias": "pc", > "is-default": true, > "cpu-max": 255, > "hotpluggable-cpus": true > }, > [...] > { > "supported-device-types": [ > "sys-bus-device", > "floppy", > "isa-device", > "ide-device" > ], > "cpu-max": 1, > "hotpluggable-cpus": true, > "name": "isapc" > }, > [...] > { > "supported-device-types": [ > "sys-bus-device", > "floppy", > "i2c-slave", > "pci-device", > "isa-device", > "ide-device" > ], > "cpu-max": 128, > "hotpluggable-cpus": true, > "name": "xenfv" > }, > [...] > { > "alias": "q35", > "supported-device-types": [ > "sys-bus-device", > "i2c-slave", > "PCIE-device", > "isa-device", > "ide-device" > ], > "cpu-max": 288, > "hotpluggable-cpus": true, > "name": "pc-q35-2.8" > }, > [...] > ] > } > > Considered alternatives > === > > Indirect mapping (machine => bus => device) > --- > > This RFC implements a mechanism to implement ax > machine-type => supported-device-types > mapping. An alternative solution I considered was to expose an > indirect mapping: > machine-type => default-bus-types > followed by > bus-type => supported-device-types. > > But exposing only the resulting supported device-types list > imposes less restrictions on how the device and bus type >
Re: [Qemu-devel] [RFC 00/15] qmp: Report supported device types on 'query-machines'
On Tue, Nov 22, 2016 at 09:18:21AM +0100, David Hildenbrand wrote: > > > > > Considered alternatives > > === > > > > Indirect mapping (machine => bus => device) > > --- > > > > This RFC implements a mechanism to implement ax > > machine-type => supported-device-types > > mapping. An alternative solution I considered was to expose an > > indirect mapping: > > machine-type => default-bus-types > > followed by > > bus-type => supported-device-types. > > > > But exposing only the resulting supported device-types list > > imposes less restrictions on how the device and bus type > > hierarchy is implemented inside QEMU. There's still a > > machine-type => bus-type => device-type > > mapping implemented internally, but it is an implementation > > detail on the current version, and not part of the > > externally-visible interface. > > > > The Implementation > > == > > > > This add a new field to MachineClass: default_bus_types, and a > > new field to BusClass: supported_device_type. > > Is it possible to modify a machine (setting some properties e.g. on the > command line), that suddenly more devices are supported? Something like > enabling an additional bus? (I assume so, because it is called "default > bus types" :) ) Yes. See the "Out of scope: Configurable buses" section on the cover letter. > > If so, the indirect mapping could be of more benefit in the long run. Yes, a new mapping may be necessary to find out which options (e.g. "usb=on") or device-types (e.g. "-device isa-fdc", "-device piix3-usb-uhci") will add new buses and make new device-types supported by the machine. But this will probably require new QMP commands, and I still want to keep the default case simple. > > Thinking about my machine at home: > > I just care about the available buses. If my machine doesn't have USB, > but PCI, I can buy a USB PCI card and make it support USB. Then I can > happily plug in USB devices. So the "default" state might at least > no longer be sufficient when wanting to plug in a USB fan on a hot > summer day ;) . > > But, with the indirect mapping, I guess we would need yet another qmp > command ... > > -- > > David -- Eduardo
Re: [Qemu-devel] [RFC 00/15] qmp: Report supported device types on 'query-machines'
Considered alternatives === Indirect mapping (machine => bus => device) --- This RFC implements a mechanism to implement ax machine-type => supported-device-types mapping. An alternative solution I considered was to expose an indirect mapping: machine-type => default-bus-types followed by bus-type => supported-device-types. But exposing only the resulting supported device-types list imposes less restrictions on how the device and bus type hierarchy is implemented inside QEMU. There's still a machine-type => bus-type => device-type mapping implemented internally, but it is an implementation detail on the current version, and not part of the externally-visible interface. The Implementation == This add a new field to MachineClass: default_bus_types, and a new field to BusClass: supported_device_type. Is it possible to modify a machine (setting some properties e.g. on the command line), that suddenly more devices are supported? Something like enabling an additional bus? (I assume so, because it is called "default bus types" :) ) If so, the indirect mapping could be of more benefit in the long run. Thinking about my machine at home: I just care about the available buses. If my machine doesn't have USB, but PCI, I can buy a USB PCI card and make it support USB. Then I can happily plug in USB devices. So the "default" state might at least no longer be sufficient when wanting to plug in a USB fan on a hot summer day ;) . But, with the indirect mapping, I guess we would need yet another qmp command ... -- David
Re: [Qemu-devel] [RFC 00/15] qmp: Report supported device types on 'query-machines'
Hi, Your series failed automatic build test. Please find the testing commands and their output below. If you have docker installed, you can probably reproduce it locally. Subject: [Qemu-devel] [RFC 00/15] qmp: Report supported device types on 'query-machines' Type: series Message-id: 1479777133-23567-1-git-send-email-ehabk...@redhat.com === TEST SCRIPT BEGIN === #!/bin/bash set -e git submodule update --init dtc # Let docker tests dump environment info export SHOW_ENV=1 export J=16 make docker-test-quick@centos6 make docker-test-mingw@fedora make docker-test-build@min-glib === TEST SCRIPT END === Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384 Switched to a new branch 'test' 5e1ecf3 qdev: Add device_class_set_bus_type() function c8ab7cb ppc: Initialize default bus lists 70590aa mips: Initialize default bus lists 896edc8 arm: Initialize default bus lists dd993a8 s390x: Initialize default bus lists 9ef506f pc: Initialize default bus lists 24f4450 pci: Introduce INTERFACE_PCIE_DEVICE interface name c39bfeb qmp: Add 'supported-device-types' field to 'query-machines' 07ce179 machine: Add MachineClass::default_buses field 7680d6c qdev: Add device_type field to BusClass 76da7a3 tests: Add rules to non-gtester qtest test cases e531dad qtest.py: Make 'binary' parameter optional 80780b6 qtest.py: make logging optional 3d3c3e5 qtest.py: Support QTEST_LOG environment variable b2326ca qemu.py: Make logging optional === OUTPUT BEGIN === Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc' Cloning into 'dtc'... Submodule path 'dtc': checked out '65cc4d2748a2c2e6f27f1cf39e07a5dbabd80ebf' BUILD centos6 make[1]: Entering directory `/var/tmp/patchew-tester-tmp-fmoi6289/src' ARCHIVE qemu.tgz ARCHIVE dtc.tgz COPYRUNNER RUN test-quick in qemu:centos6 Packages installed: SDL-devel-1.2.14-7.el6_7.1.x86_64 ccache-3.1.6-2.el6.x86_64 epel-release-6-8.noarch gcc-4.4.7-17.el6.x86_64 git-1.7.1-4.el6_7.1.x86_64 glib2-devel-2.28.8-5.el6.x86_64 libfdt-devel-1.4.0-1.el6.x86_64 make-3.81-23.el6.x86_64 package g++ is not installed pixman-devel-0.32.8-1.el6.x86_64 tar-1.23-15.el6_8.x86_64 zlib-devel-1.2.3-29.el6.x86_64 Environment variables: PACKAGES=libfdt-devel ccache tar git make gcc g++ zlib-devel glib2-devel SDL-devel pixman-devel epel-release HOSTNAME=68bc562d605e TERM=xterm MAKEFLAGS= -j16 HISTSIZE=1000 J=16 USER=root CCACHE_DIR=/var/tmp/ccache EXTRA_CONFIGURE_OPTS= V= SHOW_ENV=1 MAIL=/var/spool/mail/root PATH=/usr/lib/ccache:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PWD=/ LANG=en_US.UTF-8 TARGET_LIST= HISTCONTROL=ignoredups SHLVL=1 HOME=/root TEST_DIR=/tmp/qemu-test LOGNAME=root LESSOPEN=||/usr/bin/lesspipe.sh %s FEATURES= dtc DEBUG= G_BROKEN_FILENAMES=1 CCACHE_HASHDIR= _=/usr/bin/env Configure options: --enable-werror --target-list=x86_64-softmmu,aarch64-softmmu --prefix=/var/tmp/qemu-build/install No C++ compiler available; disabling C++ specific optional code Install prefix/var/tmp/qemu-build/install BIOS directory/var/tmp/qemu-build/install/share/qemu binary directory /var/tmp/qemu-build/install/bin library directory /var/tmp/qemu-build/install/lib module directory /var/tmp/qemu-build/install/lib/qemu libexec directory /var/tmp/qemu-build/install/libexec include directory /var/tmp/qemu-build/install/include config directory /var/tmp/qemu-build/install/etc local state directory /var/tmp/qemu-build/install/var Manual directory /var/tmp/qemu-build/install/share/man ELF interp prefix /usr/gnemul/qemu-%M Source path /tmp/qemu-test/src C compilercc Host C compiler cc C++ compiler Objective-C compiler cc ARFLAGS rv CFLAGS-O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -g QEMU_CFLAGS -I/usr/include/pixman-1-pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -fPIE -DPIE -m64 -mcx16 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -Wendif-labels -Wmissing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-all LDFLAGS -Wl,--warn-common -Wl,-z,relro -Wl,-z,now -pie -m64 -g make make install install pythonpython -B smbd /usr/sbin/smbd module supportno host CPU x86_64 host big endian no target list x86_64-softmmu aarch64-softmmu tcg debug enabled no gprof enabled no sparse enabledno strip binariesyes profiler no static build no pixmansystem SDL support yes (1.2.14) GTK support no GTK GL supportno VTE support no TLS priority NORMAL GNUTLS supportno GNUTLS rndno libgcrypt no libgcrypt kdf no nettleno nettle kdf
Re: [Qemu-devel] [RFC 00/15] qmp: Report supported device types on 'query-machines'
Hi, Your series seems to have some coding style problems. See output below for more information: Subject: [Qemu-devel] [RFC 00/15] qmp: Report supported device types on 'query-machines' Type: series Message-id: 1479777133-23567-1-git-send-email-ehabk...@redhat.com === TEST SCRIPT BEGIN === #!/bin/bash BASE=base n=1 total=$(git log --oneline $BASE.. | wc -l) failed=0 # Useful git options git config --local diff.renamelimit 0 git config --local diff.renames True commits="$(git log --format=%H --reverse $BASE..)" for c in $commits; do echo "Checking PATCH $n/$total: $(git log -n 1 --format=%s $c)..." if ! git show $c --format=email | ./scripts/checkpatch.pl --mailback -; then failed=1 echo fi n=$((n+1)) done exit $failed === TEST SCRIPT END === Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384 From https://github.com/patchew-project/qemu * [new tag] patchew/1479777133-23567-1-git-send-email-ehabk...@redhat.com -> patchew/1479777133-23567-1-git-send-email-ehabk...@redhat.com Switched to a new branch 'test' 5e1ecf3 qdev: Add device_class_set_bus_type() function c8ab7cb ppc: Initialize default bus lists 70590aa mips: Initialize default bus lists 896edc8 arm: Initialize default bus lists dd993a8 s390x: Initialize default bus lists 9ef506f pc: Initialize default bus lists 24f4450 pci: Introduce INTERFACE_PCIE_DEVICE interface name c39bfeb qmp: Add 'supported-device-types' field to 'query-machines' 07ce179 machine: Add MachineClass::default_buses field 7680d6c qdev: Add device_type field to BusClass 76da7a3 tests: Add rules to non-gtester qtest test cases e531dad qtest.py: Make 'binary' parameter optional 80780b6 qtest.py: make logging optional 3d3c3e5 qtest.py: Support QTEST_LOG environment variable b2326ca qemu.py: Make logging optional === OUTPUT BEGIN === Checking PATCH 1/15: qemu.py: Make logging optional... Checking PATCH 2/15: qtest.py: Support QTEST_LOG environment variable... Checking PATCH 3/15: qtest.py: make logging optional... WARNING: line over 80 characters #26: FILE: scripts/qtest.py:87: + socket_scm_helper=socket_scm_helper, total: 0 errors, 1 warnings, 14 lines checked Your patch has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS. Checking PATCH 4/15: qtest.py: Make 'binary' parameter optional... Checking PATCH 5/15: tests: Add rules to non-gtester qtest test cases... Checking PATCH 6/15: qdev: Add device_type field to BusClass... Checking PATCH 7/15: machine: Add MachineClass::default_buses field... Checking PATCH 8/15: qmp: Add 'supported-device-types' field to 'query-machines'... WARNING: line over 80 characters #111: FILE: tests/qmp-machine-info.py:39: +dt = vm.qmp('qom-get', path=cp, property='device-type')['return'] ERROR: line over 90 characters #119: FILE: tests/qmp-machine-info.py:47: +vm = qtest.QEMUQtestMachine(args=['-S', '-machine', machine['name']], logging=False) WARNING: line over 80 characters #141: FILE: tests/qmp-machine-info.py:69: +self.fail("extra device types: %s" % (' '.join(missing_types))) WARNING: line over 80 characters #162: FILE: tests/qmp-machine-info.py:90: +vm = qtest.QEMUQtestMachine(args=['-S', '-machine', 'none'], logging=False) total: 1 errors, 3 warnings, 189 lines checked Your patch has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS. Checking PATCH 9/15: pci: Introduce INTERFACE_PCIE_DEVICE interface name... Checking PATCH 10/15: pc: Initialize default bus lists... ERROR: do not use C99 // comments #20: FILE: hw/i386/pc_piix.c:438: +machine_class_add_default_bus(m, "floppy-bus"); //FIXME: use macro ERROR: do not use C99 // comments #21: FILE: hw/i386/pc_piix.c:439: +machine_class_add_default_bus(m, "i2c-bus"); //FIXME: use macro ERROR: do not use C99 // comments #24: FILE: hw/i386/pc_piix.c:442: +machine_class_add_default_bus(m, "IDE");//FIXME: use macro ERROR: do not use C99 // comments #32: FILE: hw/i386/pc_piix.c:1087: +machine_class_add_default_bus(m, "floppy-bus"); //FIXME: use macro ERROR: do not use C99 // comments #34: FILE: hw/i386/pc_piix.c:1089: +machine_class_add_default_bus(m, "IDE");//FIXME: use macro ERROR: do not use C99 // comments #42: FILE: hw/i386/pc_piix.c:1109: +machine_class_add_default_bus(m, "floppy-bus"); //FIXME: use macro ERROR: do not use C99 // comments #43: FILE: hw/i386/pc_piix.c:1110: +machine_class_add_default_bus(m, "i2c-bus"); //FIXME: use macro ERROR: do not use C99 // comments #46: FILE: hw/i386/pc_piix.c:1113: +machine_class_add_default_bus(m, "IDE");//FIXME: use macro ERROR: do not use C99 // comments #58: FILE: hw/i386/pc_q35.c:295: +machine_class_add_default_bus(m, "i2c-bus"); //FIXME: use macro ERROR: do not use C99 // comments #61: