On Thu, Nov 24, 2016 at 02:34:05PM +0100, Markus Armbruster wrote:
> Eduardo Habkost <ehabk...@redhat.com> writes:
>
> > On Wed, Nov 23, 2016 at 06:43:16PM +0200, Marcel Apfelbaum wrote:
> >> On 11/22/2016 03:11 AM, Eduardo Habkost wrote:
> >> > The Problem
> >> > ===========
> >> >
> >> > Currently management software has no way to find out which device
> >> > types can be plugged in a machine, unless the machine is already
> >> > initialized.
>
> I doubt it can find the precise list of device types even for an
> initialized machine.
My plan is to make it possible. See:
[RFC 06/15] qdev: Add device_type field to BusClass
I will change how it looks like on v2 (mostly due to the PCI/PCIe
issues), but basically it makes all bus objects able to tell
which device types they accept.
>
> >> Hi Eduardo,
> >> Thank you for this interesting series. I think this is a problem
> >> worth addressing.
> >>
> >> > Even after the machine is initialized, there's no way to map
> >> > existing bus types to supported device types unless management
> >> > software hardcodes the mapping between bus types and device
> >> > types.
> >> >
> >>
> >> Here I am a little lost.
> >>
> >> We are going for machine => supported devices or
> >> bus-type => supported devices?
> >
> > On this series, we go for machine-type => supported-devices.
> >
> > A bus-type => supported-devices map wouldn't work because
> > different PCIe bus instances might accept different types of
> > devices (so supported-devices depend on the specific bus
> > instance, not only on the bus-type).
> >
> > v2 will probably be more detailed. I plan to change it to:
> >
> > query-machine(machine-type) => list of BusInfo
> >
> > BusInfo would contain:
> > * bus-type
> > * bus-path
> > * accepted-device-types (list of type/interface names)
>
> Let me take a step back and explore what questions a management
> application may want to ask. I think the basic questions are:
>
> * Which devices models could be plugged now?
>
> "Now" means in the current state of the machine. Depending on "now",
> this could be hot- or cold-plug.
Yes.
>
> * Into which sockets could device model T be plugged now?
>
> Mind, I wrote "socket", not bus. Buses provide sockets, but they are
> not the only provider of sockets.
>
> We need a way to name sockets. (QOM path to device or bus, socket
> address on device or bus) could work.
Naming "sockets" was not on my model, but I think we can do that.
I will probably send RFC v2 without any socket abstraction, so
other conceptual changes I am introducing can be reviewed
(especially the PCI/PCIe mess I am diving into, right now). But I
will try to document how the design can evolve to handle sockets.
>
> Are we on the same page so far?
Yes.
>
> Related: same questions with "newly created machine of machine type M"
> instead of "now". This isn't basic, because you could always create the
> machine, then ask the basic question. But not having to create the
> machine is convenient, in particular as long as a single QEMU process
> can only ever create one machine.
>
> Related: same, with creation options.
>
> Your series provides a solution for a special non-basic case: newly
> created machine of machine type M with no optional parts, i.e. with
> -nodefaults and nothing else. Or maybe with the default set of optional
> parts, i.e. without -nodefaults, not sure.
>
> I'm not demanding you provide a solution for the general case first. It
> would be nice, but it could be too complex for a first step. I do
> recommend you design your solution for the special case with the general
> case in mind. Say, by trying to ensure the special case query returns
> something that will do or can be evolved to do for the general case,
> too.
Yep. I will keep that in mind.
>
> >> > Example: floppy support on q35 vs i440fx
> >> > ----------------------------------------
> >> >
> >> > There's no way for libvirt to find out that there's no floppy
> >> > controller on pc-q35-* machine-types by default.
> >> >
> >>
> >> Again "by default". So do we want to query the init state of a machine?
> >> What devices are there? Or what devices *can be* there?
> >
> > "by default" means what's present when using "-machine <machine>"
> > with no extra -device arguments.
>
> Not just -device. Some legacy options can also create buses. For
> instance, -device if=scsi,... creates a SCSI bus.
Right. I found that out when debugging the -nodefaults issues.
>
> > We want to know what _buses_ are always there. Which in turn lets
> > management know which _device_ types _can_ be plugged.
> >
> >>
> >> > With this series, pc-i440fx-* will report "floppy" as a supported
> >> > device type, but pc-q35-* will not.
> >> >
> >> > Example: Legacy PCI vs vs PCIe devices
> >> > --------------------------------------
> >> >
> >> > Some devices require a PCIe bus to be available, others work on
> >> > both legacy PCI and PCIe, while others work only on a legacy PCI
> >> > bus.
> >> >
> >> > Currently management software has no way to know which devices
> >> > can be added to a given machine, unless it hardcodes machine-type
> >> > names and device-types names.
> >> >
> >>
> >> Again it seems a double problem, machine => devices vs pci/pcie bus =>
> >> devices.
> >> The bus => devices match is not related to a machine type.
> >
> > A bus-type => device-type match would not depend on the
> > machine-type, but it would not be useful: different bus instances
> > can accept different device-types (and the way the bus topology
> > is configured depend on the machine-type).
> >
> >
> >>
> >> > The Proposed Interface
> >> > ======================
> >> >
> >> > This series adds a new field to the output of 'query-machines':
> >> > 'supported-device-types'. It will contain a list of QOM type
> >> > names, that can be used to find the list of device types that can
> >> > be plugged in the machine by default.
> >>
> >> What do you mean "by default"? Without bridges or part of the machine
> >> itself?
> >
> > I mean "when you just run -machine with no extra -device
> > arguments".
> >
> >>
> >> The type names reported on
> >> > the new field can then be used as the 'implements' argument on
> >> > the 'qom-list-types' command, to find out which device types can
> >> > be plugged on the machine.
> >> >
> >> > Example output
> >> > --------------
> >> >
> >> > (QEMU) query-machines
> >> > {
> >> > "return": [
> >> > [...]
> >> > {
> >> > "supported-device-types": [
> >> > "sys-bus-device"
> >>
> >>
> >> I don't know how "sys-bus-device" can help us... :)
> >
> > Yes, I added comments about it below. :)
>
> The sysbus has always been a cop-out. It's a catch-all pseudo-bus. The
> only thing sysbus devices really have in common is that they can't plug
> into any real bus.
>
> This used to be an academic issue until Alex made some sysbus devices
> pluggable. Before, "sysbus device" told you everything about
> pluggability you needed to know: not pluggable. Since then, it tells
> you nothing. We'll have to address this regression eventually. See
> "TYPE_SYS_BUS_DEVICE is too generic" below.
>
> >> > ],
> >> > "cpu-max": 1,
> >> > "hotpluggable-cpus": false,
> >> > "name": "none"
> >> > },
> >> > [...]
> >> > {
> >> > "supported-device-types": [
> >> > "sys-bus-device"
> >> > ],
> >> > "cpu-max": 1,
> >> > "hotpluggable-cpus": false,
> >> > "name": "xenpv"
> >> > },
> >> > [...]
> >> > {
> >> > "supported-device-types": [
> >> > "sys-bus-device",
> >> > "floppy",
> >> > "i2c-slave",
> >> > "pci-device",
> >> > "isa-device",
> >> > "ide-device"
> >>
> >> Is don't know is this high level classification is useful,
> >> here is an example:
> >>
> >> pvi-device is supported => then we look for all pci devices?
> >> But what if some pci devices make sense on a machine type,
> >> but not on another?
> >
> > If not all pci devices are supported, then the machine must not
> > return "pci-device" as supported. We need to define a new
> > type/interface name that would be implemented only by the
> > supported devices. e.g. "legacy-pci-device".
>
> Before PCIe, "PCI device" told you everything about pluggability you
> needed to know: can be plugged into a PCI bus. Since PCIe, it doesn't.
> We'll have to address this regression, too. See "PCI vs PCIe" below.
I am trying to clean up the PCI/PCIe mess right now, for v2.
>
> >> > ],
> >> > "name": "pc-i440fx-2.8",
> >> > "alias": "pc",
> >> > "is-default": true,
> >> > "cpu-max": 255,
> >> > "hotpluggable-cpus": true
> >> > },
> >> > [...]
> >> > {
> >> > "supported-device-types": [
> >> > "sys-bus-device",
> >> > "floppy",
> >> > "isa-device",
> >> > "ide-device"
> >> > ],
> >> > "cpu-max": 1,
> >> > "hotpluggable-cpus": true,
> >> > "name": "isapc"
> >> > },
> >> > [...]
> >> > {
> >> > "supported-device-types": [
> >> > "sys-bus-device",
> >> > "floppy",
> >> > "i2c-slave",
> >> > "pci-device",
> >> > "isa-device",
> >> > "ide-device"
> >> > ],
> >> > "cpu-max": 128,
> >> > "hotpluggable-cpus": true,
> >> > "name": "xenfv"
> >> > },
> >> > [...]
> >> > {
> >> > "alias": "q35",
> >> > "supported-device-types": [
> >> > "sys-bus-device",
> >> > "i2c-slave",
> >> > "PCIE-device",
> >> > "isa-device",
> >> > "ide-device"
> >> > ],
> >> > "cpu-max": 288,
> >> > "hotpluggable-cpus": true,
> >> > "name": "pc-q35-2.8"
> >> > },
> >> > [...]
> >> > ]
> >> > }
> >> >
> >> > Considered alternatives
> >> > =======================
> >> >
> >> > Indirect mapping (machine => bus => device)
> >> > -------------------------------------------
> >> >
> >> > This RFC implements a mechanism to implement ax
> >> > machine-type => supported-device-types
> >> > mapping. An alternative solution I considered was to expose an
> >> > indirect mapping:
> >> > machine-type => default-bus-types
> >> > followed by
> >> > bus-type => supported-device-types.
> >> >
> >>
> >> As I previously stated, I don't know if it helps. bus-type
> >> can support different devices on different archs.
> >
> > Yes. After learning a bit more about PCIe, I am convinced that a
> > bus->type => device-types mapping won't help.
> >
> >>
> >> > But exposing only the resulting supported device-types list
> >> > imposes less restrictions on how the device and bus type
> >> > hierarchy is implemented inside QEMU. There's still a
> >> > machine-type => bus-type => device-type
> >> > mapping implemented internally, but it is an implementation
> >> > detail on the current version, and not part of the
> >> > externally-visible interface.
> >> >
> >>
> >> Good, I personally don't like this "pass".
>
> Point taken.
>
> The core idea here seems to be using the QOM type hierarchy for
> representing "pluggability".
Exactly.
>
> >> > The Implementation
> >> > ==================
> [Skipping for now...]
> >> >
> >> > Limitations
> >> > ===========
> >> >
> >> > TYPE_SYS_BUS_DEVICE is too generic
> >> > ----------------------------------
> >> >
> >> > Currently all machines have a TYPE_SYS_BUS bus, meaning all
> >> > TYPE_SYS_BUS_DEVICE subclasses are reported as supported.
> >> >
> >>
> >> Agreed, this is a problem.
> >>
> >> > The current solution in this series is to report
> >> > TYPE_SYS_BUS_DEVICE as supported by all machines.
>
> Because all machines have this catch-all pseudo-bus. Reporting it
> exists is correct, but as is the information is useless.
I will probably keep returning it by default (to make the
existing information correct), but provide mechanisms for
machine-types to return more precise information.
>
> >> > But we could
> >> > gradually add arch-specific or machine-family-specific interface
> >> > names that can be used on devices that are meant to work with
> >> > only a subset of TYPE_SYS_BUS_DEVICE subclasses.
>
> Basically, encode pluggability information in the QOM type hierarchy.
>
> To gauge whether that would work I'd have to actually understand how
> exactly pluggable sysbus devices work, in particular what the conditions
> for accepting a plug are.
I can't evaluate that, either.
But note that covering 100% of the pluggability decisions for
sysbus is not one of the problems I am trying to solve right now.
I mean: if the new mechanisms can cover it too, great, but they
don't (and there's no obvious solution for it), it is out of
scope.
>
> >> > A future version of this series may remove TYPE_SYS_BUS_DEVICE
> >> > from the supported-device-types output, and return a
> >> > arch-specific or machine-family-specific interface name to
> >> > restrict management software to a subset of TYPE_SYS_BUS_DEVICE
> >> > subclasses.
> >> >
> >> > PCI vs PCIe
> >> > -----------
> >> >
> >> > Machines with PCIe buses will report INTERFACE_PCIE_DEVICE on
> >> > supported-device-types.
> >> >
> >> > Machines with legacy PCI buses will report TYPE_PCI_DEVICE on
> >> > supported-device-types.
> >> >
> >> > The problem with the current approach is that PCIe devices are
> >> > TYPE_PCI_DEVICE subclasses. The allows PCI device classes to
> >> > indicate they are PCIe-capable, but there's no obvious way to
> >> > indicate that a device is PCIe-only. This needs to be addressed
> >> > in a future version of this series.
> >> >
> >> > Suggestions are welcome.
> >>
> >>
> >> As we talked offline, I personally like an interface IPCIType
> >> with a field having 3 possible values {pci/pcie/hybrid}
> >>
> >> To understand how hybrid works we need some rules, like
> >> "pci on pci bus/pcie on pcie bus"
> >> "pcie on a non-root pcie bus/pcie otherwise
> >>
> >> I don't think we'll have a lot of rules, simple boolean fields
> >> for the interface should be enough.
> >
> > What you propose makes sense, the only difference is that the
> > boolean fields would be just interface names that can be used as
> > the "implements" argument on qom-list-types.
> >
> > e.g.:
> >
> > * Hybrid PCI device-types would implement both "legacy-pci-device" and
> > "pcie-device" interfaces.
> > * PCIe-only device-types would implement only the "pcie-device"
> > interface.
> > * Legacy-PCI-only device-types would implement only the
> > "legacy-pci-device" interface.
>
> In other words, encode "pluggability" in the QOM type hierarchy.
>
> > Then, the bus instances would have a 'accepted-device-types'
> > field:
> >
> > * A legacy PCI bus would accept only "legacy-pci-device" devices.
> > * A PCIe-only bus would accept only "pcie-device" devices.
> > * A PCIe bus that accepts legacy PCI devices (the root bus?)
> > would accept both "legacy-pci-device" and "pcie-device"
> > devices.
> >
> > Then, query-machines would return the list of bus instances that
> > machine init creates, containing the bus ID, bus type, and
> > accepted-device-types.
> >
> > Does that make sense?
>
> Yes.
>
> Our decision to have hybrid PCI/PCIe devices and buses breeds
> considerable complexity. I wish we had avoided them, but I believe it's
> too late to change now.
I'm diving into it right now. v2 will have some attempts to sort
it out.
>
> >> This still does not solve the problem that some devices makes
> >> sense only on a specific arch.
>
> Examples?
>
> > Right now, we can simply compile out arch-specific devices that
> > can never be plugged in a QEMU binary. But if we still want them
> > compiled in for some reason, we can categorize them on a
> > different type/interface name, and the corresponding machine-type
> > would tell management that their buses accept only the
> > arch-specific type/interface name.
>
> Could become messy. Can't say without looking at examples.
Can't say, either.
>
> >> > Incomplete bus lists on some machines
> >> > -------------------------------------
> >> >
> >> > With this series, not all machines classes are changed to add the
> >> > full list of device types on the 'supported-device-types'. To
> >> > allow the code to be updated gradually, qmp-machine-info.py has a
> >> > STRICT_ARCHES variable, that will make the test code require a
> >> > complete device type list only on some architectures.
>
> Ideally, pluggability information would be derived from QOM. I figure
> that's the only practical way to solve the general case.
>
> For the special case "newly created machine", deriving from QOM may be
> impractical, say because you'd have to create the machine. Then you
> have to hard-code the result somehow. Tests to validate the hardcoded
> results match the results you get when you actually create the machine
> would be required, at least once you *can* get results for a created
> machine.
Tests were implemented, see
[RFC 08/15] qmp: Add 'supported-device-types' field to 'query-machines'.
>
> >> > Out of scope: Configurable buses
> >> > --------------------------------
> >> >
> >> > There's no way to map machine options like "usb=on|off" to
> >> > device-types or buses. I plan to propose a new interface that
> >> > allows machine options to be mapped to buses/device-types later.
> >> >
> >> > Out of scope: Deciding where to plug devices
> >> > --------------------------------------------
> >> >
> >> > Once management software discovers which devices can be plugged
> >> > to a machine, it still has to discover or define where devices
> >> > can/should/will be plugged. This is out of the scope of this
> >> > series.
> >> >
> >>
> >> That's a pitty :(
> >> I was hoping this series will solve this issue. But if it prepares
> >> the grounds for it is also a good step .
> >
> > The bus ID will be in the scope of v2.
>
> I'm afraid "bus" is insufficiently general.
Does "socket" sound general enough?
>
> >> Thanks,
> >> Marcel
> >>
> >> > Out of scope: Hotplug
> >> > ---------------------
> >> >
> >> > The proposed interface is supposed to help management software
> >> > find which device types can be used when creating the VM. Device
> >> > hotplug is out of the scope of this series. However, the new
> >> > 'device-types' QOM property on bus objects could be used to find
> >> > out which device types can be plugged on the existing buses.
--
Eduardo
