Hi, It seems to me we're getting rather bogged down in how to proceed with an improved CPU hotplug (and hot unplug) interface, both generically and for ppc in particular.
So here's a somewhat more concrete suggestion of a way forward, to see if we can get some consensus. The biggest difficulty I think we're grappling with is that device-add is actually *not* a great interface to cpu hotplug. Or rather, it's not great as the _only_ interface: in order to represent the many different constraints on how cpus can be plugged on various platforms, it's natural to use a heirarchy of cpu core / socket / package types specific to the specific platform or real-world cpu package being modeled. However, for the normal case of a regular homogenous (and at least slightly para-virtualized) server, that interface is nasty for management layers because they have to know the right type to instantiate. To address this, I'm proposing this two layer interface: Layer 1: Low-level, device-add based * a new, generic cpu-package QOM type represents a group of 1 or more cpu threads which can be hotplugged as a unit * cpu-package is abstract and can't be instantiated directly * archs and/or individual platforms have specific subtypes of cpu-package which can be instantiated * for platforms attempting to be faithful representations of real hardware these subtypes would match the specific characteristics of the real hardware devices. In addition to the cpu threads, they may have other on chip devices as sub-objects. * for platforms which are paravirtual - or which have existing firmware abstractions for cpu cores/sockets/packages/whatever - these could be more abstract, but would still be tied to that platform's constraints * Depending on the platform the cpu-package object could have further internal structure (e.g. a package object representing a socket contains package objects representing each core, which in turn contain cpu objects for each thread) * Some crazy platform that has multiple daughterboards each with several multi-chip-modules each with several chips, each with several cores each with several threads could represent that too. What would be common to all the cpu-package subtypes is: * A boolean "present" attribute ("realized" might already be suitable, but I'm not certain) * A generic means of determining the number of cpu threads in the package, and enumerating those * A generic means of determining if the package is hotpluggable or not * They'd get listed in a standard place in the QOM tree This interface is suitable if you want complete control over constructing the system, including weird cases like heterogeneous machines (either totally different cpu types, or just different numbers of threads in different packages). The intention is that these objects would never look at the global cpu type or sockets/cores/threads numbers. The next level up would instead configure the packages to match those for the common case. Layer 2: Higher-level * not all machine types need support this model, but I'd expect all future versions of machine types designed for production use to do so * machine types don't construct cpu objects directly * instead they create enough cpu-package objects - of a subtype suitable for this machine - to provide maxcpus threads * the machine type would set the "present" bit on enough of the cpu packages to provide the base number of cpu threads Management layers can then manage hotplug without knowing platform specifics by using qmp to toggle the "present" bit on packages. Platforms that allow thread-level pluggability can expose a package for every thread, those that allow core-level expose a package per core, those that have even less granularity expose a package at whatever grouping they can do hotplug on. Examples: For use with pc (or q35 or whatever) machine type, I'd expect a cpu-package subtype called, say "acpi-thread" which represents a single thread in the ACPI sense. Toggling those would trigger ACPI hotplug events as cpu_add does now. For use with pseries, I'd expect a "papr-core" cpu-package subtype, which represents a single (paravirtual) core. Toggling present on this would trigger the PAPR hotplug events. A property would control the number of threads in the core (only settable before enabling present). For use with the powernv machine type (once ready for merge) I'd expect "POWER8-package" type which represents a POWER8 chip / module as close to the real hardware as we can get. It would have a fixed number of cores and threads within it as per the real hardware, and would also include xscoms and other per-module logic. From here to there: A suggested order of implementation to get there without too much risk of breaking things. 1. Fix bugs with creation / removal of CPU objects (Bharata's cpu hotplug series already has this) 2. Split creation and realization of CPU objects, so machine types must explicitly perform both steps (Bharata's series has this too) 3. Add the abstract cpu-package type, and define the generic interfaces it needs (Bharata's series has something that could be changed to this fairly easily) 4. For each machine type we care to convert: 4.1. Add platform suitable cpu-package subtypes 4.2. Convert the (latest version) machine type to instantiate packages instead of cpu threads directly 4.3. Add any necessary backwards compat goo 5. Teach libvirt how to toggle cpu-packages -- David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
signature.asc
Description: PGP signature