Re: vCPU hotplug roadmap (was: Minutes for KVM call 2013-01-15)
On Wed, Jan 30, 2013 at 11:58:56AM +0100, Andreas Färber wrote: > Am 15.01.2013 17:16, schrieb Juan Quintela: > > > > * cpu hot plug > > - use qdev propierties conected to a set of socket objects (anthony) > > - cpusets are the wrong interface (anthony) > > - make a link between cpu <-> socket instead of a propierty? > > - how far are we from being able to describe a cpu with -device? > > (didn't heare the answer, andreas?) > > - perhaps the best approach? > > - After soft-freeze, exceptions depend on the maintainer > > - After hard-freeze, no exceptions > > -device don't require a bus, just an implementation detail, we can change > > that > > - use cpuset as an intermediate step until full vision is implemented > > - several approaches from where we are now, to have something before > > we get a full solution > > > > > > At this point, Andreas agreed to write a better summary of the > > discussion and suggestions O:-) > > Got buried, here we go: > > == vCPU hot-plug user interfaces == > > === cpu_set === > > Previously available in qemu-kvm.git: > `cpu_set online` via HMP > > Pros: > * Hides QOM/qdev implementation details (afaerber) > * Thus: Doesn't depend on QOM CPUState refactoring (imammedo) > * Opens a fast route to implementing vCPU unplug in KVM (imammedo) > * Unintrusive to add and easy to obsolete/remove in future (imammedo) > * Existing virt-test cases (afaerber) > * Supported by libvirt (imammedo) > * Prevents confusing guests by hot-plugging random mix of CPUs (agraf) > > Cons: > * Cannot express topologies (ehabkost) Actually, I believe this is not the main problem (we will have exactly the same limitation if using thread-level device_add). To me, the main problem is that we are creating a new QMP command that should be eventually obsoleted by device_add. > > === device_add === > > `device_add driver=Haswell-x86_64-cpu id=qdevid` > [You can try this today and see it failing / not working.] > > Pros: > * QMP/HMP command available today and known to users (afaerber) > * Unified command for device and CPU hot-plug (imammedo) > * Would allow first doing thread-level vCPU hotplug (imammedo) > * Could be extended to support socket-level hot-plug (aliguori/imammedo) > > Cons: > * Operates on raw QOM type name unlike -cpu (afaerber) > * Needs support in libvirt for device_add driver= (imammedo) > * libvirt needs means to enumerate CPU types (imammedo) => QMP? (AF) > > Challenges: > * No CPU qbus (afaerber) > => "should work" without (aliguori) > * CPU subclasses needed for identifying type name (afaerber/imammedo) > => "Haswell-x86_64-cpu" does not exist yet, just "x86_64-cpu" > * CPU class_init for -cpu host requires KVM init (imammedo) > [suggestion by ehabkost to use kvm_arch_vcpu_init, WIP by afaerber] I don't know what you mean by "use kvm_arch_vcpu_init()". I sent a RFC following somebody's suggestion of simply make kvm_arch_init() call a function to finish the -cpu host initialization, as we can't initialize everything inside class_init. See x86_cpu_finish_host_class_init() at: Message-Id: <1357329382-20944-7-git-send-email-ehabk...@redhat.com> http://article.gmane.org/gmane.comp.emulators.qemu/186778 > * Conversion of CPU features to static properties needed (imammedo) > => device_add driver=foo,level=x,xlevel=y,... > * Alternatively conversion to global properties (imammedo) > * Cements type names - rename for 1.4? (afaerber) => permissable (alig.) > [patches for arm, m68k, openrisc, unicore32 on list] > > === qom-set === > > `qom-set` via QMP w/ link property (aliguori) > > Topology represented in QOM: > CPUSocket has-aCPUCore has-aCPUThread a.k.a. CPUState, or > CPUSocket links-to CPUCore links-to CPUThread a.k.a. CPUState > > Challenges (afaerber): > * No CPUSocket/CPUCore objects yet and may take a while to get there... > topology fields being moved to CPUState for 1.4 [done, more WIP] > * No decisions on canonical paths for CPUs: CPU? machine? unassigned? > * Duality of thread-level device types and socket-level? (afaerber) > => fine to have, e.g., quad-core Xeon 500 device (aliguori) > * CPUState is no_user (afaerber) > => need to generally drop no_user for QOM (aliguori) I would like to drop no_user on 1.5 even if we don't manage to finish CPU hotplug, as exposing the CPU objects and classes will be very useful to allow libvirt to probe for the available CPU models and features. > > === libvirt === > > libvirt's XML topology modelling is closer to today's -smp than to the > desired QOM modelling: > http://www.libvirt.org/formatcaps.html > > `virsh setvcpus ` > http://libvirt.org/sources/virshcmdref/html/sect-setvcpus.html > > == qom-cpu course of action (afaerber) == > > It was requested to have vCPU hot-plug in v1.5. > > For device_add we need to move code from cpu_init() into QOM facilities. > => QOM realize support would help [applied by aliguori] > => cleanups piggy-backed onto CPU realizef
vCPU hotplug roadmap (was: Minutes for KVM call 2013-01-15)
Am 15.01.2013 17:16, schrieb Juan Quintela: > > * cpu hot plug > - use qdev propierties conected to a set of socket objects (anthony) > - cpusets are the wrong interface (anthony) > - make a link between cpu <-> socket instead of a propierty? > - how far are we from being able to describe a cpu with -device? > (didn't heare the answer, andreas?) > - perhaps the best approach? > - After soft-freeze, exceptions depend on the maintainer > - After hard-freeze, no exceptions > -device don't require a bus, just an implementation detail, we can change > that > - use cpuset as an intermediate step until full vision is implemented > - several approaches from where we are now, to have something before > we get a full solution > > > At this point, Andreas agreed to write a better summary of the > discussion and suggestions O:-) Got buried, here we go: == vCPU hot-plug user interfaces == === cpu_set === Previously available in qemu-kvm.git: `cpu_set online` via HMP Pros: * Hides QOM/qdev implementation details (afaerber) * Thus: Doesn't depend on QOM CPUState refactoring (imammedo) * Opens a fast route to implementing vCPU unplug in KVM (imammedo) * Unintrusive to add and easy to obsolete/remove in future (imammedo) * Existing virt-test cases (afaerber) * Supported by libvirt (imammedo) * Prevents confusing guests by hot-plugging random mix of CPUs (agraf) Cons: * Cannot express topologies (ehabkost) === device_add === `device_add driver=Haswell-x86_64-cpu id=qdevid` [You can try this today and see it failing / not working.] Pros: * QMP/HMP command available today and known to users (afaerber) * Unified command for device and CPU hot-plug (imammedo) * Would allow first doing thread-level vCPU hotplug (imammedo) * Could be extended to support socket-level hot-plug (aliguori/imammedo) Cons: * Operates on raw QOM type name unlike -cpu (afaerber) * Needs support in libvirt for device_add driver= (imammedo) * libvirt needs means to enumerate CPU types (imammedo) => QMP? (AF) Challenges: * No CPU qbus (afaerber) => "should work" without (aliguori) * CPU subclasses needed for identifying type name (afaerber/imammedo) => "Haswell-x86_64-cpu" does not exist yet, just "x86_64-cpu" * CPU class_init for -cpu host requires KVM init (imammedo) [suggestion by ehabkost to use kvm_arch_vcpu_init, WIP by afaerber] * Conversion of CPU features to static properties needed (imammedo) => device_add driver=foo,level=x,xlevel=y,... * Alternatively conversion to global properties (imammedo) * Cements type names - rename for 1.4? (afaerber) => permissable (alig.) [patches for arm, m68k, openrisc, unicore32 on list] === qom-set === `qom-set` via QMP w/ link property (aliguori) Topology represented in QOM: CPUSocket has-aCPUCore has-aCPUThread a.k.a. CPUState, or CPUSocket links-to CPUCore links-to CPUThread a.k.a. CPUState Challenges (afaerber): * No CPUSocket/CPUCore objects yet and may take a while to get there... topology fields being moved to CPUState for 1.4 [done, more WIP] * No decisions on canonical paths for CPUs: CPU? machine? unassigned? * Duality of thread-level device types and socket-level? (afaerber) => fine to have, e.g., quad-core Xeon 500 device (aliguori) * CPUState is no_user (afaerber) => need to generally drop no_user for QOM (aliguori) === libvirt === libvirt's XML topology modelling is closer to today's -smp than to the desired QOM modelling: http://www.libvirt.org/formatcaps.html `virsh setvcpus ` http://libvirt.org/sources/virshcmdref/html/sect-setvcpus.html == qom-cpu course of action (afaerber) == It was requested to have vCPU hot-plug in v1.5. For device_add we need to move code from cpu_init() into QOM facilities. => QOM realize support would help [applied by aliguori] => cleanups piggy-backed onto CPU realizefn [applied to qom-cpu-next] Agreement on goal of X86CPU subclasses, but conflicts how to get there: * Refactor x86_def_t to X86CPUInfo for X86CPUClass class_init? (AF 2012) * Refactor x86_def_t to X86CPU instance_init as done for arm? * Refactor x86_def_t to class_inits? (afaerber) -> heavy merge conflicts due to bug fixes / cleanups Pro: We can get things into a consistent QOM'ish state across targets. Con: We will refactor again on top for machine-compat properties. * Keep x86_def_t within X86CPUClass as done for ppc? (WIP: afaerber) => smallest common denominator, separates x86 from cross-target work APIC ID topology fixes are being reviewed for 1.4. [merged] X86CPU wave 4 cleanups by Igor are being reviewed for 1.4. [merged] Rename CPU types according to unified --cpu scheme for 1.4? (aliguori: permissable) [patches on list] VMState series by Juan being rebased - subset for 1.4, rest for 1.5. [1.4 part on list, WIP for 1.5] Remainder is considered 1.5 material, qom-cpu-next avail. during Freeze. == Common issues (imammedo) == - back-port CPU hot-plug ACPI notification - hot-plug is not allowed on SysBus: - APIC
Minutes for KVM call 2013-01-15
* cpu hot plug - use qdev propierties conected to a set of socket objects (anthony) - cpusets are the wrong interface (anthony) - make a link between cpu <-> socket instead of a propierty? - how far are we from being able to describe a cpu with -device? (didn't heare the answer, andreas?) - perhaps the best approach? - After soft-freeze, exceptions depend on the maintainer - After hard-freeze, no exceptions -device don't require a bus, just an implementation detail, we can change that - use cpuset as an intermediate step until full vision is implemented - several approaches from where we are now, to have something before we get a full solution At this point, Andreas agreed to write a better summary of the discussion and suggestions O:-) Later, Juan. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
