also see this redhat bugzilla, about kernel bug (even with online_movable) https://bugzilla.redhat.com/show_bug.cgi?id=1320534
" Description Milan Zamazal 2016-03-23 08:38:07 EDT Description of problem: File /usr/lib/udev/rules.d/40-redhat.rules contains the following line: SUBSYSTEM=="memory", ACTION=="add", ATTR{state}=="offline", ATTR{state}="online" This ensures that hotplugged memory gets available to the system. In order to be able to hotunplug the hotplugged memory later, its state should be set to `online_movable' instead of `online'. Otherwise the system may allocate kernel pages in the hotplugged memory, preventing its removal. To avoid this problem the rule should look like SUBSYSTEM=="memory", ACTION=="add", ATTR{state}=="offline", ATTR{state}="online_movable" However, there is a kernel bug that prevents this simple change from working, see https://bugzilla.redhat.com/1314306. It's not possible to set memory blocks as online_movable in random order (it results in "Permission denied" failures), the state must be changed from the highest numbered plugged memory block to the lowest one. So either the kernel bug should be fixed before the change, or the udev rules should ensure changing the state of the memory blocks to `online_movable' in the proper order (from the highest to the lowest memory block). Version-Release number of selected component (if applicable): 219-19.el7_2.4.x86_64 " ----- Mail original ----- De: "Alexandre Derumier" <aderum...@odiso.com> À: "Wolfgang Bumiller" <w.bumil...@proxmox.com> Cc: "pve-devel" <pve-devel@pve.proxmox.com> Envoyé: Mercredi 14 Décembre 2016 17:05:20 Objet: Re: [pve-devel] qemu memory hotplug limited to 64 dimm >>Apparently I do have to explicitly mention that I'm talking about *unused* >>memory, iow. memory where the /sys/../enabled file is 0. I think this was >>working at some point on modern guest >>kernels? do you mean "online" instead "enabled" ? also here some notes I have set in the wiki online should be set by an udev rules, to auto online dimm on hotplug /lib/udev/rules.d/80-hotplug-cpu-mem.rules SUBSYSTEM=="cpu", ACTION=="add", TEST=="online", ATTR{online}=="0", ATTR{online}="1" SUBSYSTEM=="memory", ACTION=="add", TEST=="state", ATTR{state}=="offline", ATTR{state}="online" for linux kernel >= 4.7, you don't need the udev rule for memory hotplug, you only need to add this kernel parameter at boot: memhp_default_state=online Memory hot-unplug - Memory unplug don't work windows currently (<=win10 currently) - Memory unplug can't be unstable on linux (<= kernel 4.8 currently) for Linux memory unplug, you need have movable zone enabled in kernel config (not enabled by default on debian/ubuntu) CONFIG_MOVABLE_NODE=YES and "movable_node" boot kernel parameter enabled ----- Mail original ----- De: "Wolfgang Bumiller" <w.bumil...@proxmox.com> À: "Alexandre Derumier" <aderum...@odiso.com> Cc: "pve-devel" <pve-devel@pve.proxmox.com>, "dietmar" <diet...@proxmox.com> Envoyé: Mercredi 14 Décembre 2016 16:44:52 Objet: Re: [pve-devel] qemu memory hotplug limited to 64 dimm > On December 14, 2016 at 3:45 PM Alexandre DERUMIER <aderum...@odiso.com> > wrote: > > > >>3) Drop support for hot*UN*plugging altogether and simply always hot-add > >>one big dimm. ;-) > >>On that note, my guests currently hang when I unplug memory and I bet > >>that happens to all of you and nobody ever noticed because nobody needs > >>that feature anyway ;-) > > yes, it's a linux guest limitation. (qemu implementation is ok), but linux > kernel memory is not movable currently, and if it's located on a dimm, you > can't unplug it, as the kernel memory can't be move at another location. Apparently I do have to explicitly mention that I'm talking about *unused* memory, iow. memory where the /sys/../enabled file is 0. I think this was working at some point on modern guest kernels? _______________________________________________ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel _______________________________________________ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel