See PCM inline... On Fri, Jun 3, 2016 at 8:44 AM Daniel P. Berrange <berra...@redhat.com> wrote:
> On Fri, Jun 03, 2016 at 12:32:17PM +0000, Paul Michali wrote: > > Hi! > > > > I've been playing with Liberty code a bit and had some questions that I'm > > hoping Nova folks may be able to provide guidance on... > > > > If I set up a flavor with hw:mem_page_size=2048, and I'm creating > (Cirros) > > VMs with size 1024, will the scheduling use the minimum of the number of > > 1024 what units ? 1024 MB, or 1024 huge pages aka 2048 MB ? > PCM: I was using small flavor, which is 2 GB. So that's 2048 MB and the page size is 2048K, so 1024 pages? Hope I have the units right. > > huge pages available and the size requested for the VM, or will it base > > scheduling only on the number of huge pages? > > > > It seems to be doing the latter, where I had 1945 huge pages free, and > > tried to create another VM (1024) and Nova rejected the request with "no > > hosts available". > > From this I'm guessing you're meaning 1024 huge pages aka 2 GB earlier. > > Anyway, when you request huge pages to be used for a flavour, the > entire guest RAM must be able to be allocated from huge pages. > ie if you have a guest with 2 GB of RAM, you must have 2 GB worth > of huge pages available. It is not possible for a VM to use > 1.5 GB of huge pages and 500 MB of normal sized pages. > PCM: Right, so, with 2GB of RAM, I need 1024 huge pages of size 2048K. In this case, there are 1945 huge pages available, so I was wondering why it failed. Maybe I'm confusing sizes/pages? > > > Is this still the same for Mitaka? > > Yep, this use of huge pages has not changed. > > > Where could I look in the code to see how the scheduling is determined? > > Most logic related to huge pages is in nova/virt/hardware.py > > > If I use mem_page_size=large (what I originally had), should it evenly > > assign huge pages from the available NUMA nodes (there are two in my > case)? > > > > It looks like it was assigning all VMs to the same NUMA node (0) in this > > case. Is the right way to change to 2048, like I did above? > > Nova will always avoid spreading your VM across 2 host NUMA nodes, > since that gives bad performance characteristics. IOW, it will always > allocate huge pages from the NUMA node that the guest will run on. If > you explicitly want your VM to spread across 2 host NUMA nodes, then > you must tell nova to create 2 *guest* NUMA nodes for the VM. Nova > will then place each guest NUMA node, on a separate host NUMA node > and allocate huge pages from node to match. This is done using > the hw:numa_nodes=2 parameter on the flavour > PCM: Gotcha, but that was not the issue I'm seeing. With this small flavor (2GB = 1024 pages), I had 13107 huge pages initially. As I created VMs, they were *all* placed on the same NUMA node (0). As a result, when I got to more than have the available pages, Nova failed to allow further VMs, even though I had 6963 available on one compute node, and 5939 on another. It seems that all the assignments were to node zero. Someone suggested to me to set mem_page_size to 2048, and at that point it started assigning to both NUMA nodes evenly. Thanks for the help!!! Regards, PCM > > > Again, has this changed at all in Mitaka? > > Nope. Well aside from random bug fixes. > > Regards, > Daniel > -- > |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ > :| > |: http://libvirt.org -o- http://virt-manager.org > :| > |: http://autobuild.org -o- http://search.cpan.org/~danberr/ > :| > |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc > :| > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
