[Bug 614322] Re: libvirt not recognizing NUMA architecture
Thanks very much, Ralf - we will sync that package shortly. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in Ubuntu. https://bugs.launchpad.net/bugs/614322 Title: libvirt not recognizing NUMA architecture To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/614322/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 614322] Re: libvirt not recognizing NUMA architecture
I have discussed the matter with Guido Günther from the Debian team. After some testing he reenabled the numa feature in the debian package: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=648323 Would you like to follow him? ** Bug watch added: Debian Bug tracker #648323 http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=648323 -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in Ubuntu. https://bugs.launchpad.net/bugs/614322 Title: libvirt not recognizing NUMA architecture To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/614322/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 614322] Re: libvirt not recognizing NUMA architecture
Regarding the change from Opinion to Confirmed, ordinarily I'd object, but in this case I agree - this *is* a bug, whose resolution potentially depends on some other bugs. I don't object to numa being enabled if the underlying bugs get fixed, so if you have time, by all means please do work on this bug, preferably through upstream and debian. It's not an opinion. Thanks, Ralf. Note that in the last year and a half, the underlying bugs may have even been fixed upstream, so whoever looks into this, I suggest finding a test case (it's not in the changelog) and re-testing. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in Ubuntu. https://bugs.launchpad.net/bugs/614322 Title: libvirt not recognizing NUMA architecture To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/614322/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 614322] Re: libvirt not recognizing NUMA architecture
Hi, noone seems to have commented on the reason why numa is disabled in libvirt. In Ubuntu, both libnuma1 and numactl are in universe. So in order toenable those in libvirt (which is in main), we would have to do a main inclusion request (MIR) for those. But, they are also disabled in debian. It seems worth opening a bug against the debian package to inquire as to why it is disabled there. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in Ubuntu. https://bugs.launchpad.net/bugs/614322 Title: libvirt not recognizing NUMA architecture To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/614322/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 614322] Re: libvirt not recognizing NUMA architecture
Actually, the debian package's 0.7.7-3 changelog entry from Mar 2010 shows: * [b69d3cc] Revert "Enable NUMA support" since it breaks the python bindings. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in Ubuntu. https://bugs.launchpad.net/bugs/614322 Title: libvirt not recognizing NUMA architecture To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/614322/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 614322] Re: libvirt not recognizing NUMA architecture
** Changed in: libvirt (Ubuntu) Status: Opinion => Confirmed -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in Ubuntu. https://bugs.launchpad.net/bugs/614322 Title: libvirt not recognizing NUMA architecture To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/614322/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 614322] Re: libvirt not recognizing NUMA architecture
I can confirm this and I am very dissapointed that still after more than a year this issue has not been fixed. I am currently running Ubuntu precise (development, the next lts) on HP DL 585 G2 with 4 Dual Cores and thus 4 Numa cells. Numactl shows the hardware: # numactl --show policy: default preferred node: current physcpubind: 0 1 2 3 4 5 6 7 cpubind: 0 1 2 3 nodebind: 0 1 2 3 membind: 0 1 2 3 virsh does not display the numa information: # virsh nodeinfo CPU model: x86_64 CPU(s): 8 CPU frequency: 1000 MHz CPU socket(s): 4 Core(s) per socket: 2 Thread(s) per core: 1 NUMA cell(s):1 Memory size: 32948228 kB # virsh freecell 0 error: this function is not supported by the connection driver: NUMA memory information not available on this platform Just using numactl is no option if the guests are to be migrated. The numa-placement needs to be handled by libvirt! -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in Ubuntu. https://bugs.launchpad.net/bugs/614322 Title: libvirt not recognizing NUMA architecture To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/614322/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 614322] Re: libvirt not recognizing NUMA architecture
I can confirm this. Most of our hosts have 64GB RAM and 2 Intel Nehalem hexa-cores. With 3 VM's using 16GB RAM each. I noticed performance degredation and the host also swapped out a lot of memory. The swapping degraded performance dramatically. vm.swappiness=0 in /etc/sysctl.conf did not help. It seems that NUMA on Intel CPU's can be expensive because RAM needs to be transfered from other nodes. With only 1 node (socket) there is no problem. With 2 or more nodes you see slower. Without the capabilities you can prevent this behavior by pinning the vcpu's. You should spread you VM's over the available nodes: numactl --hardware | grep "node 0 cpus" node 0 cpus: 0 2 4 6 8 10 12 14 16 18 20 22 Your XML should contain something like this: 1 2 3 4 The next VM should use they other node. numactl --hardware | grep "node 1 cpus" node 0 cpus: 1 3 5 7 9 11 13 15 17 19 21 23 1 2 3 4 So you don't need the NUMA info from "virsh capabilities". We now split up our hosts by the amount of NUMA-nodes to prevent performance-degradation and swapping. -- libvirt not recognizing NUMA architecture https://bugs.launchpad.net/bugs/614322 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 614322] Re: libvirt not recognizing NUMA architecture
You can easily compile the latest libvirt version yourself: aptitude install build-essential, libxml2-dev, libgnutls-dev, libdevmapper-dev, libparted0-dev, libvirt-dev ./configure –prefix=/usr –exec-prefix=/usr –libdir=/usr/lib –includedir=/usr/include && make && make install There is a configure option "--with-numa", but I didn't use ist. libvirt recognizes the numa cells without this option. This improves the network performance, but it's still slower than KVM on CentOS and still much slower than Xen. -- libvirt not recognizing NUMA architecture https://bugs.launchpad.net/bugs/614322 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 614322] Re: libvirt not recognizing NUMA architecture
yes I see this with 10.04 and 10.10 it appears that libvirt was not built with libnuma. I also agree and would like to see this corrected. 10.04 libvirt version 0.7.5 10.10 libvirt version 0.8.3 if you need the capabilities I will be more than happy to upload -- libvirt not recognizing NUMA architecture https://bugs.launchpad.net/bugs/614322 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 614322] Re: libvirt not recognizing NUMA architecture
So, any news? Is anybody able to confirm this? Frank -- libvirt not recognizing NUMA architecture https://bugs.launchpad.net/bugs/614322 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 614322] Re: libvirt not recognizing NUMA architecture
Hi there, I used the latest RHEL6 Beta with libvirt-0.8.1-13.el6.x86_64 and the latest CentOS 5.5 with libvirt-0.6.3-33.el5_5.1. I tried Ubuntu 8.04 and 10.04 and both show about 50% of the performance that RHEL and CentOS do. This is a serious problem. No company, that wants to virtualize applications that need the performance, would choose kvm on Ubuntu as platform this way. If Ubuntu wants to be an enterprise ready distribution, they have to compete with RHEL, Xen, ESXi and things like that. Cheers, Frank -- libvirt not recognizing NUMA architecture https://bugs.launchpad.net/bugs/614322 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 614322] Re: libvirt not recognizing NUMA architecture
Thanks for the bug report. Which version are you running on redhat and centos? ** Changed in: libvirt (Ubuntu) Importance: Undecided => Wishlist ** Changed in: libvirt (Ubuntu) Status: New => Opinion -- libvirt not recognizing NUMA architecture https://bugs.launchpad.net/bugs/614322 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 614322] Re: libvirt not recognizing NUMA architecture
** Package changed: linux (Ubuntu) => libvirt (Ubuntu) -- libvirt not recognizing NUMA architecture https://bugs.launchpad.net/bugs/614322 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
