[Openstack] What is the most commonly used Hypervisor and toolset combination?
hi, all My team is trying to deploy openstack in production environment. We tried to get "libvirt + xen 3.4.3 + CenOS 5.4 + Openstack 2012.2" working, but encountered lots of issues. We already have thousands of virtual machines running in production, and that's why we are trying Xen 3.4.3 and CentOS 5.4. After we solved one problem, there comes more, which is very annoying So, my question is: In real production environment using Openstack, what's the most commonly used Hypervisor and toolset? We hope to deploy Openstack quickly, and stay in main stream. Thank u. Regards Wang Li ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] What is the most commonly used Hypervisor and toolset combination?
I'd suggest: CentOS 6.3/RHEL 6.3 + KVM + OpenStack Essex, you may replace CentOS with Ubuntu/Fedora if you want. If you are big fan of Xen or having huge legacy PV VMs, you might want to try XenServer + XenAPI + OpenStack Essex. On Wed, Jul 18, 2012 at 12:20 PM, Wang Li wrote: > hi, all > > My team is trying to deploy openstack in production environment. > > We tried to get "libvirt + xen 3.4.3 + CenOS 5.4 + Openstack 2012.2" > working, but encountered lots of issues. > > We already have thousands of virtual machines running in production, > and that's why we are trying Xen 3.4.3 and CentOS 5.4. > > After we solved one problem, there comes more, which is very > annoying > > So, my question is: > > In real production environment using Openstack, what's the most > commonly used Hypervisor and toolset? > > We hope to deploy Openstack quickly, and stay in main stream. > > > Thank u. > > Regards > > Wang Li > > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp -- Regards Huang Zhiteng ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] What is the most commonly used Hypervisor and toolset combination?
To my knowledge, if you want to use Xen, using XCP or XenServer (i.e. using XenAPI driver) is the way to go. If you look at the contributions to the drivers, you can have a good guess at who is using them. I know people are going into production on XenAPI, not heard about Xen+libvirt in production. Having said this, I have seen some fixes to Folsom around Xen + libvirt, I think from SUSE? I work for Citrix on the XenAPI driver, so I am not exactly "impartial". My testing is certainly running XenServer 6.0.2 (+ development builds of the next release) with the XenAPI driver. If you have any problems getting going with XenAPI, I am happy to help. Preferably through the ML to share the knowledge. I am currently doing some work improving the documentation. I have got a few patches in already, hopefully I get around to adding some more very soon. I hope that helps, John > -Original Message- > From: openstack-bounces+john.garbutt=eu.citrix@lists.launchpad.net > [mailto:openstack-bounces+john.garbutt=eu.citrix@lists.launchpad.net] > On Behalf Of Huang Zhiteng > Sent: Wednesday, July 18, 2012 5:47 AM > To: Wang Li > Cc: openstack@lists.launchpad.net > Subject: Re: [Openstack] What is the most commonly used Hypervisor and > toolset combination? > > I'd suggest: CentOS 6.3/RHEL 6.3 + KVM + OpenStack Essex, you may replace > CentOS with Ubuntu/Fedora if you want. If you are big fan of Xen or having > huge legacy PV VMs, you might want to try XenServer + XenAPI + OpenStack > Essex. > > On Wed, Jul 18, 2012 at 12:20 PM, Wang Li wrote: > > hi, all > > > > My team is trying to deploy openstack in production environment. > > > > We tried to get "libvirt + xen 3.4.3 + CenOS 5.4 + Openstack 2012.2" > working, but encountered lots of issues. > > > > We already have thousands of virtual machines running in > > production, and > that's why we are trying Xen 3.4.3 and CentOS 5.4. > > > > After we solved one problem, there comes more, which is very > annoying > > > > So, my question is: > > > > In real production environment using Openstack, what's the most > commonly used Hypervisor and toolset? > > > > We hope to deploy Openstack quickly, and stay in main stream. > > > > > > Thank u. > > > > Regards > > > > Wang Li > > > > > > ___ > > Mailing list: https://launchpad.net/~openstack > > Post to : openstack@lists.launchpad.net > > Unsubscribe : https://launchpad.net/~openstack > > More help : https://help.launchpad.net/ListHelp > > > > -- > Regards > Huang Zhiteng > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] What is the most commonly used Hypervisor and toolset combination?
John Garbutt wrote: > To my knowledge, if you want to use Xen, using XCP or XenServer (i.e. using > XenAPI driver) is the way to go. If you look at the contributions to the > drivers, you can have a good guess at who is using them. > > I know people are going into production on XenAPI, not heard about > Xen+libvirt in production. Having said this, I have seen some fixes to Folsom > around Xen + libvirt, I think from SUSE? > Yes, I'm slowly working on improving support for xen.org Xen via the libvirt driver and hope to have these improvements in for the Folsom release. Regards, Jim ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] What is the most commonly used Hypervisor and toolset combination?
Until recently, stating that "Ubuntu is the official distro for OpenStack" wouldn't have hurt anybody's feelings.. That's changing now, with the Fedora+RedHat/Debian guys getting everything solid on their respective distros.. Anyway! DevStack is Ubuntu+KVM (by default), All the per commit testing is Ubuntu+QEMU/KVM.. The docs are best for Ubuntu+KVM. If you're doing your first install - I would suggest sticking to that. Once you have that all figured out, moving to something less documented like XenAPI should be much much easier. Good luck! Thanks, Kiall On Wed, Jul 18, 2012 at 5:20 AM, Wang Li wrote: > hi, all > > My team is trying to deploy openstack in production environment. > > We tried to get "libvirt + xen 3.4.3 + CenOS 5.4 + Openstack > 2012.2" working, but encountered lots of issues. > > We already have thousands of virtual machines running in > production, and that's why we are trying Xen 3.4.3 and CentOS 5.4. > > After we solved one problem, there comes more, which is very > annoying > > So, my question is: > > In real production environment using Openstack, what's the most > commonly used Hypervisor and toolset? > > We hope to deploy Openstack quickly, and stay in main stream. > > > Thank u. > > Regards > > Wang Li > > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] What is the most commonly used Hypervisor and toolset combination?
+1 to Kiall's response. On Thu, Jul 19, 2012 at 2:50 AM, Kiall Mac Innes wrote: > Until recently, stating that "Ubuntu is the official distro for OpenStack" > wouldn't have hurt anybody's feelings.. That's changing now, with the > Fedora+RedHat/Debian guys getting everything solid on their respective > distros.. > > Anyway! DevStack is Ubuntu+KVM (by default), All the per commit testing is > Ubuntu+QEMU/KVM.. The docs are best for Ubuntu+KVM. If you're doing your > first install - I would suggest sticking to that. > > Once you have that all figured out, moving to something less documented > like XenAPI should be much much easier. > > Good luck! > > Thanks, > Kiall > > > On Wed, Jul 18, 2012 at 5:20 AM, Wang Li wrote: > >> hi, all >> >> My team is trying to deploy openstack in production environment. >> >> We tried to get "libvirt + xen 3.4.3 + CenOS 5.4 + Openstack >> 2012.2" working, but encountered lots of issues. >> >> We already have thousands of virtual machines running in >> production, and that's why we are trying Xen 3.4.3 and CentOS 5.4. >> >> After we solved one problem, there comes more, which is very >> annoying >> >> So, my question is: >> >> In real production environment using Openstack, what's the most >> commonly used Hypervisor and toolset? >> >> We hope to deploy Openstack quickly, and stay in main stream. >> >> >> Thank u. >> >> Regards >> >> Wang Li >> >> >> ___ >> Mailing list: https://launchpad.net/~openstack >> Post to : openstack@lists.launchpad.net >> Unsubscribe : https://launchpad.net/~openstack >> More help : https://help.launchpad.net/ListHelp >> > > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp > > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] What is the most commonly used Hypervisor and toolset combination?
That would be great Jim, I've built a cloud that uses CentOS+libvirt+Xen 4.1.3 to do GPU passthrough and I just love to be able to use libvirt with Xen, this setup makes a lot of sense to me since our main, bigger cloud is the standard libvirt+KVM, using libvirt across the board is great for us. I'm following your work closely, the GPU cloud is still using libvirt+xend but when I move to Xen 4.2 my understanding is that I will need libvirt+xl (xenlight) so I guess there's still some work to be done in libvirt there... The reason I want to move to Xen 4.2 is the GPU passthrough of NVIDIA GPUs... currently, with Xen 4.1.3, I successfully passthrough ATI GPUs only. Boris -Message d'origine- De : openstack-bounces+boris-michel.deschenes=ubisoft@lists.launchpad.net [mailto:openstack-bounces+boris-michel.deschenes=ubisoft@lists.launchpad.net] De la part de Jim Fehlig Envoyé : 18 juillet 2012 17:56 À : John Garbutt Cc : openstack@lists.launchpad.net Objet : Re: [Openstack] What is the most commonly used Hypervisor and toolset combination? John Garbutt wrote: > To my knowledge, if you want to use Xen, using XCP or XenServer (i.e. using > XenAPI driver) is the way to go. If you look at the contributions to the > drivers, you can have a good guess at who is using them. > > I know people are going into production on XenAPI, not heard about > Xen+libvirt in production. Having said this, I have seen some fixes to Folsom > around Xen + libvirt, I think from SUSE? > Yes, I'm slowly working on improving support for xen.org Xen via the libvirt driver and hope to have these improvements in for the Folsom release. Regards, Jim ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] What is the most commonly used Hypervisor and toolset combination?
Hi Boris, We have GPU passthrough working with NVIDIA GPUs in Xen 4.1.2, if I recall correctly. We don't yet have a stable Xen + Libvirt installation working, but we're looking at it. Perhaps it would be worth collaborating since it sounds like this could be a win for both of us. best, JP On Aug 16, 2012, at 9:31 AM, Boris-Michel Deschenes wrote: > That would be great Jim, > > I've built a cloud that uses CentOS+libvirt+Xen 4.1.3 to do GPU passthrough > and I just love to be able to use libvirt with Xen, this setup makes a lot of > sense to me since our main, bigger cloud is the standard libvirt+KVM, using > libvirt across the board is great for us. > > I'm following your work closely, the GPU cloud is still using libvirt+xend > but when I move to Xen 4.2 my understanding is that I will need libvirt+xl > (xenlight) so I guess there's still some work to be done in libvirt there... > > The reason I want to move to Xen 4.2 is the GPU passthrough of NVIDIA GPUs... > currently, with Xen 4.1.3, I successfully passthrough ATI GPUs only. > > Boris > > -Message d'origine- > De : openstack-bounces+boris-michel.deschenes=ubisoft@lists.launchpad.net > [mailto:openstack-bounces+boris-michel.deschenes=ubisoft@lists.launchpad.net] > De la part de Jim Fehlig > Envoyé : 18 juillet 2012 17:56 > À : John Garbutt > Cc : openstack@lists.launchpad.net > Objet : Re: [Openstack] What is the most commonly used Hypervisor and toolset > combination? > > John Garbutt wrote: >> To my knowledge, if you want to use Xen, using XCP or XenServer (i.e. using >> XenAPI driver) is the way to go. If you look at the contributions to the >> drivers, you can have a good guess at who is using them. >> >> I know people are going into production on XenAPI, not heard about >> Xen+libvirt in production. Having said this, I have seen some fixes to >> Folsom around Xen + libvirt, I think from SUSE? >> > > Yes, I'm slowly working on improving support for xen.org Xen via the libvirt > driver and hope to have these improvements in for the Folsom release. > > Regards, > Jim > > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] What is the most commonly used Hypervisor and toolset combination?
For the record, we (i.e. Citrix) are certainly looking at exposing the GPU pass-through support in XCP (a Xen.org project that includes XenAPI) and XenServer in the Grizzly timeframe. I now sit next to a lot of the Citrix Xen.org and XenServer people. We are all based in Cambridge, UK. However, you will probably get answers just as quickly by asking on the Xen mailing lists. Thanks, John > -Original Message- > From: John Paul Walters [mailto:jwalt...@isi.edu] > Sent: Thursday, August 16, 2012 4:11 PM > To: Boris-Michel Deschenes > Cc: Jim Fehlig; John Garbutt; openstack@lists.launchpad.net > Subject: Re: [Openstack] What is the most commonly used Hypervisor and > toolset combination? > > Hi Boris, > > We have GPU passthrough working with NVIDIA GPUs in Xen 4.1.2, if I recall > correctly. We don't yet have a stable Xen + Libvirt installation working, but > we're looking at it. Perhaps it would be worth collaborating since it sounds > like > this could be a win for both of us. > > best, > JP > > > On Aug 16, 2012, at 9:31 AM, Boris-Michel Deschenes wrote: > > > That would be great Jim, > > > > I've built a cloud that uses CentOS+libvirt+Xen 4.1.3 to do GPU passthrough > and I just love to be able to use libvirt with Xen, this setup makes a lot of > sense > to me since our main, bigger cloud is the standard libvirt+KVM, using libvirt > across the board is great for us. > > > > I'm following your work closely, the GPU cloud is still using libvirt+xend > > but > when I move to Xen 4.2 my understanding is that I will need libvirt+xl > (xenlight) > so I guess there's still some work to be done in libvirt there... > > > > The reason I want to move to Xen 4.2 is the GPU passthrough of NVIDIA > GPUs... currently, with Xen 4.1.3, I successfully passthrough ATI GPUs only. > > > > Boris > > > > -Message d'origine- > > De : > > openstack-bounces+boris-michel.deschenes=ubisoft.com@lists.launchpad.n > > et > > [mailto:openstack-bounces+boris-michel.deschenes=ubisoft@lists.lau > > nchpad.net] De la part de Jim Fehlig Envoyé : 18 juillet 2012 17:56 À : John > Garbutt Cc : openstack@lists.launchpad.net Objet : Re: [Openstack] What is the > most commonly used Hypervisor and toolset combination? > > > > John Garbutt wrote: > >> To my knowledge, if you want to use Xen, using XCP or XenServer (i.e. using > XenAPI driver) is the way to go. If you look at the contributions to the > drivers, > you can have a good guess at who is using them. > >> > >> I know people are going into production on XenAPI, not heard about > Xen+libvirt in production. Having said this, I have seen some fixes to Folsom > around Xen + libvirt, I think from SUSE? > >> > > > > Yes, I'm slowly working on improving support for xen.org Xen via the libvirt > driver and hope to have these improvements in for the Folsom release. > > > > Regards, > > Jim > > > > > > ___ > > Mailing list: https://launchpad.net/~openstack > > Post to : openstack@lists.launchpad.net > > Unsubscribe : https://launchpad.net/~openstack > > More help : https://help.launchpad.net/ListHelp > > > > ___ > > Mailing list: https://launchpad.net/~openstack > > Post to : openstack@lists.launchpad.net > > Unsubscribe : https://launchpad.net/~openstack > > More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] What is the most commonly used Hypervisor and toolset combination?
Sorry for the delayed response. Boris-Michel Deschenes wrote: > That would be great Jim, > > I've built a cloud that uses CentOS+libvirt+Xen 4.1.3 to do GPU passthrough > and I just love to be able to use libvirt with Xen, this setup makes a lot of > sense to me since our main, bigger cloud is the standard libvirt+KVM, using > libvirt across the board is great for us. > > I'm following your work closely, the GPU cloud is still using libvirt+xend > but when I move to Xen 4.2 my understanding is that I will need libvirt+xl > (xenlight) so I guess there's still some work to be done in libvirt there... > Yes, there is. libxl changed significantly between Xen 4.1 and soon to be released Xen 4.2, so much so that the current libvirt libxl driver won't even build against Xen 4.2. In addition, the libxl driver does not have feature parity with the legacy xen driver. So lots of work to be done, but I have limited free cycles. I'm hoping to get another body or two at SUSE to help with this work. That said, xm/xend will still be included in Xen 4.2 and can be configured as the primary tool stack, allowing you to continue using your existing setup with Xen 4.2. Regards, Jim > The reason I want to move to Xen 4.2 is the GPU passthrough of NVIDIA GPUs... > currently, with Xen 4.1.3, I successfully passthrough ATI GPUs only. > > Boris > > -Message d'origine- > De : openstack-bounces+boris-michel.deschenes=ubisoft@lists.launchpad.net > [mailto:openstack-bounces+boris-michel.deschenes=ubisoft@lists.launchpad.net] > De la part de Jim Fehlig > Envoyé : 18 juillet 2012 17:56 > À : John Garbutt > Cc : openstack@lists.launchpad.net > Objet : Re: [Openstack] What is the most commonly used Hypervisor and toolset > combination? > > John Garbutt wrote: > >> To my knowledge, if you want to use Xen, using XCP or XenServer (i.e. using >> XenAPI driver) is the way to go. If you look at the contributions to the >> drivers, you can have a good guess at who is using them. >> >> I know people are going into production on XenAPI, not heard about >> Xen+libvirt in production. Having said this, I have seen some fixes to >> Folsom around Xen + libvirt, I think from SUSE? >> >> > > Yes, I'm slowly working on improving support for xen.org Xen via the libvirt > driver and hope to have these improvements in for the Folsom release. > > Regards, > Jim > > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp > > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] What is the most commonly used Hypervisor and toolset combination?
Boris-Michel Deschenes wrote:
> John,
>
> Sorry for my late response..
>
> It would be great to collaborate, like I said, I prefer to keep the libvirt
> layer as it works great with openstack and many other techs (collectd,
> virt-manager, etc.), the virsh tool is also very useful for us.
>
> You say:
> ---
> We have GPU passthrough working with NVIDIA GPUs in Xen 4.1.2, if I recall
> correctly. We don't yet have a stable Xen + Libvirt installation working,
> but we're looking at it. Perhaps it would be worth collaborating since it
> sounds like this could be a win for both of us.
> ---
> I have Jim Fehlig in CC since this could be of interest to him.
>
> We managed to have the GPU passthrough of NVIDIA cards using Xen 4.1.2 but
> ONLY with the xenapi (actually the whole XCP toolstack), with libvirt/Xen
> 4.1.2 and even libvirt/Xen 4.1.3, I only manage to apss through radeon GPUs,
> the reason could be:
>
> 1. The inability to pass the gfx_passthru parameter through libvirt (IIRC
> this parameter passes the PCI device as the main VGA card and not a second
> one).
> 2. Bad FLR reset support (or other PCI low-level function) from the NVIDIA
> boards
>
I've noticed this issue with some Broadcom multifunction nics. No FLR,
so fallback to secondary bus reset, which is problematic if another
function is being used by a different vm.
> 3. something else entirely.
>
> Anyway, like I said, this GPU passthrough of nvidia worked well with XCP
> using xenapi but not with libvirt/Xen
>
Hmm, would be nice to get that fixed. To date, I haven't tried GPU
passthrough with Xen so I'm not familiar with the issues.
> Now, as for the libvirt/Xen setup we have, I don't know if I would call it
> stable but it does the job as a POC cloud and is actually used by real people
> with real GPU needs (for example developing on OpenCL 1.2), the main thing is
> that it seamlessly integrates with openstack (because of libvirt) and with
> the instance_type_extra_specs, you can actually add a couple of these
> "special" nodes to an existing plain KVM cloud and they will receive the
> instances requesting GPUs without any problem.
>
> the setup:
> (this only refers to compute nodes as controller nodes are un-modified)
>
> 1. Install Centos 6.2 and make your own project Zeus (transforming a centos
> in Xen)
> http://www.howtoforge.com/virtualization-with-xen-on-centos-6.2-x86_64-paravirtualization-and-hardware-virtualization
> (first page only and skip the bridge setup as openstack-nova-compute does
> this at startup). You end up with a Xen hypervisor with libvirt, the libvirt
> patch is actually a single-line config change IIRC. Pretty straight-forward.
>
> 2. Install openstack-nova from EPEL (so all this refers only to ESSEX,
> openstack 2012.1)
>
> 3. configure the compute node accordingly (libvirt_type=xen)
>
> That's the first part, at this point, you can spawn a VM, and attach a GPU
> manually with:
>
> virsh nodedev-dettach pci__02_00_01
> (edit the VM's nova libvirt.xml to add a pci node dev definition like this:
> http://docs.fedoraproject.org/en-US/Fedora/13/html/Virtualization_Guide/chap-Virtualization-PCI_passthrough.html
> )
> virsh define libvirt.xml
> virsh start instance-000x
>
> Now, this is all manual and we wish to automate this in openstack, so this is
> what I've done, I currently can launch VMs in my cloud and the passthrough
> occurs without any intervention.
>
> These files were modified from an original essex installation to make this
> possible:
>
> (on the controller)
> create a g1.small instance_type with {'free_gpus': '1'} as
> instance_type_extra_specs
> select the compute_filter filter to enforce extra_specs in scheduling (also
> the function host_passes of the filter is slightly modified so that it read
> key>=value instead of key=value... (free_gpus>=1 is good, does not need to be
> strictly equals to 1)
>
I think this has already been done for you in Folsom via the
ComputeCapabilitiesFilter and Jinwoo Suh's addition of
instance_type_extra_specs operators. See commit 90f77d71.
> (on the compute node)
> nova/virt/libvirt/gpu.py
> a new file that contains functions like detach_all_gpus, get_free_gpus,
> simple stuff
Have you considered pushing this upstream?
> using virsh and lspci
> nova/virt/libvirt/connection.py
> calls gpu.detach_all_gpus on startup (virsh nodedev-dettach)
> builds the VM libvirt.xml as normal but also adds the pci nodedev
> definition
> advertises free_gpus capabilities so that the scheduler gets it through
> host_state calls
>
> that's about it, with that we get:
>
> 1. compute nodes that detach all GPUS on startup
> 2. compute nodes that advertise the nb of free gpus to the scheduler
> 3. compute nodes that are able to build the VMs libvirt.xml with a valid,
> free GPU definition when a VM is launched
> 4. controller that runs a scheduler that knows where to send VMs (free_gpus
