Dan Yasny has uploaded a new change for review. Change subject: Signed-off-by: Dan Yasny <[email protected]> ......................................................................
Signed-off-by: Dan Yasny <[email protected]> Cisco VM-FEX support vdsm hooks Change-Id: I45a7fa46919bb39a648dff190c40618395990e91 Signed-off-by: Dan Yasny <[email protected]> --- A vdsm_hooks/vmfex/Makefile.am A vdsm_hooks/vmfex/README A vdsm_hooks/vmfex/before_vm_migrate_destination.py A vdsm_hooks/vmfex/before_vm_start.py 4 files changed, 394 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/47/7547/1 diff --git a/vdsm_hooks/vmfex/Makefile.am b/vdsm_hooks/vmfex/Makefile.am new file mode 100644 index 0000000..b921275 --- /dev/null +++ b/vdsm_hooks/vmfex/Makefile.am @@ -0,0 +1,36 @@ +# +# Copyright 2011 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# + +EXTRA_DIST = \ + before_vm_migrate_destination.py \ + before_vm_start.py \ + +install-data-local: + $(MKDIR_P) $(DESTDIR)$(vdsmhooksdir)/before_vm_start + $(INSTALL_SCRIPT) $(srcdir)/before_vm_start.py \ + $(DESTDIR)$(vdsmhooksdir)/before_vm_start/50_vmfex + $(MKDIR_P) $(DESTDIR)$(vdsmhooksdir)/before_vm_migrate_destination + $(INSTALL_SCRIPT) $(srcdir)/before_vm_migrate_destination.py \ + $(DESTDIR)$(vdsmhooksdir)/before_vm_migrate_destination/50_vmfex + +uninstall-local: uninstall-data-sudoers + $(RM) $(DESTDIR)$(vdsmhooksdir)/before_vm_start/50_vmfex + $(RM) $(DESTDIR)$(vdsmhooksdir)/before_vm_migrate_destination/50_vmfex + diff --git a/vdsm_hooks/vmfex/README b/vdsm_hooks/vmfex/README new file mode 100644 index 0000000..1eb7629 --- /dev/null +++ b/vdsm_hooks/vmfex/README @@ -0,0 +1,61 @@ +vmfex vdsm hook +=================== + +Add Cisco VM-FEX Port Profile to Virtual Machine +Sample: + vmfex=NIC1:profile1,NIC2:profile2... + + Will add 2 virtual nics attached to profile1 and profile2 using the vnic + MAC addresses specified, replacing the actual NICs assigned to the VM. + The mapping is by order of NIC<number> in the parameters, mapped to the + order of the "interface" elements in the devices of the VM xml, as well + as the NICs defined for a VM in the RHEV-M admin GUI. + +Libvirt internals: +Replace the existing interface xml: + <interface type="bridge"> + <mac address="<mac>"/> + <model type="virtio"/> + <source bridge="<logical network>"/> + </interface> + +with the following interface xml: + <interface type='network'> + <mac address='<mac>'/> + <source network='direct-pool'/> + <virtualport type='802.1Qbh'> + <parameters profileid='<Port Profile>'/> + </virtualport> + <model type='virtio'/> + </interface> + +Dynamic network with libvirt (define a NIC pool, so libvirt can assign VMs to NICs dynamically): + + <network> + <name>direct-pool</name> + <forward mode="passthrough"> + <interface dev="eth3"/> + <interface dev="eth4"/> + <interface dev="eth5"/> + <interface dev="eth6"/> + <interface dev="eth7"/> + <interface dev="eth8"/> + <interface dev="eth9"/> + <interface dev="eth10"/> + <interface dev="eth11"/> + </forward> + </network> + +Using libvirt, the network is defined like this: + + virsh net-define /tmp/direct-pool.xml + virsh net-start direct-pool + virsh net-autostart direct-pool + +(where /tmp/direct-pool.xml contains the xml above) + +(everything else is autogenerated, and shouldn't be specified +when defining a guest (but whatever is there after definition +should be left in place, e.g. the PCI address)). Note that these +interface definitions are completely static - you never need to modify +them due to migration, or starting up/shutting down the guest. diff --git a/vdsm_hooks/vmfex/before_vm_migrate_destination.py b/vdsm_hooks/vmfex/before_vm_migrate_destination.py new file mode 100755 index 0000000..a80df7f --- /dev/null +++ b/vdsm_hooks/vmfex/before_vm_migrate_destination.py @@ -0,0 +1,109 @@ +#!/usr/bin/python + +import os +import subprocess +import sys +import hooking +import traceback +import libvirtconnection + +''' +Placed in before_vm_migrate_destination + +vmfex hook on migration destination: + +Set up a dynamic NIC pool for incoming migrations to use + + <network> + <name>direct-pool</name> + <forward mode="passthrough"> + <interface dev="eth3"/> + <interface dev="eth4"/> + <interface dev="eth5"/> + <interface dev="eth6"/> + <interface dev="eth7"/> + <interface dev="eth8"/> + <interface dev="eth9"/> + <interface dev="eth10"/> + <interface dev="eth11"/> + </forward> + </network> + +Using libvirt, the network is defined like this: + + virsh net-define /tmp/direct-pool.xml + virsh net-start direct-pool + virsh net-autostart direct-pool + +(where /tmp/direct-pool.xml contains the xml above) + +(everything else is autogenerated, and shouldn't be specified +when defining a guest (but whatever is there after definition +should be left in place, e.g. the PCI address)). Note that these +interface definitions are completely static - you never need to modify +them due to migration, or starting up/shutting down the guest. + +''' + +def getUsableNics(): + # Scan localhost for physical NICs and return list of physical nics + # that have all zeroes MAC + # Example ['eth0','eth1'] + nics = [] + cmd = "find /sys/devices/pci* | grep '/net/' |grep address" + p = subprocess.Popen(cmd,stdout=subprocess.PIPE,shell=True) + s = p.stdout.read().split() + for i in s: + mac = open(i).read().strip() + if mac == '00:00:00:00:00:00': + nics.append(i.split('/')[-2]) + return nics + +def createDirectPool(conn): + xmlstr = """<network> \n <name>direct-pool</name> \n <forward mode="passthrough"> \n """ + for i in getUsableNics(): + s = '<interface dev="' + str(i) + '"/> \n' + xmlstr += s + + xmlstr += """ </forward> \n </network> """ + conn.networkDefineXML(xmlstr) + dpool = conn.networkLookupByName('direct-pool') + dpool.setAutostart(1) + dpool.create() + + + +if os.environ.has_key('vmfex'): + try: + #connect to libvirtd + conn = libvirtconnection.get('qemu:///system') + + #check for running VMs. If there are VMs running, skip creating a dNIC pool + if len(conn.listDomainsID()) == 0: + #check if direct-pool is created + if 'direct-pool' not in conn.listNetworks(): + createDirectPool(conn) + else: + if conn.networkLookupByName('direct-pool').isActive() == 0: + conn.networkLookupByName('direct-pool').create() + conn.networkLookupByName('direct-pool').setAutostart(1) + + #check if the defined dNIC pool didn't change + dpool = conn.networkLookupByName('direct-pool') + definedNics = [] + for i in dpool.split(): + if 'dev=' in i: + definedNics.append(str(i.split('=')[1].split('/')[0].strip("'"))) + #if the defined NIC pool doesn't match the usable NIC list, recreate the network + if set(definedNics) != set(getUsableNics()): + conn.networkLookupByName('direct-pool').destroy() + conn.networkLookupByName('direct-pool').undefine() + createDirectPool(conn) + + if 'direct-pool' not in conn.listNetworks(): + createDirectPool(conn) + + except: + sys.stderr.write('vmfex: [unexpected error]: %s\n' % traceback.format_exc()) + sys.exit(2) + diff --git a/vdsm_hooks/vmfex/before_vm_start.py b/vdsm_hooks/vmfex/before_vm_start.py new file mode 100755 index 0000000..9002e3f --- /dev/null +++ b/vdsm_hooks/vmfex/before_vm_start.py @@ -0,0 +1,188 @@ +#!/usr/bin/python + +import os +import subprocess +import sys +import xml +import hooking +import traceback +from xml.dom import minidom +import libvirtconnection + +''' +Placed in before_vm_start + +vmfex hook: +Add Cisco VM-FEX Port Profile to Virtual Machine +Sample: + vmfex=NIC1:profile1,NIC2:profile2... + + Will add 2 virtual nics attached to profile1 and profile2 using the vnic + MAC addresses specified, replacing the actual NICs assigned to the VM. + The mapping is by order of NIC<number> in the parameters, mapped to the + order of the "interface" elements in the devices of the VM xml, as well + as the NICs defined for a VM in the RHEV-M admin GUI. + +Libvirt internals: +Replace the existing interface xml: + <interface type="bridge"> + <mac address="<mac>"/> + <model type="virtio"/> + <source bridge="<logical network>"/> + </interface> + +with the following interface xml: + <interface type='network'> + <mac address='<mac>'/> + <source network='direct-pool'/> + <virtualport type='802.1Qbh'> + <parameters profileid='<Port Profile>'/> + </virtualport> + <model type='virtio'/> + </interface> + +Dynamic network with libvirt (define a NIC pool, so libvirt can assign VMs to NICs dynamically): + + <network> + <name>direct-pool</name> + <forward mode="passthrough"> + <interface dev="eth3"/> + <interface dev="eth4"/> + <interface dev="eth5"/> + <interface dev="eth6"/> + <interface dev="eth7"/> + <interface dev="eth8"/> + <interface dev="eth9"/> + <interface dev="eth10"/> + <interface dev="eth11"/> + </forward> + </network> + +Using libvirt, the network is defined like this: + + virsh net-define /tmp/direct-pool.xml + virsh net-start direct-pool + virsh net-autostart direct-pool + +(where /tmp/direct-pool.xml contains the xml above) + +(everything else is autogenerated, and shouldn't be specified +when defining a guest (but whatever is there after definition +should be left in place, e.g. the PCI address)). Note that these +interface definitions are completely static - you never need to modify +them due to migration, or starting up/shutting down the guest. + +''' + +def getUsableNics(): + # Scan localhost for physical NICs and return list of physical nics + # that have all zeroes MAC + # Example ['eth0','eth1'] + nics = [] + cmd = "find /sys/devices/pci* | grep '/net/' |grep address" + p = subprocess.Popen(cmd,stdout=subprocess.PIPE,shell=True) + s = p.stdout.read().split() + for i in s: + mac = open(i).read().strip() + if mac == '00:00:00:00:00:00': + nics.append(i.split('/')[-2]) + return nics + +def enumNics(devices): + # Given the domxml of VM devices, return the NIC elements numbered 1 through len()+1 + vnicmap = {} + i = 1 + for interface in devices.getElementsByTagName('interface'): + vnicmap[i] = interface + i += 1 + return vnicmap + +def createDirectPool(conn): + xmlstr = """<network> \n <name>direct-pool</name> \n <forward mode="passthrough"> \n """ + for i in getUsableNics(): + s = '<interface dev="' + str(i) + '"/> \n' + xmlstr += s + + xmlstr += """ </forward> \n </network> """ + conn.networkDefineXML(xmlstr) + dpool = conn.networkLookupByName('direct-pool') + dpool.setAutostart(1) + dpool.create() + + + +if os.environ.has_key('vmfex'): + try: + #connect to libvirtd + conn = libvirtconnection.get('qemu:///system') + + #check for running VMs. If there are VMs running, skip creating a dNIC pool + if len(conn.listDomainsID()) == 0: + #check if direct-pool is created + if 'direct-pool' not in conn.listNetworks(): + createDirectPool(conn) + else: + if conn.networkLookupByName('direct-pool').isActive() == 0: + conn.networkLookupByName('direct-pool').create() + conn.networkLookupByName('direct-pool').setAutostart(1) + + #check if the defined dNIC pool didn't change + dpool = conn.networkLookupByName('direct-pool') + definedNics = [] + for i in dpool.split(): + if 'dev=' in i: + definedNics.append(str(i.split('=')[1].split('/')[0].strip("'"))) + #if the defined NIC pool doesn't match the usable NIC list, recreate the network + if set(definedNics) != set(getUsableNics()): + conn.networkLookupByName('direct-pool').destroy() + conn.networkLookupByName('direct-pool').undefine() + createDirectPool(conn) + + if 'direct-pool' not in conn.listNetworks(): + createDirectPool(conn) + + except: + pass + + + try: + #Get the vmfex line + vmfex = os.environ['vmfex'] + + #Get the VM's xml definition + domxml = hooking.read_domxml() + devices = domxml.getElementsByTagName('devices')[0] + vnicmap = enumNics(devices) + + counter = 0 + + for entry in vmfex.split(','): + sys.stderr.write('vmfex: adding interface to profile: %s\n' % entry) + try: + nicnum = int(entry.split(':')[0][3:]) + profile = entry.split(':')[1] + mynic = vnicmap[nicnum] + #replace type="bridge" with "network" + mynic.setAttribute('type','network') + + source = mynic.getElementsByTagName('source')[0] + source.removeAttribute('bridge') + source.setAttribute('network', 'direct-pool') + + virtualport = domxml.createElement('virtualport') + virtualport.setAttribute('type', '802.1Qbh') + mynic.appendChild(virtualport) + + parameters = domxml.createElement('parameters') + parameters.setAttribute('profileid', profile) + virtualport.appendChild(parameters) + + except: + sys.stderr.write('vmfex: Error in vmfex custom property line: %s\n' % entry) + sys.exit(2) + + hooking.write_domxml(domxml) + except: + sys.stderr.write('vmfex: [unexpected error]: %s\n' % traceback.format_exc()) + sys.exit(2) + -- To view, visit http://gerrit.ovirt.org/7547 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I45a7fa46919bb39a648dff190c40618395990e91 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Dan Yasny <[email protected]> _______________________________________________ vdsm-patches mailing list [email protected] https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
