Dan Yasny has uploaded a new change for review. Change subject: Hook: Cisco VM-FEX support vdsm hooks ......................................................................
Hook: Cisco VM-FEX support vdsm hooks Now using "with" for files Replaced all file() with open() Made pyflakes happy with the libvirtconnect import statement Signed-off-by: Dan Yasny <[email protected]> Author: Dan Yasny <[email protected]> Change-Id: I29f49e7a16d88a5a7981e2c9b5d1caa2bdc568ff --- A vdsm_hooks/vmfex/Makefile.am A vdsm_hooks/vmfex/README A vdsm_hooks/vmfex/before_vm_migrate_destination.py A vdsm_hooks/vmfex/before_vm_start.py 4 files changed, 490 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/60/8760/1 diff --git a/vdsm_hooks/vmfex/Makefile.am b/vdsm_hooks/vmfex/Makefile.am new file mode 100644 index 0000000..4b18cfe --- /dev/null +++ b/vdsm_hooks/vmfex/Makefile.am @@ -0,0 +1,35 @@ +# +# Copyright 2011 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# + +EXTRA_DIST = \ + before_vm_migrate_destination.py \ + before_vm_start.py \ + +install-data-local: + $(MKDIR_P) $(DESTDIR)$(vdsmhooksdir)/before_vm_start + $(INSTALL_SCRIPT) $(srcdir)/before_vm_start.py \ + $(DESTDIR)$(vdsmhooksdir)/before_vm_start/50_vmfex + $(MKDIR_P) $(DESTDIR)$(vdsmhooksdir)/before_vm_migrate_destination + $(INSTALL_SCRIPT) $(srcdir)/before_vm_migrate_destination.py \ + $(DESTDIR)$(vdsmhooksdir)/before_vm_migrate_destination/50_vmfex + +uninstall-local: + $(RM) $(DESTDIR)$(vdsmhooksdir)/before_vm_start/50_vmfex + $(RM) $(DESTDIR)$(vdsmhooksdir)/before_vm_migrate_destination/50_vmfex diff --git a/vdsm_hooks/vmfex/README b/vdsm_hooks/vmfex/README new file mode 100644 index 0000000..81e678a --- /dev/null +++ b/vdsm_hooks/vmfex/README @@ -0,0 +1,69 @@ +vmfex vdsm hook +=================== + +Add Cisco VM-FEX Port Profile to Virtual Machine + +vmfex hook: +Add Cisco VM-FEX Port Profile to Virtual Machine +since the only unique parameter passed from the engine to the VM is the +MAC, a dictionary-like mapping of MAC to port profile will be used +Sample: +vmfex={'00:11:22:33:44:55:66':'Profile1', + '00:11:22:33:44:55:67':'Profile2',...} (one line) + +Will add 2 virtual nics attached to profile1 and profile2 using +the vnic MAC addresses specified, replacing the actual NICs assigned +to the VM. + +A VM NIC with a MAC that is not mentioned in this dictionary will not be +altered, and will remain attached to the bridge/logical network defined +for it in the engine. + +Libvirt internals: +Replace the existing interface xml: + <interface type="bridge"> + <mac address="<mac>"/> + <model type="virtio"/> + <source bridge="<logical network>"/> + </interface> + +with the following interface xml: + <interface type='network'> + <mac address='<mac>'/> + <source network='direct-pool'/> + <virtualport type='802.1Qbh'> + <parameters profileid='<Port Profile>'/> + </virtualport> + <model type='virtio'/> + </interface> + +Dynamic network with libvirt (define a NIC pool, so libvirt can assign VMs to NICs dynamically): + + <network> + <name>direct-pool</name> + <forward mode="passthrough"> + <interface dev="eth3"/> + <interface dev="eth4"/> + <interface dev="eth5"/> + <interface dev="eth6"/> + <interface dev="eth7"/> + <interface dev="eth8"/> + <interface dev="eth9"/> + <interface dev="eth10"/> + <interface dev="eth11"/> + </forward> + </network> + +Using libvirt, the network is defined like this: + + virsh net-define /tmp/direct-pool.xml + virsh net-start direct-pool + virsh net-autostart direct-pool + +(where /tmp/direct-pool.xml contains the xml above) + +(everything else is autogenerated, and shouldn't be specified +when defining a guest (but whatever is there after definition +should be left in place, e.g. the PCI address)). Note that these +interface definitions are completely static - you never need to modify +them due to migration, or starting up/shutting down the guest. diff --git a/vdsm_hooks/vmfex/before_vm_migrate_destination.py b/vdsm_hooks/vmfex/before_vm_migrate_destination.py new file mode 100755 index 0000000..07f653f --- /dev/null +++ b/vdsm_hooks/vmfex/before_vm_migrate_destination.py @@ -0,0 +1,163 @@ +#!/usr/bin/python + +import os +import sys +import traceback +import fcntl +from xml.dom import minidom +try: + #3.0 compat + import libvirtconnection + libvirtconnection +except ImportError: + #3.1 compat + from vdsm import libvirtconnection + +''' +Placed in before_vm_migrate_destination + +vmfex hook on migration destination: + +Set up a dynamic NIC pool for incoming migrations to use + + <network> + <name>direct-pool</name> + <forward mode="passthrough"> + <interface dev="eth3"/> + <interface dev="eth4"/> + <interface dev="eth5"/> + <interface dev="eth6"/> + <interface dev="eth7"/> + <interface dev="eth8"/> + <interface dev="eth9"/> + <interface dev="eth10"/> + <interface dev="eth11"/> + </forward> + </network> + +Using libvirt, the network is defined like this: + + virsh net-define /tmp/direct-pool.xml + virsh net-start direct-pool + virsh net-autostart direct-pool + +(where /tmp/direct-pool.xml contains the xml above) + +(everything else is autogenerated, and shouldn't be specified +when defining a guest (but whatever is there after definition +should be left in place, e.g. the PCI address)). Note that these +interface definitions are completely static - you never need to modify +them due to migration, or starting up/shutting down the guest. + +''' + + +def getUsableNics(): + # Scan localhost for physical NICs and return list of physical nics + # that have all zeroes MAC. These NICs are the ones that can be used + # with VMFEX. + # Example ['eth0','eth1'] + nics = [] + for root, dirs, names in os.walk('/sys/devices/'): + if 'address' in names and 'pci' in root: + with open(root + '/address', 'r') as f: + mac = f.readlines()[0].strip() + if mac == '00:00:00:00:00:00': + eth = root.split('/')[-1] + nics.append(eth) + return nics + + +def createDirectPool(conn): + with open('/var/run/vdsm/hook-vmfex.lock', 'w') as f: + fcntl.flock(f.fileno(), fcntl.LOCK_EX) + try: + xmlstr = '''<network> + <name>direct-pool</name> + <forward mode="passthrough"> + ''' + for i in getUsableNics(): + xmlstr += '<interface dev="' + i + '"/> \n' + + xmlstr += ' </forward> \n </network> ' + conn.networkDefineXML(xmlstr) + dpool = conn.networkLookupByName('direct-pool') + dpool.setAutostart(1) + dpool.create() + sys.stderr.write('vmfex: creating Direct-Pool Network \n') + sys.stderr.write(xmlstr + '\n') + finally: + fcntl.flock(f.fileno(), fcntl.LOCK_UN) + + +def removeDirectPool(conn): + with open('/var/run/vdsm/hook-vmfex.lock', 'w') as f: + fcntl.flock(f.fileno(), fcntl.LOCK_EX) + try: + dpool = conn.networkLookupByName('direct-pool') + dpool.destroy() + dpool.undefine() + sys.stderr.write('vmfex: removing direct-pool \n') + finally: + fcntl.flock(f.fileno(), fcntl.LOCK_UN) + + +def qbhInUse(conn): + for vm in conn.listDomainsID(): + domxml = minidom.parseString(conn.lookupByID(vm).XMLDesc(0)) + for vport in domxml.getElementsByTagName('virtualport'): + if vport.getAttribute('type') == '802.1Qbh': + return True + return False + + +def compareDPoolToUsableNics(conn): + #return True if currently defined direct-pool matches the set + #returned by getUsableNics() + dpool = conn.networkLookupByName('direct-pool') + definedNics = [] + dpoolxml = minidom.parseString(dpool.XMLDesc(0)) + for iface in dpoolxml.getElementsByTagName('interface'): + definedNics.append(iface.getAttribute('dev')) + if set(definedNics) == set(getUsableNics()): + return True + else: + return False + + +def handleDirectPool(conn): + #TODO: take this part and everything it uses out and into a + #separate module + + #is direct-pool defined? If not, just create it and move on + if 'direct-pool' not in conn.listNetworks(): + createDirectPool(conn) + return + #Now we know for sure that the pool exists... + #are there VMs running, and are they using 802.1Qbh? + #if yes, can't touch the existing pool + if not qbhInUse(conn): + #no VMs use the existing pool, it can be checked and updated + #is the pool the same as list of available dNICs? + if not compareDPoolToUsableNics(conn): + #not the same, something changed, recreating + removeDirectPool(conn) + createDirectPool(conn) + #no VMs running with Qbh, pool can be updated + else: + #is the pool the same as list of available dNICs? + if not compareDPoolToUsableNics(conn): + #not the same, something changed, recreating + removeDirectPool(conn) + createDirectPool(conn) + + +if 'vmfex' in os.environ: + try: + #connect to libvirtd + conn = libvirtconnection.get() + handleDirectPool(conn) + + except: + sys.stderr.write('vmfex: ERROR: %s\n' % traceback.format_exc()) + sys.exit(2) diff --git a/vdsm_hooks/vmfex/before_vm_start.py b/vdsm_hooks/vmfex/before_vm_start.py new file mode 100755 index 0000000..1ef86e4 --- /dev/null +++ b/vdsm_hooks/vmfex/before_vm_start.py @@ -0,0 +1,223 @@ +#!/usr/bin/python + +import os +import sys +import hooking +import traceback +import fcntl +import ast +from xml.dom import minidom +try: + #3.0 compat + import libvirtconnection + libvirtconnection +except ImportError: + #3.1 compat + from vdsm import libvirtconnection + +''' +Placed in before_vm_start + +vmfex hook: +Add Cisco VM-FEX Port Profile to Virtual Machine +since the only unique parameter passed from the engine to the VM is the +MAC, a dictionary-like mapping of MAC to port profile will be used +Sample: +vmfex={'00:11:22:33:44:55:66':'Profile1', + '00:11:22:33:44:55:67':'Profile2',...} (one line) + +Will add 2 virtual nics attached to profile1 and profile2 using +the vnic MAC addresses specified, replacing the actual NICs assigned +to the VM. + +A VM NIC with a MAC that is not mentioned in this dictionary will not be +altered, and will remain attached to the bridge/logical network defined +for it in the engine. + +Libvirt internals: +Replace the existing interface xml: + <interface type="bridge"> + <mac address="<mac>"/> + <model type="virtio"/> + <source bridge="<logical network>"/> + </interface> + +with the following interface xml: + <interface type='network'> + <mac address='<mac>'/> + <source network='direct-pool'/> + <virtualport type='802.1Qbh'> + <parameters profileid='<Port Profile>'/> + </virtualport> + <model type='virtio'/> + </interface> + +Dynamic network with libvirt (define a NIC pool, so libvirt can assign +VMs to NICs dynamically): + + <network> + <name>direct-pool</name> + <forward mode="passthrough"> + <interface dev="eth3"/> + <interface dev="eth4"/> + <interface dev="eth5"/> + <interface dev="eth6"/> + <interface dev="eth7"/> + <interface dev="eth8"/> + <interface dev="eth9"/> + <interface dev="eth10"/> + <interface dev="eth11"/> + </forward> + </network> + +Using libvirt, the network is defined like this: + + virsh net-define /tmp/direct-pool.xml + virsh net-start direct-pool + virsh net-autostart direct-pool + +(where /tmp/direct-pool.xml contains the xml above) + +(everything else is autogenerated, and shouldn't be specified +when defining a guest (but whatever is there after definition +should be left in place, e.g. the PCI address)). Note that these +interface definitions are completely static - you never need to modify +them due to migration, or starting up/shutting down the guest. +''' + + +def getUsableNics(): + # Scan localhost for physical NICs and return list of physical nics + # that have all zeroes MAC. These NICs are the ones that can be used + # with VMFEX. + # Example ['eth0','eth1'] + nics = [] + for root, dirs, names in os.walk('/sys/devices/'): + if 'address' in names and 'pci' in root: + with open(root + '/address', 'r') as f: + mac = f.readlines()[0].strip() + if mac == '00:00:00:00:00:00': + eth = root.split('/')[-1] + nics.append(eth) + return nics + + +def createDirectPool(conn): + with open('/var/run/vdsm/hook-vmfex.lock', 'w') as f: + fcntl.flock(f.fileno(), fcntl.LOCK_EX) + try: + xmlstr = '''<network> + <name>direct-pool</name> + <forward mode="passthrough"> + ''' + for i in getUsableNics(): + xmlstr += '<interface dev="' + i + '"/> \n' + + xmlstr += ' </forward> \n </network> ' + conn.networkDefineXML(xmlstr) + dpool = conn.networkLookupByName('direct-pool') + dpool.setAutostart(1) + dpool.create() + sys.stderr.write('vmfex: creating Direct-Pool Network \n') + sys.stderr.write(xmlstr + '\n') + finally: + fcntl.flock(f.fileno(), fcntl.LOCK_UN) + + +def removeDirectPool(conn): + with open('/var/run/vdsm/hook-vmfex.lock', 'w') as f: + fcntl.flock(f.fileno(), fcntl.LOCK_EX) + try: + dpool = conn.networkLookupByName('direct-pool') + dpool.destroy() + dpool.undefine() + sys.stderr.write('vmfex: removing direct-pool \n') + finally: + fcntl.flock(f.fileno(), fcntl.LOCK_UN) + + +def qbhInUse(conn): + for vm in conn.listDomainsID(): + domxml = minidom.parseString(conn.lookupByID(vm).XMLDesc(0)) + for vport in domxml.getElementsByTagName('virtualport'): + if vport.getAttribute('type') == '802.1Qbh': + return True + return False + + +def compareDPoolToUsableNics(conn): + #return True if currently defined direct-pool matches the set + #returned by getUsableNics() + dpool = conn.networkLookupByName('direct-pool') + definedNics = [] + dpoolxml = minidom.parseString(dpool.XMLDesc(0)) + for iface in dpoolxml.getElementsByTagName('interface'): + definedNics.append(iface.getAttribute('dev')) + if set(definedNics) == set(getUsableNics()): + return True + else: + return False + + +def handleDirectPool(conn): + #TODO: take this part and everything it uses out and into a + #separate module + + #is direct-pool defined? If not, just create it and move on + if 'direct-pool' not in conn.listNetworks(): + createDirectPool(conn) + return + #Now we know for sure that the pool exists... + #are there VMs running, and are they using 802.1Qbh? + #if yes, can't touch the existing pool + if not qbhInUse(conn): + #no VMs use the existing pool, it can be checked and updated + #is the pool the same as list of available dNICs? + if not compareDPoolToUsableNics(conn): + #not the same, something changed, recreating + removeDirectPool(conn) + createDirectPool(conn) + #no VMs running with Qbh, pool can be updated + else: + #is the pool the same as list of available dNICs? + if not compareDPoolToUsableNics(conn): + #not the same, something changed, recreating + removeDirectPool(conn) + createDirectPool(conn) + + +if 'vmfex' in os.environ: + try: + sys.stderr.write('vmfex: starting to edit VM \n') + #connect to libvirtd and handle the direct-pool network + conn = libvirtconnection.get() + handleDirectPool(conn) + #Get the vmfex line + vmfex = os.environ['vmfex'] + sys.stderr.write('vmfex: customProperty: ' + str(vmfex) + '\n') + #convert to dictionary + vmfexd = ast.literal_eval(vmfex) + #make sure the keys are lowercase + vmfexd = dict((k.lower(), v) for k, v in vmfexd.iteritems()) + #Get the VM's xml definition + domxml = hooking.read_domxml() + + for iface in domxml.getElementsByTagName('interface'): + mac = iface.getElementsByTagName('mac')[0] + macaddr = mac.getAttribute('address').lower() + if macaddr in vmfexd: + profile = vmfexd[macaddr] + iface.setAttribute('type', 'network') + source = iface.getElementsByTagName('source')[0] + source.removeAttribute('bridge') + source.setAttribute('network', 'direct-pool') + virtualport = domxml.createElement('virtualport') + virtualport.setAttribute('type', '802.1Qbh') + iface.appendChild(virtualport) + parameters = domxml.createElement('parameters') + parameters.setAttribute('profileid', profile) + virtualport.appendChild(parameters) + hooking.write_domxml(domxml) + except: + sys.stderr.write('vmfex: ERROR %s\n' % traceback.format_exc()) + sys.exit(2) -- To view, visit http://gerrit.ovirt.org/8760 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I29f49e7a16d88a5a7981e2c9b5d1caa2bdc568ff Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Dan Yasny <[email protected]> _______________________________________________ vdsm-patches mailing list [email protected] https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
