Public bug reported: If I resize an instance to a flavor with more cpus than should be possible, even more cpus than cpu_allocation_ratio would allow, then nova proceeds with the resize and the instance state goes to error and it does exist anymore in the hypervisor.
My environment: Nova 2014.1 (Icehouse) Libvirt / KVM hypervisor Ceph RBD for volume storage 1 nova-manage node, 2 nova-compute nodes (all virtualized on my laptop) Ubuntu 14.04 for all OSes except for the instance Cirros 0.3.2 for the instance /etc/nova/nova.conf on all nodes contain: allow_resize_to_same_host=true (although this didn't make a difference!) ram_allocation_ratio=0.95 cpu_allocation_ratio=5 CPU and RAM allocation_ratio limits are set but only the memory limit took effect. This is from the log below: memory limit: 1900.95 MB, free: 1388.95 MB CPUs limit not specified, defaulting to unlimited This is the log from nova-compute.log when I resize an instance to a flavor with 500 vcpus: AUDIT nova.compute.resource_tracker [-] Auditing locally available compute resources AUDIT nova.compute.resource_tracker [-] Free ram (MB): 1489 AUDIT nova.compute.resource_tracker [-] Free disk (GB): 34 AUDIT nova.compute.resource_tracker [-] Free VCPUS: 2 INFO nova.compute.resource_tracker [-] Compute_service record updated for r-ABCDEF1234:r-ABCDEF1234.vagrant.local AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Attempting claim: memory 500 MB, disk 1 GB, VCPUs 500 AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Total memory: 2001 MB, used: 512.00 MB AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] memory limit: 1900.95 MB, free: 1388.95 MB AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Total disk: 34 GB, used: 0.00 GB AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] disk limit not specified, defaulting to unlimited AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Total CPUs: 2 VCPUs, used: 0.00 VCPUs AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] CPUs limit not specified, defaulting to unlimited AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Claim successful AUDIT nova.compute.resource_tracker [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] Updating from migration 75204b82-70e9-4e13-b56f-d7b78613cac7 AUDIT nova.compute.manager [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Migrating WARNING nova.virt.libvirt.utils [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] systool is not installed WARNING nova.virt.libvirt.utils [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] systool is not installed INFO urllib3.connectionpool [-] Starting new HTTP connection (1): 192.168.55.253 INFO oslo.messaging._drivers.impl_rabbit [-] Connected to AMQP server on 192.168.55.11:5672 INFO nova.virt.libvirt.driver [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Creating image ERROR glanceclient.common.http [-] Request returned failure status. WARNING nova.compute.utils [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Can't access image : Image could not be found. INFO nova.virt.libvirt.firewall [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Called setup_basic_filtering in nwfilter INFO nova.virt.libvirt.firewall [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Ensuring static filters ERROR nova.virt.libvirt.driver [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] An error occurred while trying to launch a defined domain with xml: <domain type='qemu'> <name>instance-00000006</name> <uuid>75204b82-70e9-4e13-b56f-d7b78613cac7</uuid> <memory unit='KiB'>512000</memory> <currentMemory unit='KiB'>512000</currentMemory> <vcpu placement='static'>500</vcpu> <sysinfo type='smbios'> <system> <entry name='manufacturer'>OpenStack Foundation</entry> <entry name='product'>OpenStack Nova</entry> <entry name='version'>2014.1</entry> <entry name='serial'>cffca416-65a2-48b5-ba0c-40ad122dc9d2</entry> <entry name='uuid'>75204b82-70e9-4e13-b56f-d7b78613cac7</entry> </system> </sysinfo> <os> <type arch='x86_64' machine='pc-i440fx-trusty'>hvm</type> <boot dev='hd'/> <smbios mode='sysinfo'/> </os> <features> <acpi/> <apic/> </features> <cpu mode='host-model'> <model fallback='allow'/> </cpu> <clock offset='utc'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type='network' device='disk'> <driver name='qemu' type='raw' cache='none'/> <auth username='volumes'> <secret type='ceph' uuid='d9002041-37e4-b53c-1d9e-a5e5ee7e3466'/> </auth> <source protocol='rbd' name='volumes/volume-22a7ae26-e55b-4aad-a3a1-bbee6ecd6abb'> <host name='192.168.55.11' port='6789'/> </source> <target dev='vda' bus='virtio'/> <serial>22a7ae26-e55b-4aad-a3a1-bbee6ecd6abb</serial> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </disk> <controller type='usb' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/> </controller> <controller type='pci' index='0' model='pci-root'/> <interface type='bridge'> <mac address='fa:16:3e:e3:8f:5d'/> <source bridge='br101'/> <model type='virtio'/> <driver name='qemu'/> <filterref filter='nova-instance-instance-00000006-fa163ee38f5d'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> <serial type='file'> <source path='/var/lib/nova/instances/75204b82-70e9-4e13-b56f-d7b78613cac7/console.log'/> <target port='0'/> </serial> <serial type='pty'> <target port='1'/> </serial> <console type='file'> <source path='/var/lib/nova/instances/75204b82-70e9-4e13-b56f-d7b78613cac7/console.log'/> <target type='serial' port='0'/> </console> <input type='tablet' bus='usb'/> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <graphics type='vnc' port='-1' autoport='yes' listen='192.168.55.101' keymap='en-us'> <listen type='address' address='192.168.55.101'/> </graphics> <video> <model type='cirrus' vram='9216' heads='1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </video> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </memballoon> </devices> </domain> ERROR nova.compute.manager [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Setting instance vm_state to ERROR TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Traceback (most recent call last): TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3559, in finish_resize TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] disk_info, image) TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3530, in _finish_resize TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] block_device_info, power_on) TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 5004, in finish_migration TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] block_device_info, power_on) TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3644, in _create_domain_and_network TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] power_on=power_on) TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3547, in _create_domain TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] domain.XMLDesc(0)) TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__ TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] six.reraise(self.type_, self.value, self.tb) TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3542, in _create_domain TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] domain.createWithFlags(launch_flags) TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 179, in doit TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] result = proxy_call(self._autowrap, f, *args, **kwargs) TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 139, in proxy_call TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] rv = execute(f,*args,**kwargs) TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 77, in tworker TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] rv = meth(*args,**kwargs) TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] File "/usr/lib/python2.7/dist-packages/libvirt.py", line 896, in createWithFlags TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self) TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] libvirtError: unsupported configuration: Maximum CPUs greater than specified machine type limit TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] ERROR oslo.messaging.rpc.dispatcher [-] Exception during message handling: unsupported configuration: Maximum CPUs greater than specified machine type limit TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last): TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 133, in _dispatch_and_reply TRACE oslo.messaging.rpc.dispatcher incoming.message)) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 176, in _dispatch TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 122, in _do_dispatch TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint, method)(ctxt, **new_args) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 88, in wrapped TRACE oslo.messaging.rpc.dispatcher payload) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__ TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 71, in wrapped TRACE oslo.messaging.rpc.dispatcher return f(self, context, *args, **kw) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 280, in decorated_function TRACE oslo.messaging.rpc.dispatcher pass TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__ TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 266, in decorated_function TRACE oslo.messaging.rpc.dispatcher return function(self, context, *args, **kwargs) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 333, in decorated_function TRACE oslo.messaging.rpc.dispatcher function(self, context, *args, **kwargs) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 254, in decorated_function TRACE oslo.messaging.rpc.dispatcher break TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__ TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 235, in decorated_function TRACE oslo.messaging.rpc.dispatcher return function(self, context, *args, **kwargs) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 309, in decorated_function TRACE oslo.messaging.rpc.dispatcher e, sys.exc_info()) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__ TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 296, in decorated_function TRACE oslo.messaging.rpc.dispatcher return function(self, context, *args, **kwargs) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3571, in finish_resize TRACE oslo.messaging.rpc.dispatcher self._set_instance_error_state(context, instance['uuid']) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__ TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3559, in finish_resize TRACE oslo.messaging.rpc.dispatcher disk_info, image) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3530, in _finish_resize TRACE oslo.messaging.rpc.dispatcher block_device_info, power_on) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 5004, in finish_migration TRACE oslo.messaging.rpc.dispatcher block_device_info, power_on) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3644, in _create_domain_and_network TRACE oslo.messaging.rpc.dispatcher power_on=power_on) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3547, in _create_domain TRACE oslo.messaging.rpc.dispatcher domain.XMLDesc(0)) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__ TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3542, in _create_domain TRACE oslo.messaging.rpc.dispatcher domain.createWithFlags(launch_flags) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 179, in doit TRACE oslo.messaging.rpc.dispatcher result = proxy_call(self._autowrap, f, *args, **kwargs) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 139, in proxy_call TRACE oslo.messaging.rpc.dispatcher rv = execute(f,*args,**kwargs) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 77, in tworker TRACE oslo.messaging.rpc.dispatcher rv = meth(*args,**kwargs) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/libvirt.py", line 896, in createWithFlags TRACE oslo.messaging.rpc.dispatcher if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self) TRACE oslo.messaging.rpc.dispatcher libvirtError: unsupported configuration: Maximum CPUs greater than specified machine type limit AUDIT nova.compute.resource_tracker [-] Auditing locally available compute resources WARNING nova.compute.resource_tracker [-] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Instance not resizing, skipping migration. AUDIT nova.compute.resource_tracker [-] Free ram (MB): 989 AUDIT nova.compute.resource_tracker [-] Free disk (GB): 33 AUDIT nova.compute.resource_tracker [-] Free VCPUS: -498 WARNING nova.compute.manager [-] Found 1 in the database and 0 on the hypervisor. ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1338822 Title: CPU limit not used in instance resize Status in OpenStack Compute (Nova): New Bug description: If I resize an instance to a flavor with more cpus than should be possible, even more cpus than cpu_allocation_ratio would allow, then nova proceeds with the resize and the instance state goes to error and it does exist anymore in the hypervisor. My environment: Nova 2014.1 (Icehouse) Libvirt / KVM hypervisor Ceph RBD for volume storage 1 nova-manage node, 2 nova-compute nodes (all virtualized on my laptop) Ubuntu 14.04 for all OSes except for the instance Cirros 0.3.2 for the instance /etc/nova/nova.conf on all nodes contain: allow_resize_to_same_host=true (although this didn't make a difference!) ram_allocation_ratio=0.95 cpu_allocation_ratio=5 CPU and RAM allocation_ratio limits are set but only the memory limit took effect. This is from the log below: memory limit: 1900.95 MB, free: 1388.95 MB CPUs limit not specified, defaulting to unlimited This is the log from nova-compute.log when I resize an instance to a flavor with 500 vcpus: AUDIT nova.compute.resource_tracker [-] Auditing locally available compute resources AUDIT nova.compute.resource_tracker [-] Free ram (MB): 1489 AUDIT nova.compute.resource_tracker [-] Free disk (GB): 34 AUDIT nova.compute.resource_tracker [-] Free VCPUS: 2 INFO nova.compute.resource_tracker [-] Compute_service record updated for r-ABCDEF1234:r-ABCDEF1234.vagrant.local AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Attempting claim: memory 500 MB, disk 1 GB, VCPUs 500 AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Total memory: 2001 MB, used: 512.00 MB AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] memory limit: 1900.95 MB, free: 1388.95 MB AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Total disk: 34 GB, used: 0.00 GB AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] disk limit not specified, defaulting to unlimited AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Total CPUs: 2 VCPUs, used: 0.00 VCPUs AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] CPUs limit not specified, defaulting to unlimited AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Claim successful AUDIT nova.compute.resource_tracker [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] Updating from migration 75204b82-70e9-4e13-b56f-d7b78613cac7 AUDIT nova.compute.manager [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Migrating WARNING nova.virt.libvirt.utils [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] systool is not installed WARNING nova.virt.libvirt.utils [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] systool is not installed INFO urllib3.connectionpool [-] Starting new HTTP connection (1): 192.168.55.253 INFO oslo.messaging._drivers.impl_rabbit [-] Connected to AMQP server on 192.168.55.11:5672 INFO nova.virt.libvirt.driver [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Creating image ERROR glanceclient.common.http [-] Request returned failure status. WARNING nova.compute.utils [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Can't access image : Image could not be found. INFO nova.virt.libvirt.firewall [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Called setup_basic_filtering in nwfilter INFO nova.virt.libvirt.firewall [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Ensuring static filters ERROR nova.virt.libvirt.driver [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] An error occurred while trying to launch a defined domain with xml: <domain type='qemu'> <name>instance-00000006</name> <uuid>75204b82-70e9-4e13-b56f-d7b78613cac7</uuid> <memory unit='KiB'>512000</memory> <currentMemory unit='KiB'>512000</currentMemory> <vcpu placement='static'>500</vcpu> <sysinfo type='smbios'> <system> <entry name='manufacturer'>OpenStack Foundation</entry> <entry name='product'>OpenStack Nova</entry> <entry name='version'>2014.1</entry> <entry name='serial'>cffca416-65a2-48b5-ba0c-40ad122dc9d2</entry> <entry name='uuid'>75204b82-70e9-4e13-b56f-d7b78613cac7</entry> </system> </sysinfo> <os> <type arch='x86_64' machine='pc-i440fx-trusty'>hvm</type> <boot dev='hd'/> <smbios mode='sysinfo'/> </os> <features> <acpi/> <apic/> </features> <cpu mode='host-model'> <model fallback='allow'/> </cpu> <clock offset='utc'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type='network' device='disk'> <driver name='qemu' type='raw' cache='none'/> <auth username='volumes'> <secret type='ceph' uuid='d9002041-37e4-b53c-1d9e-a5e5ee7e3466'/> </auth> <source protocol='rbd' name='volumes/volume-22a7ae26-e55b-4aad-a3a1-bbee6ecd6abb'> <host name='192.168.55.11' port='6789'/> </source> <target dev='vda' bus='virtio'/> <serial>22a7ae26-e55b-4aad-a3a1-bbee6ecd6abb</serial> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </disk> <controller type='usb' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/> </controller> <controller type='pci' index='0' model='pci-root'/> <interface type='bridge'> <mac address='fa:16:3e:e3:8f:5d'/> <source bridge='br101'/> <model type='virtio'/> <driver name='qemu'/> <filterref filter='nova-instance-instance-00000006-fa163ee38f5d'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> <serial type='file'> <source path='/var/lib/nova/instances/75204b82-70e9-4e13-b56f-d7b78613cac7/console.log'/> <target port='0'/> </serial> <serial type='pty'> <target port='1'/> </serial> <console type='file'> <source path='/var/lib/nova/instances/75204b82-70e9-4e13-b56f-d7b78613cac7/console.log'/> <target type='serial' port='0'/> </console> <input type='tablet' bus='usb'/> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <graphics type='vnc' port='-1' autoport='yes' listen='192.168.55.101' keymap='en-us'> <listen type='address' address='192.168.55.101'/> </graphics> <video> <model type='cirrus' vram='9216' heads='1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </video> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </memballoon> </devices> </domain> ERROR nova.compute.manager [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Setting instance vm_state to ERROR TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Traceback (most recent call last): TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3559, in finish_resize TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] disk_info, image) TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3530, in _finish_resize TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] block_device_info, power_on) TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 5004, in finish_migration TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] block_device_info, power_on) TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3644, in _create_domain_and_network TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] power_on=power_on) TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3547, in _create_domain TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] domain.XMLDesc(0)) TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__ TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] six.reraise(self.type_, self.value, self.tb) TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3542, in _create_domain TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] domain.createWithFlags(launch_flags) TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 179, in doit TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] result = proxy_call(self._autowrap, f, *args, **kwargs) TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 139, in proxy_call TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] rv = execute(f,*args,**kwargs) TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 77, in tworker TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] rv = meth(*args,**kwargs) TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] File "/usr/lib/python2.7/dist-packages/libvirt.py", line 896, in createWithFlags TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self) TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] libvirtError: unsupported configuration: Maximum CPUs greater than specified machine type limit TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] ERROR oslo.messaging.rpc.dispatcher [-] Exception during message handling: unsupported configuration: Maximum CPUs greater than specified machine type limit TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last): TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 133, in _dispatch_and_reply TRACE oslo.messaging.rpc.dispatcher incoming.message)) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 176, in _dispatch TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 122, in _do_dispatch TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint, method)(ctxt, **new_args) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 88, in wrapped TRACE oslo.messaging.rpc.dispatcher payload) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__ TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 71, in wrapped TRACE oslo.messaging.rpc.dispatcher return f(self, context, *args, **kw) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 280, in decorated_function TRACE oslo.messaging.rpc.dispatcher pass TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__ TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 266, in decorated_function TRACE oslo.messaging.rpc.dispatcher return function(self, context, *args, **kwargs) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 333, in decorated_function TRACE oslo.messaging.rpc.dispatcher function(self, context, *args, **kwargs) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 254, in decorated_function TRACE oslo.messaging.rpc.dispatcher break TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__ TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 235, in decorated_function TRACE oslo.messaging.rpc.dispatcher return function(self, context, *args, **kwargs) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 309, in decorated_function TRACE oslo.messaging.rpc.dispatcher e, sys.exc_info()) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__ TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 296, in decorated_function TRACE oslo.messaging.rpc.dispatcher return function(self, context, *args, **kwargs) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3571, in finish_resize TRACE oslo.messaging.rpc.dispatcher self._set_instance_error_state(context, instance['uuid']) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__ TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3559, in finish_resize TRACE oslo.messaging.rpc.dispatcher disk_info, image) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3530, in _finish_resize TRACE oslo.messaging.rpc.dispatcher block_device_info, power_on) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 5004, in finish_migration TRACE oslo.messaging.rpc.dispatcher block_device_info, power_on) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3644, in _create_domain_and_network TRACE oslo.messaging.rpc.dispatcher power_on=power_on) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3547, in _create_domain TRACE oslo.messaging.rpc.dispatcher domain.XMLDesc(0)) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__ TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3542, in _create_domain TRACE oslo.messaging.rpc.dispatcher domain.createWithFlags(launch_flags) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 179, in doit TRACE oslo.messaging.rpc.dispatcher result = proxy_call(self._autowrap, f, *args, **kwargs) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 139, in proxy_call TRACE oslo.messaging.rpc.dispatcher rv = execute(f,*args,**kwargs) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 77, in tworker TRACE oslo.messaging.rpc.dispatcher rv = meth(*args,**kwargs) TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/libvirt.py", line 896, in createWithFlags TRACE oslo.messaging.rpc.dispatcher if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self) TRACE oslo.messaging.rpc.dispatcher libvirtError: unsupported configuration: Maximum CPUs greater than specified machine type limit AUDIT nova.compute.resource_tracker [-] Auditing locally available compute resources WARNING nova.compute.resource_tracker [-] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Instance not resizing, skipping migration. AUDIT nova.compute.resource_tracker [-] Free ram (MB): 989 AUDIT nova.compute.resource_tracker [-] Free disk (GB): 33 AUDIT nova.compute.resource_tracker [-] Free VCPUS: -498 WARNING nova.compute.manager [-] Found 1 in the database and 0 on the hypervisor. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1338822/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp