Public bug reported:
If I resize an instance to a flavor with more cpus than should be
possible, even more cpus than cpu_allocation_ratio would allow, then
nova proceeds with the resize and the instance state goes to error and
it does exist anymore in the hypervisor.
My environment:
Nova 2014.1 (Icehouse)
Libvirt / KVM hypervisor
Ceph RBD for volume storage
1 nova-manage node, 2 nova-compute nodes (all virtualized on my laptop)
Ubuntu 14.04 for all OSes except for the instance
Cirros 0.3.2 for the instance
/etc/nova/nova.conf on all nodes contain:
allow_resize_to_same_host=true (although this didn't make a difference!)
ram_allocation_ratio=0.95
cpu_allocation_ratio=5
CPU and RAM allocation_ratio limits are set but only the memory limit took
effect. This is from the log below:
memory limit: 1900.95 MB, free: 1388.95 MB
CPUs limit not specified, defaulting to unlimited
This is the log from nova-compute.log when I resize an instance to a
flavor with 500 vcpus:
AUDIT nova.compute.resource_tracker [-] Auditing locally available compute
resources
AUDIT nova.compute.resource_tracker [-] Free ram (MB): 1489
AUDIT nova.compute.resource_tracker [-] Free disk (GB): 34
AUDIT nova.compute.resource_tracker [-] Free VCPUS: 2
INFO nova.compute.resource_tracker [-] Compute_service record updated for
r-ABCDEF1234:r-ABCDEF1234.vagrant.local
AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance:
75204b82-70e9-4e13-b56f-d7b78613cac7] Attempting claim: memory 500 MB, disk 1
GB, VCPUs 500
AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance:
75204b82-70e9-4e13-b56f-d7b78613cac7] Total memory: 2001 MB, used: 512.00 MB
AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance:
75204b82-70e9-4e13-b56f-d7b78613cac7] memory limit: 1900.95 MB, free: 1388.95 MB
AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance:
75204b82-70e9-4e13-b56f-d7b78613cac7] Total disk: 34 GB, used: 0.00 GB
AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance:
75204b82-70e9-4e13-b56f-d7b78613cac7] disk limit not specified, defaulting to
unlimited
AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance:
75204b82-70e9-4e13-b56f-d7b78613cac7] Total CPUs: 2 VCPUs, used: 0.00 VCPUs
AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance:
75204b82-70e9-4e13-b56f-d7b78613cac7] CPUs limit not specified, defaulting to
unlimited
AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance:
75204b82-70e9-4e13-b56f-d7b78613cac7] Claim successful
AUDIT nova.compute.resource_tracker [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] Updating
from migration 75204b82-70e9-4e13-b56f-d7b78613cac7
AUDIT nova.compute.manager [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance:
75204b82-70e9-4e13-b56f-d7b78613cac7] Migrating
WARNING nova.virt.libvirt.utils [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] systool is
not installed
WARNING nova.virt.libvirt.utils [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] systool is
not installed
INFO urllib3.connectionpool [-] Starting new HTTP connection (1): 192.168.55.253
INFO oslo.messaging._drivers.impl_rabbit [-] Connected to AMQP server on
192.168.55.11:5672
INFO nova.virt.libvirt.driver [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance:
75204b82-70e9-4e13-b56f-d7b78613cac7] Creating image
ERROR glanceclient.common.http [-] Request returned failure status.
WARNING nova.compute.utils [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance:
75204b82-70e9-4e13-b56f-d7b78613cac7] Can't access image : Image could not be
found.
INFO nova.virt.libvirt.firewall [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance:
75204b82-70e9-4e13-b56f-d7b78613cac7] Called setup_basic_filtering in nwfilter
INFO nova.virt.libvirt.firewall [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance:
75204b82-70e9-4e13-b56f-d7b78613cac7] Ensuring static filters
ERROR nova.virt.libvirt.driver [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] An error
occurred while trying to launch a defined domain with xml: <domain type='qemu'>
<name>instance-00000006</name>
<uuid>75204b82-70e9-4e13-b56f-d7b78613cac7</uuid>
<memory unit='KiB'>512000</memory>
<currentMemory unit='KiB'>512000</currentMemory>
<vcpu placement='static'>500</vcpu>
<sysinfo type='smbios'>
<system>
<entry name='manufacturer'>OpenStack Foundation</entry>
<entry name='product'>OpenStack Nova</entry>
<entry name='version'>2014.1</entry>
<entry name='serial'>cffca416-65a2-48b5-ba0c-40ad122dc9d2</entry>
<entry name='uuid'>75204b82-70e9-4e13-b56f-d7b78613cac7</entry>
</system>
</sysinfo>
<os>
<type arch='x86_64' machine='pc-i440fx-trusty'>hvm</type>
<boot dev='hd'/>
<smbios mode='sysinfo'/>
</os>
<features>
<acpi/>
<apic/>
</features>
<cpu mode='host-model'>
<model fallback='allow'/>
</cpu>
<clock offset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
<devices>
<emulator>/usr/bin/qemu-system-x86_64</emulator>
<disk type='network' device='disk'>
<driver name='qemu' type='raw' cache='none'/>
<auth username='volumes'>
<secret type='ceph' uuid='d9002041-37e4-b53c-1d9e-a5e5ee7e3466'/>
</auth>
<source protocol='rbd'
name='volumes/volume-22a7ae26-e55b-4aad-a3a1-bbee6ecd6abb'>
<host name='192.168.55.11' port='6789'/>
</source>
<target dev='vda' bus='virtio'/>
<serial>22a7ae26-e55b-4aad-a3a1-bbee6ecd6abb</serial>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04'
function='0x0'/>
</disk>
<controller type='usb' index='0'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01'
function='0x2'/>
</controller>
<controller type='pci' index='0' model='pci-root'/>
<interface type='bridge'>
<mac address='fa:16:3e:e3:8f:5d'/>
<source bridge='br101'/>
<model type='virtio'/>
<driver name='qemu'/>
<filterref filter='nova-instance-instance-00000006-fa163ee38f5d'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03'
function='0x0'/>
</interface>
<serial type='file'>
<source
path='/var/lib/nova/instances/75204b82-70e9-4e13-b56f-d7b78613cac7/console.log'/>
<target port='0'/>
</serial>
<serial type='pty'>
<target port='1'/>
</serial>
<console type='file'>
<source
path='/var/lib/nova/instances/75204b82-70e9-4e13-b56f-d7b78613cac7/console.log'/>
<target type='serial' port='0'/>
</console>
<input type='tablet' bus='usb'/>
<input type='mouse' bus='ps2'/>
<input type='keyboard' bus='ps2'/>
<graphics type='vnc' port='-1' autoport='yes' listen='192.168.55.101'
keymap='en-us'>
<listen type='address' address='192.168.55.101'/>
</graphics>
<video>
<model type='cirrus' vram='9216' heads='1'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02'
function='0x0'/>
</video>
<memballoon model='virtio'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x05'
function='0x0'/>
</memballoon>
</devices>
</domain>
ERROR nova.compute.manager [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance:
75204b82-70e9-4e13-b56f-d7b78613cac7] Setting instance vm_state to ERROR
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
Traceback (most recent call last):
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3559, in
finish_resize
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
disk_info, image)
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3530, in
_finish_resize
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
block_device_info, power_on)
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 5004,
in finish_migration
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
block_device_info, power_on)
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3644,
in _create_domain_and_network
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
power_on=power_on)
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3547,
in _create_domain
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
domain.XMLDesc(0))
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line
68, in __exit__
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
six.reraise(self.type_, self.value, self.tb)
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3542,
in _create_domain
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
domain.createWithFlags(launch_flags)
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 179, in doit
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
result = proxy_call(self._autowrap, f, *args, **kwargs)
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 139, in
proxy_call
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
rv = execute(f,*args,**kwargs)
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 77, in tworker
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
rv = meth(*args,**kwargs)
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
File "/usr/lib/python2.7/dist-packages/libvirt.py", line 896, in createWithFlags
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
libvirtError: unsupported configuration: Maximum CPUs greater than specified
machine type limit
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
ERROR oslo.messaging.rpc.dispatcher [-] Exception during message handling:
unsupported configuration: Maximum CPUs greater than specified machine type
limit
TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 133,
in _dispatch_and_reply
TRACE oslo.messaging.rpc.dispatcher incoming.message))
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 176,
in _dispatch
TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint,
method, ctxt, args)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 122,
in _do_dispatch
TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint,
method)(ctxt, **new_args)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/exception.py", line 88, in wrapped
TRACE oslo.messaging.rpc.dispatcher payload)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68,
in __exit__
TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value,
self.tb)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/exception.py", line 71, in wrapped
TRACE oslo.messaging.rpc.dispatcher return f(self, context, *args, **kw)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 280, in
decorated_function
TRACE oslo.messaging.rpc.dispatcher pass
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68,
in __exit__
TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value,
self.tb)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 266, in
decorated_function
TRACE oslo.messaging.rpc.dispatcher return function(self, context, *args,
**kwargs)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 333, in
decorated_function
TRACE oslo.messaging.rpc.dispatcher function(self, context, *args, **kwargs)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 254, in
decorated_function
TRACE oslo.messaging.rpc.dispatcher break
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68,
in __exit__
TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value,
self.tb)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 235, in
decorated_function
TRACE oslo.messaging.rpc.dispatcher return function(self, context, *args,
**kwargs)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 309, in
decorated_function
TRACE oslo.messaging.rpc.dispatcher e, sys.exc_info())
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68,
in __exit__
TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value,
self.tb)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 296, in
decorated_function
TRACE oslo.messaging.rpc.dispatcher return function(self, context, *args,
**kwargs)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3571, in
finish_resize
TRACE oslo.messaging.rpc.dispatcher self._set_instance_error_state(context,
instance['uuid'])
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68,
in __exit__
TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value,
self.tb)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3559, in
finish_resize
TRACE oslo.messaging.rpc.dispatcher disk_info, image)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3530, in
_finish_resize
TRACE oslo.messaging.rpc.dispatcher block_device_info, power_on)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 5004, in
finish_migration
TRACE oslo.messaging.rpc.dispatcher block_device_info, power_on)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3644, in
_create_domain_and_network
TRACE oslo.messaging.rpc.dispatcher power_on=power_on)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3547, in
_create_domain
TRACE oslo.messaging.rpc.dispatcher domain.XMLDesc(0))
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68,
in __exit__
TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value,
self.tb)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3542, in
_create_domain
TRACE oslo.messaging.rpc.dispatcher domain.createWithFlags(launch_flags)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 179, in doit
TRACE oslo.messaging.rpc.dispatcher result = proxy_call(self._autowrap, f,
*args, **kwargs)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 139, in proxy_call
TRACE oslo.messaging.rpc.dispatcher rv = execute(f,*args,**kwargs)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 77, in tworker
TRACE oslo.messaging.rpc.dispatcher rv = meth(*args,**kwargs)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/libvirt.py", line 896, in createWithFlags
TRACE oslo.messaging.rpc.dispatcher if ret == -1: raise libvirtError
('virDomainCreateWithFlags() failed', dom=self)
TRACE oslo.messaging.rpc.dispatcher libvirtError: unsupported configuration:
Maximum CPUs greater than specified machine type limit
AUDIT nova.compute.resource_tracker [-] Auditing locally available compute
resources
WARNING nova.compute.resource_tracker [-] [instance:
75204b82-70e9-4e13-b56f-d7b78613cac7] Instance not resizing, skipping migration.
AUDIT nova.compute.resource_tracker [-] Free ram (MB): 989
AUDIT nova.compute.resource_tracker [-] Free disk (GB): 33
AUDIT nova.compute.resource_tracker [-] Free VCPUS: -498
WARNING nova.compute.manager [-] Found 1 in the database and 0 on the
hypervisor.
** Affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1338822
Title:
CPU limit not used in instance resize
Status in OpenStack Compute (Nova):
New
Bug description:
If I resize an instance to a flavor with more cpus than should be
possible, even more cpus than cpu_allocation_ratio would allow, then
nova proceeds with the resize and the instance state goes to error and
it does exist anymore in the hypervisor.
My environment:
Nova 2014.1 (Icehouse)
Libvirt / KVM hypervisor
Ceph RBD for volume storage
1 nova-manage node, 2 nova-compute nodes (all virtualized on my laptop)
Ubuntu 14.04 for all OSes except for the instance
Cirros 0.3.2 for the instance
/etc/nova/nova.conf on all nodes contain:
allow_resize_to_same_host=true (although this didn't make a difference!)
ram_allocation_ratio=0.95
cpu_allocation_ratio=5
CPU and RAM allocation_ratio limits are set but only the memory limit took
effect. This is from the log below:
memory limit: 1900.95 MB, free: 1388.95 MB
CPUs limit not specified, defaulting to unlimited
This is the log from nova-compute.log when I resize an instance to a
flavor with 500 vcpus:
AUDIT nova.compute.resource_tracker [-] Auditing locally available compute
resources
AUDIT nova.compute.resource_tracker [-] Free ram (MB): 1489
AUDIT nova.compute.resource_tracker [-] Free disk (GB): 34
AUDIT nova.compute.resource_tracker [-] Free VCPUS: 2
INFO nova.compute.resource_tracker [-] Compute_service record updated for
r-ABCDEF1234:r-ABCDEF1234.vagrant.local
AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance:
75204b82-70e9-4e13-b56f-d7b78613cac7] Attempting claim: memory 500 MB, disk 1
GB, VCPUs 500
AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance:
75204b82-70e9-4e13-b56f-d7b78613cac7] Total memory: 2001 MB, used: 512.00 MB
AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance:
75204b82-70e9-4e13-b56f-d7b78613cac7] memory limit: 1900.95 MB, free: 1388.95 MB
AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance:
75204b82-70e9-4e13-b56f-d7b78613cac7] Total disk: 34 GB, used: 0.00 GB
AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance:
75204b82-70e9-4e13-b56f-d7b78613cac7] disk limit not specified, defaulting to
unlimited
AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance:
75204b82-70e9-4e13-b56f-d7b78613cac7] Total CPUs: 2 VCPUs, used: 0.00 VCPUs
AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance:
75204b82-70e9-4e13-b56f-d7b78613cac7] CPUs limit not specified, defaulting to
unlimited
AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance:
75204b82-70e9-4e13-b56f-d7b78613cac7] Claim successful
AUDIT nova.compute.resource_tracker [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] Updating
from migration 75204b82-70e9-4e13-b56f-d7b78613cac7
AUDIT nova.compute.manager [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance:
75204b82-70e9-4e13-b56f-d7b78613cac7] Migrating
WARNING nova.virt.libvirt.utils [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] systool is
not installed
WARNING nova.virt.libvirt.utils [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] systool is
not installed
INFO urllib3.connectionpool [-] Starting new HTTP connection (1):
192.168.55.253
INFO oslo.messaging._drivers.impl_rabbit [-] Connected to AMQP server on
192.168.55.11:5672
INFO nova.virt.libvirt.driver [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance:
75204b82-70e9-4e13-b56f-d7b78613cac7] Creating image
ERROR glanceclient.common.http [-] Request returned failure status.
WARNING nova.compute.utils [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance:
75204b82-70e9-4e13-b56f-d7b78613cac7] Can't access image : Image could not be
found.
INFO nova.virt.libvirt.firewall [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance:
75204b82-70e9-4e13-b56f-d7b78613cac7] Called setup_basic_filtering in nwfilter
INFO nova.virt.libvirt.firewall [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance:
75204b82-70e9-4e13-b56f-d7b78613cac7] Ensuring static filters
ERROR nova.virt.libvirt.driver [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] An error
occurred while trying to launch a defined domain with xml: <domain type='qemu'>
<name>instance-00000006</name>
<uuid>75204b82-70e9-4e13-b56f-d7b78613cac7</uuid>
<memory unit='KiB'>512000</memory>
<currentMemory unit='KiB'>512000</currentMemory>
<vcpu placement='static'>500</vcpu>
<sysinfo type='smbios'>
<system>
<entry name='manufacturer'>OpenStack Foundation</entry>
<entry name='product'>OpenStack Nova</entry>
<entry name='version'>2014.1</entry>
<entry name='serial'>cffca416-65a2-48b5-ba0c-40ad122dc9d2</entry>
<entry name='uuid'>75204b82-70e9-4e13-b56f-d7b78613cac7</entry>
</system>
</sysinfo>
<os>
<type arch='x86_64' machine='pc-i440fx-trusty'>hvm</type>
<boot dev='hd'/>
<smbios mode='sysinfo'/>
</os>
<features>
<acpi/>
<apic/>
</features>
<cpu mode='host-model'>
<model fallback='allow'/>
</cpu>
<clock offset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
<devices>
<emulator>/usr/bin/qemu-system-x86_64</emulator>
<disk type='network' device='disk'>
<driver name='qemu' type='raw' cache='none'/>
<auth username='volumes'>
<secret type='ceph' uuid='d9002041-37e4-b53c-1d9e-a5e5ee7e3466'/>
</auth>
<source protocol='rbd'
name='volumes/volume-22a7ae26-e55b-4aad-a3a1-bbee6ecd6abb'>
<host name='192.168.55.11' port='6789'/>
</source>
<target dev='vda' bus='virtio'/>
<serial>22a7ae26-e55b-4aad-a3a1-bbee6ecd6abb</serial>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04'
function='0x0'/>
</disk>
<controller type='usb' index='0'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01'
function='0x2'/>
</controller>
<controller type='pci' index='0' model='pci-root'/>
<interface type='bridge'>
<mac address='fa:16:3e:e3:8f:5d'/>
<source bridge='br101'/>
<model type='virtio'/>
<driver name='qemu'/>
<filterref filter='nova-instance-instance-00000006-fa163ee38f5d'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03'
function='0x0'/>
</interface>
<serial type='file'>
<source
path='/var/lib/nova/instances/75204b82-70e9-4e13-b56f-d7b78613cac7/console.log'/>
<target port='0'/>
</serial>
<serial type='pty'>
<target port='1'/>
</serial>
<console type='file'>
<source
path='/var/lib/nova/instances/75204b82-70e9-4e13-b56f-d7b78613cac7/console.log'/>
<target type='serial' port='0'/>
</console>
<input type='tablet' bus='usb'/>
<input type='mouse' bus='ps2'/>
<input type='keyboard' bus='ps2'/>
<graphics type='vnc' port='-1' autoport='yes' listen='192.168.55.101'
keymap='en-us'>
<listen type='address' address='192.168.55.101'/>
</graphics>
<video>
<model type='cirrus' vram='9216' heads='1'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02'
function='0x0'/>
</video>
<memballoon model='virtio'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x05'
function='0x0'/>
</memballoon>
</devices>
</domain>
ERROR nova.compute.manager [req-960d005b-0ca5-434b-96dc-e1e7575aef0f
49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance:
75204b82-70e9-4e13-b56f-d7b78613cac7] Setting instance vm_state to ERROR
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
Traceback (most recent call last):
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3559, in
finish_resize
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
disk_info, image)
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3530, in
_finish_resize
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
block_device_info, power_on)
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 5004,
in finish_migration
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
block_device_info, power_on)
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3644,
in _create_domain_and_network
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
power_on=power_on)
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3547,
in _create_domain
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
domain.XMLDesc(0))
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line
68, in __exit__
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
six.reraise(self.type_, self.value, self.tb)
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3542,
in _create_domain
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
domain.createWithFlags(launch_flags)
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 179, in doit
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
result = proxy_call(self._autowrap, f, *args, **kwargs)
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 139, in
proxy_call
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
rv = execute(f,*args,**kwargs)
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 77, in tworker
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
rv = meth(*args,**kwargs)
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
File "/usr/lib/python2.7/dist-packages/libvirt.py", line 896, in createWithFlags
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed',
dom=self)
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
libvirtError: unsupported configuration: Maximum CPUs greater than specified
machine type limit
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]
ERROR oslo.messaging.rpc.dispatcher [-] Exception during message handling:
unsupported configuration: Maximum CPUs greater than specified machine type
limit
TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 133,
in _dispatch_and_reply
TRACE oslo.messaging.rpc.dispatcher incoming.message))
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 176,
in _dispatch
TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint,
method, ctxt, args)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 122,
in _do_dispatch
TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint,
method)(ctxt, **new_args)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/exception.py", line 88, in wrapped
TRACE oslo.messaging.rpc.dispatcher payload)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68,
in __exit__
TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value,
self.tb)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/exception.py", line 71, in wrapped
TRACE oslo.messaging.rpc.dispatcher return f(self, context, *args, **kw)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 280, in
decorated_function
TRACE oslo.messaging.rpc.dispatcher pass
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68,
in __exit__
TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value,
self.tb)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 266, in
decorated_function
TRACE oslo.messaging.rpc.dispatcher return function(self, context, *args,
**kwargs)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 333, in
decorated_function
TRACE oslo.messaging.rpc.dispatcher function(self, context, *args,
**kwargs)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 254, in
decorated_function
TRACE oslo.messaging.rpc.dispatcher break
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68,
in __exit__
TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value,
self.tb)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 235, in
decorated_function
TRACE oslo.messaging.rpc.dispatcher return function(self, context, *args,
**kwargs)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 309, in
decorated_function
TRACE oslo.messaging.rpc.dispatcher e, sys.exc_info())
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68,
in __exit__
TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value,
self.tb)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 296, in
decorated_function
TRACE oslo.messaging.rpc.dispatcher return function(self, context, *args,
**kwargs)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3571, in
finish_resize
TRACE oslo.messaging.rpc.dispatcher
self._set_instance_error_state(context, instance['uuid'])
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68,
in __exit__
TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value,
self.tb)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3559, in
finish_resize
TRACE oslo.messaging.rpc.dispatcher disk_info, image)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3530, in
_finish_resize
TRACE oslo.messaging.rpc.dispatcher block_device_info, power_on)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 5004, in
finish_migration
TRACE oslo.messaging.rpc.dispatcher block_device_info, power_on)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3644, in
_create_domain_and_network
TRACE oslo.messaging.rpc.dispatcher power_on=power_on)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3547, in
_create_domain
TRACE oslo.messaging.rpc.dispatcher domain.XMLDesc(0))
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68,
in __exit__
TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value,
self.tb)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3542, in
_create_domain
TRACE oslo.messaging.rpc.dispatcher domain.createWithFlags(launch_flags)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 179, in doit
TRACE oslo.messaging.rpc.dispatcher result = proxy_call(self._autowrap,
f, *args, **kwargs)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 139, in proxy_call
TRACE oslo.messaging.rpc.dispatcher rv = execute(f,*args,**kwargs)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 77, in tworker
TRACE oslo.messaging.rpc.dispatcher rv = meth(*args,**kwargs)
TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/libvirt.py", line 896, in createWithFlags
TRACE oslo.messaging.rpc.dispatcher if ret == -1: raise libvirtError
('virDomainCreateWithFlags() failed', dom=self)
TRACE oslo.messaging.rpc.dispatcher libvirtError: unsupported configuration:
Maximum CPUs greater than specified machine type limit
AUDIT nova.compute.resource_tracker [-] Auditing locally available compute
resources
WARNING nova.compute.resource_tracker [-] [instance:
75204b82-70e9-4e13-b56f-d7b78613cac7] Instance not resizing, skipping migration.
AUDIT nova.compute.resource_tracker [-] Free ram (MB): 989
AUDIT nova.compute.resource_tracker [-] Free disk (GB): 33
AUDIT nova.compute.resource_tracker [-] Free VCPUS: -498
WARNING nova.compute.manager [-] Found 1 in the database and 0 on the
hypervisor.
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1338822/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
