** Changed in: nova
Status: Fix Committed => Fix Released
** Changed in: nova
Milestone: None => kilo-1
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1349978
Title:
hard reboot doesn't re-create instance folder
Status in OpenStack Compute (Nova):
Fix Released
Bug description:
I'm running stateless Nova compute servers (booted with ram_fs mounted
on /). If I power-cycle a server it forgets all its instances but they
could be recovered by hard-rebooting, except hard-rebooting a libvirt
instance assumes the instance's folder already exists (ie.
/var/lib/nova/instances/e4e4a4a3-1036-4734-9c0b-3bb34c88b8b6/). When
nova tries to re-create the XML file it causes this stack trace:
2014-07-29 16:02:41.250 2922 ERROR oslo.messaging.rpc.dispatcher [-]
Exception during message handling: [Errno 2] No such file or directory:
'/var/lib/nova/instances/e4e4a4a3-1036-4734-9c0b-3bb34c88b8b6/libvirt.xml'
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher Traceback
(most recent call last):
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 133,
in _dispatch_and_reply
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher
incoming.message))
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 176,
in _dispatch
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher return
self._do_dispatch(endpoint, method, ctxt, args)
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 122,
in _do_dispatch
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher result =
getattr(endpoint, method)(ctxt, **new_args)
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/exception.py", line 88, in wrapped
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher payload)
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68,
in __exit__
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher
six.reraise(self.type_, self.value, self.tb)
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/exception.py", line 71, in wrapped
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher return
f(self, context, *args, **kw)
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 274, in
decorated_function
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher pass
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68,
in __exit__
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher
six.reraise(self.type_, self.value, self.tb)
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 260, in
decorated_function
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher return
function(self, context, *args, **kwargs)
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 327, in
decorated_function
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher
function(self, context, *args, **kwargs)
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 303, in
decorated_function
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher e,
sys.exc_info())
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68,
in __exit__
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher
six.reraise(self.type_, self.value, self.tb)
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 290, in
decorated_function
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher return
function(self, context, *args, **kwargs)
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2649, in
reboot_instance
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher
self._set_instance_obj_error_state(context, instance)
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68,
in __exit__
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher
six.reraise(self.type_, self.value, self.tb)
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2630, in
reboot_instance
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher
bad_volumes_callback=bad_volumes_callback)
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1999, in
reboot
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher
block_device_info)
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2092, in
_hard_reboot
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher
write_to_disk=True)
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3455, in
to_xml
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher
libvirt_utils.write_to_file(xml_path, xml)
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/utils.py", line 526, in
write_to_file
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher with
open(path, 'w') as f:
2014-07-29 16:02:41.250 2922 TRACE oslo.messaging.rpc.dispatcher IOError:
[Errno 2] No such file or directory:
'/var/lib/nova/instances/e4e4a4a3-1036-4734-9c0b-3bb34c88b8b6/libvirt.xml'
If I manually create the directory then hard-reboot will re-create
everything else and work as expected.
OS: Ubuntu 14.04, 64-bit
nova-compute version: icehouse 2014.1.1
libvirt: v1.2.2
Using Ceph for storage: v0.80.1
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1349978/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
