Public bug reported: http://logs.openstack.org/73/461473/3/check/gate-tempest-dsvm-neutron- full-ubuntu-xenial/a277636/console.html#_2017-05-02_18_16_55_064917
http://logs.openstack.org/73/461473/3/check/gate-tempest-dsvm-neutron- full-ubuntu- xenial/a277636/logs/screen-n-cpu.txt.gz#_May_02_17_49_07_771248 May 02 17:49:07.771248 ubuntu-xenial-rax-ord-8683720 nova-compute[23706]: ERROR oslo_messaging.rpc.server [req-f281fc56-b69f-4e1a-a6d0-752871138ace tempest-ServerRescueTestJSON-709146246 tempest-ServerRescueTestJSON-709146246] Exception during message handling ERROR oslo_messaging.rpc.server Traceback (most recent call last): ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 157, in _process_incoming ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/exception_wrapper.py", line 77, in wrapped ERROR oslo_messaging.rpc.server function_name, call_dict, binary) ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ ERROR oslo_messaging.rpc.server self.force_reraise() ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/exception_wrapper.py", line 68, in wrapped ERROR oslo_messaging.rpc.server return f(self, context, *args, **kw) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/compute/manager.py", line 187, in decorated_function ERROR oslo_messaging.rpc.server LOG.warning(msg, e, instance=instance) ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ ERROR oslo_messaging.rpc.server self.force_reraise() ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/compute/manager.py", line 156, in decorated_function ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/compute/utils.py", line 660, in decorated_function ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/compute/manager.py", line 215, in decorated_function ERROR oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info()) ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ ERROR oslo_messaging.rpc.server self.force_reraise() ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/compute/manager.py", line 203, in decorated_function ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/compute/manager.py", line 3406, in rescue_instance ERROR oslo_messaging.rpc.server reason=_("Driver Error: %s") % e) ERROR oslo_messaging.rpc.server InstanceNotRescuable: Instance 4869b462-c3cf-4437-8c94-1d0dcd5fff8b cannot be rescued: Driver Error: failed to connect to monitor socket: No such process ERROR oslo_messaging.rpc.server This is the libvirt guest qemu log: http://logs.openstack.org/73/461473/3/check/gate-tempest-dsvm-neutron- full-ubuntu-xenial/a277636/logs/libvirt/qemu/instance-00000055.txt.gz This is where the guest failed to connect to the monitor in the libvirtd logs: http://logs.openstack.org/73/461473/3/check/gate-tempest-dsvm-neutron- full-ubuntu- xenial/a277636/logs/libvirt/libvirtd.txt.gz#_2017-05-02_17_49_05_050 Shortly before that: http://logs.openstack.org/73/461473/3/check/gate-tempest-dsvm-neutron- full-ubuntu- xenial/a277636/logs/libvirt/libvirtd.txt.gz#_2017-05-02_17_49_05_047 2017-05-02 17:49:05.047+0000: 1193: debug : processMonitorEOFEvent:4541 : Monitor connection to 'instance-00000055' closed without SHUTDOWN event; assuming the domain crashed ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1688038 Title: test_rescued_vm_add_remove_security_group fails with "InstanceNotRescuable: Instance 4869b462-c3cf-4437-8c94-1d0dcd5fff8b cannot be rescued: Driver Error: failed to connect to monitor socket: No such process" Status in OpenStack Compute (nova): New Bug description: http://logs.openstack.org/73/461473/3/check/gate-tempest-dsvm-neutron- full-ubuntu-xenial/a277636/console.html#_2017-05-02_18_16_55_064917 http://logs.openstack.org/73/461473/3/check/gate-tempest-dsvm-neutron- full-ubuntu- xenial/a277636/logs/screen-n-cpu.txt.gz#_May_02_17_49_07_771248 May 02 17:49:07.771248 ubuntu-xenial-rax-ord-8683720 nova-compute[23706]: ERROR oslo_messaging.rpc.server [req-f281fc56-b69f-4e1a-a6d0-752871138ace tempest-ServerRescueTestJSON-709146246 tempest-ServerRescueTestJSON-709146246] Exception during message handling ERROR oslo_messaging.rpc.server Traceback (most recent call last): ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 157, in _process_incoming ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/exception_wrapper.py", line 77, in wrapped ERROR oslo_messaging.rpc.server function_name, call_dict, binary) ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ ERROR oslo_messaging.rpc.server self.force_reraise() ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/exception_wrapper.py", line 68, in wrapped ERROR oslo_messaging.rpc.server return f(self, context, *args, **kw) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/compute/manager.py", line 187, in decorated_function ERROR oslo_messaging.rpc.server LOG.warning(msg, e, instance=instance) ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ ERROR oslo_messaging.rpc.server self.force_reraise() ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/compute/manager.py", line 156, in decorated_function ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/compute/utils.py", line 660, in decorated_function ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/compute/manager.py", line 215, in decorated_function ERROR oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info()) ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ ERROR oslo_messaging.rpc.server self.force_reraise() ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/compute/manager.py", line 203, in decorated_function ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/compute/manager.py", line 3406, in rescue_instance ERROR oslo_messaging.rpc.server reason=_("Driver Error: %s") % e) ERROR oslo_messaging.rpc.server InstanceNotRescuable: Instance 4869b462-c3cf-4437-8c94-1d0dcd5fff8b cannot be rescued: Driver Error: failed to connect to monitor socket: No such process ERROR oslo_messaging.rpc.server This is the libvirt guest qemu log: http://logs.openstack.org/73/461473/3/check/gate-tempest-dsvm-neutron- full-ubuntu-xenial/a277636/logs/libvirt/qemu/instance-00000055.txt.gz This is where the guest failed to connect to the monitor in the libvirtd logs: http://logs.openstack.org/73/461473/3/check/gate-tempest-dsvm-neutron- full-ubuntu- xenial/a277636/logs/libvirt/libvirtd.txt.gz#_2017-05-02_17_49_05_050 Shortly before that: http://logs.openstack.org/73/461473/3/check/gate-tempest-dsvm-neutron- full-ubuntu- xenial/a277636/logs/libvirt/libvirtd.txt.gz#_2017-05-02_17_49_05_047 2017-05-02 17:49:05.047+0000: 1193: debug : processMonitorEOFEvent:4541 : Monitor connection to 'instance-00000055' closed without SHUTDOWN event; assuming the domain crashed To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1688038/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp