Public bug reported:
I am seeing a behavior, while deleting nova instance it returns status as an
Error and fails to delete itself.
So scenario, where I see this behavior is as follow:
Create a nova VM instance.
Create a cinder volume
Attach this volume to nova vm instance and after that wait for volume to go to
‘In-use’ state.
Mount the created partition to VM
Unmount the created partition
Detach the volume from VM instance and after that wait for volume to go to
‘Available’ state
Delete the volume
Delete the nova VM instance ==> It fails at this step, because status of server
instance is set to Error.
Based on logs I also see Rabbitmq error, which might explain the problem.
3908 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
Traceback (most r ecent call last):
3909 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/
python2.7/dist-packages/oslo/messaging/_drivers/impl_rabbit.py", line 648, in
ensure
3910 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
return method (*args, **kwargs)
3911 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/
python2.7/dist-packages/oslo/messaging/_drivers/impl_rabbit.py", line 753, in
_publish
3912 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
publisher = c ls(self.conf, self.channel, topic, **kwargs)
3913 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/
python2.7/dist-packages/oslo/messaging/_drivers/impl_rabbit.py", line 420, in
__init__
3914 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
super(NotifyP ublisher, self).__init__(conf, channel, topic, **kwargs)
3915 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/
python2.7/dist-packages/oslo/messaging/_drivers/impl_rabbit.py", line 396, in
__init__
3916 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
**options)
3917 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/
python2.7/dist-packages/oslo/messaging/_drivers/impl_rabbit.py", line 339, in
__init__
3918 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
self.reconnec t(channel)
2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit File
"/usr/lib/ python2.7/dist-packages/oslo/messaging/_drivers/impl_rabbit.py",
line 423, in reconnect
3920 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
super(NotifyP ublisher, self).reconnect(channel)
3921 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/
python2.7/dist-packages/oslo/messaging/_drivers/impl_rabbit.py", line 347, in
reconnect
3922 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
routing_key=s elf.routing_key)
3923 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/ python2.7/dist-packages/kombu/messaging.py", line 84, in
__init__
3924 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
self.revive(s elf._channel)
3925 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/ python2.7/dist-packages/kombu/messaging.py", line 218, in
revive
3926 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
self.declare( )
3927 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/ python2.7/dist-packages/kombu/messaging.py", line 104, in
declare
3928 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
self.exchange .declare()
3929 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/ python2.7/dist-packages/kombu/entity.py", line 166, in
declare
3930 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
nowait=nowait , passive=passive,
3931 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/ python2.7/dist-packages/amqp/channel.py", line 613, in
exchange_declare
3932 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
self._send_me thod((40, 10), args)
3933 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/ python2.7/dist-packages/amqp/abstract_channel.py", line 56,
in _send_method
3934 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
self.channel_ id, method_sig, args, content,
3935 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/ python2.7/dist-packages/amqp/method_framing.py", line 221,
in write_method
3936 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
write_frame(1 , channel, payload)
3937 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/ python2.7/dist-packages/amqp/transport.py", line 177, in
write_frame
3938 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
frame_type, c hannel, size, payload, 0xce,
3939 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/ python2.7/dist-packages/eventlet/greenio.py", line 307, in
sendall
3940 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
tail = self.s end(data, flags)
3941 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/ python2.7/dist-packages/eventlet/greenio.py", line 293, in
send
3942 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
total_sent += fd.send(data[total_sent:], flags)
3943 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
error: [Errno 32] Broken pipe
3944 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
3945 <182>Sep 29 16:14:57 node-6 nova-api 2014-09-29 16:14:57.084 29278 INFO
oslo.messaging._d rivers.impl_rabbit
[req-e1e3acf2-2e42-40bb-bcb9-839151099a96 ] Reconnecting to AMQP serve r on
192.168.9.5:5672
** Affects: nova
Importance: Undecided
Status: New
** Attachment added: "Nova log"
https://bugs.launchpad.net/bugs/1375408/+attachment/4219462/+files/nova-all.log
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1375408
Title:
nova instance delete issue
Status in OpenStack Compute (Nova):
New
Bug description:
I am seeing a behavior, while deleting nova instance it returns status as an
Error and fails to delete itself.
So scenario, where I see this behavior is as follow:
Create a nova VM instance.
Create a cinder volume
Attach this volume to nova vm instance and after that wait for volume to go
to ‘In-use’ state.
Mount the created partition to VM
Unmount the created partition
Detach the volume from VM instance and after that wait for volume to go to
‘Available’ state
Delete the volume
Delete the nova VM instance ==> It fails at this step, because status of
server instance is set to Error.
Based on logs I also see Rabbitmq error, which might explain the problem.
3908 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
Traceback (most r ecent call last):
3909 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/
python2.7/dist-packages/oslo/messaging/_drivers/impl_rabbit.py", line 648, in
ensure
3910 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
return method (*args, **kwargs)
3911 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/
python2.7/dist-packages/oslo/messaging/_drivers/impl_rabbit.py", line 753, in
_publish
3912 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
publisher = c ls(self.conf, self.channel, topic, **kwargs)
3913 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/
python2.7/dist-packages/oslo/messaging/_drivers/impl_rabbit.py", line 420, in
__init__
3914 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
super(NotifyP ublisher, self).__init__(conf, channel, topic, **kwargs)
3915 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/
python2.7/dist-packages/oslo/messaging/_drivers/impl_rabbit.py", line 396, in
__init__
3916 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
**options)
3917 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/
python2.7/dist-packages/oslo/messaging/_drivers/impl_rabbit.py", line 339, in
__init__
3918 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
self.reconnec t(channel)
2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/
python2.7/dist-packages/oslo/messaging/_drivers/impl_rabbit.py", line 423, in
reconnect
3920 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
super(NotifyP ublisher, self).reconnect(channel)
3921 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/
python2.7/dist-packages/oslo/messaging/_drivers/impl_rabbit.py", line 347, in
reconnect
3922 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
routing_key=s elf.routing_key)
3923 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/ python2.7/dist-packages/kombu/messaging.py", line 84, in
__init__
3924 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
self.revive(s elf._channel)
3925 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/ python2.7/dist-packages/kombu/messaging.py", line 218, in
revive
3926 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
self.declare( )
3927 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/ python2.7/dist-packages/kombu/messaging.py", line 104, in
declare
3928 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
self.exchange .declare()
3929 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/ python2.7/dist-packages/kombu/entity.py", line 166, in
declare
3930 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
nowait=nowait , passive=passive,
3931 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/ python2.7/dist-packages/amqp/channel.py", line 613, in
exchange_declare
3932 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
self._send_me thod((40, 10), args)
3933 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/ python2.7/dist-packages/amqp/abstract_channel.py", line
56, in _send_method
3934 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
self.channel_ id, method_sig, args, content,
3935 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/ python2.7/dist-packages/amqp/method_framing.py", line 221,
in write_method
3936 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
write_frame(1 , channel, payload)
3937 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/ python2.7/dist-packages/amqp/transport.py", line 177, in
write_frame
3938 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
frame_type, c hannel, size, payload, 0xce,
3939 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/ python2.7/dist-packages/eventlet/greenio.py", line 307, in
sendall
3940 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
tail = self.s end(data, flags)
3941 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
File "/usr/lib/ python2.7/dist-packages/eventlet/greenio.py", line 293, in
send
3942 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
total_sent += fd.send(data[total_sent:], flags)
3943 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
error: [Errno 32] Broken pipe
3944 2014-09-29 16:14:57.080 29278 TRACE oslo.messaging._drivers.impl_rabbit
3945 <182>Sep 29 16:14:57 node-6 nova-api 2014-09-29 16:14:57.084 29278 INFO
oslo.messaging._d rivers.impl_rabbit
[req-e1e3acf2-2e42-40bb-bcb9-839151099a96 ] Reconnecting to AMQP serve r on
192.168.9.5:5672
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1375408/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
