Public bug reported:
If a VM with a floating IP is deleted while concurrently deleting the
floating IP (i.e. "nova delete" and "neutron floatingip-delete") the VM
may go into error state.
Nova receives a 500 error code from neutron because neutron fails to
delete the port and the following stack is observed:
2015-11-04 21:48:37.447 17275 ERROR neutron.api.v2.resource
[req-764172bc-4938-440d-9e6a-dd4c0f120f1b ] delete failed
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource Traceback (most
recent call last):
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/neutron/api/v2/resource.py", line 83,
in resource
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource result =
method(request=request, **args)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/neutron/api/v2/base.py", line 490, in
delete
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource
obj_deleter(request.context, id, **kwargs)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/oslo_db/api.py", line 131, in wrapper
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource return f(*args,
**kwargs)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/neutron/plugins/ml2/plugin.py", line
1272, in delete_port
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource context, id,
do_notify=False)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/neutron/db/l3_dvr_db.py", line 274,
in disassociate_floatingips
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource
do_notify=do_notify)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/neutron/db/l3_db.py", line 1361, in
disassociate_floatingips
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource context,
port_id)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/neutron/db/l3_db.py", line 1088, in
disassociate_floatingips
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource 'router_id':
None})
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 483,
in __exit__
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource self.rollback()
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line
60, in __exit__
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource
compat.reraise(exc_type, exc_value, exc_tb)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 480,
in __exit__
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource self.commit()
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 385,
in commit
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource
self._prepare_impl()
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 365,
in _prepare_impl
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource
self.session.flush()
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line
1986, in flush
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource
self._flush(objects)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line
2104, in _flush
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource
transaction.rollback(_capture_exception=True)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line
60, in __exit__
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource
compat.reraise(exc_type, exc_value, exc_tb)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line
2068, in _flush
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource
flush_context.execute()
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line
372, in execute
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource
rec.execute(self)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line
526, in execute
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource uow
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line
61, in save_obj
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource mapper, table,
update)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line
537, in _emit_update_statements
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource
(table.description, len(update), rows))
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource StaleDataError:
UPDATE statement on table 'floatingips' expected to update 1 row(s); 0 were
matched.
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource
Version: stable/kilo
** Affects: neutron
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1513242
Title:
StaleDataError in disassociate_floatingips
Status in neutron:
New
Bug description:
If a VM with a floating IP is deleted while concurrently deleting the
floating IP (i.e. "nova delete" and "neutron floatingip-delete") the
VM may go into error state.
Nova receives a 500 error code from neutron because neutron fails to
delete the port and the following stack is observed:
2015-11-04 21:48:37.447 17275 ERROR neutron.api.v2.resource
[req-764172bc-4938-440d-9e6a-dd4c0f120f1b ] delete failed
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource Traceback (most
recent call last):
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/neutron/api/v2/resource.py", line 83,
in resource
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource result =
method(request=request, **args)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/neutron/api/v2/base.py", line 490, in
delete
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource
obj_deleter(request.context, id, **kwargs)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/oslo_db/api.py", line 131, in wrapper
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource return
f(*args, **kwargs)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/neutron/plugins/ml2/plugin.py", line
1272, in delete_port
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource context, id,
do_notify=False)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/neutron/db/l3_dvr_db.py", line 274,
in disassociate_floatingips
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource
do_notify=do_notify)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/neutron/db/l3_db.py", line 1361, in
disassociate_floatingips
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource context,
port_id)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/neutron/db/l3_db.py", line 1088, in
disassociate_floatingips
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource 'router_id':
None})
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 483,
in __exit__
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource
self.rollback()
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line
60, in __exit__
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource
compat.reraise(exc_type, exc_value, exc_tb)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 480,
in __exit__
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource self.commit()
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 385,
in commit
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource
self._prepare_impl()
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 365,
in _prepare_impl
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource
self.session.flush()
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line
1986, in flush
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource
self._flush(objects)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line
2104, in _flush
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource
transaction.rollback(_capture_exception=True)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line
60, in __exit__
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource
compat.reraise(exc_type, exc_value, exc_tb)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line
2068, in _flush
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource
flush_context.execute()
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line
372, in execute
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource
rec.execute(self)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line
526, in execute
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource uow
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line
61, in save_obj
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource mapper,
table, update)
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource File
"/opt/neutron/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line
537, in _emit_update_statements
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource
(table.description, len(update), rows))
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource StaleDataError:
UPDATE statement on table 'floatingips' expected to update 1 row(s); 0 were
matched.
2015-11-04 21:48:37.447 17275 TRACE neutron.api.v2.resource
Version: stable/kilo
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1513242/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
