Reviewed: https://review.openstack.org/290922 Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=2dc9d67f6e0fbec8272dc1dfdb594d05378536d5 Submitter: Jenkins Branch: master
commit 2dc9d67f6e0fbec8272dc1dfdb594d05378536d5 Author: Kevin Benton <ke...@benton.pub> Date: Wed Mar 9 16:08:06 2016 -0800 Construct exceptions before passing to retryrequest The IP allocation failure constructions weren't being constructed before being passed to RetryRequest so they would fail to be reraised after the retries were exhausted. This patch just constructs them and adjusts a few of the unit tests to execute the full retry life-cycle. Closes-Bug: #1555387 Change-Id: I1c1ea488886ee3f4790eaa53911ea5845f327334 ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1555387 Title: ipam driver not using retryrequest correctly Status in neutron: Fix Released Bug description: The IPAM driver is not passing a constructed exception into RetryRequest so the reraise on retry exhaustion fails: 2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource [req-6e0e721f-12df-4132-81ef-f1b96875cdd7 c_rally_3c9bcbbd_m5PryrFI -] create failed 2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource Traceback (most recent call last): 2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource File "/opt/stack/neutron/neutron/api/v2/resource.py", line 84, in resource 2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource result = method(request=request, **args) 2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource File "/opt/stack/neutron/neutron/api/v2/base.py", line 410, in create 2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource return self._create(request, body, **kwargs) 2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 148, in wrapper 2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource ectxt.value = e.inner_exc 2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource self.force_reraise() 2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb) 2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource File "<string>", line 2, in reraise 2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource TypeError: exceptions must be old-style classes or derived from BaseException, not type 2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api [req-d357eb93-048c-425b-b6de-f860be1d767a c_rally_3c9bcbbd_IA8UUZUc -] DB exceeded retry limit. 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api Traceback (most recent call last): 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 138, in wrapper 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api return f(*args, **kwargs) 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/opt/stack/neutron/neutron/api/v2/base.py", line 521, in _create 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api obj = do_create(body) 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/opt/stack/neutron/neutron/api/v2/base.py", line 503, in do_create 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api request.context, reservation.reservation_id) 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api self.force_reraise() 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api six.reraise(self.type_, self.value, self.tb) 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/opt/stack/neutron/neutron/api/v2/base.py", line 496, in do_create 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api return obj_creator(request.context, **kwargs) 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/opt/stack/neutron/neutron/db/l3_hamode_db.py", line 463, in create_router 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api self.delete_router(context, router_dict['id']) 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api self.force_reraise() 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api six.reraise(self.type_, self.value, self.tb) 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/opt/stack/neutron/neutron/db/l3_hamode_db.py", line 457, in create_router 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api context, router_db)[1] 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/opt/stack/neutron/neutron/db/l3_hamode_db.py", line 443, in _create_ha_interfaces_and_ensure_network 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api creator, dep_getter, dep_creator, dep_id_attr) 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/opt/stack/neutron/neutron/common/utils.py", line 707, in create_object_with_dependency 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api ctx.reraise = False 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api self.force_reraise() 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api six.reraise(self.type_, self.value, self.tb) 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/opt/stack/neutron/neutron/common/utils.py", line 696, in create_object_with_dependency 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api result = creator(dependency) 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/opt/stack/neutron/neutron/db/l3_hamode_db.py", line 394, in _create_ha_interfaces 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api l3_port_check=False) 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api self.force_reraise() 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api six.reraise(self.type_, self.value, self.tb) 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/opt/stack/neutron/neutron/db/l3_hamode_db.py", line 388, in _create_ha_interfaces 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api router.tenant_id) 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/opt/stack/neutron/neutron/db/l3_hamode_db.py", line 375, in add_ha_port 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api deletion, content) 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/opt/stack/neutron/neutron/db/common_db_mixin.py", line 55, in safe_creation 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api obj = create_fn() 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/opt/stack/neutron/neutron/plugins/common/utils.py", line 161, in create_port 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api return core_plugin.create_port(context, {'port': port_data}) 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/opt/stack/neutron/neutron/plugins/ml2/plugin.py", line 1123, in create_port 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api result, mech_context = self._create_port_db(context, port) 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/opt/stack/neutron/neutron/plugins/ml2/plugin.py", line 1094, in _create_port_db 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api result = super(Ml2Plugin, self).create_port(context, port) 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/opt/stack/neutron/neutron/db/db_base_plugin_v2.py", line 1237, in create_port 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api port_id) 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/opt/stack/neutron/neutron/db/ipam_pluggable_backend.py", line 179, in allocate_ips_for_port_and_store 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api revert_on_fail=False) 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api self.force_reraise() 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api six.reraise(self.type_, self.value, self.tb) 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/opt/stack/neutron/neutron/db/ipam_pluggable_backend.py", line 163, in allocate_ips_for_port_and_store 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api ips = self._allocate_ips_for_port(context, port_copy) 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/opt/stack/neutron/neutron/db/ipam_pluggable_backend.py", line 235, in _allocate_ips_for_port 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api return self._ipam_allocate_ips(context, ipam_driver, p, ips) 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/opt/stack/neutron/neutron/db/ipam_pluggable_backend.py", line 135, in _ipam_allocate_ips 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api "external system for %s"), addresses) 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api self.force_reraise() 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api six.reraise(self.type_, self.value, self.tb) 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/opt/stack/neutron/neutron/db/ipam_pluggable_backend.py", line 120, in _ipam_allocate_ips 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api context, ipam_driver, port, ip_list) 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/opt/stack/neutron/neutron/db/ipam_pluggable_backend.py", line 91, in _ipam_allocate_single_ip 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api port, subnet), 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/opt/stack/neutron/neutron/db/ipam_pluggable_backend.py", line 80, in _ipam_try_allocate_ip 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api return ipam_subnet.allocate(ip_request) 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/opt/stack/neutron/neutron/ipam/drivers/neutrondb_ipam/driver.py", line 350, in allocate 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api auto_generated) 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/opt/stack/neutron/neutron/ipam/drivers/neutrondb_ipam/driver.py", line 219, in _allocate_specific_ip 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api session, db_range, first_ip=first_ip, last_ip=last_ip) 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api File "/opt/stack/neutron/neutron/ipam/drivers/neutrondb_ipam/db_api.py", line 167, in update_range 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api raise db_exc.RetryRequest(ipam_exc.IPAllocationFailed) 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api RetryRequest 2016-03-09 16:06:10.210 98260 ERROR oslo_db.api 2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource [req-d357eb93-048c-425b-b6de-f860be1d767a c_rally_3c9bcbbd_IA8UUZUc -] create failed 2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource Traceback (most recent call last): 2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource File "/opt/stack/neutron/neutron/api/v2/resource.py", line 84, in resource 2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource result = method(request=request, **args) 2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource File "/opt/stack/neutron/neutron/api/v2/base.py", line 410, in create 2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource return self._create(request, body, **kwargs) 2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 148, in wrapper 2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource ectxt.value = e.inner_exc 2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource self.force_reraise() 2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb) 2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource File "<string>", line 2, in reraise 2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource TypeError: exceptions must be old-style classes or derived from BaseException, not type 2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1555387/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp