Public bug reported: neutron-vpn-agent can generate stack traces with AttributeError: 'NoneType' object has no attribute 'ipv4' in two different locations in ipsec.py, in add_nat_rule() and remove_nat_rule(), during sync() operations while site connections are being created.
Here is an example stack trace (based on a Liberty distribution, but I believe this is still an issue in master/newton): 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last): 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/venv/neutron-20160426T025546Z/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher executor_callback)) 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/venv/neutron-20160426T025546Z/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher executor_callback) 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/venv/neutron-20160426T025546Z/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 129, in _do_dispatch 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args) 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/venv/neutron-20160426T025546Z/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/ipsec.py", line 675, in vpnservice_updated 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher self.sync(context, [router] if router else []) 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/venv/neutron-20160426T025546Z/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher return f(*args, **kwargs) 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/venv/neutron-20160426T025546Z/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/ipsec.py", line 830, in sync 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher self._delete_vpn_processes(sync_router_ids, router_ids) 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/venv/neutron-20160426T025546Z/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/ipsec.py", line 860, in _delete_vpn_processes 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher self.destroy_process(process_id) 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/venv/neutron-20160426T025546Z/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/ipsec.py", line 728, in destroy_process 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher self._update_nat(vpnservice, self.remove_nat_rule) 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/venv/neutron-20160426T025546Z/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/ipsec.py", line 664, in _update_nat 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher top=True) 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/venv/neutron-20160426T025546Z/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/ipsec.py", line 629, in remove_nat_rule 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher iptables_manager.ipv4['nat'].remove_rule(chain, rule, top=top) 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher AttributeError: 'NoneType' object has no attribute 'ipv4' 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher ** Affects: neutron Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1575402 Title: VPNaaS update NAT rules can generate stack trace Status in neutron: New Bug description: neutron-vpn-agent can generate stack traces with AttributeError: 'NoneType' object has no attribute 'ipv4' in two different locations in ipsec.py, in add_nat_rule() and remove_nat_rule(), during sync() operations while site connections are being created. Here is an example stack trace (based on a Liberty distribution, but I believe this is still an issue in master/newton): 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last): 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/venv/neutron-20160426T025546Z/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher executor_callback)) 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/venv/neutron-20160426T025546Z/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher executor_callback) 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/venv/neutron-20160426T025546Z/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 129, in _do_dispatch 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args) 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/venv/neutron-20160426T025546Z/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/ipsec.py", line 675, in vpnservice_updated 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher self.sync(context, [router] if router else []) 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/venv/neutron-20160426T025546Z/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher return f(*args, **kwargs) 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/venv/neutron-20160426T025546Z/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/ipsec.py", line 830, in sync 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher self._delete_vpn_processes(sync_router_ids, router_ids) 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/venv/neutron-20160426T025546Z/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/ipsec.py", line 860, in _delete_vpn_processes 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher self.destroy_process(process_id) 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/venv/neutron-20160426T025546Z/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/ipsec.py", line 728, in destroy_process 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher self._update_nat(vpnservice, self.remove_nat_rule) 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/venv/neutron-20160426T025546Z/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/ipsec.py", line 664, in _update_nat 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher top=True) 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/venv/neutron-20160426T025546Z/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/ipsec.py", line 629, in remove_nat_rule 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher iptables_manager.ipv4['nat'].remove_rule(chain, rule, top=top) 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher AttributeError: 'NoneType' object has no attribute 'ipv4' 2016-04-26 20:26:27.894 28022 ERROR oslo_messaging.rpc.dispatcher To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1575402/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp