** Changed in: neutron/kilo Status: Fix Committed => Fix Released
** Tags removed: kilo-rc-potential -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1444201 Title: change in ipset elements breaks agent Status in OpenStack Neutron (virtual network service): Fix Committed Status in neutron kilo series: Fix Released Bug description: If an ipset element count changes (e.g. due to https://review.openstack.org/#/c/170328/), the previous elements will cause the create command to fail even though the -exist flag is being passed. This prevents the agent from setting up the ipsets correctly. Example exception: 015-03-30 15:31:40.887 DEBUG oslo_concurrency.lockutils [req-13ed77d8-6224-44ba-a2a9-becaa991663c None None] Releasing file lock "/opt/stack/data/neutron/lock/neutron- ipset" after holding it for 0.005s release /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:227 2015-03-30 15:31:40.888 DEBUG oslo_concurrency.lockutils [req-13ed77d8-6224-44ba-a2a9-becaa991663c None None] Lock "ipset" released by "set_members" :: held 0.005s inne r /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:456 2015-03-30 15:31:40.888 ERROR neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-13ed77d8-6224-44ba-a2a9-becaa991663c None None] Error while processing VIF ports 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Traceback (most recent call last): 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent File "/opt/stack/neutron/neutron/plugins/openvswitch/agent/ovs_neutron_agent.p y", line 1586, in rpc_loop 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent ovs_restarted) 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent File "/opt/stack/neutron/neutron/plugins/openvswitch/agent/ovs_neutron_agent.p y", line 1350, in process_network_ports 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent port_info.get('updated', set())) 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent File "/opt/stack/neutron/neutron/agent/securitygroups_rpc.py", line 360, in se tup_port_filters 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent self.prepare_devices_filter(new_devices) 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent File "/opt/stack/neutron/neutron/agent/securitygroups_rpc.py", line 219, in de corated_function 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent *args, **kwargs) 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent File "/opt/stack/neutron/neutron/agent/securitygroups_rpc.py", line 244, in pr epare_devices_filter 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent security_groups, security_group_member_ips) 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__ 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent self.gen.next() 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent File "/opt/stack/neutron/neutron/agent/firewall.py", line 106, in defer_apply 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent self.filter_defer_apply_off() 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent File "/opt/stack/neutron/neutron/agent/linux/iptables_firewall.py", line 659, in filter_defer_apply_off 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent self.unfiltered_ports) 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent File "/opt/stack/neutron/neutron/agent/linux/iptables_firewall.py", line 155, in _setup_chains_apply 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent self._setup_chain(port, INGRESS_DIRECTION) 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent File "/opt/stack/neutron/neutron/agent/linux/iptables_firewall.py", line 182, in _setup_chain 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent self._add_rules_by_security_group(port, DIRECTION) 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent File "/opt/stack/neutron/neutron/agent/linux/iptables_firewall.py", line 411, in _add_rules_by_security_group 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent self._update_ipset_members(remote_sg_ids) 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent File "/opt/stack/neutron/neutron/agent/linux/iptables_firewall.py", line 448, in _update_ipset_members 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent self.ipset.set_members(sg_id, ip_version, current_ips) 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 445, in inner 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent return f(*args, **kwargs) 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent File "/opt/stack/neutron/neutron/agent/linux/ipset_manager.py", line 74, in set_members 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent self._create_set(set_name, ethertype) 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent File "/opt/stack/neutron/neutron/agent/linux/ipset_manager.py", line 123, in _create_set 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent self._apply(cmd) 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent File "/opt/stack/neutron/neutron/agent/linux/ipset_manager.py", line 132, in _apply 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent self.execute(cmd_ns, run_as_root=True, process_input=input) 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent File "/opt/stack/neutron/neutron/agent/linux/utils.py", line 137, in execute 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent raise RuntimeError(m) 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent RuntimeError: 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Command: ['ipset', 'create', '-exist', u'IPv45d5e4955-d64c-40af-853e', 'hash:ip', 'family', 'inet', 'hashsize', '2048', 'maxelem', '131072'] 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Exit code: 1 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Stdin: 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Stdout: 2015-03-30 15:31:40.888 32755 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Stderr: ipset v6.20.1: Set cannot be created: set with the same name already exists To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1444201/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp