from:"Xavier León"

Re: [openstack-dev] [neutron] iptables routes are not being injected to router namespace

2015-01-23 Thread Xavier León

On Thu, Jan 22, 2015 at 10:57 PM, Brian Haley  wrote:
> On 01/22/2015 02:35 PM, Kevin Benton wrote:
>> Right, there are two bugs here. One is in whatever went wrong with 
>> defer_apply
>> and one is with this exception handling code. I would allow the fix to go in 
>> for
>> the exception handling and then file another bug for the actual underlying
>> defer_apply bug.
>
> What went wrong with defer_apply() was caused by oslo.concurrency - version
> 1.4.1 seems to fix the problem, see https://review.openstack.org/#/c/149400/
> (thanks Ihar!)
>
> Xavier - can you update your oslo.concurrency to that version and verify it
> helps?  It seems to work in my config.

Yes. Updating to oslo.concurrency 1.4.1 fixed this problem.

Thanks,
Xavi

>
> Then the change in the other patchset could be applied, along with a test that
> triggers exceptions so this gets caught.
>
> Thanks,
>
> -Brian
>
>> On Thu, Jan 22, 2015 at 10:32 AM, Brian Haley > > wrote:
>>
>> On 01/22/2015 01:06 PM, Kevin Benton wrote:
>> > There was a bug for this already.
>> > https://bugs.launchpad.net/bugs/1413111
>>
>> Thanks Kevin.  I added more info to it, but don't think the patch 
>> proposed there
>> is correct.  Something in the iptables manager defer_apply() code isn't
>> quite right.
>>
>> -Brian
>>
>>
>> > On Thu, Jan 22, 2015 at 9:07 AM, Brian Haley > 
>> > >> wrote:
>> >
>> > On 01/22/2015 10:17 AM, Carl Baldwin wrote:
>> > > I think this warrants a bug report.  Could you file one with 
>> what you
>> > > know so far?
>> >
>> > Carl,
>> >
>> > Seems as though a recent change introduced a bug.  This is on a 
>> devstack
>> > I just created today, at l3/vpn-agent startup:
>> >
>> > 2015-01-22 11:55:07.961 4203 TRACE neutron.agent.l3.agent 
>> Traceback (most
>> > recent call last):
>> > 2015-01-22 11:55:07.961 4203 TRACE neutron.agent.l3.agent   File
>> > "/opt/stack/neutron/neutron/common/utils.py", line 342, in call
>> > 2015-01-22 11:55:07.961 4203 TRACE neutron.agent.l3.agent 
>> return
>> > func(*args, **kwargs)
>> > 2015-01-22 11:55:07.961 4203 TRACE neutron.agent.l3.agent   File
>> > "/opt/stack/neutron/neutron/agent/l3/agent.py", line 584, in
>> process_router
>> > 2015-01-22 11:55:07.961 4203 TRACE neutron.agent.l3.agent
>> >  self._process_external(ri)
>> > 2015-01-22 11:55:07.961 4203 TRACE neutron.agent.l3.agent   File
>> > "/opt/stack/neutron/neutron/agent/l3/agent.py", line 576, in
>> _process_external
>> > 2015-01-22 11:55:07.961 4203 TRACE neutron.agent.l3.agent
>> >  self._update_fip_statuses(ri, existing_floating_ips, fip_statuses)
>> > 2015-01-22 11:55:07.961 4203 TRACE neutron.agent.l3.agent
>> UnboundLocalError:
>> > local variable 'existing_floating_ips' referenced before assignment
>> > 2015-01-22 11:55:07.961 4203 TRACE neutron.agent.l3.agent
>> > Traceback (most recent call last):
>> >   File 
>> "/usr/local/lib/python2.7/dist-packages/eventlet/greenpool.py",
>> line
>> > 82, in _spawn_n_impl
>> > func(*args, **kwargs)
>> >   File "/opt/stack/neutron/neutron/agent/l3/agent.py", line 1093, 
>> in
>> > _process_router_update
>> > self._process_router_if_compatible(router)
>> >   File "/opt/stack/neutron/neutron/agent/l3/agent.py", line 1047, 
>> in
>> > _process_router_if_compatible
>> > self._process_added_router(router)
>> >   File "/opt/stack/neutron/neutron/agent/l3/agent.py", line 1056, 
>> in
>> > _process_added_router
>> > self.process_router(ri)
>> >   File "/opt/stack/neutron/neutron/common/utils.py", line 345, in 
>> call
>> > self.logger(e)
>> >   File
>> "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line
>> > 82, in __exit__
>> > six.reraise(self.type_, self.value, self.tb)
>> >   File "/opt/stack/neutron/neutron/common/utils.py", line 342, in 
>> call
>> > return func(*args, **kwargs)
>> >   File "/opt/stack/neutron/neutron/agent/l3/agent.py", line 584, in
>> > process_router
>> > self._process_external(ri)
>> >   File "/opt/stack/neutron/neutron/agent/l3/agent.py", line 576, in
>> > _process_external
>> > self._update_fip_statuses(ri, existing_floating_ips, 
>> fip_statuses)
>> > UnboundLocalError: local variable 'existing_floating_ips' 
>> referenced
>> before
>> > assignment
>> >
>> > Since that's happening while we're holding the iptables lock I'm 
>> assuming
>> > no rules are being applied.
>> >
>> > I'm looking into it now, w

Re: [openstack-dev] [neutron] iptables routes are not being injected to router namespace

2015-01-21 Thread Xavier León

On Tue, Jan 20, 2015 at 10:32 PM, Brian Haley  wrote:
> On 01/20/2015 09:20 AM, Xavier León wrote:
>> Hi all,
>>
>> we've been doing some tests with openstack kilo and found
>> out a problem: iptables routes are not being injected to the
>> router namespace.
>>
>> Scenario:
>> - a private network NOT connected to the outside world.
>> - a router with only one interface connected to the private network.
>> - a vm instance connected to the private network as well.
>>
>> From inside the instance, we try to get some information from the
>> metadata service with curl:
>>
>> $ curl http://169.254.169.254
>> curl: (7) couldn't connect to host
>>
>> With the same set up in juno, there was no such problem and
>> metadata information is shown.
>>
>> The request is not filtered at the instance and hits the router
>> namespace (checked with tcpdump). However, when looking
>> from the controller at the iptables rules at the router, they appear
>> empty.
>>
>> stack@devstack: ~$ sudo ip netns exec
>> qrouter-d4ec737a-c5fb-4f5b-8bd0-1b5353bbade3 iptables-save
> 
>
>> # Generated by iptables-save v1.4.21 on Tue Jan 20 14:05:48 2015
>> *filter
>> :INPUT ACCEPT [5:914]
>> :FORWARD ACCEPT [0:0]
>> :OUTPUT ACCEPT [10:868]
>> COMMIT
>
> Are you sure the l3-agent is running?  You should have seen wrapped rules from
> it in most of these tables, for example:
>
> # Generated by iptables-save v1.4.21 on Tue Jan 20 16:29:19 2015
> *filter
> :INPUT ACCEPT [34:10882]
> :FORWARD ACCEPT [0:0]
> :OUTPUT ACCEPT [1:84]
> :neutron-filter-top - [0:0]
> :neutron-l3-agent-FORWARD - [0:0]
> :neutron-l3-agent-INPUT - [0:0]
> :neutron-l3-agent-OUTPUT - [0:0]
> :neutron-l3-agent-local - [0:0]
> [...]

Yes, the l3-agent is up and running. I see these rules when executing
the same test in juno but not in kilo. FYI, it's a all-in-one devstack
deployment.

>
> I would check the log files for any errors.

There are no errors in the logs.

After digging a bit more, we have seen that setting the config value
of enable_isolated_metadata to True (default: False) in dhcp_agent.ini
solves the problem in our scenario.
However, this change in configuration was not necessary before (our
tests passed in juno for that matter with that setting to False). So
we were wondering if there has been a change in how the metadata
service is accessed in such scenarios, a new issue because of the l3
agent refactoring or any other problem in our setup we haven't
narrowed yet.

Cheers,
Xavi

>
> -Brian
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron] iptables routes are not being injected to router namespace

2015-01-20 Thread Xavier León

Hi all,

we've been doing some tests with openstack kilo and found
out a problem: iptables routes are not being injected to the
router namespace.

Scenario:
- a private network NOT connected to the outside world.
- a router with only one interface connected to the private network.
- a vm instance connected to the private network as well.

>From inside the instance, we try to get some information from the
metadata service with curl:

$ curl http://169.254.169.254
curl: (7) couldn't connect to host

With the same set up in juno, there was no such problem and
metadata information is shown.

The request is not filtered at the instance and hits the router
namespace (checked with tcpdump). However, when looking
from the controller at the iptables rules at the router, they appear
empty.

stack@devstack: ~$ sudo ip netns exec
qrouter-d4ec737a-c5fb-4f5b-8bd0-1b5353bbade3 iptables-save
# Generated by iptables-save v1.4.21 on Tue Jan 20 14:05:48 2015
*raw
:PREROUTING ACCEPT [12:1334]
:OUTPUT ACCEPT [10:868]
COMMIT
# Completed on Tue Jan 20 14:05:48 2015
# Generated by iptables-save v1.4.21 on Tue Jan 20 14:05:48 2015
*nat
:PREROUTING ACCEPT [10:913]
:INPUT ACCEPT [3:493]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
COMMIT
# Completed on Tue Jan 20 14:05:48 2015
# Generated by iptables-save v1.4.21 on Tue Jan 20 14:05:48 2015
*mangle
:PREROUTING ACCEPT [12:1334]
:INPUT ACCEPT [5:914]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [10:868]
:POSTROUTING ACCEPT [10:868]
COMMIT
# Completed on Tue Jan 20 14:05:48 2015
# Generated by iptables-save v1.4.21 on Tue Jan 20 14:05:48 2015
*filter
:INPUT ACCEPT [5:914]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [10:868]
COMMIT

Is this some problem related to the refactoring of the l3 agent?
Any pointer to what might be the problem here? I can provide
more information on the subject if necessary to reproduce this.
Any input would be appreciated.

Cheers,
Xavi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] iptables routes are not being injected to router namespace

Re: [openstack-dev] [neutron] iptables routes are not being injected to router namespace

[openstack-dev] [neutron] iptables routes are not being injected to router namespace

3 matches

Site Navigation

Mail list logo

Footer information