Re: [Openstack-operators] Neutron DHCP agent local routes

2016-04-25 Thread Neil Jerram 
On 22/04/16 12:22, Dan Sneddon wrote:
> Yes, there is an unfortunate possibility of confusion around the word
> 'subnet'. To some, that means a network segment, or VLAN, and to some
> that just means the specific IP subnet. Neutron takes the latter view,
> and treats multiple subnets as multiple IP ranges on the same VLAN
> (which Neutron calls a 'network').
>
> So if you add multiple subnets to a network, then it is assumed that
> means multiple subnets on one physical network, or VLAN (even when
> using VXLAN or GRE, there is an internal VLAN segment ID).

Agreed.  Prior to the 'routed networks' work, a Neutron Network by 
definition provides L2 adjacency between all the instances that are 
attached to that Network.  Given that, I think the interpretation is 
unavoidable that multiple subnets are just IP ranges that are all 
directly connected.  And hence the host_routes code that I cited is correct.

> There is ongoing work to address the use case where you have a set of
> networks that belongs to one tenant, but they are different subnets on
> different VLANs with routing between them. See the 'routed networks'
> work by Carl Baldwin et. al. That seems to be what the OP was going for
> by adding multiple subnets to one network.

With routed networks, though, the situation changes.  When a Network has 
'l2_adjacency = False', instances with IPs from different subnets are 
probably not directly connected.  Hence, once the l2_adjacency field 
exists, I think we will want to conditionalize that code on it, i.e.:

 if self.network.l2_adjacency and subnet.ip_version == 4:
 host_routes.extend(["%s,0.0.0.0" % (s.cidr) for s in
 self.network.subnets
 if (s.ip_version == 4 and
 s.cidr != subnet.cidr)])

Regards,
Neil


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Neutron DHCP agent local routes

2016-04-22 Thread Dan Sneddon 
Yes, there is an unfortunate possibility of confusion around the word
'subnet'. To some, that means a network segment, or VLAN, and to some
that just means the specific IP subnet. Neutron takes the latter view,
and treats multiple subnets as multiple IP ranges on the same VLAN
(which Neutron calls a 'network').

So if you add multiple subnets to a network, then it is assumed that
means multiple subnets on one physical network, or VLAN (even when
using VXLAN or GRE, there is an internal VLAN segment ID).

There is ongoing work to address the use case where you have a set of
networks that belongs to one tenant, but they are different subnets on
different VLANs with routing between them. See the 'routed networks'
work by Carl Baldwin et. al. That seems to be what the OP was going for
by adding multiple subnets to one network.

Alternately, the desired end state can probably be achieved by simply
using one subnet per network, but using a separate network for each
192.168.10.x, 192.168.11.x, and 192.168.12.x.

-- 
Dan Sneddon |  Principal OpenStack Engineer
dsned...@redhat.com |  redhat.com/openstack
650.254.4025|  dsneddon:irc   @dxs:twitter

On 04/22/2016 05:49 AM, Kevin Benton wrote:
> But if the subnets are on the same VLAN it should work. That's what
> that patch is designed for. There is (currently) an assumption in
> Neutron that subnets in the same network are on the same L2 segment. So
> if a VM wants to communicate with a VM in another subnet on the same
> network, it can cut out the router and use the direct route by ARPing
> for the VM on its own.
> 
> The default is for VMs to only get an IP from one of the subnets on a
> multi subnet network. So if I understand your topology, it exactly
> matches what that patch intended to address (eliminate the hairpinning
> via the router for same network communication).
> 
> Does your router do something special that skipping it breaks things?
> 
> On Apr 22, 2016 05:19, "Remco"  > wrote:
> 
> Hi Neil,
> 
> Well that explains...
> My networking setup does not conform to the commit message in such
> a way that there are multiple subnets on the network, however the
> instance does not have an IP address in all of them but only in one.
> I guess the only option for now is to patch this code, as a
> workaround.The reason for having multiple subnets on the same L2
> segment is a migration scenario. So I guess I'll just patch this as
> the situation will disappear in the near future as normally subnets
> are separated by L2 domains (vlans) in our setup.
> 
> Thanks!
> Remco
> 
> On Fri, Apr 22, 2016 at 1:16 PM, Neil Jerram
> > wrote:
> 
> On 22/04/16 12:03, Remco wrote:
> > Hi Neil,
> >
> > Thanks.
> > The ip route output is as following, i guess the 0.0.0.0 gateway is 
> only
> > listed by cloud-init:
> >
> > debian@instance:/$ ip route
> > default via 192.168.10.1 dev eth0
> > 192.168.10.0/24 
>  dev eth0  scope link
> > 169.254.169.254 via 192.168.10.1 dev eth0
> > 192.168.11.0/24 
>  dev eth0  scope link
> > 192.168.12.0/24 
>  dev eth0  proto kernel  scope
> > link  src 192.168.10.2
> >
> > (ip addresses are altered for security reasons).
> 
> Thanks for these clarifications.
> 
> > I'm not sure what creates these routes. I have two suspects: 
> cloud-init
> > and DHCP. As the same issue is observed on instances without 
> cloud-init
> > this rules out cloud-init.
> > We see the same issue on both Windows and Linux instances.
> 
> OK, I think you're seeing the effect of this DHCP agent code, from
> neutron/agent/linux/dhcp.py:
> 
>  host_routes.extend(["%s,0.0.0.0" % (s.cidr)
> for s in
>  self.network.subnets
>  if (s.ip_version == 4 and
>  s.cidr != subnet.cidr)])
> 
> AFAICS there is no obvious knob for suppressing this logic.
> 
> The code was added in commit 6dce817c7c2, and the commit
> message says:
> 
> =8<=
> Provide routes for neighbor IPv4 subnets
> 
> Network may contain several Subnets.  In this case all these
> subnets are
> accessible via same link and two VMs with addresses from different
> Subnets may talk to each other directly, bypassing default
> router.  For
> this to work, "neighbour Subnets" should have entry in VM's routing
>

Re: [Openstack-operators] Neutron DHCP agent local routes

2016-04-22 Thread Kevin Benton 
But if the subnets are on the same VLAN it should work. That's what that
patch is designed for. There is (currently) an assumption in Neutron that
subnets in the same network are on the same L2 segment. So if a VM wants to
communicate with a VM in another subnet on the same network, it can cut out
the router and use the direct route by ARPing for the VM on its own.

The default is for VMs to only get an IP from one of the subnets on a multi
subnet network. So if I understand your topology, it exactly matches what
that patch intended to address (eliminate the hairpinning via the router
for same network communication).

Does your router do something special that skipping it breaks things?
On Apr 22, 2016 05:19, "Remco"  wrote:

> Hi Neil,
>
> Well that explains...
> My networking setup does not conform to the commit message in such a way
> that there are multiple subnets on the network, however the instance does
> not have an IP address in all of them but only in one.
> I guess the only option for now is to patch this code, as a workaround.The
> reason for having multiple subnets on the same L2 segment is a migration
> scenario. So I guess I'll just patch this as the situation will disappear
> in the near future as normally subnets are separated by L2 domains (vlans)
> in our setup.
>
> Thanks!
> Remco
>
> On Fri, Apr 22, 2016 at 1:16 PM, Neil Jerram 
> wrote:
>
>> On 22/04/16 12:03, Remco wrote:
>> > Hi Neil,
>> >
>> > Thanks.
>> > The ip route output is as following, i guess the 0.0.0.0 gateway is only
>> > listed by cloud-init:
>> >
>> > debian@instance:/$ ip route
>> > default via 192.168.10.1 dev eth0
>> > 192.168.10.0/24  dev eth0  scope link
>> > 169.254.169.254 via 192.168.10.1 dev eth0
>> > 192.168.11.0/24  dev eth0  scope link
>> > 192.168.12.0/24  dev eth0  proto kernel  scope
>> > link  src 192.168.10.2
>> >
>> > (ip addresses are altered for security reasons).
>>
>> Thanks for these clarifications.
>>
>> > I'm not sure what creates these routes. I have two suspects: cloud-init
>> > and DHCP. As the same issue is observed on instances without cloud-init
>> > this rules out cloud-init.
>> > We see the same issue on both Windows and Linux instances.
>>
>> OK, I think you're seeing the effect of this DHCP agent code, from
>> neutron/agent/linux/dhcp.py:
>>
>>  host_routes.extend(["%s,0.0.0.0" % (s.cidr) for s in
>>  self.network.subnets
>>  if (s.ip_version == 4 and
>>  s.cidr != subnet.cidr)])
>>
>> AFAICS there is no obvious knob for suppressing this logic.
>>
>> The code was added in commit 6dce817c7c2, and the commit message says:
>>
>> =8<=
>> Provide routes for neighbor IPv4 subnets
>>
>> Network may contain several Subnets.  In this case all these subnets are
>> accessible via same link and two VMs with addresses from different
>> Subnets may talk to each other directly, bypassing default router.  For
>> this to work, "neighbour Subnets" should have entry in VM's routing
>> tables.  RFC3442 describes this situation:
>>
>> In some cases more than one IP subnet may be configured on a link.
>> In such cases, a host whose IP address is in one IP subnet in the
>> link could communicate directly with a host whose IP address is in a
>> different IP subnet on the same link. In cases where a client is
>> being assigned an IP address on an IP subnet on such a link, for each
>> IP subnet in the link other than the IP subnet on which the client
>> has been assigned the DHCP server MAY be configured to specify a
>> router IP address of 0.0.0.0.
>>
>> When network contains more that one IPv4 subnets, report these subnets
>> in 'classless-static-routes' DHCP option.
>>
>> DocImpact
>>
>> Change-Id: Ifcf1d99e0f0136bf52b8d13675b7ccfd48005fab
>> Closes-Bug: #1372885
>> =8<=
>>
>> So I guess the next question is: in what way does your networking setup
>> not conform to the assumptions in that commit message?
>>
>> Regards,
>> Neil
>>
>>
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Neutron DHCP agent local routes

2016-04-22 Thread Neil Jerram 
On 22/04/16 12:03, Remco wrote:
> Hi Neil,
>
> Thanks.
> The ip route output is as following, i guess the 0.0.0.0 gateway is only
> listed by cloud-init:
>
> debian@instance:/$ ip route
> default via 192.168.10.1 dev eth0
> 192.168.10.0/24  dev eth0  scope link
> 169.254.169.254 via 192.168.10.1 dev eth0
> 192.168.11.0/24  dev eth0  scope link
> 192.168.12.0/24  dev eth0  proto kernel  scope
> link  src 192.168.10.2
>
> (ip addresses are altered for security reasons).

Thanks for these clarifications.

> I'm not sure what creates these routes. I have two suspects: cloud-init
> and DHCP. As the same issue is observed on instances without cloud-init
> this rules out cloud-init.
> We see the same issue on both Windows and Linux instances.

OK, I think you're seeing the effect of this DHCP agent code, from 
neutron/agent/linux/dhcp.py:

 host_routes.extend(["%s,0.0.0.0" % (s.cidr) for s in
 self.network.subnets
 if (s.ip_version == 4 and
 s.cidr != subnet.cidr)])

AFAICS there is no obvious knob for suppressing this logic.

The code was added in commit 6dce817c7c2, and the commit message says:

=8<=
Provide routes for neighbor IPv4 subnets

Network may contain several Subnets.  In this case all these subnets are
accessible via same link and two VMs with addresses from different
Subnets may talk to each other directly, bypassing default router.  For
this to work, "neighbour Subnets" should have entry in VM's routing
tables.  RFC3442 describes this situation:

In some cases more than one IP subnet may be configured on a link.
In such cases, a host whose IP address is in one IP subnet in the
link could communicate directly with a host whose IP address is in a
different IP subnet on the same link. In cases where a client is
being assigned an IP address on an IP subnet on such a link, for each
IP subnet in the link other than the IP subnet on which the client
has been assigned the DHCP server MAY be configured to specify a
router IP address of 0.0.0.0.

When network contains more that one IPv4 subnets, report these subnets
in 'classless-static-routes' DHCP option.

DocImpact

Change-Id: Ifcf1d99e0f0136bf52b8d13675b7ccfd48005fab
Closes-Bug: #1372885
=8<=

So I guess the next question is: in what way does your networking setup 
not conform to the assumptions in that commit message?

Regards,
Neil


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Neutron DHCP agent local routes

2016-04-22 Thread Remco 
Hi Neil,

Thanks.
The ip route output is as following, i guess the 0.0.0.0 gateway is only
listed by cloud-init:

debian@instance:/$ ip route
default via 192.168.10.1 dev eth0
192.168.10.0/24 dev eth0  scope link
169.254.169.254 via 192.168.10.1 dev eth0
192.168.11.0/24 dev eth0  scope link
192.168.12.0/24 dev eth0  proto kernel  scope link  src 192.168.10.2

(ip addresses are altered for security reasons).

I'm not sure what creates these routes. I have two suspects: cloud-init and
DHCP. As the same issue is observed on instances without cloud-init this
rules out cloud-init.
We see the same issue on both Windows and Linux instances.

We do not have host routes in the subnet config.

Thanks,
Remco.

On Fri, Apr 22, 2016 at 12:06 PM, Neil Jerram 
wrote:

> On 22/04/16 10:59, Remco wrote:
> > Hi,
>
> Hi Remco,
>
> > We are running Liberty, with provider networks. We use the Neutron DHCP
> > agent to configure IPv4 and IPv6 addresses for instances. This is all
> > working great, however we sometimes need to put multiple subnets on one
> > VLAN interface (for legacy reasons). This is in fact also working,
> > however it configures some unwanted routes.
> >
> > For example, the following subnets exist in a Neutron network:
> >
> > 192.168.10.0/24 
> > 192.168.11.0/24 
> > 192.168.12.0/24 
> >
> > I.e. if the IP address obtained is 192.168.10.8/24
> > , the following routes are inserted:
>
> Do you mean on the instance?
>
> > 192.168.10.0/24  with gateway 0.0.0.0
> > 192.168.11.0/24  with gateway 0.0.0.0
> > 192.168.12.0/24  with gateway 0.0.0.0
>
> Does 'with gateway 0.0.0.0' mean directly connected?  What precisely
> does 'ip r' show?
>
> (I've never seen, e.g., '192.168.10.0/24 via 0.0.0.0' before, and don't
> know how I would interpret that.  So guessing you mean '192.168.10.0/24
> dev eth0'.)
>
> > The first route is obviously not an issue, as this route is directly
> > connected, but the other routes are plain wrong and stop traffic from
> > being routed to the default gateway.
> >
> > Does anyone know how to disable this behavior? I only want a default
> > gateway obtained from DHCP and I do not need any further routes.
>
> Do you know what is creating the .11 and .12 routes?  Unless you have
> host routes in your subnet config, I would expect the instance only to
> create a route for its DHCP-issued address, and hence only for .10.
>
> Neil
>
>
>
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Neutron DHCP agent local routes

2016-04-22 Thread Neil Jerram 
On 22/04/16 10:59, Remco wrote:
> Hi,

Hi Remco,

> We are running Liberty, with provider networks. We use the Neutron DHCP
> agent to configure IPv4 and IPv6 addresses for instances. This is all
> working great, however we sometimes need to put multiple subnets on one
> VLAN interface (for legacy reasons). This is in fact also working,
> however it configures some unwanted routes.
>
> For example, the following subnets exist in a Neutron network:
>
> 192.168.10.0/24 
> 192.168.11.0/24 
> 192.168.12.0/24 
>
> I.e. if the IP address obtained is 192.168.10.8/24
> , the following routes are inserted:

Do you mean on the instance?

> 192.168.10.0/24  with gateway 0.0.0.0
> 192.168.11.0/24  with gateway 0.0.0.0
> 192.168.12.0/24  with gateway 0.0.0.0

Does 'with gateway 0.0.0.0' mean directly connected?  What precisely 
does 'ip r' show?

(I've never seen, e.g., '192.168.10.0/24 via 0.0.0.0' before, and don't 
know how I would interpret that.  So guessing you mean '192.168.10.0/24 
dev eth0'.)

> The first route is obviously not an issue, as this route is directly
> connected, but the other routes are plain wrong and stop traffic from
> being routed to the default gateway.
>
> Does anyone know how to disable this behavior? I only want a default
> gateway obtained from DHCP and I do not need any further routes.

Do you know what is creating the .11 and .12 routes?  Unless you have 
host routes in your subnet config, I would expect the instance only to 
create a route for its DHCP-issued address, and hence only for .10.

Neil



___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators