Reviewed: https://review.openstack.org/283790
Committed:
https://git.openstack.org/cgit/openstack/neutron/commit/?id=4df8d9a7016ab20fce235833d792b89309ec98a7
Submitter: Jenkins
Branch: master
commit 4df8d9a7016ab20fce235833d792b89309ec98a7
Author: Kevin Benton <ke...@benton.pub>
Date: Mon Feb 22 16:41:45 2016 -0800
Make agent interface plugging utilize network MTU
This changes the 'plug' and 'plug_new' interfaces of the
LinuxInterfaceDriver to accept an MTU argument. It then
updates the dhcp agent and l3 agent to pass the MTU that
is set on the network that the port belongs to. This allows
it to take into account the overhead calculations that are
done for encapsulation types.
It's necessary for the L3 agent to have the MTU because it
must recognize when fragmentation is needed so it can fragment
or generate an ICMP error.
It's necessary for the DHCP agent to have the MTU so it doesn't
interfere when it plugs into a bridge with a larger than 1500
MTU (the bridge would reduce its MTU to match the agent).
If an operator sets 'network_device_mtu', the value of that
will be used instead to preserve previous behavior.
Closes-Bug: #1549470
Closes-Bug: #1542108
Closes-Bug: #1542475
DocImpact: Neutron agents now support arbitrary MTU
configurations on each network (including
jumbo frames). This is accomplished by checking
the MTU value defined for each network on which
it is wiring VIFs.
Co-Authored-By: Matt Kassawara <mkassaw...@gmail.com>
Change-Id: Ic091fa78dfd133179c71cbc847bf955a06cb248a
** Changed in: neutron
Status: In Progress => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1542475
Title:
MTU concerns for the Open vSwitch agent
Status in neutron:
Fix Released
Bug description:
I ran some experiments with the Open vSwitch (OVS) agent [1] to
determine the source of MTU problems and offer a potential solution.
The environment for these experiments contains the following items:
1) A physical (underlying) network supporting MTU of 1500 or 9000 bytes.
2) One controller node running the neutron server, OVS agent, L3 agent, DHCP
agent, metadata agent, and OVS provider network bridge br-ex.
3) One compute node running the Open vSwitch agent.
4) A neutron provider/public network.
5) A neutron self-service/private network.
6) A neutron router between the provider and self-service networks.
7) The self-service network uses the VXLAN protocol with IPv4 endpoints which
adds 50 bytes of overhead.
8) An instance on the self-service network with a floating IP address from an
allocation pool on the provider network.
Background:
1. Interfaces (or ports) on OVS bridges such as those for overlay
network tunnels appear to use an arbitrarily large MTU. Thus, OVS
bridges and tunnel interfaces somewhat inherit the MTU of physical
network interfaces. For example, if OVS uses the IP address of eth0
for a tunnel overlay network endpoint and eth0 has a 1500 MTU, the
tunnel interface can only send packets with a payload of up to 1500
bytes including overlay protocol overhead.
2. OVS creates interfaces (ports) in the host namespace and moves them
to the appropriate namespace(s) rather than creating veth pairs
between namespaces.
3. For Linux bridge devices such as those on the compute node that
implement security groups, Linux assumes a 1500 MTU and changes the
MTU to the lowest MTU of any port on the bridge. For example, a bridge
without ports has a 1500 MTU. If eth0 has a 9000 MTU and you add it as
a port on the bridge, the bridge changes to a 9000 MTU. If eth1 has a
1500 MTU and you add it as a port on the bridge, the bridge changes to
a 1500 MTU.
4. Only devices that operate at layer-3 can participate in path MTU
discovery (PMTUD). Therefore, a change of MTU in a layer-2 device such
as a bridge or veth pair causes that device to discard packets larger
than the smallest MTU.
Observations:
1. For any physical network MTU, the port for the self-service network
router interface (qr) in the router namespace (qrouter) has a 1500
MTU. Background item (2) prevents a MTU disparity at layer-2 between
the router namespace and OVS bridge br-int. If a packet from the
provider network to the instance has a payload larger than 1500 bytes,
the router can send an ICMP message to the source telling it to use a
1500 MTU. However, the correct MTU for a private network using the
VXLAN overlay protocol should account for 50 bytes of overhead. Thus,
OVS fragments the packet over the tunnel and reassembles it on the
compute node containing the instance.
2. For a physical network MTU larger than 1500, the port for the
provider network router gateway (qg) in the router namespace (qrouter)
has a 1500 MTU. Background item (2) prevents a MTU disparity between
the router namespace and OVS provider network bridge br-ex. If a
packet from the provider network to the instance has a payload larger
than 1500 bytes, the router can send an ICMP messages to the source
telling it to use a 1500 MTU regardless of the private network overlay
protocol. Thus, the agent cannot realize a physical network MTU larger
than 1500.
3. If a provider or private network uses DHCP, the port in the DHCP
namespace has a 1500 MTU for any physical network MTU.
4. The Linux bridge that implements security groups on the compute
node lacks any ports on physical network interfaces. Background item
(3) causes the bridge to assume a 1500 MTU. Nova actually manages this
bridge and creates a veth pair between it and the Open vSwitch bridge
br-int. Both ends of the veth pair have a 1500 MTU. Background item
(1) indicates that the OVS bridge br-int could have a larger MTU.
Thus, OVS discards packets inbound to instances with a payload larger
than 1500 bytes.
5. Instances must use a MTU value the accounts for overlay protocol
overhead. Neutron currently offers a way to provide a correct value
via DHCP. However, considering observation item (4), providing a MTU
value larger than 1500 causes a disparity at layer-2 between the VM
and tap interface port on the Linux bridge that implements security
groups on the compute node. Thus, the bridge discards packets outbound
from instances with a payload larger than 1500 bytes.
6. The nova 'network_device_mtu' option controls the MTU of all
devices that it manages in observation items (4) and (5). For example,
using a value of 9000 causes the bridge, veth pair, and tap to have a
9000 MTU. Combining this option with providing the correct value to
instances via DHCP essentially resolves MTU problems on compute nodes.
Potential solution:
1. The port for the self-service network router interface (qr) in the
router namespace (qrouter) must use the MTU of the physical network
accounting for any overlay protocol overhead. For example, if the
physical network has a 9000 MTU and the private network uses the VXLAN
overlay protocol, the port must have a 8950 MTU.
2. The port for the provider network router gateway (qg) in the router
namespace (qrouter) must use the MTU of the physical network. For
example, if the physical network has a 9000 MTU, the port must have a
9000 MTU. If the provider network uses an overlay protocol, the MTU of
the port must also account for any overhead.
3. For networks using DHCP, the port in the DHCP namespace (qdhcp)
should use the MTU of the network on which it provides services
accounting for any overlay protocol overhead.
4. The Linux bridge that implements security groups on the compute
node and all ports on it must use the MTU of the physical network
accounting for any overlay protocol overhead.
[1] http://lists.openstack.org/pipermail/openstack-
dev/2016-January/084241.html
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1542475/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
