Jesse, We can raise the MTU of the underlying network until it reaches the maximum value for the physical equipment, a common situation for 10+ Gbps data center networks. For example, if the physical equipment supports a maximum 9000 MTU, we can't further increase it to account for overlay protocol overhead. We can easily reduce the VM MTU to account for outbound traffic, but inbound traffic assumes it can use 9000 bytes and therefore requires PMTUD... or (grudgingly) reducing the MTU of all hosts on the physical network... effectively moving the problem rather than solving it.
I think we need to solve this problem sooner than later to prevent the current loathing by operators using conventional OpenStack neutron drivers from seeping into OVN before it has a chance at success. Right now, the ability to eliminate (often obscure) problems for any underlying network MTU provides a huge selling point, at least for OpenStack. Matt On Tue, Feb 2, 2016 at 5:25 PM, Jesse Gross <je...@kernel.org> wrote: > On Wed, Jan 27, 2016 at 5:02 PM, Matt Kassawara <mkassaw...@gmail.com> > wrote: > > Hi, > > > > Conventional OpenStack Networking service (neutron) drivers such as Open > > vSwitch and Linux bridge have a long history of MTU issues involving > overlay > > networks. For example, consider the following typical virtual network > > architecture: > > > > 1) A provider network using native Ethernet or 802.1q tagging connects > the > > physical and virtual network infrastructure. Typically uses a 1500 or > 9000 > > MTU. > > 2) A private network using an overlay such as VXLAN or GRE. Overlay > protocol > > overhead reduces the effective MTU available to a VM. > > 3) A VM attaching to the private network. > > 4) A virtual router connecting the provider and private networks. > > > > For a packet outbound from a VM, we can simply set the MTU of the network > > interface in the VM to account for overlay protocol overhead. For > example, > > if the underlying network supports a 1500 MTU, a VM attaching to a VXLAN > > overlay network should use a 1450 MTU. We can provide this MTU to the VM > via > > DHCP, RA, or manual configuration of the interface. > > > > For a packet inbound to a VM, the situation becomes more interesting. The > > host sending the packet usually resides outside of the overlay network > and > > attempts to use the MTU available to it, typically 1500 or greater. The > > provider network also uses a 1500 MTU. The virtual router contains > > interfaces on the provider network using a 1500 MTU and the private > network > > using a 1450 MTU. As the packet attempts to traverse the router, it hits > the > > MTU disparity between 1500 and 1450 which causes the provider interface > to > > originate an ICMP message for PMTU discovery containing the 1450 MTU. The > > host receives this message and sends the packet again using the 1450 MTU. > > > > Currently, both the Open vSwitch and Linux bridge drivers incorrectly > place > > MTU disparities in layer-2 components which cannot originate ICMP > messages > > and drop packets that exceed the MTU. > > > > Using the native layer-3 implementation, how does OVN handle MTU > disparities > > between provider networks and private networks using an overlay protocol? > > As you say, the right way to handle this is to emit ICMP messages for > PMTUD or worst case do fragmentation. Unfortunately, at the Linux > kernel level there isn't a good way to handle this currently this with > any kind of programmable datapath, which is why the issue is common to > all of the various implementations. The only way that I have seen this > really handled is to use a separate namespace containing a Linux > router which isn't really all that satisfactory. > > Long term, we'll need to implement some primitives in the OVS datapath > to detect over-MTU packets so that OVN can react appropriately. > However, it's likely that this will require some architectural changes > so it's probably not likely to happen in the immediate future. > > I should also mention that another way to handle this is to raise the > MTU of the underlay network so that you don't reduce the MTU as seen > by the VM/outside world. However, I realize that this does not apply > to all situations. >
_______________________________________________ discuss mailing list discuss@openvswitch.org http://openvswitch.org/mailman/listinfo/discuss
