Re: [ovs-dev] Path MTU discovery on GRE interfaces

[Matthias May via dev]

On 24/06/2021 05:51, Jesse Gross wrote:
> On Wed, Jun 23, 2021 at 10:06 AM Ben Pfaff  > wrote:
> 
> [updating Jesse's email address]
> 
> On Wed, Jun 23, 2021 at 04:48:29PM +0200, Matthias May via dev wrote:
> > I'm currently fighting with issues where TCP/UDP frames that are larger 
> than the MTU of a GRE tunnel are dropped.
> > I'm aware of the whys and how to work around the issue, but while 
> looking for solutions i stumbled over the fact that:
> > * [1] added PMTUD support to OVS
> > * [2] disabled/removed with v1.9.0 respectively v1.10.0 the feature
> >
> > Even after some significant time looking through the history i haven't 
> found a reason why this was removed, just
> that it
> > was removed.
> >
> > I started some preliminary work to add PMTUD support to OVS (again), 
> but the fact that it was removed 8 years ago
> seems
> > to me like a red flag to not do it (again).
> >
> > Could someone fluent with the OVS history from 8 years ago shed some 
> light on why PMTUD support was dropped?
> > Any pointers to a thread on this topic?
> 
> It was a layering violation.  This caused problems like, for example,
> not having a good IP address to send the "frag needed" message from.
> 
> 
> In terms of the history, I believe what happened is that PMTUD support was 
> added before the kernel module was
> upstreamed. When we later submitted the code upstream, we knew that it would 
> not fly due to the layering violations so
> support was removed before submitting.
> 
> However, as Dan mentioned, I believe that check_pkt_len can be used to 
> implement essentially the same behavior and it is
> upstream as it is more generic. It should still only be used in the context 
> of an L3 operation to avoid introducing the
> same layering issues though.


Thank you for your input.
I haven't done anything with check_pkt_len yet, but this seems promising.

Currently i simply ignore the DF bit and force fragmentation on the tunnel 
between the two sites.
After all, the proper solution is to set the MTU correctly on all involved 
devices.
--> I already don't have an issue for proper devices.

This is to work around some "industrial" devices that simply don't have the 
option to reduce the MTU, don't implement
DHCP option 26, and where the performance hit by doing fragmentation is too 
high.
IMO these devices are broken already. Being forced to have to work with them 
means that breaking L2/L3 layering to make
them work a bit better are probably the least of the issue(s).

BR
Matthias

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] Path MTU discovery on GRE interfaces

[Matthias May via dev]

On 23/06/2021 19:06, Ben Pfaff wrote:
> [updating Jesse's email address]
> 
> On Wed, Jun 23, 2021 at 04:48:29PM +0200, Matthias May via dev wrote:
>> I'm currently fighting with issues where TCP/UDP frames that are larger than 
>> the MTU of a GRE tunnel are dropped.
>> I'm aware of the whys and how to work around the issue, but while looking 
>> for solutions i stumbled over the fact that:
>> * [1] added PMTUD support to OVS
>> * [2] disabled/removed with v1.9.0 respectively v1.10.0 the feature
>>
>> Even after some significant time looking through the history i haven't found 
>> a reason why this was removed, just that it
>> was removed.
>>
>> I started some preliminary work to add PMTUD support to OVS (again), but the 
>> fact that it was removed 8 years ago seems
>> to me like a red flag to not do it (again).
>>
>> Could someone fluent with the OVS history from 8 years ago shed some light 
>> on why PMTUD support was dropped?
>> Any pointers to a thread on this topic?
> 
> It was a layering violation.  This caused problems like, for example,
> not having a good IP address to send the "frag needed" message from.
> 
> Jesse may remember more.
> 

OK i guessed as much.
I was thinking about what address to use when there is none available.
Maye the least breaking thing would be to just use as source of the ICMP 
unreachable the destination to which the frames
are addressed.

BR
Matthias

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] Path MTU discovery on GRE interfaces

[Jesse Gross]

On Wed, Jun 23, 2021 at 10:06 AM Ben Pfaff  wrote:

> [updating Jesse's email address]
>
> On Wed, Jun 23, 2021 at 04:48:29PM +0200, Matthias May via dev wrote:
> > I'm currently fighting with issues where TCP/UDP frames that are larger
> than the MTU of a GRE tunnel are dropped.
> > I'm aware of the whys and how to work around the issue, but while
> looking for solutions i stumbled over the fact that:
> > * [1] added PMTUD support to OVS
> > * [2] disabled/removed with v1.9.0 respectively v1.10.0 the feature
> >
> > Even after some significant time looking through the history i haven't
> found a reason why this was removed, just that it
> > was removed.
> >
> > I started some preliminary work to add PMTUD support to OVS (again), but
> the fact that it was removed 8 years ago seems
> > to me like a red flag to not do it (again).
> >
> > Could someone fluent with the OVS history from 8 years ago shed some
> light on why PMTUD support was dropped?
> > Any pointers to a thread on this topic?
>
> It was a layering violation.  This caused problems like, for example,
> not having a good IP address to send the "frag needed" message from.
>

In terms of the history, I believe what happened is that PMTUD support was
added before the kernel module was upstreamed. When we later submitted the
code upstream, we knew that it would not fly due to the layering violations
so support was removed before submitting.

However, as Dan mentioned, I believe that check_pkt_len can be used to
implement essentially the same behavior and it is upstream as it is more
generic. It should still only be used in the context of an L3 operation to
avoid introducing the same layering issues though.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] Path MTU discovery on GRE interfaces

[Dan Williams]

On Wed, 2021-06-23 at 10:06 -0700, Ben Pfaff wrote:
> [updating Jesse's email address]
> 
> On Wed, Jun 23, 2021 at 04:48:29PM +0200, Matthias May via dev wrote:
> > I'm currently fighting with issues where TCP/UDP frames that are
> > larger than the MTU of a GRE tunnel are dropped.
> > I'm aware of the whys and how to work around the issue, but while
> > looking for solutions i stumbled over the fact that:
> > * [1] added PMTUD support to OVS
> > * [2] disabled/removed with v1.9.0 respectively v1.10.0 the feature
> > 
> > Even after some significant time looking through the history i
> > haven't found a reason why this was removed, just that it
> > was removed.
> > 
> > I started some preliminary work to add PMTUD support to OVS
> > (again), but the fact that it was removed 8 years ago seems
> > to me like a red flag to not do it (again).
> > 
> > Could someone fluent with the OVS history from 8 years ago shed
> > some light on why PMTUD support was dropped?
> > Any pointers to a thread on this topic?
> 
> It was a layering violation.  This caused problems like, for example,
> not having a good IP address to send the "frag needed" message from.

See also Aaron Conole's recent attempt to do some fragmentation
handling when delivering to OVS ports with a smaller MTU. 

Since the tunnels have a smaller MTU for encapsulated traffic by
necessity, things that need to send through the tunnel (like a
container) must have a smaller MTU. But when something outside of the
container's host sends a large UDP packet to the container, OVS fails
to deliver that packet to the container's OVS port because its MTU is
too small.

We finally landed on using check_pkt_len to detect this condition and
punt the ICMP reply to ovn-controller, but check_pkt_len isn't easily
hardware offloadable :( And it would be great to just fragment this
traffic to the right MTU in the first place, rather than have to send
an ICMP reply or punt the fragmentation up to a controller.

Dan

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] Path MTU discovery on GRE interfaces

[Ben Pfaff]

[updating Jesse's email address]

On Wed, Jun 23, 2021 at 04:48:29PM +0200, Matthias May via dev wrote:
> I'm currently fighting with issues where TCP/UDP frames that are larger than 
> the MTU of a GRE tunnel are dropped.
> I'm aware of the whys and how to work around the issue, but while looking for 
> solutions i stumbled over the fact that:
> * [1] added PMTUD support to OVS
> * [2] disabled/removed with v1.9.0 respectively v1.10.0 the feature
> 
> Even after some significant time looking through the history i haven't found 
> a reason why this was removed, just that it
> was removed.
> 
> I started some preliminary work to add PMTUD support to OVS (again), but the 
> fact that it was removed 8 years ago seems
> to me like a red flag to not do it (again).
> 
> Could someone fluent with the OVS history from 8 years ago shed some light on 
> why PMTUD support was dropped?
> Any pointers to a thread on this topic?

It was a layering violation.  This caused problems like, for example,
not having a good IP address to send the "frag needed" message from.

Jesse may remember more.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] Path MTU discovery on GRE interfaces

[Matthias May via dev]

Hi Jesse, Hi List

I'm currently fighting with issues where TCP/UDP frames that are larger than 
the MTU of a GRE tunnel are dropped.
I'm aware of the whys and how to work around the issue, but while looking for 
solutions i stumbled over the fact that:
* [1] added PMTUD support to OVS
* [2] disabled/removed with v1.9.0 respectively v1.10.0 the feature

Even after some significant time looking through the history i haven't found a 
reason why this was removed, just that it
was removed.

I started some preliminary work to add PMTUD support to OVS (again), but the 
fact that it was removed 8 years ago seems
to me like a red flag to not do it (again).

Could someone fluent with the OVS history from 8 years ago shed some light on 
why PMTUD support was dropped?
Any pointers to a thread on this topic?

BR
Matthias

[1] https://mail.openvswitch.org/pipermail/ovs-git/2010-March/009936.html
[2] https://www.openvswitch.org/releases/NEWS-2.15.0.txt

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev