Re: [ovs-dev] Odd gre_sys MTU setting (2.6.x vs 2.8.x)

2018-01-16 Thread James Page 
On Tue, 16 Jan 2018 at 16:26 James Page  wrote:

> On Tue, 16 Jan 2018 at 16:23 James Page  wrote:
>
>>
>> On Tue, 16 Jan 2018 at 15:47 Eric Garver  wrote:
>> [...]
>>
>>> > Re-tested with the mainline kernel; gre_sys device is still configured
>>> with
>>> > a 1472 MTU, however I was then able to increase it using ip link set
>>> > gre_sys mtu , confirming that the kernel applied hardware limit
>>> was not
>>> > longer being enforced - however looks like the IFLA_MTU settings
>>> provided
>>> > from OVS are still being ignored by the kernel.
>>>
>>> What was the value of ? Did you try 66535 (what OVS uses)? Any dmesg
>>> for the OVS case?
>>>
>>
>> Good question:
>>
>> # ip link set gre_sys mtu 65535
>> RTNETLINK answers: Invalid argument
>> # ip link set gre_sys mtu 9000
>> # ip link
>> [...]
>> 18: gre_sys@NONE:  mtu 9000 qdisc
>> pfifo_fast master ovs-system state UNKNOWN mode DEFAULT group default qlen
>> 1000
>> link/ether ea:38:bd:61:29:4d brd ff:ff:ff:ff:ff:ff
>>
>> so the value set by OVS was not accepted using ip as well.
>>
>
> 65490 was the largest MTU value I am able to set via the ip command
>

And to confirm no error messages in dmesg.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] Odd gre_sys MTU setting (2.6.x vs 2.8.x)

2018-01-16 Thread James Page 
On Tue, 16 Jan 2018 at 16:23 James Page  wrote:

>
> On Tue, 16 Jan 2018 at 15:47 Eric Garver  wrote:
> [...]
>
>> > Re-tested with the mainline kernel; gre_sys device is still configured
>> with
>> > a 1472 MTU, however I was then able to increase it using ip link set
>> > gre_sys mtu , confirming that the kernel applied hardware limit was
>> not
>> > longer being enforced - however looks like the IFLA_MTU settings
>> provided
>> > from OVS are still being ignored by the kernel.
>>
>> What was the value of ? Did you try 66535 (what OVS uses)? Any dmesg
>> for the OVS case?
>>
>
> Good question:
>
> # ip link set gre_sys mtu 65535
> RTNETLINK answers: Invalid argument
> # ip link set gre_sys mtu 9000
> # ip link
> [...]
> 18: gre_sys@NONE:  mtu 9000 qdisc
> pfifo_fast master ovs-system state UNKNOWN mode DEFAULT group default qlen
> 1000
> link/ether ea:38:bd:61:29:4d brd ff:ff:ff:ff:ff:ff
>
> so the value set by OVS was not accepted using ip as well.
>

65490 was the largest MTU value I am able to set via the ip command.

>
>
>
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] Odd gre_sys MTU setting (2.6.x vs 2.8.x)

2018-01-16 Thread James Page 
On Tue, 16 Jan 2018 at 15:47 Eric Garver  wrote:
[...]

> > Re-tested with the mainline kernel; gre_sys device is still configured
> with
> > a 1472 MTU, however I was then able to increase it using ip link set
> > gre_sys mtu , confirming that the kernel applied hardware limit was
> not
> > longer being enforced - however looks like the IFLA_MTU settings provided
> > from OVS are still being ignored by the kernel.
>
> What was the value of ? Did you try 66535 (what OVS uses)? Any dmesg
> for the OVS case?
>

Good question:

# ip link set gre_sys mtu 65535
RTNETLINK answers: Invalid argument
# ip link set gre_sys mtu 9000
# ip link
[...]
18: gre_sys@NONE:  mtu 9000 qdisc
pfifo_fast master ovs-system state UNKNOWN mode DEFAULT group default qlen
1000
link/ether ea:38:bd:61:29:4d brd ff:ff:ff:ff:ff:ff

so the value set by OVS was not accepted using ip as well.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] Odd gre_sys MTU setting (2.6.x vs 2.8.x)

2018-01-16 Thread Eric Garver 
On Tue, Jan 16, 2018 at 06:27:23AM +, James Page wrote:
> On Mon, 15 Jan 2018 at 18:55 Christian Ehrhardt <
> christian.ehrha...@canonical.com> wrote:
> 
> > On Mon, Jan 15, 2018 at 5:43 PM, Eric Garver  wrote:
> > > On Mon, Jan 15, 2018 at 11:32:32AM -0500, Eric Garver wrote:
> > >> On Mon, Jan 15, 2018 at 03:17:28PM +, James Page wrote:
> > >> > Hi Eric
> > >> >
> > >> > On Mon, 15 Jan 2018 at 16:54 Eric Garver  wrote:
> > >> >
> > >> > > On Sat, Jan 13, 2018 at 03:57:08PM +, James Page wrote:
> > >> > > > Hi Ben s
> > >> > > >
> > >> > > > On Sat, 13 Jan 2018 at 14:55 James Page 
> > wrote:
> > >> > > >
> > >> > > > > OK, I sent a patch:
> > >> > > > >> https://patchwork.ozlabs.org/patch/860192/
> > >> > > > >
> > >> > > > >
> > >> > > > > Thanks Ben - I'll pull this patch into the Ubuntu packages and
> > test
> > >> > > this
> > >> > > > > weekend.
> > >> > > > >
> > >> > > >
> > >> > > >  I pulled your patch in ontop of our current 2.8.1 packages and
> > >> > > re-tested;
> > >> > > > I'm not seeing a difference in behaviour with the patch in place.
> > I
> > >> > > removed
> > >> > > > the vport_gre and ip_gre kernel modules to force a re-creation of
> > the
> > >> > > > device on restart of OVS as well as trying a reboot of the test
> > machine:
> > >> > > >
> > >> > > > 7: gre0@NONE:  mtu 1476 qdisc noop state DOWN mode
> > DEFAULT group
> > >> > > > default qlen 1
> > >> > > > link/gre 0.0.0.0 brd 0.0.0.0
> > >> > > > 8: gretap0@NONE:  mtu 1462 qdisc noop state
> > DOWN
> > >> > > mode
> > >> > > > DEFAULT group default qlen 1000
> > >> > > > link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
> > >> > > > 9: gre_sys@NONE:  mtu 1472 qdisc
> > >> > > > pfifo_fast master ovs-system state UNKNOWN mode DEFAULT group
> > default
> > >> > > qlen
> > >> > > > 1000
> > >> > > > link/ether 52:ad:6d:89:bb:54 brd ff:ff:ff:ff:ff:ff
> > >> > >
> > >> > > Are there any errors is dmesg? It's possible setlink won't accept
> > >> > > UINT16_MAX. grep for "Invalid MTU".
> > >> > >
> > >> >
> > >> > Not under 4.4 but I do under 4.13:
> > >> >
> > >> > [6.794079] gre: GRE over IPv4 demultiplexor driver
> > >> > [6.797621] ip_gre: GRE over IPv4 tunneling driver
> > >> > [6.798326] gre_sys: Invalid MTU 65535 requested, hw max 1500
> > >> >
> > >> > Looks like the kernel is limiting to 1500.
> > >>
> > >> I would expect setting with ip-link to fail as well.
> > >> What does the below show?:
> > >>
> > >>   $ ip link set dev gre_sys mtu 65535
> > >
> > > Ugh. This is a separate kernel bug fixed by this commit:
> > >
> > > commit cfddd4c33c254954927942599d299b3865743146 <(386)%20574-3146>
> > > Author: Xin Long 
> > > Date:   Mon Dec 18 14:24:35 2017 +0800
> > >
> > > ip_gre: remove the incorrect mtu limit for ipgre tap
> >
> > Thanks Eric, that matches my findings, glad that there seems to be an
> > accepted fix already.
> > But it is fairly recent and only in since 4.15-rc8 levels afaik.
> >
> > But OTOH its description at [1] reads pretty much like my notes so far.
> >
> > @James - do you think you could test a super-recent mainline kernel
> > build from [2] in regard to this issue?
> >
> 
> Thanks all
> 
> Re-tested with the mainline kernel; gre_sys device is still configured with
> a 1472 MTU, however I was then able to increase it using ip link set
> gre_sys mtu , confirming that the kernel applied hardware limit was not
> longer being enforced - however looks like the IFLA_MTU settings provided
> from OVS are still being ignored by the kernel.

What was the value of ? Did you try 66535 (what OVS uses)? Any dmesg
for the OVS case?
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] Odd gre_sys MTU setting (2.6.x vs 2.8.x)

2018-01-15 Thread James Page 
On Mon, 15 Jan 2018 at 18:55 Christian Ehrhardt <
christian.ehrha...@canonical.com> wrote:

> On Mon, Jan 15, 2018 at 5:43 PM, Eric Garver  wrote:
> > On Mon, Jan 15, 2018 at 11:32:32AM -0500, Eric Garver wrote:
> >> On Mon, Jan 15, 2018 at 03:17:28PM +, James Page wrote:
> >> > Hi Eric
> >> >
> >> > On Mon, 15 Jan 2018 at 16:54 Eric Garver  wrote:
> >> >
> >> > > On Sat, Jan 13, 2018 at 03:57:08PM +, James Page wrote:
> >> > > > Hi Ben s
> >> > > >
> >> > > > On Sat, 13 Jan 2018 at 14:55 James Page 
> wrote:
> >> > > >
> >> > > > > OK, I sent a patch:
> >> > > > >> https://patchwork.ozlabs.org/patch/860192/
> >> > > > >
> >> > > > >
> >> > > > > Thanks Ben - I'll pull this patch into the Ubuntu packages and
> test
> >> > > this
> >> > > > > weekend.
> >> > > > >
> >> > > >
> >> > > >  I pulled your patch in ontop of our current 2.8.1 packages and
> >> > > re-tested;
> >> > > > I'm not seeing a difference in behaviour with the patch in place.
> I
> >> > > removed
> >> > > > the vport_gre and ip_gre kernel modules to force a re-creation of
> the
> >> > > > device on restart of OVS as well as trying a reboot of the test
> machine:
> >> > > >
> >> > > > 7: gre0@NONE:  mtu 1476 qdisc noop state DOWN mode
> DEFAULT group
> >> > > > default qlen 1
> >> > > > link/gre 0.0.0.0 brd 0.0.0.0
> >> > > > 8: gretap0@NONE:  mtu 1462 qdisc noop state
> DOWN
> >> > > mode
> >> > > > DEFAULT group default qlen 1000
> >> > > > link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
> >> > > > 9: gre_sys@NONE:  mtu 1472 qdisc
> >> > > > pfifo_fast master ovs-system state UNKNOWN mode DEFAULT group
> default
> >> > > qlen
> >> > > > 1000
> >> > > > link/ether 52:ad:6d:89:bb:54 brd ff:ff:ff:ff:ff:ff
> >> > >
> >> > > Are there any errors is dmesg? It's possible setlink won't accept
> >> > > UINT16_MAX. grep for "Invalid MTU".
> >> > >
> >> >
> >> > Not under 4.4 but I do under 4.13:
> >> >
> >> > [6.794079] gre: GRE over IPv4 demultiplexor driver
> >> > [6.797621] ip_gre: GRE over IPv4 tunneling driver
> >> > [6.798326] gre_sys: Invalid MTU 65535 requested, hw max 1500
> >> >
> >> > Looks like the kernel is limiting to 1500.
> >>
> >> I would expect setting with ip-link to fail as well.
> >> What does the below show?:
> >>
> >>   $ ip link set dev gre_sys mtu 65535
> >
> > Ugh. This is a separate kernel bug fixed by this commit:
> >
> > commit cfddd4c33c254954927942599d299b3865743146 <(386)%20574-3146>
> > Author: Xin Long 
> > Date:   Mon Dec 18 14:24:35 2017 +0800
> >
> > ip_gre: remove the incorrect mtu limit for ipgre tap
>
> Thanks Eric, that matches my findings, glad that there seems to be an
> accepted fix already.
> But it is fairly recent and only in since 4.15-rc8 levels afaik.
>
> But OTOH its description at [1] reads pretty much like my notes so far.
>
> @James - do you think you could test a super-recent mainline kernel
> build from [2] in regard to this issue?
>

Thanks all

Re-tested with the mainline kernel; gre_sys device is still configured with
a 1472 MTU, however I was then able to increase it using ip link set
gre_sys mtu , confirming that the kernel applied hardware limit was not
longer being enforced - however looks like the IFLA_MTU settings provided
from OVS are still being ignored by the kernel.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] Odd gre_sys MTU setting (2.6.x vs 2.8.x)

2018-01-15 Thread Christian Ehrhardt 
On Mon, Jan 15, 2018 at 5:43 PM, Eric Garver  wrote:
> On Mon, Jan 15, 2018 at 11:32:32AM -0500, Eric Garver wrote:
>> On Mon, Jan 15, 2018 at 03:17:28PM +, James Page wrote:
>> > Hi Eric
>> >
>> > On Mon, 15 Jan 2018 at 16:54 Eric Garver  wrote:
>> >
>> > > On Sat, Jan 13, 2018 at 03:57:08PM +, James Page wrote:
>> > > > Hi Ben s
>> > > >
>> > > > On Sat, 13 Jan 2018 at 14:55 James Page  wrote:
>> > > >
>> > > > > OK, I sent a patch:
>> > > > >> https://patchwork.ozlabs.org/patch/860192/
>> > > > >
>> > > > >
>> > > > > Thanks Ben - I'll pull this patch into the Ubuntu packages and test
>> > > this
>> > > > > weekend.
>> > > > >
>> > > >
>> > > >  I pulled your patch in ontop of our current 2.8.1 packages and
>> > > re-tested;
>> > > > I'm not seeing a difference in behaviour with the patch in place. I
>> > > removed
>> > > > the vport_gre and ip_gre kernel modules to force a re-creation of the
>> > > > device on restart of OVS as well as trying a reboot of the test 
>> > > > machine:
>> > > >
>> > > > 7: gre0@NONE:  mtu 1476 qdisc noop state DOWN mode DEFAULT group
>> > > > default qlen 1
>> > > > link/gre 0.0.0.0 brd 0.0.0.0
>> > > > 8: gretap0@NONE:  mtu 1462 qdisc noop state DOWN
>> > > mode
>> > > > DEFAULT group default qlen 1000
>> > > > link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
>> > > > 9: gre_sys@NONE:  mtu 1472 qdisc
>> > > > pfifo_fast master ovs-system state UNKNOWN mode DEFAULT group default
>> > > qlen
>> > > > 1000
>> > > > link/ether 52:ad:6d:89:bb:54 brd ff:ff:ff:ff:ff:ff
>> > >
>> > > Are there any errors is dmesg? It's possible setlink won't accept
>> > > UINT16_MAX. grep for "Invalid MTU".
>> > >
>> >
>> > Not under 4.4 but I do under 4.13:
>> >
>> > [6.794079] gre: GRE over IPv4 demultiplexor driver
>> > [6.797621] ip_gre: GRE over IPv4 tunneling driver
>> > [6.798326] gre_sys: Invalid MTU 65535 requested, hw max 1500
>> >
>> > Looks like the kernel is limiting to 1500.
>>
>> I would expect setting with ip-link to fail as well.
>> What does the below show?:
>>
>>   $ ip link set dev gre_sys mtu 65535
>
> Ugh. This is a separate kernel bug fixed by this commit:
>
> commit cfddd4c33c254954927942599d299b3865743146
> Author: Xin Long 
> Date:   Mon Dec 18 14:24:35 2017 +0800
>
> ip_gre: remove the incorrect mtu limit for ipgre tap

Thanks Eric, that matches my findings, glad that there seems to be an
accepted fix already.
But it is fairly recent and only in since 4.15-rc8 levels afaik.

But OTOH its description at [1] reads pretty much like my notes so far.

@James - do you think you could test a super-recent mainline kernel
build from [2] in regard to this issue?

[1]: 
https://github.com/torvalds/linux/commit/cfddd4c33c254954927942599d299b3865743146
[2]: http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.15-rc8/
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] Odd gre_sys MTU setting (2.6.x vs 2.8.x)

2018-01-15 Thread Christian Ehrhardt 
> But that does not disable the central check.
> And this only relies on the max_mtu values in the device struct

gre's own check would be called by
dev.c:dev_set_mtu
  * core check here*
  -> __dev_set_mtu
-> ops->ndo_change_mtu

But the check that refuses us is before the call to gre so it never gets there.

> I still have to look what gro actually sets on initial setup, but atm
> I'd assume it could do some default which might be 1500 or so and due
> to that limits it later on our try to increase the value.

gre uses ether_setup to set up the device which sets sane ethernet defaults.
But in this case this will be
»···dev->mtu»···»···= ETH_DATA_LEN;
»···dev->min_mtu»···»···= ETH_MIN_MTU;
»···dev->max_mtu»···»···= ETH_DATA_LEN;

so max = initial mtu, and that is not big enough to increase later.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] Odd gre_sys MTU setting (2.6.x vs 2.8.x)

2018-01-15 Thread Eric Garver 
On Mon, Jan 15, 2018 at 11:32:32AM -0500, Eric Garver wrote:
> On Mon, Jan 15, 2018 at 03:17:28PM +, James Page wrote:
> > Hi Eric
> > 
> > On Mon, 15 Jan 2018 at 16:54 Eric Garver  wrote:
> > 
> > > On Sat, Jan 13, 2018 at 03:57:08PM +, James Page wrote:
> > > > Hi Ben s
> > > >
> > > > On Sat, 13 Jan 2018 at 14:55 James Page  wrote:
> > > >
> > > > > OK, I sent a patch:
> > > > >> https://patchwork.ozlabs.org/patch/860192/
> > > > >
> > > > >
> > > > > Thanks Ben - I'll pull this patch into the Ubuntu packages and test
> > > this
> > > > > weekend.
> > > > >
> > > >
> > > >  I pulled your patch in ontop of our current 2.8.1 packages and
> > > re-tested;
> > > > I'm not seeing a difference in behaviour with the patch in place. I
> > > removed
> > > > the vport_gre and ip_gre kernel modules to force a re-creation of the
> > > > device on restart of OVS as well as trying a reboot of the test machine:
> > > >
> > > > 7: gre0@NONE:  mtu 1476 qdisc noop state DOWN mode DEFAULT group
> > > > default qlen 1
> > > > link/gre 0.0.0.0 brd 0.0.0.0
> > > > 8: gretap0@NONE:  mtu 1462 qdisc noop state DOWN
> > > mode
> > > > DEFAULT group default qlen 1000
> > > > link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
> > > > 9: gre_sys@NONE:  mtu 1472 qdisc
> > > > pfifo_fast master ovs-system state UNKNOWN mode DEFAULT group default
> > > qlen
> > > > 1000
> > > > link/ether 52:ad:6d:89:bb:54 brd ff:ff:ff:ff:ff:ff
> > >
> > > Are there any errors is dmesg? It's possible setlink won't accept
> > > UINT16_MAX. grep for "Invalid MTU".
> > >
> > 
> > Not under 4.4 but I do under 4.13:
> > 
> > [6.794079] gre: GRE over IPv4 demultiplexor driver
> > [6.797621] ip_gre: GRE over IPv4 tunneling driver
> > [6.798326] gre_sys: Invalid MTU 65535 requested, hw max 1500
> > 
> > Looks like the kernel is limiting to 1500.
> 
> I would expect setting with ip-link to fail as well.
> What does the below show?:
> 
>   $ ip link set dev gre_sys mtu 65535

Ugh. This is a separate kernel bug fixed by this commit:

commit cfddd4c33c254954927942599d299b3865743146
Author: Xin Long 
Date:   Mon Dec 18 14:24:35 2017 +0800

ip_gre: remove the incorrect mtu limit for ipgre tap
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] Odd gre_sys MTU setting (2.6.x vs 2.8.x)

2018-01-15 Thread Christian Ehrhardt 
>> Looks like the kernel is limiting to 1500.
>
> I would expect setting with ip-link to fail as well.
> What does the below show?:
>
>   $ ip link set dev gre_sys mtu 65535

For your question - we just talked about that and yes on the new
kernel that fails as well if set via ip.

But those checks are newer than kernel 4.4

Since
commit 61e84623ace35ce48975e8f90bbbac7557c43d61
Author: Jarod Wilson 
Date:   Fri Oct 7 22:04:33 2016 -0400

net: centralize net_device min/max MTU checking

Any call is checked in net/core/dev.c and this is the message you saw.
Now there is another related change
commit a52ad514fdf3b8a57ca4322c92d2d8d5c6182485
Author: Jarod Wilson 
Date:   Fri Oct 7 22:04:34 2016 -0400

net: deprecate eth_change_mtu, remove usage

Which essentially removes all the per device checks in favor of the
new central check.
In there it mentions that:
 ... All callers have been audited for calls to alloc_etherdev* or
ether_setup directly, which means they all have a valid dev->min_mtu
and dev->max_mtu ...

Working theories for me atm are:
- gre formerly had no such check at al land worked, but the central
check kicks in and refuses to set the higher MTU
- This would either be a kernel bug on gre, or on the new OVS in case
the netlink based creation would be supposed to add something to set
the max_mtu accordingly.

The checks on MTU change essentially are those in core and a call to
the net_device_ops.ndo_change_mtu callback.
That is implemented in ip_tunnel_change_mtu by gre.
in 4.4 this was checking as:
if (new_mtu < 68 ||
new_mtu > 0xFFF8 - dev->hard_header_len - t_hlen)
return -EINVAL;

The new check is not that much different it seems:
int max_mtu = 0xFFF8 - dev->hard_header_len - t_hlen;

if (new_mtu < ETH_MIN_MTU)
return -EINVAL;

if (new_mtu > max_mtu) {
if (strict)
return -EINVAL;

But that does not disable the central check.
And this only relies on the max_mtu values in the device struct

I still have to look what gro actually sets on initial setup, but atm
I'd assume it could do some default which might be 1500 or so and due
to that limits it later on our try to increase the value.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] Odd gre_sys MTU setting (2.6.x vs 2.8.x)

2018-01-15 Thread Eric Garver 
On Mon, Jan 15, 2018 at 03:17:28PM +, James Page wrote:
> Hi Eric
> 
> On Mon, 15 Jan 2018 at 16:54 Eric Garver  wrote:
> 
> > On Sat, Jan 13, 2018 at 03:57:08PM +, James Page wrote:
> > > Hi Ben s
> > >
> > > On Sat, 13 Jan 2018 at 14:55 James Page  wrote:
> > >
> > > > OK, I sent a patch:
> > > >> https://patchwork.ozlabs.org/patch/860192/
> > > >
> > > >
> > > > Thanks Ben - I'll pull this patch into the Ubuntu packages and test
> > this
> > > > weekend.
> > > >
> > >
> > >  I pulled your patch in ontop of our current 2.8.1 packages and
> > re-tested;
> > > I'm not seeing a difference in behaviour with the patch in place. I
> > removed
> > > the vport_gre and ip_gre kernel modules to force a re-creation of the
> > > device on restart of OVS as well as trying a reboot of the test machine:
> > >
> > > 7: gre0@NONE:  mtu 1476 qdisc noop state DOWN mode DEFAULT group
> > > default qlen 1
> > > link/gre 0.0.0.0 brd 0.0.0.0
> > > 8: gretap0@NONE:  mtu 1462 qdisc noop state DOWN
> > mode
> > > DEFAULT group default qlen 1000
> > > link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
> > > 9: gre_sys@NONE:  mtu 1472 qdisc
> > > pfifo_fast master ovs-system state UNKNOWN mode DEFAULT group default
> > qlen
> > > 1000
> > > link/ether 52:ad:6d:89:bb:54 brd ff:ff:ff:ff:ff:ff
> >
> > Are there any errors is dmesg? It's possible setlink won't accept
> > UINT16_MAX. grep for "Invalid MTU".
> >
> 
> Not under 4.4 but I do under 4.13:
> 
> [6.794079] gre: GRE over IPv4 demultiplexor driver
> [6.797621] ip_gre: GRE over IPv4 tunneling driver
> [6.798326] gre_sys: Invalid MTU 65535 requested, hw max 1500
> 
> Looks like the kernel is limiting to 1500.

I would expect setting with ip-link to fail as well.
What does the below show?:

  $ ip link set dev gre_sys mtu 65535
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] Odd gre_sys MTU setting (2.6.x vs 2.8.x)

2018-01-15 Thread James Page 
Hi Eric

On Mon, 15 Jan 2018 at 16:54 Eric Garver  wrote:

> On Sat, Jan 13, 2018 at 03:57:08PM +, James Page wrote:
> > Hi Ben s
> >
> > On Sat, 13 Jan 2018 at 14:55 James Page  wrote:
> >
> > > OK, I sent a patch:
> > >> https://patchwork.ozlabs.org/patch/860192/
> > >
> > >
> > > Thanks Ben - I'll pull this patch into the Ubuntu packages and test
> this
> > > weekend.
> > >
> >
> >  I pulled your patch in ontop of our current 2.8.1 packages and
> re-tested;
> > I'm not seeing a difference in behaviour with the patch in place. I
> removed
> > the vport_gre and ip_gre kernel modules to force a re-creation of the
> > device on restart of OVS as well as trying a reboot of the test machine:
> >
> > 7: gre0@NONE:  mtu 1476 qdisc noop state DOWN mode DEFAULT group
> > default qlen 1
> > link/gre 0.0.0.0 brd 0.0.0.0
> > 8: gretap0@NONE:  mtu 1462 qdisc noop state DOWN
> mode
> > DEFAULT group default qlen 1000
> > link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
> > 9: gre_sys@NONE:  mtu 1472 qdisc
> > pfifo_fast master ovs-system state UNKNOWN mode DEFAULT group default
> qlen
> > 1000
> > link/ether 52:ad:6d:89:bb:54 brd ff:ff:ff:ff:ff:ff
>
> Are there any errors is dmesg? It's possible setlink won't accept
> UINT16_MAX. grep for "Invalid MTU".
>

Not under 4.4 but I do under 4.13:

[6.794079] gre: GRE over IPv4 demultiplexor driver
[6.797621] ip_gre: GRE over IPv4 tunneling driver
[6.798326] gre_sys: Invalid MTU 65535 requested, hw max 1500

Looks like the kernel is limiting to 1500.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] Odd gre_sys MTU setting (2.6.x vs 2.8.x)

2018-01-15 Thread Eric Garver 
On Sat, Jan 13, 2018 at 03:57:08PM +, James Page wrote:
> Hi Ben s
> 
> On Sat, 13 Jan 2018 at 14:55 James Page  wrote:
> 
> > OK, I sent a patch:
> >> https://patchwork.ozlabs.org/patch/860192/
> >
> >
> > Thanks Ben - I'll pull this patch into the Ubuntu packages and test this
> > weekend.
> >
> 
>  I pulled your patch in ontop of our current 2.8.1 packages and re-tested;
> I'm not seeing a difference in behaviour with the patch in place. I removed
> the vport_gre and ip_gre kernel modules to force a re-creation of the
> device on restart of OVS as well as trying a reboot of the test machine:
> 
> 7: gre0@NONE:  mtu 1476 qdisc noop state DOWN mode DEFAULT group
> default qlen 1
> link/gre 0.0.0.0 brd 0.0.0.0
> 8: gretap0@NONE:  mtu 1462 qdisc noop state DOWN mode
> DEFAULT group default qlen 1000
> link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
> 9: gre_sys@NONE:  mtu 1472 qdisc
> pfifo_fast master ovs-system state UNKNOWN mode DEFAULT group default qlen
> 1000
> link/ether 52:ad:6d:89:bb:54 brd ff:ff:ff:ff:ff:ff

Are there any errors is dmesg? It's possible setlink won't accept
UINT16_MAX. grep for "Invalid MTU".
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] Odd gre_sys MTU setting (2.6.x vs 2.8.x)

2018-01-13 Thread James Page 
Hi Ben s

On Sat, 13 Jan 2018 at 14:55 James Page  wrote:

> OK, I sent a patch:
>> https://patchwork.ozlabs.org/patch/860192/
>
>
> Thanks Ben - I'll pull this patch into the Ubuntu packages and test this
> weekend.
>

 I pulled your patch in ontop of our current 2.8.1 packages and re-tested;
I'm not seeing a difference in behaviour with the patch in place. I removed
the vport_gre and ip_gre kernel modules to force a re-creation of the
device on restart of OVS as well as trying a reboot of the test machine:

7: gre0@NONE:  mtu 1476 qdisc noop state DOWN mode DEFAULT group
default qlen 1
link/gre 0.0.0.0 brd 0.0.0.0
8: gretap0@NONE:  mtu 1462 qdisc noop state DOWN mode
DEFAULT group default qlen 1000
link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
9: gre_sys@NONE:  mtu 1472 qdisc
pfifo_fast master ovs-system state UNKNOWN mode DEFAULT group default qlen
1000
link/ether 52:ad:6d:89:bb:54 brd ff:ff:ff:ff:ff:ff
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] Odd gre_sys MTU setting (2.6.x vs 2.8.x)

2018-01-13 Thread James Page 
On Fri, 12 Jan 2018 at 20:45 Ben Pfaff  wrote:

> > > > Most likely you are seeing this bug:
> > > >
> > > >   https://bugzilla.redhat.com/show_bug.cgi?id=1488484
> > > >
> > > > It's actually an issue with the kernel GRE driver ignoring IFLA_MTU
> when
> > > > the device is created.
>

Great - figured someone must have seen this already!
[...]


> OK, I sent a patch:
> https://patchwork.ozlabs.org/patch/860192/


Thanks Ben - I'll pull this patch into the Ubuntu packages and test this
weekend.

Cheers

James
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] Odd gre_sys MTU setting (2.6.x vs 2.8.x)

2018-01-12 Thread Ben Pfaff 
On Fri, Jan 12, 2018 at 01:48:00PM -0500, Eric Garver wrote:
> On Fri, Jan 12, 2018 at 10:18:42AM -0800, Ben Pfaff wrote:
> > On Fri, Jan 12, 2018 at 10:51:11AM -0500, Eric Garver wrote:
> > > On Fri, Jan 12, 2018 at 12:33:23PM +, James Page wrote:
> > > > Hi
> > > > 
> > > > We're currently trying to debug an issue we're seeing when using 
> > > > OpenStack
> > > > with OVS 2.8.1 (see [0]).
> > > > 
> > > > When using GRE tunnels, we're seeing broken/slow performance due to what
> > > > appears to be an incorrect MTU setting on the gre_sys device; for 
> > > > previous
> > > > OVS releases (2.6.1 specifically but I think this applies back to 2.5.x 
> > > > as
> > > > well) this device was created with an MTU of 65490; however under 2.8.1 
> > > > it
> > > > gets created with an MTU of 1472, which when used with GRE networks with
> > > > larger MTU settings causes problems with fragmentation and broken path 
> > > > MTU
> > > > discovery.
> > > > 
> > > > I've tested under Linux 4.4 and 4.13 and see the problem with both 
> > > > kernels.
> > > > 
> > > > Its possible to workaround the issue by setting the MTU on the gre_sys
> > > > device directly, but this change will get reset back to 1472 on a 
> > > > restart
> > > > of openvswitch.
> > > > 
> > > > I know the mtu_request feature was introduced around 2.7.0 - groking the
> > > > code I can't see where the MTU of this device is actually configured...
> > > > 
> > > > Any help much appreciated
> > > > 
> > > > James
> > > > 
> > > > [0] https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1742505
> > > 
> > > Most likely you are seeing this bug:
> > > 
> > >   https://bugzilla.redhat.com/show_bug.cgi?id=1488484
> > > 
> > > It's actually an issue with the kernel GRE driver ignoring IFLA_MTU when
> > > the device is created.
> > 
> > Can OVS work around this by following up its RTM_NEWLINK by an
> > (otherwise redundant) RTM_SETLINK?
> 
> Yes.
> 
> > (And should it do so?)
> 
> I don't see any negative effect other than another round trip to the
> kernel. Worth noting that the other tunnels (vxlan, geneve) are not
> affected.

OK, I sent a patch:
https://patchwork.ozlabs.org/patch/860192/
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] Odd gre_sys MTU setting (2.6.x vs 2.8.x)

2018-01-12 Thread Eric Garver 
On Fri, Jan 12, 2018 at 10:18:42AM -0800, Ben Pfaff wrote:
> On Fri, Jan 12, 2018 at 10:51:11AM -0500, Eric Garver wrote:
> > On Fri, Jan 12, 2018 at 12:33:23PM +, James Page wrote:
> > > Hi
> > > 
> > > We're currently trying to debug an issue we're seeing when using OpenStack
> > > with OVS 2.8.1 (see [0]).
> > > 
> > > When using GRE tunnels, we're seeing broken/slow performance due to what
> > > appears to be an incorrect MTU setting on the gre_sys device; for previous
> > > OVS releases (2.6.1 specifically but I think this applies back to 2.5.x as
> > > well) this device was created with an MTU of 65490; however under 2.8.1 it
> > > gets created with an MTU of 1472, which when used with GRE networks with
> > > larger MTU settings causes problems with fragmentation and broken path MTU
> > > discovery.
> > > 
> > > I've tested under Linux 4.4 and 4.13 and see the problem with both 
> > > kernels.
> > > 
> > > Its possible to workaround the issue by setting the MTU on the gre_sys
> > > device directly, but this change will get reset back to 1472 on a restart
> > > of openvswitch.
> > > 
> > > I know the mtu_request feature was introduced around 2.7.0 - groking the
> > > code I can't see where the MTU of this device is actually configured...
> > > 
> > > Any help much appreciated
> > > 
> > > James
> > > 
> > > [0] https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1742505
> > 
> > Most likely you are seeing this bug:
> > 
> >   https://bugzilla.redhat.com/show_bug.cgi?id=1488484
> > 
> > It's actually an issue with the kernel GRE driver ignoring IFLA_MTU when
> > the device is created.
> 
> Can OVS work around this by following up its RTM_NEWLINK by an
> (otherwise redundant) RTM_SETLINK?

Yes.

> (And should it do so?)

I don't see any negative effect other than another round trip to the
kernel. Worth noting that the other tunnels (vxlan, geneve) are not
affected.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] Odd gre_sys MTU setting (2.6.x vs 2.8.x)

2018-01-12 Thread Ben Pfaff 
On Fri, Jan 12, 2018 at 10:51:11AM -0500, Eric Garver wrote:
> On Fri, Jan 12, 2018 at 12:33:23PM +, James Page wrote:
> > Hi
> > 
> > We're currently trying to debug an issue we're seeing when using OpenStack
> > with OVS 2.8.1 (see [0]).
> > 
> > When using GRE tunnels, we're seeing broken/slow performance due to what
> > appears to be an incorrect MTU setting on the gre_sys device; for previous
> > OVS releases (2.6.1 specifically but I think this applies back to 2.5.x as
> > well) this device was created with an MTU of 65490; however under 2.8.1 it
> > gets created with an MTU of 1472, which when used with GRE networks with
> > larger MTU settings causes problems with fragmentation and broken path MTU
> > discovery.
> > 
> > I've tested under Linux 4.4 and 4.13 and see the problem with both kernels.
> > 
> > Its possible to workaround the issue by setting the MTU on the gre_sys
> > device directly, but this change will get reset back to 1472 on a restart
> > of openvswitch.
> > 
> > I know the mtu_request feature was introduced around 2.7.0 - groking the
> > code I can't see where the MTU of this device is actually configured...
> > 
> > Any help much appreciated
> > 
> > James
> > 
> > [0] https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1742505
> 
> Most likely you are seeing this bug:
> 
>   https://bugzilla.redhat.com/show_bug.cgi?id=1488484
> 
> It's actually an issue with the kernel GRE driver ignoring IFLA_MTU when
> the device is created.

Can OVS work around this by following up its RTM_NEWLINK by an
(otherwise redundant) RTM_SETLINK?  (And should it do so?)
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] Odd gre_sys MTU setting (2.6.x vs 2.8.x)

2018-01-12 Thread Eric Garver 
On Fri, Jan 12, 2018 at 12:33:23PM +, James Page wrote:
> Hi
> 
> We're currently trying to debug an issue we're seeing when using OpenStack
> with OVS 2.8.1 (see [0]).
> 
> When using GRE tunnels, we're seeing broken/slow performance due to what
> appears to be an incorrect MTU setting on the gre_sys device; for previous
> OVS releases (2.6.1 specifically but I think this applies back to 2.5.x as
> well) this device was created with an MTU of 65490; however under 2.8.1 it
> gets created with an MTU of 1472, which when used with GRE networks with
> larger MTU settings causes problems with fragmentation and broken path MTU
> discovery.
> 
> I've tested under Linux 4.4 and 4.13 and see the problem with both kernels.
> 
> Its possible to workaround the issue by setting the MTU on the gre_sys
> device directly, but this change will get reset back to 1472 on a restart
> of openvswitch.
> 
> I know the mtu_request feature was introduced around 2.7.0 - groking the
> code I can't see where the MTU of this device is actually configured...
> 
> Any help much appreciated
> 
> James
> 
> [0] https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1742505

Most likely you are seeing this bug:

  https://bugzilla.redhat.com/show_bug.cgi?id=1488484

It's actually an issue with the kernel GRE driver ignoring IFLA_MTU when
the device is created.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev