Re: Caching learned MSS/MTU values

[Hannes Frederic Sowa]

Hi Fred!

On Fri, Oct 25, 2013 at 11:05:14PM +, Templin, Fred L wrote:
> > -Original Message-
> > From: ipv6-ops-bounces+fred.l.templin=boeing@lists.cluenet.de 
> > [mailto:ipv6-ops-
> > bounces+fred.l.templin=boeing@lists.cluenet.de] On Behalf Of Hannes 
> > Frederic Sowa
> > Sent: Friday, October 25, 2013 1:49 PM
> > To: Templin, Fred L
> > Cc: Jason Fesler; IPv6 operators forum
> > Subject: Re: Caching learned MSS/MTU values
> > 
> > On Fri, Oct 18, 2013 at 10:23:00PM +, Templin, Fred L wrote:
> > > Hi Hannes,
> > >
> > > > Oh, that is interesting. I'll have a look at the weekend.
> > >
> > > OK. I had to roll another version to make some minor
> > > changes - see:
> > >
> > > http://linkupnetworks.com/seal/sealv2-0.2.tgz
> > > http://www.ietf.org/id/draft-templin-intarea-seal-64.txt
> > >
> > > I will let it rest for now, so this would be the version to
> > > start looking at. Let me know if there are any questions or
> > > comments.
> > 
> > Do you plan to convert them to git for easier patch management? I would
> > highly recommend it
> 
> Sure - do you have instructions on how to do this?

For git I would recommend reading some tutorial.
http://git-scm.com/documentation should help.

As these changes will have to go through net-next I would recommend cloning
the net-next repository: 
https://git.kernel.org/cgit/linux/kernel/git/davem/net-next.git

I would create a branch based on v3.10 and then git apply the patch (be aware
that git apply does not allow fuzz, so use git apply --reject if it does not
apply cleanly).

Check all the rejects and bring it in a compileable (better yet, stable
state). Then check if you can even break the giant patch up into smaller
independent parts. If it gets merged into the kernel you have to check
that each patch independently compiles and does not leave the logic in
some broken state (so that git bisect is still possible).

You could use gitk to stage seperate hunks and commit them independently.
Sending the patches for review as is as simple as a git send-email then.

Even if you don't plan to send it out for review soon this will make
rebasing the patch on more recent kernels more easily.

Greetings,

  Hannes

RE: Caching learned MSS/MTU values

[Templin, Fred L]

Hi Hannes,

> -Original Message-
> From: ipv6-ops-bounces+fred.l.templin=boeing@lists.cluenet.de 
> [mailto:ipv6-ops-
> bounces+fred.l.templin=boeing@lists.cluenet.de] On Behalf Of Hannes 
> Frederic Sowa
> Sent: Friday, October 25, 2013 1:49 PM
> To: Templin, Fred L
> Cc: Jason Fesler; IPv6 operators forum
> Subject: Re: Caching learned MSS/MTU values
> 
> On Fri, Oct 18, 2013 at 10:23:00PM +, Templin, Fred L wrote:
> > Hi Hannes,
> >
> > > Oh, that is interesting. I'll have a look at the weekend.
> >
> > OK. I had to roll another version to make some minor
> > changes - see:
> >
> > http://linkupnetworks.com/seal/sealv2-0.2.tgz
> > http://www.ietf.org/id/draft-templin-intarea-seal-64.txt
> >
> > I will let it rest for now, so this would be the version to
> > start looking at. Let me know if there are any questions or
> > comments.
> 
> Do you plan to convert them to git for easier patch management? I would
> highly recommend it

Sure - do you have instructions on how to do this?

> (it would also be necessary if you strive upstream inclusion).

The code is still a long ways off from proposing for upstream
inclusion, but I will definitely keep this in mind.

> Could you tell me an exact date or commit id where you cloned these
> files from?

I pulled the tarball from:

https://www.kernel.org/pub/linux/kernel/v3.0/linux-3.10.12.tar.xz

and made the edits directly in those files.

> Regarding IPv6 PMTU issues: it seems we had some rather broken behaviour
> in linux for some time. In some cases races could lead to complete
> blackholes to some IPv6 destinations for minutes. In the end it seemed
> pretty good reproduceable and I wonder why not many more people did
> report bugs.  I hope the relevant patches will land in one of the next
> stable kernels. It is not always a bad configured bad firewall. ;)

Yes, there are at least two things that would help - RFC4821 for
hosts and SEAL for tunnels.

Thanks - Fred
fred.l.temp...@boeing.com
 
> Greetings,
> 
>   Hannes

Re: Caching learned MSS/MTU values

[Hannes Frederic Sowa]

On Fri, Oct 18, 2013 at 10:23:00PM +, Templin, Fred L wrote:
> Hi Hannes,
> 
> > Oh, that is interesting. I'll have a look at the weekend.
> 
> OK. I had to roll another version to make some minor
> changes - see:
> 
> http://linkupnetworks.com/seal/sealv2-0.2.tgz
> http://www.ietf.org/id/draft-templin-intarea-seal-64.txt
> 
> I will let it rest for now, so this would be the version to
> start looking at. Let me know if there are any questions or
> comments.

Do you plan to convert them to git for easier patch management? I would
highly recommend it (it would also be necessary if you strive upstream
inclusion).

Could you tell me an exact date or commit id where you cloned these
files from?

Regarding IPv6 PMTU issues: it seems we had some rather broken behaviour
in linux for some time. In some cases races could lead to complete
blackholes to some IPv6 destinations for minutes. In the end it seemed
pretty good reproduceable and I wonder why not many more people did
report bugs.  I hope the relevant patches will land in one of the next
stable kernels. It is not always a bad configured bad firewall. ;)

Greetings,

  Hannes

RE: Caching learned MSS/MTU values

[Templin, Fred L]

Hi Hannes,

> Oh, that is interesting. I'll have a look at the weekend.

OK. I had to roll another version to make some minor
changes - see:

http://linkupnetworks.com/seal/sealv2-0.2.tgz
http://www.ietf.org/id/draft-templin-intarea-seal-64.txt

I will let it rest for now, so this would be the version to
start looking at. Let me know if there are any questions or
comments.

Thanks - Fred
fred.l.temp...@boeing.com

Re: Caching learned MSS/MTU values

[Hannes Frederic Sowa]

Hi!

On Fri, Oct 18, 2013 at 04:35:39PM +, Templin, Fred L wrote:
> > There is basis support for mtu probing for tcp. It is currently
> > deactivated by
> > default: cat /proc/sys/net/ipv4/tcp_mtu_probing => 0
> > 
> > Guess it had not the seen the testing it needs to activate it by
> > default.
> 
> Yes, I had heard that there was an off-by-default linux implementation
> of RFC4821. I also heard that it was not yet fully compliant to the
> spec, but that was a while ago now and it may have gotten better by now?

Going through the git logs it does not seem to get a lot of commits. Maybe it
is just stable, I don't know. I'll activate it on my boxes now to see if
something breaks.

The committed patch droped the note that it is not fully spec compliant but
seems to be based on the -05 draft.

> > I still have to take a closer look at SEAL. Thanks for the reminder. ;)
> 
> Sure. I have recently published an alpha linux implementation of SEAL:
> 
> http://www.ietf.org/mail-archive/web/ipv6/current/msg19114.html
> 
> It is still in early phases and does not yet fully implement the spec
> but does implement the core RFC4821 path MTU probing and fragmentation
> requirements for several different varieties of tunnels. The code is
> also quite ugly, and I would welcome any help on cleaning it up and/or
> implementing more features in the spec.

Oh, that is interesting. I'll have a look at the weekend.

Thanks,

  Hannes

RE: Caching learned MSS/MTU values

[Templin, Fred L]

Hi Hannes,

> -Original Message-
> From: Hannes Frederic Sowa [mailto:han...@stressinduktion.org]
> Sent: Friday, October 18, 2013 9:24 AM
> To: Templin, Fred L
> Cc: Jason Fesler; IPv6 operators forum
> Subject: Re: Caching learned MSS/MTU values
> 
> On Fri, Oct 18, 2013 at 03:17:28PM +, Templin, Fred L wrote:
> > Hi,
> >
> > > -Original Message-
> > > From: ipv6-ops-bounces+fred.l.templin=boeing@lists.cluenet.de
> > > [mailto:ipv6-ops-
> bounces+fred.l.templin=boeing@lists.cluenet.de] On
> > > Behalf Of Hannes Frederic Sowa
> > > Sent: Friday, October 18, 2013 12:31 AM
> > > To: Jason Fesler
> > > Cc: IPv6 operators forum
> > > Subject: Re: Caching learned MSS/MTU values
> > >
> > > On Thu, Oct 17, 2013 at 09:05:24AM -0700, Jason Fesler wrote:
> > > > I'm once again considering trying to improve on the test-ipv6.com
> > > PMTUD
> > > > failure detection. Due to limitations on the client side I can't
> use
> > > raw
> > > > sockets to generate test packets. The client is JavaScript and
> runs
> > > in a
> > > > browser; all I can do is try fetching urls from multiple
> locations,
> > > each
> > > > with a different MTU.
> > > >
> > > > I know that the various operating systems tend to cache any PMTUD
> > > issues
> > > > that they can detect; future connections to that destination will
> use
> > > > smaller packets accordingly. What I can not see to find is an
> > > adequate
> > > > description of what granularity this gets cached with. /128? /64?
> > > Also, I
> > > > the absence of Packet Too Big messages, what does each OS do?
> > >
> > > Linux, too, does cache on /128 basis. In the absence of PTB the
> > > connection
> > > will get stuck. ;)
> >
> > Right, and we are observing non-negligible cases where PTBs are
> either
> > not delivered or lost somewhere along the way. That is why there is a
> > growing push for wider deployment of RFC4821 for end systems, and why
> > I am investing my time in developing SEAL for tunnels.
> 
> There is basis support for mtu probing for tcp. It is currently
> deactivated by
> default: cat /proc/sys/net/ipv4/tcp_mtu_probing => 0
> 
> Guess it had not the seen the testing it needs to activate it by
> default.

Yes, I had heard that there was an off-by-default linux implementation
of RFC4821. I also heard that it was not yet fully compliant to the
spec, but that was a while ago now and it may have gotten better by now?

> I still have to take a closer look at SEAL. Thanks for the reminder. ;)

Sure. I have recently published an alpha linux implementation of SEAL:

http://www.ietf.org/mail-archive/web/ipv6/current/msg19114.html

It is still in early phases and does not yet fully implement the spec
but does implement the core RFC4821 path MTU probing and fragmentation
requirements for several different varieties of tunnels. The code is
also quite ugly, and I would welcome any help on cleaning it up and/or
implementing more features in the spec.

Thanks - Fred
fred.l.temp...@boeing.com

> Greetings,
> 
>   Hannes

Re: Caching learned MSS/MTU values

[Hannes Frederic Sowa]

On Fri, Oct 18, 2013 at 03:17:28PM +, Templin, Fred L wrote:
> Hi,
> 
> > -Original Message-
> > From: ipv6-ops-bounces+fred.l.templin=boeing@lists.cluenet.de
> > [mailto:ipv6-ops-bounces+fred.l.templin=boeing@lists.cluenet.de] On
> > Behalf Of Hannes Frederic Sowa
> > Sent: Friday, October 18, 2013 12:31 AM
> > To: Jason Fesler
> > Cc: IPv6 operators forum
> > Subject: Re: Caching learned MSS/MTU values
> > 
> > On Thu, Oct 17, 2013 at 09:05:24AM -0700, Jason Fesler wrote:
> > > I'm once again considering trying to improve on the test-ipv6.com
> > PMTUD
> > > failure detection. Due to limitations on the client side I can't use
> > raw
> > > sockets to generate test packets. The client is JavaScript and runs
> > in a
> > > browser; all I can do is try fetching urls from multiple locations,
> > each
> > > with a different MTU.
> > >
> > > I know that the various operating systems tend to cache any PMTUD
> > issues
> > > that they can detect; future connections to that destination will use
> > > smaller packets accordingly. What I can not see to find is an
> > adequate
> > > description of what granularity this gets cached with. /128? /64?
> > Also, I
> > > the absence of Packet Too Big messages, what does each OS do?
> > 
> > Linux, too, does cache on /128 basis. In the absence of PTB the
> > connection
> > will get stuck. ;)
> 
> Right, and we are observing non-negligible cases where PTBs are either
> not delivered or lost somewhere along the way. That is why there is a
> growing push for wider deployment of RFC4821 for end systems, and why
> I am investing my time in developing SEAL for tunnels.

There is basis support for mtu probing for tcp. It is currently deactivated by
default: cat /proc/sys/net/ipv4/tcp_mtu_probing => 0

Guess it had not the seen the testing it needs to activate it by default.

I still have to take a closer look at SEAL. Thanks for the reminder. ;)

Greetings,

  Hannes

RE: Caching learned MSS/MTU values

[Templin, Fred L]

Hi,

> -Original Message-
> From: ipv6-ops-bounces+fred.l.templin=boeing@lists.cluenet.de
> [mailto:ipv6-ops-bounces+fred.l.templin=boeing@lists.cluenet.de] On
> Behalf Of Hannes Frederic Sowa
> Sent: Friday, October 18, 2013 12:31 AM
> To: Jason Fesler
> Cc: IPv6 operators forum
> Subject: Re: Caching learned MSS/MTU values
> 
> On Thu, Oct 17, 2013 at 09:05:24AM -0700, Jason Fesler wrote:
> > I'm once again considering trying to improve on the test-ipv6.com
> PMTUD
> > failure detection. Due to limitations on the client side I can't use
> raw
> > sockets to generate test packets. The client is JavaScript and runs
> in a
> > browser; all I can do is try fetching urls from multiple locations,
> each
> > with a different MTU.
> >
> > I know that the various operating systems tend to cache any PMTUD
> issues
> > that they can detect; future connections to that destination will use
> > smaller packets accordingly. What I can not see to find is an
> adequate
> > description of what granularity this gets cached with. /128? /64?
> Also, I
> > the absence of Packet Too Big messages, what does each OS do?
> 
> Linux, too, does cache on /128 basis. In the absence of PTB the
> connection
> will get stuck. ;)

Right, and we are observing non-negligible cases where PTBs are either
not delivered or lost somewhere along the way. That is why there is a
growing push for wider deployment of RFC4821 for end systems, and why
I am investing my time in developing SEAL for tunnels.

Thanks - Fred
fred.l.temp...@boeing.com

> 
> Greetings,
> 
>   Hannes

Re: Caching learned MSS/MTU values

[Jason Fesler]

Everyone,

Thank you.

Re: Caching learned MSS/MTU values

[Hannes Frederic Sowa]

On Thu, Oct 17, 2013 at 09:05:24AM -0700, Jason Fesler wrote:
> I'm once again considering trying to improve on the test-ipv6.com PMTUD
> failure detection. Due to limitations on the client side I can't use raw
> sockets to generate test packets. The client is JavaScript and runs in a
> browser; all I can do is try fetching urls from multiple locations, each
> with a different MTU.
> 
> I know that the various operating systems tend to cache any PMTUD issues
> that they can detect; future connections to that destination will use
> smaller packets accordingly. What I can not see to find is an adequate
> description of what granularity this gets cached with. /128? /64?   Also, I
> the absence of Packet Too Big messages, what does each OS do?

Linux, too, does cache on /128 basis. In the absence of PTB the connection
will get stuck. ;)

Greetings,

  Hannes

Re: Caching learned MSS/MTU values

[Ignatios Souvatzis]

On Thu, Oct 17, 2013 at 09:05:24AM -0700, Jason Fesler wrote:

> I know that the various operating systems tend to cache any PMTUD issues
> that they can detect; future connections to that destination will use
> smaller packets accordingly. What I can not see to find is an adequate
> description of what granularity this gets cached with. /128? /64?

NetBSD does it on a per-address base - that is, I think, per /128. It's 
an additional entry in the routing table, like this:

2a01:170:1012:77::25   fe80::20d:61ff:fe46:50ad%xennet0 UGHD
2   27   1280  xennet0

-is

Re: Caching learned MSS/MTU values

[Tassos Chatzithomaoglou]

I see that WinXP keeps the /128 for the last 32 entries.

C:\Documents and Settings\Tassos>ipv6 rc
2a02:1788:4fd::b2ff:5201 via 4/fe80::218:18ff:fef6:5b54
 src 4/2a02:2148:82:6000:1430:f026:c55a:b75b
 PMTU 1500
2a02:1788:2fd::ab via 4/fe80::218:18ff:fef6:5b54
 src 4/2a02:2148:82:6000:1430:f026:c55a:b75b
 PMTU 1500
2a02:2148:2fff:ff02::3e01:2619 via 4/fe80::218:18ff:fef6:5b54
 src 4/2a02:2148:82:6000:1430:f026:c55a:b75b
 PMTU 1500


C:\Documents and Settings\Tassos>netsh interface ipv6 show global
Querying active state...

General Global Parameters
-
Default Hop Limit   : 128 hops
Neighbor Cache Limit: 256 entries per interface
Route Cache Limit   : 32 entries
Reassembly Limit: 16770400 bytes


Win7 seems to keep /128 for the last 128 entries.

C:\Users\Tassos>netsh interface ipv6 show destinationcache

Interface 10: Local Area Connection


PMTU Destination Address   Next Hop Address
 - -
1492 2001:0:5ef5:79fb:1c3e:d863:c447:6318  fe80::200:ff:fe00:0
1492 2001:0:9d38:6ab8:4a3:31f0:411e:6346   fe80::200:ff:fe00:0
1492 2001:0:9d38:6ab8:10c5:11cb:b08a:5623  fe80::200:ff:fe00:0
1492 2001:0:9d38:6ab8:3485:2015:a1ca:a777  fe80::200:ff:fe00:0
1492 2001:0:9d38:6abd:1cbd:1b0c:e0f1:34d3  fe80::200:ff:fe00:0
1492 2001:0:9d38:90d7:20a2:147a:9a93:4e6b  fe80::200:ff:fe00:0
1500 2a02:2149:8104:3100:4484:bf55:88d6:bcc3   
2a02:2149:8104:3100:4484:bf55:88d6:bcc
1492 fe80::200:ff:fe00:0   fe80::200:ff:fe00:0
1492 2001:0:9d38:90d7:2020:3f27:a9ec:39ce  fe80::200:ff:fe00:0
1492 2001:0:5ef5:79fd:873:3f92:ac11:1ee6   fe80::200:ff:fe00:0

C:\Users\Tassos>netsh interface ipv6 show global
Querying active state...

General Global Parameters
-
Default Hop Limit   : 128 hops
Neighbor Cache Limit: 256 entries per interface
Route Cache Limit   : 128 entries per compartment


--
Tassos

Jason Fesler wrote on 17/10/2013 19:05:
> I'm once again considering trying to improve on the test-ipv6.com 
>  PMTUD failure detection. Due to limitations on the 
> client side I can't use raw sockets to generate test packets. The client is 
> JavaScript and runs in a browser; all I can do is try fetching urls from 
> multiple locations, each with a different MTU. 
>
> I know that the various operating systems tend to cache any PMTUD issues that 
> they can detect; future connections to that destination will use smaller 
> packets accordingly. What I can not see to find is an adequate description of 
> what granularity this gets cached with. /128? /64?   Also, I the absence of 
> Packet Too Big messages, what does each OS do?
>
> If anyone has pointers, please share. It will affect what and how I can 
> improve the site, given the restrictions I have with the client side.
>
>
>
>
> -- 
>  Jason Fesler, email/jabber mailto:jfes...@gigo.com>> 
> resume: http://jfesler.com
>  "Give a man fire, and he'll be warm for a day;
>  set a man on fire, and he'll be warm for the rest of his life."
>
>