Short version: Ben Stasiewicz's research into significant
(33/404 = 8%) levels of PMTUD failure in IPv6 -
are tunnels causing a lot of this?
Geoff Huston's investigation of an IPv6 problem
presumably caused by one or more tunnels.
Tunnel ingress routers have a tough time
generating valid PTBs - I think they would need
to cache the initial parts of the packets
before they were encapsulated in order to be
able to generate a valid PTB to the sending host
if a PTB came back from a router in the tunnel
due to an encapsulted packet exceeding next-hop
MTU.
PTBs not making it correctly through NAT to the
sending host? Google finds Norton Security on
a PC blocking IPv6 PTBs.
None of this makes me think that PTB-based
PMTUD must be abandoned and completely replaced
with end-to-end PMTUD, such as RFC 4821.
Hi Dan,
Thanks for your message, in which you wrote, in a different order:
> Sounds like you want a research paper?
Yes. Fortunately Matthew Luckie wrote to the MTU list, pointing
to the work of one of his students - Ben Stasiewicz:
http://listserver.internetnz.net.nz/pipermail/ipv6-techsig/2009-October/000708.html
http://www.wand.net.nz/~bss7/pmtud-ben-final.pdf
This concerns a test of IPv6 websites. 0.11% of the Alexa top
million websites had an IPv6 version. PMTUD problems were found to
a significant degree (figures quoted below), and it seems that some
servers had been configured to use a small packet size to avoid
such problems.
There's a discussion:
http://listserver.internetnz.net.nz/pipermail/ipv6-techsig/2009-October/date.html#708
>> Where is the occurrence of this documented in anything more than the
>> anecdotal way you describe it above? I don't see any reason why it
>> would always be hush-hush.
>
> A recent accidental block was discussed in the thread
> http://www.gossamer-threads.com/lists/nsp/ipv6/20779, where Google
> was accidentally blocking ICMPv6 PTB.
>
> I have heard of many similar problems where ICMPv6 PTB is being
> blocked; these take hours and occasionally days to resolve.
> Some of those can be attributed to growing pains of IPv6,
> though -- perhaps a too-low limit for ICMPv6-per-second
> from a router, and that just needed to be increased, or an overly-
> aggressive ICMPv6 filter.
>
> All modern routers allow limiting the number of ICMPs per second
> they send. So if there is some reason they're sending a lot of ICMPs,
> such as during an attack or when the interface is full of non-attack
> traffic, they won't be able to send a legitimate PTB. Forwarding real
> traffic is more important than filtering ICMPs.
>
> The Cisco command is "ip icmp rate-limit unreachable", Juniper's is
> icmpv4-rate-limit. These limits are actively discussed on operations
> lists, which implies those limits are actively used on the Internet.
OK - thanks for pointing to this Google incident.
What you describe doesn't seem to me to be a fundamental problem
with RFC 1191 / RFC 1981 PMTUD. If there was a fundamental
problem, I would expect it to appear frequently in IPv4, and not
only affect PTBs but other ICMP messages, though I guess the PTBs
are the ones which would cause most trouble if they were either
filtered out, or not sent in the first place.
My understanding of Tony's message:
http://www.ietf.org/mail-archive/web/rrg/current/msg05820.html
is that he argues that that the PTB approach to PMTUD - and indeed
anything which relies on ICMP messages - is iretrievably broken,
and so new end-to-end protocols would be needed, such as RFC 4821.
But I an yet to read anything which convinces me of this.
>> Maybe many UDP-based applications do this already - locally doing the
>> same thing that RFC 4821 suggests, but without sharing any
>> information with other packetization layers.
>
> draft-petithuguenin-behave-stun-pmtud does it for UDP. I know one
> video application that plans to use the technique described in
> that I-D.
OK.
>> But TCP is a protocol which frequently is ready to send "long"
>> packets - as long as its local MSS allows.
>
> And fixing PTMUD is often done by tweaking the MSS. Cisco
> equipment has long supported that functionality (and it was
> mentioned as a quick fix in the thread I cited above). This
> "fixes" PMTUD failures. Cisco command is "ip tcp adjust-mss",
> Juniper commands are 'set flow all-tcp-mss' and 'set flow tcp-mss'.
If PTBs are either being dropped due to filtering, or not being
generated (such as due to competition with other ICMP messages
during an attack) and this is causing a significant problem with
PMTUD, then it seems the possible solutions are:
1 - Stop the filtering and configure the routers so (within
practical limits) attacks don't stop the generation of
PTBs.
2 - Adjust down MSS on sending hosts to fix the PMTU problems
for TCP - but not other protocols.
3 - Have TCP and other packetization layers in the stack and
applications no longer reliant on PTBs, but implementing
RFC 4821 fully, or performing similar end-to-end PMTUD
independently, as you noted above was being done for a
video app.
1 would clearly produce the best outcomes.
Benedikt, at the end of http://www.gossamer-threads.com/lists/nsp/ipv6/20779
strongly objected to fixing the MSS at a defensively low value.
Likewise Daniel Griggs:
http://listserver.internetnz.net.nz/pipermail/ipv6-techsig/2009-October/000714.html
Geoff Huston thinks the problem may be caused by tunnels or PTBs
not getting back to sending hosts behind NAT:
http://listserver.internetnz.net.nz/pipermail/ipv6-techsig/2009-October/000721.html
I hadn't thought of NAT boxes . . . . The NAT box would need to
look into the PTB, find the part of the original packet - IP header
and next 8 bytes at least - and then figure out which host behind it
to send the PTB to. Presumably any self-respecting NAT box would
do this. However, perhaps if actual PTB usage is minimal due to
manual or default low MTU/MSS settings, then perhaps there could
be wide enough adoption of NAT boxes which don't do this, before
anyone notices the problem sufficient to cause the developers
to add proper PTB handling.
Here is a Google report on "Norton Internet Security was disabling
all inbound ICMPv6 traffic not part of an existing session (i.e.
traffic that was not part of a response to outbound traffic
initiated by the host).":
http://sites.google.com/site/ipv6center/icmpv6-is-non-optional
On tunnels, Geoff Huston wrote, in part:
This additional header overhead implies that the tunnel's
MTU is smaller than the "raw" interface MTU. The second
problem with a tunnel is that there may be further tunnels
"inside" the tunnel, so that the tunnel ingress is not
necessarily aware of the true tunnel MTU. The third problem
is that the routing of the interior of the tunnel may change,
so that the tunnel MTU may be variable.
But for a tunnel to support PMTUD, I think the entry router needs
to maintain some kind of cache of recently sent packets so it can
construct a valid PTB to the sending host, when an encapsulated
packet in the tunnel hits an MTU limit.
This could be very expensive for large tunnels. How long is the
fragment of each packet to be held?
But if there is a condition that prevents the source from
receiving packet-too-big ICMPv6 messages then the algorithm
fails, and the application may hang when full-sized TCP packets
are passed through the network. In some cases this may happen
at a point well distanced from the two endpoints of the TCP
session, so that the ICMPv6 filtering may be occurring at a
point that is not under the control of the source or the
destination.
This is the basic reason why so many web server systems are
averse to configuring themselves as dual stack IPv4 and IPv6
servers. The problem is that through no fault of their own in
the local configuration of the IPv6 server, and through no
fault in the configuration of the IPv6 client, there are
situations where the application fails, even though every part
of the system appears to be functioning.
The first item on Geoff's list of solution is to fix the problem
in the network - whatever is dropping or failing to generate PTBs.
He contemplates changes to clients and servers, and in the end,
for practical reasons, to get his dual-stack server working
reliably, he drops the MTU to 1400:
Waiting for every filter in the Internet to do the right thing
with ICMP messages may well be a fruitless task, and adding
further complexity into applications or the TCP protocol
behaviour seems to take the long way around the problem. The
most effective approach appears to be the simplest one as well
- whether you are a dual stack client, or a dual stack server,
the best way to get more reliable service under these rather
strange corner cases is to drop your MTU.
This gets things going for now, but allows bad tunnels, PTB
filtering etc. to persist without complaint. If Geoff Huston
lacks the time and inclination to figure out where the errant
tunnel is and provide the necessary feedback to its
administrators - who is going to do it?
So do we pull our horns in, wind back MTU/MSS values to
something timid and safe - and then forever be locked into
these limits even when most of the paths to other hosts around
the world support 9kbyte jumboframes? This is just hobbling and
blinkering our hosts.
Tony seems to think we should give up on ICMP PTBs and expect OS
and app developers to implement RFC 4821.
I think that networks which filter out PTBs should change their
ways.
If there are tunnels not supporting PMTUD, then these need to be
fixed or changed.
Assuming IPv6 becomes much more widely adopted, these tunnel
problems will tend to go away, since I assume they are mainly
IPv6 over IPv4 tunnels. So I think the prevalence of PMTUD
problems in IPv6, assuming it can be largely attributed to bad
tunnels, is no reason to abandon PTB-based PMTUD.
Ben Stasiewicz found:
371 PMTUD_SUCCESS OK 371
214 RX_TOOSMALL
22 RX_NODATA ----] Fail 33
13 TCP_NOCONN ]
9 PMTUD_FAIL ----]
5 TCP_RST ]
2 RX_NOACK ----]
1 TCP_ERROR
The 214 servers classed as "RX_TOSMALL" were those where the
HTTP server returned packets too small to perform the PMTUD
test. Perhaps this was a tweak to avoid PMTUD problems which
are apparently not uncommon (33/404) in IPv6 at present.
But what is the incidence of such troubles in IPv4?
- Robin http://www.firstpr.com.au/ip/ivip/
_______________________________________________
rrg mailing list
rrg@irtf.org
http://www.irtf.org/mailman/listinfo/rrg
