On 6-Nov-2005, at 21:26, Dean Anderson wrote:
BTW, forgot to say thanks for adding the part about load balancing.
Thanks for bringing it up.
On Fri, 4 Nov 2005, Joe Abley wrote:
I don't have a good reference to cite, but it is my experience that
per-packet load balancing over parallel circuits leads to levels of
packet reordering which are low enough that serious TCP performance
degradation does not result. It may also be that the relative
adjacency of out-of-order segments (e.g. reliably within a window)
allows common TCP implementations to cope well, whereas topologies
which result in greater dispersion of segments cause the pipeline to
stall more frequently.
It is not severe in either case. Indeed, so far unicast TCP is
concerned,
diverse links are also parallel. It is only with Anycast that the
parallelism
is split. The only difference is in the number of IP-visible hops.
However even
seemingly "one-hop parallel" IP links may be run over frame relay,
atm, or mpls
links that are of unequal latency. The TCP issues are the same in
either case.
Let's follow this diversion in a little more detail. Precise
modelling of TCP over a complete, general set of network conditions
is a horrendous problem (at least for me) and I don't pretend to be
an experienced researcher in this area. However, I think there are
some coarse, high-level observations we can probably agree on which
will help here.
In the case of multiple, similarly-loaded links between a pair of
routers, the paths between source and unicast destination will be
markedly similar in terms of the metrics which have an impact on TCP
performance (e.g. propagation latency, jitter, segment re-ordering).
In the case of links which connect to different routers which are not
topologically close the paths between source and unicast destination
is much more likely to be different over each path.
TCP functions most effectively when the variation in network
conditions between source and destination is of relatively low
frequency; this allows TCP to adapt its sending rate such that the
pipeline remains reasonably full.
As network conditions change (e.g. a congestion point appears, and
packets become delayed or dropped) TCP adapts its sending behaviour
to maximise throughput. This change cannot happen instantaneously,
since all the signalling and inference happens in-band. Effective
transmission for TCP relies on the fact that network conditions tend
not to change abruptly, and hence incremental modifications in
sending behaviour can keep a TCP session tuned without the window
draining and everything restarting from scratch.
A topology which involves successive segments following paths which
are markedly different represents the pathological case of abrupt
network change; not only is the change abrupt, but it happens with
near peak frequency.
Second, the TCP performance issue noted is minor, not severe.
I have operational experience of the case in hand, and I stand by my
description. Perhaps we differ in our interpretation of the word
"severe".
This seems vague to me. The operational experience you cite is
over parallel
links, and in that case you assert the TCP performance issue is
minor, correct?
I think there is widespread support for this (I think I see you
agreeing with it, in fact :-)
Are you asserting that diverse links suffer more severe TCP
performance issues
than do parallel links? My experience is different. Both are
minor. Neither
case suffers non-operation, nor anything close to non-operation. I
define
"severe" as user visible delays of several seconds or more. Load-
splitting
(parallel or diverse) results in 3% unnecessary TCP
retransmissions, without any
user visible delays.
It certainly seems possible to do PPLB across paths from a single
router which land in different upstream ISPs, and see minimal harm as
you describe above. I am not accusing you of making this stuff up.
However, it is my experience (and that of others here) that this is
the exception rather than the rule.
Third, the use of Equal cost paths is not limited to BGP nor does
is it
dependent on BGP exit selection algorithm.
Indeed, the section on PPLB specifically does not refer to BGP.
Perhaps I misread something. I read the PPLB discussion in section
4.4.3 "Equal Cost Paths", discussing specifically BGP equal cost
paths.
It mentions BGP with the specific intention of mentioning that equal-
cost, multi-path topologies are possible with BGP. It then goes on to
mention other routing protocols in which opportunities to acquire
ECMP routes are far more common (in every BGP implementation I've
seen, there is no multi-path capability by default: you have to
explicitly turn it on).
Its not
terribly clear when you start discsussing PPLB that the problem is
more general
than one of BGP exit policy.
It seems clear to me. However, please feel free to suggest text to
make that more clear.
The phrase "node selection" is intended in this document to indicate
a selected path through one or more interconnected routing systems to
a particular anycast node. It is certainly not a description of some
mechanism by which a client can select a particular node for service.
I think the document is quite clear on this point, but if you have
specific examples of ambiguous text I would be happy to review them.
I gave an example. Your above description is vague and incorrect,
but you
plainly do not see it.
I think you are being over-literal, and judging from the response (or
non-response) of others I think your objections to this turn of
phrase are not widely shared.
Note that the process we are participating in is one of rough
consensus and not universal agreement. This is not the ITU :-)
Joe
_________________________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/grow.html
web archive: http://darkwing.uoregon.edu/~llynch/grow/
