For many use cases, reordering tolerance (on "hosts" in IETF terminology) is
possible and very useful for some intermediate nodes (routers, switches).
But #1:
Ordering would be always a requirement for some use cases:
1. Real-time communication (especially anything interactive). This use case has
no time to wait "maybe the packet would arrive later".
2. High-speed transmission accelerated in NIC. It is prohibitively expensive to
have such a big memory just for reordering functionality.
But #2:
Reordering is a big problem for stateful devices ("middleboxes" in IETF
terminology) - they cannot have such a huge memory and they should not create
latency.
If intermediate nodes should support ordering for some applications and cannot
distinguish between applications,
then all efforts to improve reordering tolerance for some applications would
have minimal usefulness.
This
> write an explicit RFC giving guidance how much reordering is deemed acceptable
Should be radically different "per use case or application".
Ed/
-----Original Message-----
From: Sebastian Moeller <[email protected]>
Sent: Saturday, February 8, 2025 15:03
To: Michael Eriksson <[email protected]>
Cc: Martin Thomson <[email protected]>; Greg White <[email protected]>;
IETF QUIC WG <[email protected]>; [email protected]
Subject: [tsvwg] Re: Robustness to packet reordering
> On 8. Feb 2025, at 11:38, Michael Eriksson
> <[email protected]> wrote:
>
> Well, QUIC receivers can easily handle an almost unlimited amount of
> reordering.
Quick note, if we are not talking about multiplexing within one 5 tuple*, then
I believe there is quite some use cases where the data is only usable if all
segments have been received. So out of order delivery might not actually give
any big advantage in application latency. E.g. if we consider the case with a
slow bottleneck (e.g. WiFi) in an otherwise fast path, there the resequencing
will mostly result in creating line speed bursts from the receiver side of the
bottleneck, if the ratio between bottleneck rate and rate on the reminder of
the path the latency gained for doing out of order delivery instead of
re-sequencing is rather mild.
*) I understand the rationale for adding multiplexing to (each) protocol layer,
but occasionally maybe using multiplexing via the lower layer's mechanism is a
viable option...
> The problem is the senders, and in particular the loss detection and
> congestion controller. As has already been mentioned in the thread, the
> default packet reordering threshold (kPacketThreshold) is only three packets.
>
> It is of course possible to adapt the threshold dynamically as the RFC
> mentions, and with Ack Frequency you can also reduce the number of (spurious)
> acks.
>
> Until it is verified that "all" production QUIC stacks implement dynamic
> packet reordering thresholds, it seems unwise to completely turn off L2
> in-order delivery.
I note that endpoints (EP) and intermediate hops (IH) have different desires in
regards to reordering... EPs really want as little reordering as possible even
if they can deal with it (as resequencing will translate into latency), while
IHs want as much freedom as possible and as little work*.
So IMHO if the IETF truly feels that more re-ordering is tolerable we should
write an explicit RFC giving guidance how much reordering is deemed acceptable,
"completely turn off L2 in-order delivery" seems not in the interest of end
points... I guess giving guidance in units of time might be helpful unless tied
to something an IH might not be privy to like path-RTT/OWD.
*) This is why IHs typically do not seem to restrict reordering to flows that
actually are affected by a link layer packet loss but apply it unconditionally
to all traffic. Getting information about which packet aggregates are
fate-sharing and hence would gain something from re-seqiencing can be
computationally costly to impossible.
Sebastian
>
> /me
>
> From: David Schinazi <[email protected]>
> Subject: [tsvwg] Re: Robustness to packet reordering
> Date: Fri, 7 Feb 2025 00:14:28 +0100 (Central European Standard Time)
>
>> Having the link-layer delay packets to provide them in-order to the
>> transport layer was helpful multiple decades ago when TCP implementations
>> had more naive algorithms and were very resource-constrained. Given how
>> modern implementations work, reordering at the link-layer is almost always
>> harmful. TCP and QUIC stacks can handle reordering well, thanks to both
>> protocol features and implementations having more memory to work with. The
>> delay induced by this reordering will generally cause more harm than good.
>> Furthermore, one of the biggest motivators for QUIC was to break
>> head-of-line blocking and allow out-of-order delivery to the application
>> layer. We have large bodies of data showing that this improves performance.
>> Please disable reordering at the 5G layer.
>>
>> David
>>
>> On Thu, Feb 6, 2025 at 2:15 PM Christian Huitema <[email protected]> wrote:
>>
>> On 2/6/2025 12:55 PM, Martin Thomson wrote:
>> >
>> > On Fri, Feb 7, 2025, at 04:59, Greg White wrote:
>> >> This is an important topic relating to the expectations and
>> >> requirements that transport protocols place on layer 2 protocols.
>> >> In layer 2 standards bodies that I've been involved in, it has
>> >> been understood that "the upper layers" expect in-order delivery,
>> > As far as QUIC goes, it is sensitive to reordering in the network. Some
>> > reordering will be interpreted as damage (Christian cited the relevant
>> > parts) and performance suffers in a few minor way when things arrive out
>> > of order (ACKs are less efficient, data needs to be held, memory accesses
>> > are less likely to be contiguous, etc...).
>> >
>> > However, the idea that the network might seek to "fix" these problems,
>> > when doing so necessarily involves extra work and delays, is not a good
>> > trade. Stuff that is delayed to "fix" a reordering that happened might
>> > delay signals that the QUIC stack could use, even if some data needs to be
>> > held at the endpoint. QUIC packets contain many things, some of which
>> > don't need to be strictly ordered to be useful.
>>
>> Applications that are sensitive to delays will break their traffic
>> into multiple QUIC streams. In case of packet loss, only one of those
>> streams will be blocked, the others will be delivered without "head
>> of queue blocking". Implementing L2 correction will make the response
>> worse, not better, because all the streams will be delayed for the
>> duration of the
>> L2 correction instead of just one.
>>
>> -- Christian Huitema
>>
>
>
