For L4S traffic, there "is" a "one shoe fits all"... One of the Prague requirements is that the transport layer must be resilient to out of order delivery (and adapt to the extent of reordering, RACK-like), so that NWs don't need to delay/queue/burst packets to bring them back in order...
So, for bearers that carry L4S (and optionally selective other low latency traffic), there is preferably no back-in-order functionality... The Non-L4S bearers or other network pipes/queues can keep this as an optimization as it will not carry L4S traffic. If it is a legacy pipe, without L4S separated, the queue latency/burstiness will be big anyways. Regards, Koen. -----Original Message----- From: Ingemar Johansson S <[email protected]> Sent: Friday, February 7, 2025 9:27 AM To: Christian Huitema <[email protected]>; Martin Thomson <[email protected]>; Neal Cardwell <[email protected]>; David Schinazi <[email protected]> Cc: Greg White <[email protected]>; [email protected]; [email protected] Subject: [tsvwg] Re: Robustness to packet reordering CAUTION: This is an external email. Please be very careful when clicking links or opening attachments. See the URL nok.it/ext for additional information. Hi And thanks all for the response. I guess there is no "one shoe fits" all here and it is a matter of the lesser of two evils. The in-sequence delivery from lower layers in 3GPP is mainly because of legacy TCP that could not handle more than 3 dupACKs. This is in my opinion becoming less practical, with higher link throughput as it implies larger and larger memory in terminals and cellular network nodes. RACK is interesting as it makes it possible to lift the requirements for in sequence delivery. Then, I think that it can be worth to find some middle ground between completely "out of order" and "in order". In 5G the two features that are relevant in this case and can cause out of order delivery are: + Retransmissions on the MAC layer: This can occur for ~10% of the packet, depending on how the link adaptation block error rate (BLER) target is tuned. Given that radio channels can vary dynamically, the BLER can also vary. More than one retransmission may be needed to recover a block of data. Each retransmission causes an extra delay of a few milliseconds. + Retransmissions on RLC layer: Less common than MAC layer retransmissions and can occur if the block of data cannot be recovered within a few MAC layer retransmissions. RLC reTx can also occur if NACKs of blocks of data on the MAC layer are erroneously interpreted as ACKs. The RLC reTx causes 10s of milliseconds extra delay depending on how timers are configured. The middle ground can here be to ensure in-sequence delivery to cover most of the retransmissions on the MAC layer and forward RLC retransmissions out of order. This can make also legacy TCP "3-dupACK" stacks work in an acceptable way most of the time. As regards to delay jitter. Delay jitter is unfortunately something that one need to live with. Delay jitter occurs for many other reasons than the two listed below, this causes at least the legacy Cubic HyStart to behave badly. The outcome is that one should be careful when using delay metrics for congestion control purposes. Replay protection in IP sec is one thing to consider, however I lack the knowledge on this. /Ingemar > -----Original Message----- > From: Christian Huitema <[email protected]> > Sent: Friday, 7 February 2025 06:30 > To: Martin Thomson <[email protected]>; Neal Cardwell > <[email protected]>; David Schinazi <[email protected]> > Cc: Greg White <[email protected]>; Ingemar Johansson S > <[email protected]>; [email protected]; [email protected] > Subject: Re: [tsvwg] Re: Robustness to packet reordering > > > On 2/6/2025 8:15 PM, Martin Thomson wrote: > > On Fri, Feb 7, 2025, at 13:53, Neal Cardwell wrote: > >> [...] (X less than, say, 10ms), then it may be worth the user's 5G > >> cell phone modem buffering received packets for X milliseconds to > >> try to deliver in-order data to the receiver if this can be done in > >> a low-latency manner. > > These decisions have knock-on effects (what if every hop in the > > chain > were to do the same?), but that does sound reasonable on the surface > of it. That is, there might be some value where spending that time on > restoring order is more productive than exposing the receiver to the > out of order stuff. It's possible that retransmissions on a short hop > of a link fit that. It's possible that reordering at that layer is - > in the aggregate - more efficient, though I'm less convinced of that. > > Actually, +- 10ms does have bad effects. It creates jitter, and it > will throw time-based congestion control algorithms out of whack. I > saw that first hand when testing Cubic, and especially Hystart, on an > LTE connection. Hystart will mistake a jitter event for a sudden > increase of the queuing delay, get out of Slow-Start early, and then > the Cubic algorithm will take a long time to recover. The occasional > 10ms for L2 error correction may seem right at first sight, the error > correction might help some ancient implementation of TCP, but it does > hurt some modern implementations of Cubic and Hystart. > > > However, I would suggest that this is only true if the 10ms would > otherwise be gainfully occupied by the receiver OR the receiver would > not be able to use the data 10ms faster. If the receiver is eager for > work to do, then why deprive it of that opportunity? If you don't > know, why presume? > > Yes. When the network tries to "help the end points", that ends up > counter productive more often than not. > > -- Christian Huitema
