GREASE-reordering is a brilliant idea, +1 to that. (Particularly for writes that need to be broken up into packets anyway, and probably also for anything still buffered when more writes come in.)
If the end result is that UDP 443 on server side is excluded from reordering buffers, that’s a clear win regardless of any other network decisions that may or may not happen. Best regards, Jake From: Ian Swett <[email protected]> Date: Thu,2021-07-29 at 7:43 PM To: Zaheduzzaman Sarker <[email protected]> Cc: Sergio Garcia Murillo <[email protected]>, "[email protected]" <[email protected]>, Roberto Peon <[email protected]>, Matt Joras <[email protected]>, "Das, Dibakar" <[email protected]>, Alan Frindell <[email protected]>, "[email protected]" <[email protected]>, Anna Brunström <[email protected]>, David Schinazi <[email protected]>, "[email protected]" <[email protected]>, "[email protected]" <[email protected]>, Kirill Pugin <[email protected]> Subject: Re: [Mops] Reminder: Video Ingest over QUIC Side Meeting Friday 7/30 18:00 UTC TLDR: Ideally, handling reordering on the receiving host will reduce user latency and be cheaper for everyone. Changing the TCP ecosystem is slow, so I can't advise immediately doing this for TCP traffic, but doing this now for QUIC(and maybe documenting it in Manageability?) seems viable. If we wait longer, the ecosystem will be ossified. I'm not sure anyone is willing to introduce intentional reordering to GREASE the loss detection system and ensure implementations adapt, but I'll discuss if that's something we would do for some traffic when sending multi-packet bursts. I believe the vast majority of QUIC applications would benefit from increased reordering if it reduced latency and buffering. In TCP, you can detect reordering from the sequence number. But for QUIC, the metadata is encrypted, so a given network can only attempt to reduce reordering in a much narrower context, which is much less useful. Add into that the fact you don't know the application, and the benefits of delaying packets within the network/path become even more difficult to reason about. To frame this in statistical terms, the sum of multiple normal distributions is a normal distribution with the sum of the means and a sum of the variances(https://en.wikipedia.org/wiki/Sum_of_normally_distributed_random_variables<https://urldefense.com/v3/__https:/en.wikipedia.org/wiki/Sum_of_normally_distributed_random_variables__;!!GjvTz_vk!CYEeHbhK7C7RyDCja4rhiCVtdTRzsDAvDFa_mKJVustvWtuOr6WXbuj53eCLHh4$>). Since the standard deviation is the square root of the variance, the standard deviation is dominated by the largest source of variation, and if each link introduces some variance, it's really only the one that introduces the most variation that matters. Given no link knows whether it's the dominant source of variation on the path, I'd suggest every link pass along packets as soon as possible. Obviously, link-layer error correction and other approaches which increase reliability without introducing delay are still welcome and valuable. Thanks, Ian On Thu, Jul 29, 2021 at 2:10 PM Zaheduzzaman Sarker <[email protected]<mailto:[email protected]>> wrote: I would say this is not specific to media over QUIC. However, a potential solution to this could boost the deployment of potential radio technologies ( for example - Unacknowledgement Mode (UM) of RLC in cellular networks) which might actually help low latency media usecases. BR Zahed On 2021-07-29, 22:35, "QUIC on behalf of Sergio Garcia Murillo" <[email protected]<mailto:[email protected]> on behalf of [email protected]<mailto:[email protected]>> wrote: Just for clarification, if this needs to be handled, this should be handled at QUIC level and it is not an specific problem for the Media over QUIC protocol, right? El jue., 29 jul. 2021 18:15, Holland, Jake <[email protected]<mailto:[email protected]>> escribió: Hi Anna, (And replacing Nathalie’s email with the correct one, hopefully) That was not quite my understanding, no. Sorry if I opened this discussion poorly. In Nathalie’s initial response, she clarified that it’s configurable, including the option to disable the reordering buffer. However, I understood this as an option available to the service provider operating the MP-DCCP split path proxies that tunnel the carried traffic, not necessarily to the endpoints or the applications having their traffic split. Depending on how it’s deployed, this may or may not be a problem for particular endpoints, but I couldn’t tell if there was an obvious or easy way for an application to select the use case (particularly one that is just trying to use QUIC and transparently getting traffic-splitting delivery) in a way that the operator will understand, so I thought it best to raise the point for discussion. Best, Jake From: Anna Brunström <[email protected]<mailto:[email protected]>> Date: Wed,2021-07-28 at 5:31 PM To: Matt Joras <[email protected]<mailto:[email protected]>>, "Holland, Jake" <[email protected]<mailto:[email protected]>> Cc: David Schinazi <[email protected]<mailto:[email protected]>>, Roberto Peon <[email protected]<mailto:[email protected]>>, "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>>, "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>>, Ian Swett <[email protected]<mailto:[email protected]>>, "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>>, "Das, Dibakar" <[email protected]<mailto:[email protected]>>, Alan Frindell <[email protected]<mailto:[email protected]>>, "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>>, Kirill Pugin <[email protected]<mailto:[email protected]>> Subject: RE: [Mops] Reminder: Video Ingest over QUIC Side Meeting Friday 7/30 18:00 UTC I think multipath and L2 are a bit different as multipath may lead to large amounts of reordering and delay variations. Not all applications/implementations may handle this well. But you should of course also have the possibility to just pass the packets on, and I agree that in many cases this will be the best. In the context of MP-DCCP, both scheduling and reordering are modular components so you can certainly choose to pass the packets on without delay. The need for supporting this is also stated in the draft. But for the multipath case, I think it is useful to also have the ability to reduce reordering if needed. For ATSSS I guess weather that is desirable or not may be controlled by the selected mode. @Jake: if you understood our conversation in ANRW as suggesting that MP-DCCP should always reorder, that was a misunderstanding. Best, Anna From: Matt Joras <[email protected]<mailto:[email protected]>> Sent: den 29 juli 2021 01:58 To: Holland, Jake <[email protected]<mailto:[email protected]>> Cc: David Schinazi <[email protected]<mailto:[email protected]>>; Roberto Peon <[email protected]<mailto:[email protected]>>; Anna Brunström <[email protected]<mailto:[email protected]>>; [email protected]<mailto:[email protected]>; [email protected]<mailto:[email protected]>; Ian Swett <[email protected]<mailto:[email protected]>>; [email protected]<mailto:[email protected]>; Das, Dibakar <[email protected]<mailto:[email protected]>>; Alan Frindell <[email protected]<mailto:[email protected]>>; [email protected]<mailto:[email protected]>; Kirill Pugin <[email protected]<mailto:[email protected]>> Subject: Re: [Mops] Reminder: Video Ingest over QUIC Side Meeting Friday 7/30 18:00 UTC Speaking as an individual, a couple comments inline adding to what Jake is saying. On Wed, Jul 28, 2021 at 3:58 PM Holland, Jake <[email protected]<mailto:[email protected]>> wrote: And yet, we still can see network buffering to maintain packet ordering in new work today, including work specifically targeted at QUIC. In the “CCID5” ANRW talk from Monday’s 2nd session, a reordering buffer was mentioned in the transparent proxy for the MP-DCCP tunnel they’re wrapping QUIC packets in to get multipath traffic-splitting during transport within wireless carriers (a proposal coming soon to tsvwg): https://irtf.org/anrw/2021/program.html#:~:text=nathalie%20romo%20moreno<https://urldefense.com/v3/__https:/irtf.org/anrw/2021/program.html*:*:text=nathalie*20romo*20moreno__;I34lJQ!!GjvTz_vk!AVpyqSAj3HYNjnpUTryB_6DdElZmzNy4D78JApMaTuSanBUHBhhvFtmh1HJdILQ$> I asked specifically about the reordering buffer at the mic, since this kind of thing has made me sad before. (There was also a side discussion in the chat.) I got the impression they believe they’re seeing some benefits to apps by including this reordering buffer in the network, citing the wider range of reordering that you get for split-path transport (as compared to the L2 forwarding case). I would question whether any application-level benefits could be measured by the network intermediaries. We (i.e., endpoint owners) have a hard enough time surmising application-level improvements from _endpoint_ transport metrics, let alone trying to intuit those as a network intermediary without any application context whatsoever. Making network-level decisions based on limited information is a recipe for "optimizations" that seem good on paper and for a limited set of metrics but are worse in reality. This is exactly the kind of thing that has led to the proliferation of TCP session terminating PEPs, which have overall been a real impedance to making improvements with TCP on the Internet. If there’s endpoint implementations that aren’t doing a good enough job here and therefore we’re seeing new network deployments that introduce buffering because their measurements indicate it’s helpful on common use cases, it could be worthwhile to get this straightened out before it gets too normalized and we start having networks reintroduce hol-blocking in a way that hurts some QUIC use cases in the name of helping others. Although I expect this can and should be solved at the endpoints, if there is data showing that the ordering solves a real problem with current implementations, reasonable people can reasonably conclude that network buffering to maintain packet order is a good idea. (Note: UDP client-side port might be an option for a network-visible signal that might “just work” for many (hopefully most?) ordering schemes to avoid buffering for ordering, but it might need some API support and it might lead to some other kinds of ugly nat problems if there’s too many flows doing it...) +Anna, Nathalie and Markus. Hopefully they can comment on this also. Best, Jake From: David Schinazi <[email protected]<mailto:[email protected]>> Date: Wed,2021-07-28 at 2:16 PM To: Roberto Peon <[email protected]<mailto:[email protected]>> Cc: Ian Swett <[email protected]<mailto:[email protected]>>, "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>>, "Das, Dibakar" <[email protected]<mailto:[email protected]>>, Alan Frindell <[email protected]<mailto:[email protected]>>, "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>>, Kirill Pugin <[email protected]<mailto:[email protected]>> Subject: Re: [Mops] Reminder: Video Ingest over QUIC Side Meeting Friday 7/30 18:00 UTC Why would we need a signal here? This applies to all traffic, be it TCP QUIC or anything else. Firmwares introducing latency to reorder packets was a reaction to bad implementations of TCP from a long time ago that have been fixed in systems that care about performance. In today's world, L2 is better off delivering any and all packets in the order they arrive instead of introducing buffer bloat. David On Wed, Jul 28, 2021 at 1:24 PM Roberto Peon <[email protected]<mailto:[email protected]>> wrote: The ideal would be to have public bits that were intended to be interpreted by (as you say, visible to) those layers so any L2 could adapted appropriately without reinventing the wheel. It isn’t just the local radio firmware that one needs to worry about—it is also the basestation(s) that may be “helping”. Separately, but also important, is being able to get signals from the application about what tradeoffs should be at the network. I believe that this dovetails into many of the multipath issues, btw. A couple potentially interesting params are: A bit to say please don’t HoL block Some kind of mechanism(s) to bound retries (e.g. “don’t retry bit”, but that is obviously not as expressive as throw out packet older than X microseconds) -=R From: QUIC <[email protected]<mailto:[email protected]>> on behalf of Ian Swett <[email protected]<mailto:[email protected]>> Date: Wednesday, July 28, 2021 at 12:42 PM To: "Das, Dibakar" <[email protected]<mailto:[email protected]>> Cc: Alan Frindell <[email protected]<mailto:[email protected]>>, "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>>, "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>>, Kirill Pugin <[email protected]<mailto:[email protected]>> Subject: Re: Reminder: Video Ingest over QUIC Side Meeting Friday 7/30 18:00 UTC Hi, I can't answer for Alan, but my belief is yes. Client wifi stacks sometimes also do some reordering(and introduce the corresponding latency), so if we could design an indication that in-order delivery has no value, it could be fairly widely applicable. That being said, I don't know what the right mechanism is? Would we need something visible to a network or can we get away with a socket option that propagates to the local 5G network or Wifi firmware when possible? Ian On Wed, Jul 28, 2021 at 3:15 PM Das, Dibakar <[email protected]<mailto:[email protected]>> wrote: Hi Kirill, Alan, I could not attend the call this week and wont be able to attend this side meeting either. But I had a general question about the performance of all such QUIC based protocols over wireless. Typically, the 5G and WiFI MAC layers deliver frames in-order which sort of recreates the HOL blocking problem at lower layers. I would expect this to in turn prevent the QUIC protocol to achieve its full performance gains at least in some congested network scenarios. Considering that in-order delivery is made optional in 5G PDCP, I was wondering if there could be a value to have some signaling defined in the QUIC (or RUSH ?) protocol that would allow lower layers to make better decision about whether to enable/disable in-order delivery for certain streams. I apologize in advance if this is not the right venue to ask questions. Regards, Dibakar From: QUIC <[email protected]<mailto:[email protected]>> On Behalf Of Alan Frindell Sent: Wednesday, July 28, 2021 8:42 AM To: [email protected]<mailto:[email protected]>; [email protected]<mailto:[email protected]>; QUIC WG <[email protected]<mailto:[email protected]>>; [email protected]<mailto:[email protected]> Cc: Kirill Pugin <[email protected]<mailto:[email protected]>> Subject: Reminder: Video Ingest over QUIC Side Meeting Friday 7/30 18:00 UTC Video Ingest over QUIC Side Meeting Friday 7/30 18:00 UTC / 11 Pacific Link to draft agenda and video conference details: https://github.com/afrind/draft-rush/blob/main/meeting-materials/agenda.2021.07.03.md<https://urldefense.com/v3/__https:/github.com/afrind/draft-rush/blob/main/meeting-materials/agenda.2021.07.03.md__;!!GjvTz_vk!E0SzSsjcIQqc-TDdIf5-y7XjoWfnEA-7r9fdRAjEKZXc1GYhGomlKIXMwmDZ0Ls$> -Alan När du skickar e-post till Karlstads universitet behandlar vi dina personuppgifter<https://urldefense.com/v3/__https:/www.kau.se/gdpr__;!!GjvTz_vk!AVpyqSAj3HYNjnpUTryB_6DdElZmzNy4D78JApMaTuSanBUHBhhvFtmhpcriyO0$>. When you send an e-mail to Karlstad University, we will process your personal data<https://urldefense.com/v3/__https:/www.kau.se/en/gdpr__;!!GjvTz_vk!AVpyqSAj3HYNjnpUTryB_6DdElZmzNy4D78JApMaTuSanBUHBhhvFtmhrH9ZMhY$>. -- Mops mailing list [email protected]<mailto:[email protected]> https://www.ietf.org/mailman/listinfo/mops<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/mops__;!!GjvTz_vk!CYEeHbhK7C7RyDCja4rhiCVtdTRzsDAvDFa_mKJVustvWtuOr6WXbuj51KXgZqY$>
