Hi David, see below.
From: QUIC <[email protected]> on behalf of David Schinazi <[email protected]> Date: Saturday, 24. October 2020 at 03:20 To: Mirja Kuehlewind <[email protected]> Cc: IETF QUIC WG <[email protected]>, Matt Joras <[email protected]> Subject: Re: "Proxying" multipath usecases and application scheduling On Fri, Oct 23, 2020 at 5:54 PM Mirja Kuehlewind <[email protected]<mailto:[email protected]>> wrote: Hi Matt, I think this discussion rather would belong on the MASQUE list and I guess we already had some of this before chartering MASQUE but let me reply to a couple of comments below. As per the MASQUE charter, multipath is out of scope for MASQUE. I'd personally suggest we keep all multipath QUIC conversations on the QUIC list for now. The point I was making is that most of the discussion below (so far) is not about multipath but about use of QUIC as a tunneling protocol with a proxy function sitting in the network which is what MASQUE is looking at. From: QUIC <[email protected]<mailto:[email protected]>> on behalf of Matt Joras <[email protected]<mailto:[email protected]>> Date: Friday, 23. October 2020 at 22:10 To: IETF QUIC WG <[email protected]<mailto:[email protected]>> Subject: "Proxying" multipath usecases and application scheduling Following up from Martin Thomson's excellent summary of the BoF-esque multipath QUIC interim, I thought I'd start a discussion extracting a couple things I saw as important takeaways from the presentation and subsequent questions. As David noted, there were essentially two categories of usecases presented. The first, embodied by the Apple and Alibaba presentations, were essentially existing applications that can benefit from a transport with the capability to simultaneously utilize multiple paths. For these it seemed everyone agreed that in order for multiple paths to be useful the application has to have input into the scheduling decisions on these paths. For example, the application knows how it wants to trade off latency of delivery and overall throughput of delivery for application data. Application input is needed if and only if there is actually a tradeoff to make. However, there are many scenarios where one can actually chose between a link that is better than the other, regarding both delay and latency, and then the decision is easy and the same for all applications. Could you elaborate? Choosing between links applies to connection migration, multipath is a much more complex beast. Yes, that is definitely also something to consider with connection migration and as I said below making the right choice when to switch to a new path is not easy (also with migration when used for handovers). Having the possibility to use paths simultaneous for a while and get measurement data for both paths can help a lot. However, just because you are able to use multiple paths simultaneously that does not mean that you have to do that all the time in all scenarios. There are many scenarios where you rather pick the best one (until you have reached threshold indicating performance degradation). Again sending some probing traffic on the other path is beneficial to make the handover decision. The second set of usecases utilized multipath "in the network". ATSSS and the two variants of Hybrid Access Networks presented fall into this category. In this scheme application data is transparently proxied over multiple paths. This would include things like TCP flows, which could be intercepted in the traditional way TCP PEPs operate, as well as UDP (including QUIC) datagrams. Intercepting TCP and tunneling is far not the same thing. There are many functions today that tunnel your traffic in some way in some network. Traffic is rerouted or EMCP is used to choose a path. These functions are needed for network management and network resilience and it’s explicitly the role of the end-to-end transport to adapt to changing network conditions and handle these situations. Both types of traffic would be scheduled along the multiple paths towards the "other end" of the proxy, where it presumably re-enters the general Internet towards the destination endpoint. Prior to the meeting this latter set of usecases always felt concerning to me, and the meeting did an excellent job of crystallizing my concerns. As others have said, a great benefit of QUIC has been its ability to "cut through" the meddling of things like TCP PEPs, which are often making transport-level decisions that harm the goals of the endpoint application. While the in-network multipath schemes cannot meddle with QUIC flows as much as TCP PEPs, I am still very concerned about the consequences of utilizing them. Again for me this is by far not the same and threating everything that is done with your flows in the network the same seems just oversimplifying it. We seem to have established that application input is necessary to achieve the benefits from utilizing multiple paths, as there is not one "best" way to schedule data. There are cases where there is actually a best way, whenever one path is better than the other without having to make tradeoff, and as I said in my other mail its usually not only application input that is needed to make a decision. E.g. Christoph brought this example that mobile data should be avoided and of course it also depends on the network condition, and that’s where congestion control comes into play. These proxyng solutions have no way to act on application concerns, and rather have to devise scheduling policies solely from network-level information, ignorant of signals from the application. Does this not inevitably lead to pathological scheduling decisions and applications which are helpless to do anything about it? The obvious counterpoint is that these proxies, through advent of being embedded "in the network", have access to information which the endpoint does not, e.g. about the state of the paths. An argument can be made that because of this they can make proactive decisions, rather than relying on the endpoint to react to changes. This is of course true, but completely ignores the fact that without the application context, it is impossible for the proxy to make an optimal scheduling decision. Again there are scenarios where there is clearly a better path, or where there is a strong desire for the network to avoid usage of a certain. E.g. in the hybrid access case, some operators lease one of the links from another operator. I think it’s understandable that they only offer dual connectivity if they have a way to minimize usage of the more expensive line. There are many scenario where network management decision need to be made independent of application characteristics. And regarding ATSSS is already exists (based on MPTCP for TCP and ATSSS-LL for other traffic) and will be used, similar as other techniques that impact routing. I think it’s understandable that operator try to utilize all available resources and usually that actually means that more capacity and a better is available for the user/the operator’s costumer. We can argue about the best technical way to make use of these resources but a) the final decision about ATSSS will be made in 3GPP and not the IETF and b) having an explicit proxy setup where the client can decide to use as it or not, as it is done also in other masque use case, seems to me actually a good approach (or at minimum better than what we have right now). When you say "ATSSS will be used", do you have references? I'm not aware of any smartphone provider planning to support ATSSS but I might have missed it. ATSSS is 5G, however, the MPTCP/TCP converter based solution is deployed for hybrid access as also standardized by BBF. If the only benefit to running such proxies "in the network" is their extended knowledge of network conditions, wouldn't a better path forward to have endpoints engaging in "native" multipath? To solve the information gap, i.e. the fact that the "network knows more", could we not develop mechanisms for signalling this information proactively to endpoints? I guess you can make the same proposal in the other direction: why cannot the endpoints tell the network what the application needs? I think this is a discussion to have in the APN BoF, however, we had this discussion many times and the short answer is trust. Might probably is a longer answer as well and there is a lot of disagreement. End-user privacy? I only meant to say I don’t think we should have this discussion here and now. Now these concerns may not sound QUIC-specific, but I think it is still relevant to the working group for the following reason. Usecases are very important towards driving design of something like a multipath protocol. It seems to me that one of the two major categories of usecases for a multipath QUIC is, by design, incompatible with some of the core tenets of QUIC as an application-enhancing transport protocol. I think that is a way to broad statement and far too simplified. There have been so many discussions in the IETF including in the QUIC working and we clearly don’t have agreement about what the right level of cooperation is because there is just not a simple solution. I fully understand the pain we had with TCP. I’ve been working so long on ECN and am still sad how this good idea git killed in its infancy because of a buggy home router implementation that badly interfered. However, I personally also still believe that there is a way to further optimize and better utilize the resource we have if there is some cooperation between the network and endpoints. Anyway that’s my view on a different topic and not the point of this discussion. As Martin said, we obviously can't stop anyone from utilizing multipath in this way, and it of course does not serve as a reason to not design a multipath QUIC, but it seems to me something we should all keep in mind as we consider such designs. I think we should not consider this to make a decision on multipath support in QUIC because as you said you can’t stop it. The only thing you can achieve for ATSSS is that 3GPP may design some proprietary way to realize this function which will further increase complexity in an already complex system. However, that also an different discussion to have. I’m been trying here to mainly explain what 3GPP is doing on ATSSS in order to get this group some understand of what the requirements are. However, I mainly just think we should simply have multipath support in QUIC because a modern protocol should be able to utilize the fact that more and more devices have connectivity to multiple network paths. I think using path migration for handovers is a bit a hack and not the real solution because it’s actually not easy for an endpoint without further network knowledge to detect when it is the right time to switch. Having a more complete multipath feature were you have two congestion controllers and can get measurements from both paths does help that problem and we do have good experience with that for MPTCP. I'll point out that "we should simply have multipath support in QUIC" is very optimistic, I really do not expect this to be simple. By “simple” I meant the decision to add multipath support to QUIC. Still, adding multipath support to QUIC is actually much simpler than is was for MPTCP as QUIC is already halve way there. When we talk about adding multipath support, that means the signaling to setup and use multiple paths simultaneous. As Olivier explained already many times, there is no desire to standardize path managers and schedulers. However, these are also parts when we can actually directly leverage the experience gained with MPTCP and what I see from actual deployments is that the path manager will be simply as most scenarios only have one interface at the server and a client with two interfaces which usually opens the second subpath as soon as the interface comes available; and there is a small set of schedulers that actually serve most use cases well, as shown in Christoph’s presentation. We seems to disagree in this group also about the complexity multipath support would add. Yes very feature you add, adds new lines to your code (somethings even features your remove can add new code…), however, if people think this is an important argument for the decisions we should further discuss that and more objectively assess the proposed approaches to that respect, rather than exchanging options or guesses. So far the discussion has focusd on use cases and the question if someone would use the feature if it would be there (I believe the answer to that question is yes) and also if there are enough people who would actively work on that (I also believe the answer to that question is also yes but there again seems to be some disagreement). I definitely would welcome more discussion about the proposed approaches at this point! Mirja David Mirja I am very curious to hear others' opinions in this direction and thanks for sitting through this wall of text. Matt Joras
