If you want to send unreliable data with sequencing information, it
might be simpler to use STREAM frame in "unreliable" mode than to use
Datagram frames.
On 11/26/2020 3:34 AM, [email protected] wrote:
Dear all,
sry for hijacking this conversation. I’m not very familiar with the
different multipath designs for QUIC, however I want to draw attention
to multipath re-ordering which probably becomes important when
multipath is combined with DATAGRAM.
As long as multipath QUIC is operated with strict reliability (similar
to TCP), re-ordering on receiver side is a simple process known from
MPTCP. Introducing unreliable DATAGRAM transmission makes it more
challenging on receiver side to maintain the packet order, because it
is not easy to differentiate between delayed and lost packets. To
avoid HoL, a multipath re-ordering process may benefit from having
connection and path sequencing. In
https://tools.ietf.org/html/draft-amend-iccrg-multipath-reordering-01
<https://tools.ietf.org/html/draft-amend-iccrg-multipath-reordering-01>
we intend to describe this in section 5.6, how fast packet loss
detection can be applied using these different packet sequence spaces.
Still the description is meaningless und will be updated until next
IETF, however we have successfully implemented this approach in a
MP-DCCP prototype, which faces similar challenges in terms of
re-ordering. That means, fast packet loss detection is very beneficial
for the receiver re-ordering process to not lose time until an
outstanding packet is assumed lost.
Br
Markus
**
*From:*QUIC <[email protected]> *On Behalf Of *Jana Iyengar
*Sent:* Mittwoch, 25. November 2020 04:35
*To:* Christian Huitema <[email protected]>
*Cc:* IETF QUIC WG <[email protected]>; Kazuho Oku <[email protected]>
*Subject:* Packet number spaces in multipath (was Re: What to do about
multipath in QUIC)
(I'm taking Spencer's suggestion to spin off a new thread.)
Christian, Kazuho,
Slowly catching up on this, and apologies if I'm missing anything that
was previously discussed in the centi-thread earlier.
If I understand the design correctly, it makes sense to me, and is
very close to what we had implemented in Chromium a while ago.
Having thought about this problem several times in the past, I'd like
to share a few points that come to mind.
First though, a point on terminology: the receiver maintains a
separate "ReceivedPackets" for each CID, probably for each CID
sequence number (CSN). Let's please not call this a SACK Dashboard, to
avoid confusion.
On the question of sending more than 2^32 packets, I think that
resetting the packet number (PN) is ok on new CIDs. I don't see why a
sender would need to maintain continuity across multiple paths
anyways, since the CC and loss recovery contexts are going to be
different across paths. A sender _could_ still maintain these packets
in the same "SentPackets" structure if it wants to, it would need an
internal representation of CSN+PN to key off.
ACK Frames:
------------------
Kazuho pointed out that when acknowledging, the ACK frame format
should include CSN. I agree. I would argue for a design where a
receiver uses an ACK frame per CSN (and encodes the CSN explicitly).
There are multiple values for doing this, the primary being that you
benefit from compression when PNs are contiguous within a CSN.
Return Path:
-----------------
There are other ways to decide which return path to send an ACK on
this, but I would propose that a receiver respond on the most recently
active forward path. That is, the path on which a packet was most
recently received. This has the natural effect that a sender that
wants to distribute traffic in a particular way also causes ACKs to be
distributed similarly across the corresponding reverse paths.
RTT measurements:
---------------------------
The return path for ACK frames will impact RTT measurements. That is
fine. It is more important that information reach the sender as soon
as possible than that it should not affect RTT measurements; we can
fix the sender to measure and compensate as necessary. The estimated
RTT statistics reflect the distribution of samples, and if both paths
are being used, then the SmoothedRTT will reflect the expected value
based on the traffic distribution across paths.
That said, it might be useful to track some new stats, especially
about how much later is a "late ack" -- an ACK frame that contains no
useful information -- is received. I'd have to think a bit more about
this, but I think we can devise a stat here. This gives us useful
information on the longest return path, which we can then explicitly
use as part of the PTO computations, to compensate for the fact that
the RTT is based on the shortest return path. (I would _not_ use this
stat in the time-based loss detection timer, but PTO ought to be fine.)
- jana
On Tue, Nov 17, 2020 at 9:42 AM Christian Huitema <[email protected]
<mailto:[email protected]>> wrote:
I have been thinking about variations of that. I think we are
making progress here.
If we follow your design, we get two constraints:
1) That the receive maintains an acknowledgement list based on the
CID through which the packets are received.
2) That the senders guarantee that the same sequence number will
not be used more than once with a specific CID.
The main implementation cost is for receivers. They have to
allocate and maintain a "SACK Dashboard" in the context of each
CID that they issue.
Senders have lots of control. For example, the "only once"
condition is also met if a simple sender uses a single number
space, as long as it does not send more than 2^32 packets. That
makes the design reasonably easy to implement for constrained
implementations.
Zero length CID are still possible, but that means the receiver
supports only one PN space per sender. Multipath is not
impossible, but you end up managing a single RTT and a single
recovery structure. Not very good, but similar to what happens if
multipath is implemented at the IP level.
There is still an issue for coordinating the take down of a path.
Suppose that a client was using both Wi-Fi and LTE, and moves out
of Wi-Fi range. The server will find out eventually that the
packets sent to the Wi-Fi path are never acknowledged, but that
may take some time. It would be better if the client could send a
message saying something like "Abandon this path". That's not the
same semantic as "retire this CID". We need a new frame for that.
"Abandon this path" is an extreme case. There are half-way steps,
like manage the relative priority of a path.
