Mirja,
I'm referring to what Christian was summarizing below. Separate
PN spaces but path ID is implicit as the sequence number of the
connection ID, and ACKs reflect this sequence number.
- jana
On Wed, Dec 2, 2020 at 3:02 AM Mirja Kuehlewind
<[email protected]
<mailto:[email protected]>> wrote:
Hi Jana,
can you maybe confirm what you mean by “the design” below
just to make sure we are all on the same page: Is that
different PN spaces per path, but using the same key on all
paths with CIDs as part of the nonce?
Thanks!
Mirja
*From: *QUIC <[email protected]
<mailto:[email protected]>> on behalf of Jana Iyengar
<[email protected] <mailto:[email protected]>>
*Date: *Wednesday, 25. November 2020 at 04:35
*To: *Christian Huitema <[email protected]
<mailto:[email protected]>>
*Cc: *IETF QUIC WG <[email protected] <mailto:[email protected]>>,
Kazuho Oku <[email protected] <mailto:[email protected]>>
*Subject: *Packet number spaces in multipath (was Re: What to
do about multipath in QUIC)
(I'm taking Spencer's suggestion to spin off a new thread.)
Christian, Kazuho,
Slowly catching up on this, and apologies if I'm missing
anything that was previously discussed in the centi-thread
earlier.
If I understand the design correctly, it makes sense to me,
and is very close to what we had implemented in Chromium a
while ago.
Having thought about this problem several times in the past,
I'd like to share a few points that come to mind.
First though, a point on terminology: the receiver maintains
a separate "ReceivedPackets" for each CID, probably for each
CID sequence number (CSN). Let's please not call this a SACK
Dashboard, to avoid confusion.
On the question of sending more than 2^32 packets, I think
that resetting the packet number (PN) is ok on new CIDs. I
don't see why a sender would need to maintain continuity
across multiple paths anyways, since the CC and loss
recovery contexts are going to be different across paths. A
sender _could_ still maintain these packets in the same
"SentPackets" structure if it wants to, it would need an
internal representation of CSN+PN to key off.
ACK Frames:
------------------
Kazuho pointed out that when acknowledging, the ACK frame
format should include CSN. I agree. I would argue for a
design where a receiver uses an ACK frame per CSN (and
encodes the CSN explicitly). There are multiple values for
doing this, the primary being that you benefit from
compression when PNs are contiguous within a CSN.
Return Path:
-----------------
There are other ways to decide which return path to send an
ACK on this, but I would propose that a receiver respond on
the most recently active forward path. That is, the path on
which a packet was most recently received. This has the
natural effect that a sender that wants to distribute
traffic in a particular way also causes ACKs to be
distributed similarly across the corresponding reverse paths.
RTT measurements:
---------------------------
The return path for ACK frames will impact RTT measurements.
That is fine. It is more important that information reach the
sender as soon as possible than that it should not affect RTT
measurements; we can fix the sender to measure and compensate
as necessary. The estimated RTT statistics reflect the
distribution of samples, and if both paths are being used,
then the SmoothedRTT will reflect the expected value based on
the traffic distribution across paths.
That said, it might be useful to track some new stats,
especially about how much later is a "late ack" -- an
ACK frame that contains no useful information -- is received.
I'd have to think a bit more about this, but I think we can
devise a stat here. This gives us useful information on the
longest return path, which we can then explicitly use as part
of the PTO computations, to compensate for the fact that the
RTT is based on the shortest return path. (I would _not_ use
this stat in the time-based loss detection timer, but PTO
ought to be fine.)
- jana
On Tue, Nov 17, 2020 at 9:42 AM Christian Huitema
<[email protected] <mailto:[email protected]>> wrote:
I have been thinking about variations of that. I think we
are making progress here.
If we follow your design, we get two constraints:
1) That the receive maintains an acknowledgement list
based on the CID through which the packets are received.
2) That the senders guarantee that the same sequence
number will not be used more than once with a specific CID.
The main implementation cost is for receivers. They have
to allocate and maintain a "SACK Dashboard" in the
context of each CID that they issue.
Senders have lots of control. For example, the "only
once" condition is also met if a simple sender uses a
single number space, as long as it does not send more
than 2^32 packets. That makes the design reasonably easy
to implement for constrained implementations.
Zero length CID are still possible, but that means the
receiver supports only one PN space per sender. Multipath
is not impossible, but you end up managing a single RTT
and a single recovery structure. Not very good, but
similar to what happens if multipath is implemented at
the IP level.
There is still an issue for coordinating the take down of
a path. Suppose that a client was using both Wi-Fi and
LTE, and moves out of Wi-Fi range. The server will find
out eventually that the packets sent to the Wi-Fi path
are never acknowledged, but that may take some time. It
would be better if the client could send a message saying
something like "Abandon this path". That's not the same
semantic as "retire this CID". We need a new frame for that.
"Abandon this path" is an extreme case. There are
half-way steps, like manage the relative priority of a path.
