Jana and all,
I just have a concern about such a design using sequence number of the
connection ID as the path ID.
Assume that the server is willing to offer two available connection IDs
at any time, and that the client is using both at the same time for
multipath transfer. It uses the CID with seq num 0 on path A and the CID
with seq num 1 on path B. Then, for any reason (like privacy), the
client wants to change the connection ID used on path B (using seq num
1). However, the host cannot retire the CID with seq num 1 without
retiring the CID with seq num 0 too. If the server does not want to
provide additional connection IDs and the client is not willing to reuse
CID with seq num 1 on path A, the client is stuck with the CID with seq
num 0 on path A and cannot use the path B anymore.
This is why I believe we should not link the path ID to the sequence
number of the Connection ID (because it is a monotonically increasing
sequence number), and rather have a separate space for them.
Best regards,
Quentin
Mirja,
I'm referring to what Christian was summarizing below. Separate PN
spaces but path ID is implicit as the sequence number of the
connection ID, and ACKs reflect this sequence number.
- jana
On Wed, Dec 2, 2020 at 3:02 AM Mirja Kuehlewind
<[email protected] <mailto:[email protected]>>
wrote:
Hi Jana,
can you maybe confirm what you mean by “the design” below just to
make sure we are all on the same page: Is that different PN spaces
per path, but using the same key on all paths with CIDs as part of
the nonce?
Thanks!
Mirja
*From: *QUIC <[email protected]
<mailto:[email protected]>> on behalf of Jana Iyengar
<[email protected] <mailto:[email protected]>>
*Date: *Wednesday, 25. November 2020 at 04:35
*To: *Christian Huitema <[email protected]
<mailto:[email protected]>>
*Cc: *IETF QUIC WG <[email protected] <mailto:[email protected]>>, Kazuho
Oku <[email protected] <mailto:[email protected]>>
*Subject: *Packet number spaces in multipath (was Re: What to do
about multipath in QUIC)
(I'm taking Spencer's suggestion to spin off a new thread.)
Christian, Kazuho,
Slowly catching up on this, and apologies if I'm missing anything
that was previously discussed in the centi-thread earlier.
If I understand the design correctly, it makes sense to me, and is
very close to what we had implemented in Chromium a while ago.
Having thought about this problem several times in the past, I'd
like to share a few points that come to mind.
First though, a point on terminology: the receiver maintains a
separate "ReceivedPackets" for each CID, probably for each CID
sequence number (CSN). Let's please not call this a SACK
Dashboard, to avoid confusion.
On the question of sending more than 2^32 packets, I think that
resetting the packet number (PN) is ok on new CIDs. I don't see
why a sender would need to maintain continuity across multiple
paths anyways, since the CC and loss recovery contexts are going
to be different across paths. A sender _could_ still maintain
these packets in the same "SentPackets" structure if it wants to,
it would need an internal representation of CSN+PN to key off.
ACK Frames:
------------------
Kazuho pointed out that when acknowledging, the ACK frame format
should include CSN. I agree. I would argue for a design where a
receiver uses an ACK frame per CSN (and encodes the CSN
explicitly). There are multiple values for doing this, the primary
being that you benefit from compression when PNs are contiguous
within a CSN.
Return Path:
-----------------
There are other ways to decide which return path to send an ACK on
this, but I would propose that a receiver respond on the most
recently active forward path. That is, the path on which a packet
was most recently received. This has the natural effect that a
sender that wants to distribute traffic in a particular way also
causes ACKs to be distributed similarly across the corresponding
reverse paths.
RTT measurements:
---------------------------
The return path for ACK frames will impact RTT measurements. That
is fine. It is more important that information reach the sender as
soon as possible than that it should not affect RTT measurements;
we can fix the sender to measure and compensate as necessary. The
estimated RTT statistics reflect the distribution of samples, and
if both paths are being used, then the SmoothedRTT will reflect
the expected value based on the traffic distribution across paths.
That said, it might be useful to track some new stats, especially
about how much later is a "late ack" -- an ACK frame that contains
no useful information -- is received. I'd have to think a bit more
about this, but I think we can devise a stat here. This gives us
useful information on the longest return path, which we can then
explicitly use as part of the PTO computations, to compensate for
the fact that the RTT is based on the shortest return path. (I
would _not_ use this stat in the time-based loss detection timer,
but PTO ought to be fine.)
- jana
On Tue, Nov 17, 2020 at 9:42 AM Christian Huitema
<[email protected] <mailto:[email protected]>> wrote:
I have been thinking about variations of that. I think we are
making progress here.
If we follow your design, we get two constraints:
1) That the receive maintains an acknowledgement list based on
the CID through which the packets are received.
2) That the senders guarantee that the same sequence number
will not be used more than once with a specific CID.
The main implementation cost is for receivers. They have to
allocate and maintain a "SACK Dashboard" in the context of
each CID that they issue.
Senders have lots of control. For example, the "only once"
condition is also met if a simple sender uses a single number
space, as long as it does not send more than 2^32 packets.
That makes the design reasonably easy to implement for
constrained implementations.
Zero length CID are still possible, but that means the
receiver supports only one PN space per sender. Multipath is
not impossible, but you end up managing a single RTT and a
single recovery structure. Not very good, but similar to what
happens if multipath is implemented at the IP level.
There is still an issue for coordinating the take down of a
path. Suppose that a client was using both Wi-Fi and LTE, and
moves out of Wi-Fi range. The server will find out eventually
that the packets sent to the Wi-Fi path are never
acknowledged, but that may take some time. It would be better
if the client could send a message saying something like
"Abandon this path". That's not the same semantic as "retire
this CID". We need a new frame for that.
"Abandon this path" is an extreme case. There are half-way
steps, like manage the relative priority of a path.
