Hi Ekr, thanks for your prompt reply.
> Thanks for your note. I don't think your proposal will be an > improvement. It destroys information which could otherwise be used for > improve round-trip and loss estimation (cf. the difference between > QUIC and TCP ACKs). What information is destroyed that was there before? You can still use handshake metadata in place of record sequence numbers and do the same analysis as before. Also, with QUIC and TCP, we're mainly talking about throughput optimization at the application stage, where DTLS doesn't ACK anyway. > Second, it prevents the receiver from saying > some non-sensical things like acknowledging part of a received packet. > It's of course true that the current design allows you to ACK >non-received packets, but that's much more straightforward to detect. I don't think this is actually a complication: In both cases you have a list of messages you've sent which are tagged somehow -- either by a record sequence number or handshake metadata which at this point can be treated opaque -- and if you receive an ACK for a non-existent tag, you ignore it or fail, whatever the policy. Beyond that, I consider the more fine-grained ACKing at the handshake level a benefit, because it allows to acknowledge parts of records containing multiple handshake messages only some of which could be processed. Example: +---------------------------------+ | Seq No 1 |---------------> Received | Fragment 0..a | +---------------------------------+ +---------------------------------+ lost | Seq No 1 |----------x | Fragment a+1..b | +---------------------------------+ +-----------------+----------+ | Seq No 1 | Seq no 2 |-------------------> Received | Fragment b+1..N | complete | +-----------------+----------+ If this happens with an implementation which only supports contiguous reassembly (I think GnuTLS does it this way, or at least used to), it will drop the second fragment, but the second message might still be buffered and hence ACK'ed. With record-level ACKs, such fine-grained ACKing isn't possible, and HS Seq No 2 will need to be resent even though it has been received. > I don't think your proposal will be an improvement. I uphold this: ACKing at the handshake level does not lead to complications while (a) easing memory efficient implementations for IoT devices and (b) increasing conceptual clarity by avoiding the current semantic ambiguity of record level ACK with dependency on implementation-specific handshake level details like buffering and reassembly strategy. > Yes, I agree that you need to only ACK data which you have processed. > I would be happy to take a PR to clarify this point. It's actually a MUST only, not a need only, which is an important point because it means that ACKing can _not_ be implemented as an automatic mechanism at the record layer. I'll file a PR once the discussion has concluded. Best, Hanno ________________________________ From: Eric Rescorla <e...@rtfm.com> Sent: Friday, February 28, 2020 2:44 PM To: Hanno Becker <hanno.bec...@arm.com> Cc: tls@ietf.org <tls@ietf.org> Subject: Re: [TLS] Record-level ACKs in DTLS 1.3 Hanno, Thanks for your note. I don't think your proposal will be an improvement. It destroys information which could otherwise be used for improve round-trip and loss estimation (cf. the difference between QUIC and TCP ACKs). Second, it prevents the receiver from saying some non-sensical things like acknowledging part of a received packet. It's of course true that the current design allows you to ACK non-received packets, but that's much more straightforward to detect. I agree that the current design requires keeping somewhat more state during the period when no ACKs have been received. However, your proposal actually requires retaining a similar data structure once they have been received. Some detailed comments below. > Hi, > > TL;DR > This is all about various aspects of how ACKs work in DTLS 1.3: > - The DTLS 1.3 specification requires clarification regarding > when ACKs should be sent. > - Record-level ACKs make efficient implementations for IoT > devices harder. I argue that handshake-level ACKs reduce > implementation complexity and allow for optimized > implementations. > > Details: > To illustrate, consider the following flight exchange, where the > second and third message in the second flight get reordered: > > Client Server > +----------+ > | Seq No 1 |--------------------------> Received > | Rec No a | > +----------+ > +---------------+ > Received <------------------------| Seq No 1 | > | Rec No b | > +---------------+ > +---------------+ > +-------------| Seq No 2 | > | | Rec No c | > | +---------------+ > | > | +---------------+ > Received <------------------------| Seq No 3 | > | | Rec No d | > | +---------------+ > +-----------+ | > | ACK |-------------------------> > | RecNo ??? | | > +-----------+ | > | > | > | > Received <----------+ > > The specification recommends that the client SHOULD send an ACK > when it receives out-of-order handshake message - here message (3,d) > while awaiting (2,c). However: > > Question: > Which records does the client acknowledge in the ACK? > > This is in fact implementation-dependent: > The client must only acknowledge record (3,d) if it has buffered it. > Implementations which don't implement out of order buffering of handshake > messages (to save RAM and ROM) must not acknowledge messages that weren't > buffered. The reason is that otherwise the server is mislead in which messages > need resending and which don't. > > Conclusion 1: > If ACKs must only be sent for records which contain handshake messages > which were actually fed into the flight buffering / reassembly module > of the implementation, the specification should clearly say so. > I consider this point to be prone to misinterpretation, potentially > leading to incompatible implementations, because ACKs acknowledge > receipt at record granularity, and yet they must be sent only under > some assumptions on how their content was processed. Yes, I agree that you need to only ACK data which you have processed. I would be happy to take a PR to clarify this point. > As I understand, the simplest implementation of the retransmission state > machine using record-level ACKs works by buffering copies of records sent > in a flight and retransmits those that don't get acknowledged. The record > sequence numbers of the original and retransmitted records are internally > associated with the opaque record content, so that ACKs for both the original > and the retransmitted records can be recognized. > > In this approach, the record contents are opaque to the sender's > retransmission > state machine, which simplifies the implementation. However, it has major > drawbacks: > > (1) It requires buffering the entire flight, which incurs high memory > overhead that can be significant on constrained devices. > > Implementations should be allowed to not buffer handshake > messages but re-generate them on the fly through a callback whenever > retransmission is needed. Note that many handshake messages cannot be re-generated without extensive caching (for instance, ServerHello). However, as long as the messages are generated consistently (a requirement in any case), then you in fact need-not buffer them in memory; you merely keep the record -> offset mappings as an edit list. > Two examples: > - Consider the Certificate message: The raw certificate > (chain) must reside in RAM/ROM already, and the Certificate message > could easily be re-regenerated from that. Buffering handshake messages > instead creates significant and unnecessary overhead. > > - This is of increasing importance with the advent of post-quantum > cryptography, which comes with significantly larger key material. > > (2) It doesn't allow switching the MTU for retransmission. > > Once the MTU may change for retransmission, the sender cannot keep > track anymore of a single set of record sequence numbers per message such > that an ACK for any of them confirms receipt of the message. > > Of course, no implementation is strictly _forced_ to follow the above > approach, > but the current record-level ACKs design significantly hardens any other > approach: > For example, retransmission through callbacks or support for MTU switching > requires maintaining the tuples (record seq nr, lists of handshake fragments) > for all transmissions of the last flight. (Note that, in particular, just > remembering the mapping for the last transmission isn't enough, since an > acknowledgement for an earlier transmission might arrive late.) Well, sort of. Nothing forbids duplicate transmission, so just remembering the last transmission is mostly just inefficient. > Conclusion 2: > Handshake-level ACKs simplify the retransmission state machine > for non-buffering implementations on constrained devices, while > not hardening buffering implementations using record-level ACKs > so far. I'm aware of the lateness of this proposal, but would be > happy if the group would discuss and consider handshake-level ACKs > instead of record-level ACKs. As noted above, I do not believe we should make this change. -Ekr IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
_______________________________________________ TLS mailing list TLS@ietf.org https://www.ietf.org/mailman/listinfo/tls