Hi Panos, I realize that TTLB might correlate well for some types of web content, but it's important to recognize that lots of web content is badly bloated (if you can tolerate the invective, this is a pretty good look at the situation, with numbers: https://infrequently.org/series/performance-inequality/).
I don't want to call out your employer's properties in particular, but at over 3M and with relatively few connections, handshakes really don't play much into page load performance. That might be typical, but just being typical doesn't mean that it's a case we should be optimizing for. The 72K page I linked above looks very different. There, your paper shows a 20-25% hit on TTLB. TTFB is likely more affected due to the way congestion controllers work and the fact that you never leave slow start. Cheers, Martin On Fri, Mar 8, 2024, at 13:56, Kampanakis, Panos wrote: > Thx Deirdre for bringing it up. > > David, > > ACK. I think the overall point of our paper is that application > performance is more closely related to PQ TTLB than PQ TTFB/handshake. > > Snippet from the paper > > *> Google’s PageSpeed Insights [12] uses a set of metrics to measure > the user experience and webpage performance. The First Contentful Paint > (FCP), Largest Contentful Paint (LCP), First Input Delay (FID), > Interaction to Next Paint (INP), Total Blocking Time (TBT), and > Cumulative Layout Shift (CLS) metrics include this work’s TTLB along > with other client-side, browser application-specific execution delays. > The PageSpeed Insights TTFB metric measures the total time up to the > point the first byte of data makes it to the client. So, PageSpeed > Insights TTFB is like this work’s TTFB/TLS handshake time with > additional network delays like DNS lookup, redirect, service worker > startup, and request time.* > > Specifically about the Web, TTLB (as defined in the paper) is directly > related to FCP, LCP, FID, INP, TBT, CLS, which are 6 of the 7 metrics > in Google’s PageSpeed Insights. We don’t want to declare that TTLB is > the ultimate metric, but intuitively, I think it is a better indicator > when it comes to application performance than TTFB. > > That does not intend to underestimate the importance of the studies on > handshake performance which was crucial to identify the best performing > new KEMs and signatures. It also does not intend to underestimate the > importance of slimming down PQ TLS 1.3 handshakes as much as possible. > > Side note about Rob’s point: > We have not collected QUIC TTLB data yet, but I want to say that the > paper’s TTLB experimental results could more or less be extended to > QUIC be subtracting one RTT. OK, I don’t have experimental measurements > to prove it yet. So I will only make this claim and stop until I have > more data. > > > > *From:* TLS <tls-boun...@ietf.org> *On Behalf Of * David Benjamin > *Sent:* Thursday, March 7, 2024 3:41 PM > *To:* Deirdre Connolly <durumcrustu...@gmail.com> > *Cc:* TLS@ietf.org > *Subject:* RE: [EXTERNAL] [TLS] Time to first byte vs time to last byte > > *CAUTION*: This email originated from outside of the organization. Do > not click links or open attachments unless you can confirm the sender > and know the content is safe. > > > This is good work, but we need to be wary of getting too excited about > TTLB, and then declaring performance solved. Ultimately, TTLB simply > dampens the impact of postquantum by mixing in the > (handshake-independent) time to do the bulk transfer. The question is > whether that reflects our goals. > > Ultimately, the thing that matters is overall application performance, > which can be complex to measure because you actually have to try that > application. Metrics like TTLB, TTFB, etc., are isolated to one > connection and thus easier to measure, and without checking each > application one by one. But they're only as valuable as they are > predictors of overall application performance. For TTLB, both the > magnitude and desirability of dampening effect are application-specific: > > If your goal is transferring a large file on the backend, such that you > really only care when the operation is complete, then yes, TTLB is a > good proxy for application system performance. You just care about > throughput in that case. Moreover, in such applications, if you are > transferring a lot of data, the dampening effect not only reflects > reality but is larger. > > However, interactive, user-facing applications are different. There, > TTLB is a poor proxy for application performance. For example, on the > web, performance is determined more by how long it takes to display a > meaningful webpage to the user. (We often call this the time to "first > contentful paint".) Now, that is a very high-level metric that is > impacted by all sorts of things, such as whether this is a repeat > visit, page structure, etc. So it is hard to immediately translate that > back down to TLS. But it is frequently much closer to the TTFB side of > the spectrum than the TTLB side. And indeed, we have been seeing > impacts from PQ to our high-level metrics on mobile. > > There's also a pretty natural intuition for this: since there is much > more focus on latency than throughput, optimizing an interactive > application often involves trying to reduce the amount of traffic on > the critical path. The more the application does so, the less accurate > TTLB's dampening effect is, and the closer we trend towards TTFB. (Of > course, some optimizations in this space involve making fewer > connections, etc. But the point here was to give a rough intuition.) > > On Thu, Mar 7, 2024 at 2:58 PM Deirdre Connolly > <durumcrustu...@gmail.com> wrote: >> "At the 2024 Workshop on Measurements, Attacks, and Defenses for the Web >> (MADweb), we presented a paper¹ advocating time to last byte (TTLB) as a >> metric for assessing the total impact of data-heavy, quantum-resistant >> algorithms such as ML-KEM and ML-DSA on real-world TLS 1.3 connections. Our >> paper shows that the new algorithms will have a much lower net effect on >> connections that transfer sizable amounts of data than they do on the TLS >> 1.3 handshake itself." >> >> https://www.amazon.science/blog/delays-from-post-quantum-cryptography-may-not-be-so-bad >> >> ¹ >> https://www.amazon.science/publications/the-impact-of-data-heavy-post-quantum-tls-1-3-on-the-time-to-last-byte-of-real-world-connections/ >> >> >> _______________________________________________ >> TLS mailing list >> TLS@ietf.org >> https://www.ietf.org/mailman/listinfo/tls > _______________________________________________ > TLS mailing list > TLS@ietf.org > https://www.ietf.org/mailman/listinfo/tls _______________________________________________ TLS mailing list TLS@ietf.org https://www.ietf.org/mailman/listinfo/tls
