> > Given that especially for the web, CDNs used much higher initcwnds,
Please, let us not assume every website is behind a CDN. > let's focus on Figure 10. Based on Fig 10, 50-100KB of data over a PQ > connection, the TTLB would be 10-15% slower for 1Mbps and 200ms RTT. At > higher speeds, this percentage is much less (1-1.5% based on Fig 9b), but > let's focus on the slow link. > > If we consider the same case for handshake, then the PQ handshake slowdown > is 30-35% which definitely looks like a very impactful slowdown. A 10-15% > for the TTLB is much less, but someone could argue that even that is a > significant slowdown. Note we are still in a slow link, so even the > classical conn transferring 72KB is probably suffering. To quantify that I > looked at my data from these experiments. A classical connection TTLB for > 50-100KB of data at 1Mbps and 200ms RTT and 0% loss was ~1.25s. This is not > shown in the paper because I only included text about the 10% loss case. > 1.25s for a 72KB page to start getting rendered on a browser over a > classical conn vs 1.25*1.15=1.44s for a PQ one. I am not sure any user > waiting for 1.25s will close the browser at 1.44s. > > Btw, the Google PageSpeed Insights TTFB metric which includes (DNS lookup, > redirects and more) considers 0.8s - 1.8s as "Needs improvement". In our > experiments, the handshake time for 1Mbps and 200ms RTT amounted to 436ms > and 576ms for the classical and PQ handshakes respectively. I am not sure > the extra 140ms (30-35% slowdown) for the PQ handshake would even throw the > Google PageSpeed Insights TTFB metric to the "Needs improvement" category. > > > > -----Original Message----- > From: Martin Thomson <m...@lowentropy.net> > Sent: Thursday, March 7, 2024 10:26 PM > To: Kampanakis, Panos <kpa...@amazon.com>; David Benjamin < > david...@chromium.org>; Deirdre Connolly <durumcrustu...@gmail.com>; Rob > Sayre <say...@gmail.com> > Cc: TLS@ietf.org; Childs-Klein, Will <chi...@amazon.com> > Subject: RE: [EXTERNAL] [TLS] Time to first byte vs time to last byte > > CAUTION: This email originated from outside of the organization. Do not > click links or open attachments unless you can confirm the sender and know > the content is safe. > > > > Hi Panos, > > I realize that TTLB might correlate well for some types of web content, > but it's important to recognize that lots of web content is badly bloated > (if you can tolerate the invective, this is a pretty good look at the > situation, with numbers: > https://infrequently.org/series/performance-inequality/). > > I don't want to call out your employer's properties in particular, but at > over 3M and with relatively few connections, handshakes really don't play > much into page load performance. That might be typical, but just being > typical doesn't mean that it's a case we should be optimizing for. > > The 72K page I linked above looks very different. There, your paper shows > a 20-25% hit on TTLB. TTFB is likely more affected due to the way > congestion controllers work and the fact that you never leave slow start. > > Cheers, > Martin > > On Fri, Mar 8, 2024, at 13:56, Kampanakis, Panos wrote: > > Thx Deirdre for bringing it up. > > > > David, > > > > ACK. I think the overall point of our paper is that application > > performance is more closely related to PQ TTLB than PQ TTFB/handshake. > > > > Snippet from the paper > > > > *> Google’s PageSpeed Insights [12] uses a set of metrics to measure > > the user experience and webpage performance. The First Contentful > > Paint (FCP), Largest Contentful Paint (LCP), First Input Delay (FID), > > Interaction to Next Paint (INP), Total Blocking Time (TBT), and > > Cumulative Layout Shift (CLS) metrics include this work’s TTLB along > > with other client-side, browser application-specific execution delays. > > The PageSpeed Insights TTFB metric measures the total time up to the > > point the first byte of data makes it to the client. So, PageSpeed > > Insights TTFB is like this work’s TTFB/TLS handshake time with > > additional network delays like DNS lookup, redirect, service worker > > startup, and request time.* > > > > Specifically about the Web, TTLB (as defined in the paper) is directly > > related to FCP, LCP, FID, INP, TBT, CLS, which are 6 of the 7 metrics > > in Google’s PageSpeed Insights. We don’t want to declare that TTLB is > > the ultimate metric, but intuitively, I think it is a better indicator > > when it comes to application performance than TTFB. > > > > That does not intend to underestimate the importance of the studies on > > handshake performance which was crucial to identify the best > > performing new KEMs and signatures. It also does not intend to > > underestimate the importance of slimming down PQ TLS 1.3 handshakes as > much as possible. > > > > Side note about Rob’s point: > > We have not collected QUIC TTLB data yet, but I want to say that the > > paper’s TTLB experimental results could more or less be extended to > > QUIC be subtracting one RTT. OK, I don’t have experimental > > measurements to prove it yet. So I will only make this claim and stop > > until I have more data. > > > > > > > > *From:* TLS <tls-boun...@ietf.org> *On Behalf Of * David Benjamin > > *Sent:* Thursday, March 7, 2024 3:41 PM > > *To:* Deirdre Connolly <durumcrustu...@gmail.com> > > *Cc:* TLS@ietf.org > > *Subject:* RE: [EXTERNAL] [TLS] Time to first byte vs time to last > > byte > > > > *CAUTION*: This email originated from outside of the organization. Do > > not click links or open attachments unless you can confirm the sender > > and know the content is safe. > > > > > > This is good work, but we need to be wary of getting too excited about > > TTLB, and then declaring performance solved. Ultimately, TTLB simply > > dampens the impact of postquantum by mixing in the > > (handshake-independent) time to do the bulk transfer. The question is > > whether that reflects our goals. > > > > Ultimately, the thing that matters is overall application performance, > > which can be complex to measure because you actually have to try that > > application. Metrics like TTLB, TTFB, etc., are isolated to one > > connection and thus easier to measure, and without checking each > > application one by one. But they're only as valuable as they are > > predictors of overall application performance. For TTLB, both the > > magnitude and desirability of dampening effect are application-specific: > > > > If your goal is transferring a large file on the backend, such that > > you really only care when the operation is complete, then yes, TTLB is > > a good proxy for application system performance. You just care about > > throughput in that case. Moreover, in such applications, if you are > > transferring a lot of data, the dampening effect not only reflects > > reality but is larger. > > > > However, interactive, user-facing applications are different. There, > > TTLB is a poor proxy for application performance. For example, on the > > web, performance is determined more by how long it takes to display a > > meaningful webpage to the user. (We often call this the time to "first > > contentful paint".) Now, that is a very high-level metric that is > > impacted by all sorts of things, such as whether this is a repeat > > visit, page structure, etc. So it is hard to immediately translate > > that back down to TLS. But it is frequently much closer to the TTFB > > side of the spectrum than the TTLB side. And indeed, we have been > > seeing impacts from PQ to our high-level metrics on mobile. > > > > There's also a pretty natural intuition for this: since there is much > > more focus on latency than throughput, optimizing an interactive > > application often involves trying to reduce the amount of traffic on > > the critical path. The more the application does so, the less accurate > > TTLB's dampening effect is, and the closer we trend towards TTFB. (Of > > course, some optimizations in this space involve making fewer > > connections, etc. But the point here was to give a rough intuition.) > > > > On Thu, Mar 7, 2024 at 2:58 PM Deirdre Connolly > > <durumcrustu...@gmail.com> wrote: > >> "At the 2024 Workshop on Measurements, Attacks, and Defenses for the > Web (MADweb), we presented a paper¹ advocating time to last byte (TTLB) as > a metric for assessing the total impact of data-heavy, quantum-resistant > algorithms such as ML-KEM and ML-DSA on real-world TLS 1.3 connections. Our > paper shows that the new algorithms will have a much lower net effect on > connections that transfer sizable amounts of data than they do on the TLS > 1.3 handshake itself." > >> > >> https://www.amazon.science/blog/delays-from-post-quantum-cryptography > >> -may-not-be-so-bad > >> > >> ¹ > >> https://www.amazon.science/publications/the-impact-of-data-heavy-post > >> -quantum-tls-1-3-on-the-time-to-last-byte-of-real-world-connections/ > >> > >> > >> _______________________________________________ > >> TLS mailing list > >> TLS@ietf.org > >> https://www.ietf.org/mailman/listinfo/tls > > _______________________________________________ > > TLS mailing list > > TLS@ietf.org > > https://www.ietf.org/mailman/listinfo/tls > _______________________________________________ > TLS mailing list > TLS@ietf.org > https://www.ietf.org/mailman/listinfo/tls >
_______________________________________________ TLS mailing list TLS@ietf.org https://www.ietf.org/mailman/listinfo/tls
