> I believe TLS handshake time is not included here. I’m using the > Resource Timing API > <https://developer.mozilla.org/en-US/docs/Web/API/Resource_Timing_API> > to measure the time-to-first-byte for a request that I’m sending to > retrieve a static file. The resource loading phases > <https://developer.mozilla.org/en-US/docs/Web/API/Resource_Timing_API/Using_the_Resource_Timing_API> > section of the documentation explicitly shows the different stages for > DNS Lookup, TCP connection establishment, etc. I’m using the > difference between requestStart and responseStart values. This value > is deemed to be the same as time-to-first-byte > <https://stackoverflow.com/questions/6533456/time-to-first-byte-with-javascript> > seen in the inspector’s network tab.
This does not seem completely ludicrous, at least :) > We’re using this static file > <https://fonts.gstatic.com/l/font?kit=KFOmCnqEu92Fr1Me4GZNCzcPK4I&skey=a0a0114a1dcab3ac&v=v20> > that is hosted on a google CDN. We tried multiple different files, and > this one had the lowest latency in both locations that we tested it > (I’m in Toronto, and my colleague Sina is in San Francisco). Ah, so that's why that request showed up :) Curious to know why you picked this instead of, say, something from speed.cloudflare.com (since you're using that for the speed tests anyway)? > @Toke Høiland-Jørgensen >> Your test does a decent job and comes pretty close, at least >> in Chromium (about 800 Mbps which is not too far off at the application >> layer, considering I have a constant 100Mbps flow in the background >> taking up some of the bandwidth). Firefox seems way off (one test said >> 500Mbps the other >1000). > > > The way I’m measuring download is that I make multiple simultaneous > requests to cloudflare’s backend requesting 100MB files. Their backend > simply returns a file that has “0”s in the body repeated until 100MB > of file is generated. Then I use readable streams > <https://developer.mozilla.org/en-US/docs/Web/API/Streams_API/Using_readable_streams> > to make multiple measurements of (total bytes downloaded, timestamp). > Then I fit a line to the measurements collected, and the slope of that > line is the calculated bandwidth. For gigabit connections, this > download happens very quickly, and it may be the case that not a lot > of points are collected, in which case the fitted line is not accurate > and one might get overly-huge bandwidths as is the >1000 case in ur > Firefox browser. I think this might be fixed if we increase the > download time. Currently it’s 5s, maybe changing that to 10-20s would > help. I think in general it’d be a good feature to have a "more > advanced options” feature that allows the user to adjust some > parameters of the connection (such as number of parallel connections, > download scenario’s duration, upload scenario’s duration, etc.) Yeah, I think running the test for longer will help; 5s is not nearly enough to saturate a connection, especially not as the link speed increases. > The reason I do this line-fitting is because I want to get rid of the > bandwidth ramp-up time when the download begins. Yeah, allowing some ramp-up time before determining the bandwidth seems reasonable, but it's not generally possible to just pick a static number of (say) seconds to chop off... Having the graph over time helps sanity-check things, though. Also, a continuous graph of latency samples over time (for the whole duration, including idle/upload/download) is usually very instructive when plotting such a test. > Real-time Bandwidth Reporting > Using readable-streams also allows for instantaneous bandwidth > reporting (maybe using average of a moving window) similar to what > fast.com <http://fast.com/> or speedtest.net <http://speedtest.net/> > do, but I unfortunately am not able to do the same thing with upload, > since getting progress on http uploads adds some pre-flight OPTIONS > requests which cloudflare’s speedtest backend > <https://speed.cloudflare.com/> doesn’t allow those requests. For this > test we are directly hitting cloudflare’s backend, you can see this in > the network tab: > > Our download is by sending an http GET request to this endpoint: > https://speed.cloudflare.com/__down?bytes=100000000 > <https://speed.cloudflare.com/__down?bytes=100000000> and our upload > is done by sending and http POST request to this endpoint: > https://speed.cloudflare.com/__up <https://speed.cloudflare.com/__up> > > Since we are using cloudflare’s backend we are limited by what they > allow us to do. The test at speed.cloudflare.com does seem to plot real-time upload bandwidth; is that a privileged operation for themselves, or something? > I did try making my own worker which essentially does the same thing > as cloudflare’s speedtest backend (They do have this template worker > <https://github.com/cloudflare/worker-speedtest-template> that for the > most part does the same thing.) I modified that worker a bit so that > it allows http progress on upload for real-time measurements, but we > hit another wall with that: we could not saturate gigabit internet > connections. Turns out that cloudflare has tiers for the workers and > the bundle tier that we are using doesn’t get the most priority in > terms of bandwidth, so we could only get up to 600mbps measurements. > Their own speed test is hosted on an enterprise tier, which is around > $6-7k USD and is way too expensive. We are however, requesting for a > demo from them, and it’s an ongoing progress. > > So since we can measure instantaneous download speeds but not upload > speeds, we don’t report it for either one. But I can still make the > adjustment to report it for download at least. Sure, better than nothing. > @Toke Høiland-Jørgensen >> How do you calculate the jitter score? It's not obvious how you get from >> the percentiles to the jitter. > > Jitter here is the standard deviation of the latency measurements in > each stage. Is this a good definition? If you're reporting standard deviation, I'd just label it as such. One good definition for jitter is IPDV: https://en.wikipedia.org/wiki/Packet_delay_variation > @Toke Høiland-Jørgensen >> Also, what are the shields below the grade supposed to mean? Do they >> change depending on the result? On which criteria? > > > They do change! The criteria are listed below. Note that in the criteria > below: > Latency is calculated as the maximum median of latency across all three > stages. > Latency with Jitter is calculated as the maximum of (median + std) across > all three stages. > > Web Browsing: > Downlink: > 1mbps > Uplink: > 1mbps > > Audio Calls: > Downlink: > 3mbps > Uplink: > 3mbps > Latency: < 150ms > Latency with Jitter: < 200ms > > 4K Video Streaming: > Downlink: > 40mbps > > Video Conferencing: > Downlink: > 2.5mbps > Uplink: > 2.5mbps > Latency: < 150ms > Latency with Jitter: < 200ms > > Online Gaming: > Downlink: > 3mbps > Uplink: > 0.5mbps > Latency: < 100ms > Latency with Jitter: < 150ms > > For the bufferbloat grade we use the same criteria as DSL reports > <http://www.dslreports.com/faq/17930>. Right, cool; explaining this on the page might be useful ;) > @Toke Høiland-Jørgensen >> As far as the web page itself is concerned, holy cross-domain script >> deluge, Batman! I'm one of those people who run uMatrix in my regular >> Firefox session, and I disallow all cross-site script loads by default. >> I had to allow 15(!) different cross-site requests, *and* whitelist a >> few domains in my ad blocker as well to even get the test to run. Please >> fix this! I get that you can't completely avoid cross-domain requests >> due to the nature of the test, but why is a speedtest pulling in scripts >> from 'shopify.com' and three different ad tracking networks? > > > Hahahah this is because we’re using Shopify App Proxies > <https://shopify.dev/tutorials/display-data-on-an-online-store-with-an-application-proxy-app-extension>. > It’s a technique we use to import assets from our main store, and make > it appear such that this is part of our main store whereas in reality > it’s a separately-hosted application. This allows us to bring in the > header, footer and the chatbot. This is a really good point though, I > wonder what we can do with that. Just having the test be on a separate (static) page would be the obvious fix ;) -Toke _______________________________________________ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat