Re: [Bloat] Progress with latency-under-load tool
Attached is the initial version of the loadlatency tool. I'm getting some rather interesting results from it already, although it does take a very long time to run. It works under Linux, Cygwin and MacOS X on both little- and big-endian machines (and between machines of different byte-sexes), and therefore it should also work under FreeBSD and other UNIXes. I haven't yet tried compiling it for iOS or Android. It produces useful results even when one of the machines is rather old and slow, despite using a proper PRNG for traffic generation. My ancient Pentium-MMX proved capable of generating at least 4.4 MB/s of traffic steadily, and probably produces spikes of even greater bandwidth. Anything of Y2K vintage or newer should be able to saturate it's network with this. There are four measures produced: Upload Capacity, Download Capacity, Link Responsiveness and Flow Smoothness. All of these are reported in "bigger is better" units to help deal with Layers 8 and 9. The Capacity calculations are a bit complex to understand. For each flow (in either direction), the average goodput is measured over the entire lifetime of the flow. All of the flows in each direction for that scenario are the aggregated by taking the harmonic mean and multiplying by the number of flows; this biases the total downwards if the flows were mutually unfair. Finally, the relevant measures across all scenarios are aggregated using the harmonic mean, thus biasing the overall measure towards cases of under-utilisation. The totals are then reported in binary kilobytes per second. The Link Responsiveness measure is much easier. I simply take the maximum latency of the "pinger" channel (which is also used for commanding flows to stop) and invert it to produce a measure in Hertz. This is rounded down to the next integer for display; if you see a zero here (which is surprisingly likely), it means that a latency of more than one second was encountered at some point during the entire test. The Flow Smoothness measure refers to the application-level data inter-arrival timings. The maximum delay between data chunks arriving is measured across all flows in all scenarios, but excludes the first megabyte of each flow so as to avoid picking up the RTT during TCP startup. Here are some numbers I got from a test over a switched 100base-TX LAN: Upload Capacity: 1018 KiB/s Download Capacity: 2181 KiB/s Link Responsiveness: 2 Hz Flow Smoothness: 1 Hz Horrible, isn't it? I deliberately left these machines with standard configurations in order to show that. - Jonathan loadlatency.tar.gz Description: GNU Zip compressed data ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Random idea in reaction to all the discussion of TCPflavours - timestamps?
On 18 Mar, 2011, at 8:49 pm, Fred Baker wrote: >> How about trying to push for a default, that the logical egress buffers are >> limited to say 90% of the physical capacity, and only ECN-enabled flows may >> use the remaining 10% when they get marked... > > Lots of questions in that; 90% of the buffers in what? In a host, in a > router, on a card in a router, in a queue's configured maximum depth, what? > One would need some pedagogic support in the form of simulations - why 90% vs > 91% vs 10% vs whatever? > >> Someone has to set an incentive for using ECN, unfortunately... > > Yes. From my perspective, the right approach is probably more like > introducing a mark/drop threshold and a drop threshold. Taking the model that > every M time units we "do something" like > > if queue depth exceeds > reduce M > drop something > else if queue depth exceeds > reduce M > select something > if it is ECN-capable, > mark it congestion-experienced > else > drop it > else is below > increase M > > the advantage of ECN traffic is that it is less likely to be dropped. That > might be a reasonable approach. That does actually seem reasonable. What's the betting that HW vendors still say it's too complicated? :-D I think we can come up with some simple empirical rules for choosing queue sizes. I may be half-remembering something VJ wrote, but here's a starting point: 0) Buffering more than 1 second of data is always unacceptable. 1) Measure (or estimate) the RTT of a full-sized packet over the exit link and back, then add 100ms for typical Internet latency, calling this total T1. If T1 is more than 500ms, clamp it to 500ms. Calculate T2 to be twice T1; this will be at most 1000ms. 2) Measure (or estimate) the throughput BW of the exit link, in bytes per second. 3) Calculate ideal queue length (in bytes) Q1 as T1 * BW, and the maximal queue length Q2 as T2 * BW. These may optionally be rounded to the nearest multiple of a whole packet size, if that is convenient for the hardware. 4) If the link quality is strongly time-varying, eg. mobile wireless, recalculate Q1 and Q2 as above regularly. 5) If the link speed depends on the type of equipment at the other end, the quality of cabling, or other similar factors, use the actual negotiated link speed when calculating BW. When these factors change, recalculate as above. I would take the "hysteresis limit" to be an empty queue for the above algorithm. - Jonathan ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Progress with latency-under-load tool
On 03/20/2011 21:45, Jonathan Morton wrote: [..] Here are some numbers I got from a test over a switched 100base-TX LAN: Upload Capacity: 1018 KiB/s Download Capacity: 2181 KiB/s Link Responsiveness: 2 Hz Flow Smoothness: 1 Hz Horrible, isn't it? I deliberately left these machines with standard configurations in order to show that. Perhaps a tangential 2 cents from me, but I'm unclear how helpful Hertz is as a unit of measurement for the challenge of raising awareness of bufferbloat. I suspect data networking people (whether network designers, admins, product managers, etc) don't think of IP networks as having characteristics measured in Hertz. And I'd imagine most ISP helpdesks wont know how to parse customer complaints along the lines of "your service doesn't give me enough Hertz". Perhaps your tool could also add a parenthetical 'translation' of the responsiveness and smoothness values into something expressed in milliseconds? cheers, gja ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Progress with latency-under-load tool
grenville armitage writes: > On 03/20/2011 21:45, Jonathan Morton wrote: > [..] >> Here are some numbers I got from a test over a switched 100base-TX LAN: >> >> Upload Capacity: 1018 KiB/s >>Download Capacity: 2181 KiB/s >> Link Responsiveness: 2 Hz >> Flow Smoothness: 1 Hz >> >> Horrible, isn't it? I deliberately left these machines with standard >> configurations in order to show that. > > Perhaps a tangential 2 cents from me, but I'm unclear how helpful > Hertz is as a unit of measurement for the challenge of raising > awareness of bufferbloat. I suspect data networking people (whether > network designers, admins, product managers, etc) don't think of IP > networks as having characteristics measured in Hertz. And I'd imagine > most ISP helpdesks wont know how to parse customer complaints along > the lines of "your service doesn't give me enough Hertz". Perhaps > your tool could also add a parenthetical 'translation' of the > responsiveness and smoothness values into something expressed in > milliseconds? Concur. And resolution past 3 significant digits from there seems possible with posix timers. I regret that I have not had time to look over the code yet this weekend. How'd they do debloated? > > cheers, > gja > ___ > Bloat-devel mailing list > bloat-de...@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat-devel -- Dave Taht http://nex-6.taht.net ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
[Bloat] Some results of the latency under load tool
Jonathan Morton writes: > Here are some numbers I got from a test over a switched 100base-TX LAN: > > Upload Capacity: 1018 KiB/s > Download Capacity: 2181 KiB/s > Link Responsiveness: 2 Hz > Flow Smoothness: 1 Hz > > Horrible, isn't it? I deliberately left these machines with standard > configurations in order to show that. I successfully compiled and ran this on both an debian squeeze/arm based platform (an openrd), and a 64 bit ubuntu 10.10. It generated some warnings on 64 bit. Run on my partially debloated wireless/wired network[1][2] via the the openrd->nano-m->nano-m->davepc path (180Mbit wireless) over IPv6 (wow, transparent IPv6! Thanks!), over IPv6, server=davepc Scenario 1: 0 uploads, 1 downloads... 8024 KiB/s down, 12.71 Hz smoothness Scenario 2: 1 uploads, 0 downloads... 6526 KiB/s up, 5.52 Hz smoothness Scenario 3: 0 uploads, 2 downloads... 6703 KiB/s down, 19.07 Hz smoothness Scenario 4: 1 uploads, 1 downloads... 3794 KiB/s up, 4084 KiB/s down, 1.82 Hz smoothness Scenario 5: 2 uploads, 0 downloads... 7503 KiB/s up, 4.11 Hz smoothness Scenario 6: 0 uploads, 3 downloads... 7298 KiB/s down, 2.08 Hz smoothness Scenario 7: 1 uploads, 2 downloads... 4323 KiB/s up, 4690 KiB/s down, 1.38 Hz smoothness My pure wireless and more debloated network is down right now[2] I'm still watching it run other scenarios. Comments/Suggestions: A) From a "quicker results" perspective, do the heaviest network tests first, then the lightest tests, then the rest? :) B) Due to the dependence on the gnu scientific library it seems unlikely this will build on openwrt. Also, if there is heavy math anywhere in this app, it needs to be done outside the testing loop on a non FPU enabled processor such as most arms to not skew the test. It ate 17% of cpu on the arm, and ~8% of cpu on davepc. Similarly there are rng's in the Linux kernel and hardware sources, I'm not sure to what extent you need randomness (I looked at the code only briefly) C) Do you have this in a repository somewhere? (I use github). I would certainly love to have more tools like this living together in harmony. Needn't be github. D) It's never been clear to me what resolution gettimeofday has on most platforms, and the posix clock_gettime and the corresponding clock_getres(clockid_t clk_id, struct timespec *res) Can make that clear. CLOCK_REALTIME also discards any clock adjustments in the interval for even higher potential accuracy. (On the interval anyway, not perhaps against wall-clock time) Given the period of this test, do you think that the precision can only be 1ms? It's certainly my hope that most systems today implement posix timers. I can produce a patch if you want. Thanks for such an interesting tool! -- Dave Taht http://nex-6.taht.net [1] The openrd has a very small txqueuelen (4) and dma-tx-queue of 20 on the GigE card, which is connected at 100Mbit to the nano-m5s, which DOES NOT have my ath9k patch or eBDP enabled but does have a small txqueuelen, the laptop is running debloat-testing (txqueulen 4, dma tx queue 64), connected via 100Mbit to the other nano-m5 radio. tcp cubic throughout. ECN and SACK/DSACK are enabled but there is no shaping in place. The wireless radios inbetween DO tend to show bloat under worse conditions than this test. (I note that debloat-testings patches to the e1000 series ethernet driver do not allow dma tx queues of less than 64 as yet) [2] for my network diagram see: http://www.bufferbloat.net/projects/bloat/wiki/Experiment_-_Bloated_LAGN_vs_debloated_WNDR5700 This is an old test I need to repeat it with debloat testing. ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Progress with latency-under-load tool
On 20 Mar, 2011, at 10:33 pm, grenville armitage wrote: >> Here are some numbers I got from a test over a switched 100base-TX LAN: >> >>Upload Capacity: 1018 KiB/s >> Download Capacity: 2181 KiB/s >> Link Responsiveness: 2 Hz >>Flow Smoothness: 1 Hz >> >> Horrible, isn't it? I deliberately left these machines with standard >> configurations in order to show that. > > Perhaps a tangential 2 cents from me, but I'm unclear how helpful Hertz is as > a unit of measurement for the challenge of raising awareness of bufferbloat. Customers won't be asking for "more Hertz" - or at least, none that have any sense. They'll be asking for "more smoothness" for their video streams, or "more responsiveness" for their online games. They already ask for "more bandwidth", not "more megabits per second". Hertz makes sense as a unit because, when talking about latency or transmission delays, shorter times are better, but people understand "bigger is better" much more easily. Hard drives used to measure their seek times in milliseconds too, but now they are measured in IOPS instead (a trend mostly associated with SSDs, which have IOPS numbers several orders of magnitude better than mechanical drives). Let me manually translate that for you, though. That Responsiveness rating of 2Hz means that the practical RTT went above 334ms - and this on a switched Fast Ethernet being driven by good-quality 1996-vintage hardware on one side and a cheap nettop on the other. It actually reflects not pure latency as 'ping' would have measured it, but a packet loss and the time required to recover from it. And the "smoothness" rating actually contrived to be *worse*, at somewhere north of 500ms. At Fast Ethernet speeds, that implies megabytes of buffering, on what really is a very old computer. It's a clear sign of something very broken in the network stack, especially as I get broadly similar results (with much higher throughput numbers) when I run the same test on GigE hardware with much more powerful computers (which can actually saturate GigE). You really don't want to see what I got on my 3G test runs. I got 0.09 Hz from a single flow, and these tests run all the way up to 32 flows. I think the modem switched down into GPRS mode for a few minutes as well, even though there was no obvious change in propagation conditions. > And resolution past 3 significant digits from there seems > possible with posix timers. If the latency was staying in the single-digit milliseconds as it should be on a LAN, you'd have three s.f. with just integers. I do print the smoothness numbers out to 2 decimal places for the individual scenarios, though - these are the numbers meant for investigating problems: Scenario 1: 0 uploads, 1 downloads... 1343 KiB/s down, 31.52 Hz smoothness Scenario 2: 1 uploads, 0 downloads... 3670 KiB/s up, 22.24 Hz smoothness Scenario 3: 0 uploads, 2 downloads... 2077 KiB/s down, 19.70 Hz smoothness Scenario 4: 1 uploads, 1 downloads... 2855 KiB/s up, 322 KiB/s down, 6.44 Hz smoothness That's from a different test, where you can see the effect of a WLAN and having Vegas on one side of the connection. With that said, since the single-digit results are so ubiquitous, perhaps some extra precision is warranted after all. Perhaps I can take the opportunity to squash some more minor bugs, and add an interpretation of the goodput in terms of gigabytes per month. > How'd they do debloated? I'm still investigating that, partly as the older hardware wasn't yet set up with kernels capable of running advanced qdiscs. It takes a while to compile a kernel on a Pentium-MMX. I'm also really not sure where to get debloated drivers for a VIA Rhine or a Sun GEM. ;-) Mind you, such a thing may not be needed, if the device drivers are already slim so that cutting txqueuelen is sufficient. I can't run debloated 3G tests - I don't have access to the buffer controls on the base tower. :-( - Jonathan ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Random idea in reaction to all the discussion of TCPflavours - timestamps?
On Sun, 20 Mar 2011, Jonathan Morton wrote: I think we can come up with some simple empirical rules for choosing queue sizes. I may be half-remembering something VJ wrote, but here's a starting point: 0) Buffering more than 1 second of data is always unacceptable. what about satellite links? my understanding is that the four round trips to geosync orbit (request up, down, reply up down) result in approximatly 1 sec round trip. David Lang 1) Measure (or estimate) the RTT of a full-sized packet over the exit link and back, then add 100ms for typical Internet latency, calling this total T1. If T1 is more than 500ms, clamp it to 500ms. Calculate T2 to be twice T1; this will be at most 1000ms. 2) Measure (or estimate) the throughput BW of the exit link, in bytes per second. 3) Calculate ideal queue length (in bytes) Q1 as T1 * BW, and the maximal queue length Q2 as T2 * BW. These may optionally be rounded to the nearest multiple of a whole packet size, if that is convenient for the hardware. 4) If the link quality is strongly time-varying, eg. mobile wireless, recalculate Q1 and Q2 as above regularly. 5) If the link speed depends on the type of equipment at the other end, the quality of cabling, or other similar factors, use the actual negotiated link speed when calculating BW. When these factors change, recalculate as above. I would take the "hysteresis limit" to be an empty queue for the above algorithm. - Jonathan ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Some results of the latency under load tool
On 20 Mar, 2011, at 11:50 pm, Dave Täht wrote: > B) Due to the dependence on the gnu scientific library it seems unlikely > this will build on openwrt. Also, if there is heavy math anywhere in > this app, it needs to be done outside the testing loop on a non FPU > enabled processor such as most arms to not skew the test. I'm aware of the limitations of older ARM CPUs, as I have had to work with them on occasion. I don't have any with Ethernet at home though. The maths inside the main loops is limited to calculating time differences, comparing them and determining when to stop - there may be some optimisation possible there, but it's not ridiculous. The main loops are actually slightly elastic, because recv() will give whatever data is available at the time it is called, and the fattest calculations are only done once per loop. Under normal circumstances, that should be every 1.2 KB, but it scales up to 64KB if the CPU temporarily can't keep up (or if a lot of data becomes available in one go). > It ate 17% of cpu on the arm, and ~8% of cpu on davepc. Looks tolerable to me. Even my ancient CISC PC can trigger the symptoms well enough. > Similarly there are rng's in the Linux kernel and hardware sources, I'm > not sure to what extent you need randomness (I looked at the code only > briefly) The spew() function uses the PRNG to generate incompressible traffic, since compressibility does matter to some network technologies. I used GSL purely to get access to a very fast PRNG that still produces good numbers - it's just a shame that this is a relatively large, general-purpose library. I originally intended to checksum the data and thus ensure that it arrived intact, but I had to delete that feature to get around a race condition. > So you are saying lower values of hz is horrible? Yes. 0.02 Hz means the application was waiting 50 seconds for data somehow. - Jonathan ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Progress with latency-under-load tool
Jonathan Morton writes: > On 20 Mar, 2011, at 10:33 pm, grenville armitage wrote: > > Customers won't be asking for "more Hertz" - or at least, none that > have any sense. They'll be asking for "more smoothness" for their > video streams, or "more responsiveness" for their online games. They > already ask for "more bandwidth", not "more megabits per second". I think despite your quest for a marketing number, that also reporting actual RTT would be helpful (and horrifying, as I just got a .02HZ rating on the still ongoing test...). I still can't claim to fully understand what this test is measuring. Scenario 8: 2 uploads, 1 downloads... 4743 KiB/s up, 3422 KiB/s down, 0.02 Hz smoothness Scenario 9: 3 uploads, 0 downloads... 7558 KiB/s up, 1.39 Hz smoothness Scenario 10: 0 uploads, 4 downloads... 6719 KiB/s down, 2.08 Hz smoothness Scenario 11: 1 uploads, 3 downloads... 4372 KiB/s up, 4067 KiB/s down, 1.30 Hz smoothness The interesting outlier thus far is 8... I'm tempted to stop the test now and recompile for testing 3 u/l 3 d/l first Even better, we could use a different name for your usage of hz or smoothness here. Mortons? (There, I named it for you. You cannot be accused of ego for using this new unit now. Enjoy. ) > Hertz makes sense as a unit because, when talking about latency or > transmission delays, shorter times are better, but people understand > "bigger is better" much more easily. Hard drives used to measure > their seek times in milliseconds too, but now they are measured in > IOPS instead (a trend mostly associated with SSDs, which have IOPS > numbers several orders of magnitude better than mechanical drives). > > Let me manually translate that for you, though. That Responsiveness > rating of 2Hz means that the practical RTT went above 334ms - and this > on a switched Fast Ethernet being driven by good-quality 1996-vintage > hardware on one side and a cheap nettop on the other. It actually > reflects not pure latency as 'ping' would have measured it, but a > packet loss and the time required to recover from it. Your explanation here is good. Perhaps it could be an automated textual description. > And the "smoothness" rating actually contrived to be *worse*, at > somewhere north of 500ms. At Fast Ethernet speeds, that implies > megabytes of buffering, on what really is a very old computer. Txqueuelen = 1000 = ~1.5Mbyte (and I'm struggling with using consistent decimal or base 2 units this month) Mibibyte? Normal dma tx ring size ranges from 64 to 512, can be seen sometimes with ethtool -g device > It's a clear sign of something very broken in the network stack, > especially as I get broadly similar results (with much higher > throughput numbers) when I run the same test on GigE hardware with > much more powerful computers (which can actually saturate GigE). > > You really don't want to see what I got on my 3G test runs. I got > 0.09 Hz from a single flow, and these tests run all the way up to 32 > flows. I think the modem switched down into GPRS mode for a few > minutes as well, even though there was no obvious change in > propagation conditions. > -- Dave Taht http://nex-6.taht.net ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Random idea in reaction to all the discussion of TCPflavours - timestamps?
On 21 Mar, 2011, at 12:18 am, da...@lang.hm wrote: >> 0) Buffering more than 1 second of data is always unacceptable. > > what about satellite links? my understanding is that the four round trips to > geosync orbit (request up, down, reply up down) result in approximatly 1 sec > round trip. That is true, but it doesn't require more than a full second of buffering, just lots of FEC to avoid packet loss on the link. At those timescales, you want the flow to look smooth, not bursty. Bursty is normal at 100ms timescales. What I've heard is that most consumer satellite links use split-TCP anyway (proxy boxes at each end) thus relieving the Internet at large from coping with an unusual problem. However, it also seems likely that backbone satellite links exist which do not use this technique. I heard something about South America, maybe? Anyway, with a 1-second RTT, the formula comes out to max 1 second of buffering because of the clamping. - Jonathan ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Progress with latency-under-load tool
d...@taht.net (Dave Täht) writes: > The interesting outlier thus far is 8... I'm tempted to stop the test > now and recompile for testing 3 u/l 3 d/l first Scenario 12: 2 uploads, 2 downloads... 4725 KiB/s up, 3645 KiB/s down, 0.81 Hz smoothness ... Still running tests ... > Txqueuelen = 1000 = ~1.5Mbyte (and I'm struggling with using consistent > decimal or base 2 units this month) Mibibyte? > > Normal dma tx ring size ranges from 64 to 512, can be seen sometimes > with ethtool -g device With wireless you also have TX_RETRIES = 13 for these (ath9) devices drivers under test. (Reduced to 4 in the less bloated version I can't test right now) You also have contention from other devices, and the occasional complete retrain from a base figure of 1Mbit/sec up, while devices struggle to regain sync. There are (presumably helpful) effects of packet aggregation. You also have modulo issues against the size of the tx queues, an interaction with the network flow scheduler (what triggers filling the tx queue?), SACK/DSACK and lost ACK retransmits, and probably more issues than I've been able to come up with offhand. -- Dave Taht http://nex-6.taht.net ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Random idea in reaction to all the discussion of TCPflavours - timestamps?
Jonathan Morton writes: > On 21 Mar, 2011, at 12:18 am, da...@lang.hm wrote: > >>> 0) Buffering more than 1 second of data is always unacceptable. Well, in the case of the DTN, it's required. We're not testing interplanetary networks here, (rather, an artificially induced one extending out well beyond the moon!) but it bears a little thinking about. -- Dave Taht http://nex-6.taht.net ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Progress with latency-under-load tool
On 21 Mar, 2011, at 12:47 am, Dave Täht wrote: >> The interesting outlier thus far is 8... I'm tempted to stop the test >> now and recompile for testing 3 u/l 3 d/l first > > Scenario 12: 2 uploads, 2 downloads... 4725 KiB/s up, 3645 KiB/s down, 0.81 > Hz smoothness > > ... Still running tests ... I should probably explain what the scenarios are. Every combination of up/down flows is tried for each of 1, 2, 3, 4 total flows. That's 14 scenarios covering basic downloading and uploading through to heavy whole-family use. Then I jump ahead to 32 flows, with 32/0, 31/1, 16/16, 1/31, 0/32 splits, to simulate BitTorrent style traffic. Total 19 scenarios. - Jonathan ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Progress with latency-under-load tool
Jonathan Morton writes: > On 21 Mar, 2011, at 12:47 am, Dave Täht wrote: > >>> The interesting outlier thus far is 8... I'm tempted to stop the test >>> now and recompile for testing 3 u/l 3 d/l first >> >> Scenario 12: 2 uploads, 2 downloads... 4725 KiB/s up, 3645 KiB/s >> down, 0.81 Hz smoothness >> >> ... Still running tests ... > > I should probably explain what the scenarios are. > > Every combination of up/down flows is tried for each of 1, 2, 3, 4 > total flows. That's 14 scenarios covering basic downloading and > uploading through to heavy whole-family use. > > Then I jump ahead to 32 flows, with 32/0, 31/1, 16/16, 1/31, 0/32 > splits, to simulate BitTorrent style traffic. > > Total 19 scenarios. I'll still argue that starting with the worst case scenarios and working down would be best... 3/3 perhaps Scenario 12: 2 uploads, 2 downloads... 4725 KiB/s up, 3645 KiB/s down, 0.81 Hz smoothness Scenario 13: 3 uploads, 1 downloads... 4917 KiB/s up, 3175 KiB/s down, 0.59 Hz smoothness > > - Jonathan > -- Dave Taht http://nex-6.taht.net ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Random idea in reaction to all the discussion of TCPflavours - timestamps?
On 03/21/2011 09:50, Dave Täht wrote: [..] We're not testing interplanetary networks here, (rather, an artificially induced one extending out well beyond the moon!) but it bears a little thinking about. Perhaps an idea for presenting bufferbloat visually? Draw a picture of the space around the Earth, with circles around the earth whose diameters are proportional to bufferbloat-induced equivalent RTT across different ISP links, or different consumer hardware, etc. "Bufferbloat puts New York on the far side of the moon!" might be a tagline to get people's attention ;) cheers, gja ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Random idea in reaction to all the discussion of TCPflavours - timestamps?
On 21 Mar, 2011, at 12:50 am, Dave Täht wrote: 0) Buffering more than 1 second of data is always unacceptable. > > Well, in the case of the DTN, it's required. > > We're not testing interplanetary networks here, (rather, an artificially > induced one extending out well beyond the moon!) but it bears a little > thinking about. It appears that most practical solutions for a DTN involve explicit store-and-forward techniques, and these would be irrelevant at the IP routing level. - Jonathan ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Random idea in reaction to all the discussion of TCPflavours - timestamps?
grenville armitage writes: > On 03/21/2011 09:50, Dave Täht wrote: > [..] > >> We're not testing interplanetary networks here, (rather, an >> artificially induced one extending out well beyond the moon!) but it >> bears a little thinking about. > > Perhaps an idea for presenting bufferbloat visually? Draw a picture of > the space around the Earth, with circles around the earth whose > diameters are proportional to bufferbloat-induced equivalent RTT > across different ISP links, or different consumer hardware, > etc. "Bufferbloat puts New York on the far side of the moon!" might be > a tagline to get people's attention ;) That is similar to one of the ideas I had while prototyping the cosmic background bufferbloat detector. (Since stalled out due issues with mapping ntp data types to postgres and postgis data types) There are some excellent caida and network geography maps that do something like this, and it would make the point, thoroughly See, for example: http://www.mail-archive.com/bloat@lists.bufferbloat.net/msg00050.html And the graphic at the middle left down (linked from the above) at: http://personalpages.manchester.ac.uk/staff/m.dodge/cybergeography//atlas/geographic.html Several other ideas in the above worth thinking about. I mentally see a map of the US pulsating like a old winamp graphic eq plugin, with vertical 3D bars over each location extending into space while the earth rotates under the terminator > > cheers, > gja > > ___ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat -- Dave Taht http://nex-6.taht.net ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Random idea in reaction to all the discussion of TCPflavours - timestamps?
On 21 Mar, 2011, at 1:04 am, Dave Täht wrote: >>> We're not testing interplanetary networks here, (rather, an >>> artificially induced one extending out well beyond the moon!) but it >>> bears a little thinking about. >> >> Perhaps an idea for presenting bufferbloat visually? Draw a picture of >> the space around the Earth, with circles around the earth whose >> diameters are proportional to bufferbloat-induced equivalent RTT >> across different ISP links, or different consumer hardware, >> etc. "Bufferbloat puts New York on the far side of the moon!" might be >> a tagline to get people's attention ;) > > That is similar to one of the ideas I had while prototyping the cosmic > background bufferbloat detector. Come with us. Leave your wheat fields. Leave your back-street shops, your fishing boats. Leave your offices in the tall skyscrapers. Leave all that is routine and commonplace, And come with us at once. Come with us, To where Man has never been, But where he will go, As certain as the passing of time. Come with us to the Moon. The rocket is waiting... -- The Orb, "The Passing Of Time", Orblivion ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Random idea in reaction to all the discussion of TCPflavours - timestamps?
Jonathan Morton writes: > On 21 Mar, 2011, at 1:04 am, Dave Täht wrote: > We're not testing interplanetary networks here, (rather, an artificially induced one extending out well beyond the moon!) but it bears a little thinking about. >>> >>> Perhaps an idea for presenting bufferbloat visually? Draw a picture of >>> the space around the Earth, with circles around the earth whose >>> diameters are proportional to bufferbloat-induced equivalent RTT >>> across different ISP links, or different consumer hardware, >>> etc. "Bufferbloat puts New York on the far side of the moon!" might be >>> a tagline to get people's attention ;) >> >> That is similar to one of the ideas I had while prototyping the cosmic >> background bufferbloat detector. > > Come with us. > Leave your wheat fields. > Leave your back-street shops, your fishing boats. > Leave your offices in the tall skyscrapers. > Leave all that is routine and commonplace, > And come with us at once. > > Come with us, > To where Man has never been, > But where he will go, > As certain as the passing of time. > > Come with us to the Moon. > The rocket is waiting... > > -- The Orb, "The Passing Of Time", Orblivion Pass the kool-aid, will ya? I hear owsley left a legacy. http://articles.latimes.com/2011/mar/15/local/la-me-owsley-stanley-20110315 -- Dave Taht http://nex-6.taht.net ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Random idea in reaction to all the discussion of TCPflavours - timestamps?
d...@taht.net (Dave Täht) writes: > Jonathan Morton writes: > >> On 21 Mar, 2011, at 1:04 am, Dave Täht wrote: >>> That is similar to one of the ideas I had while prototyping the cosmic >>> background bufferbloat detector. >> >> Come with us. >> Leave your wheat fields. >> Leave your back-street shops, your fishing boats. >> Leave your offices in the tall skyscrapers. >> Leave all that is routine and commonplace, >> And come with us at once. >> >> Come with us, >> To where Man has never been, >> But where he will go, >> As certain as the passing of time. >> >> Come with us to the Moon. >> The rocket is waiting... >> >> -- The Orb, "The Passing Of Time", Orblivion > > Pass the kool-aid, will ya? I hear owsley left a legacy. > > http://articles.latimes.com/2011/mar/15/local/la-me-owsley-stanley-20110315 In so naming the project a cosmic background bufferbloat detector I was referring to this project: http://en.wikipedia.org/wiki/Discovery_of_cosmic_microwave_background_radiation That said, redoing the horn in that in day-glo colors strikes me as a good logo idea, if I ever get the time to get back on the project... /me passes back the kool-aid -- Dave Taht http://nex-6.taht.net ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Progress with latency-under-load tool
Jonathan Morton writes: > On 21 Mar, 2011, at 12:47 am, Dave Täht wrote: > >>> The interesting outlier thus far is 8... I'm tempted to stop the test >>> now and recompile for testing 3 u/l 3 d/l first I found the cause of the outlier, the arm box had a hiccough on its hard disk controller for some reason around then. Or at least that appears to be the case, but judging from this: Scenario 16: 1 uploads, 31 downloads... 4401 KiB/s up, 4111 KiB/s down, 0.59 Hz smoothness Scenario 17: 16 uploads, 16 downloads... 6108 KiB/s up, 2103 KiB/s down, 0.00 Hz smoothness Maybe not. Me thinks we need 3 digits of precision here. -- Dave Taht http://nex-6.taht.net ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
[Bloat] The complete results from the latencyload test.
Instead of dribs and drabs. I don't know when the test started, but this took a very long time. Server responding, beginning test... MinRTT: 1.1ms Scenario 1: 0 uploads, 1 downloads... 8024 KiB/s down, 12.71 Hz smoothness Scenario 2: 1 uploads, 0 downloads... 6526 KiB/s up, 5.52 Hz smoothness Scenario 3: 0 uploads, 2 downloads... 6703 KiB/s down, 19.07 Hz smoothness Scenario 4: 1 uploads, 1 downloads... 3794 KiB/s up, 4084 KiB/s down, 1.82 Hz smoothness Scenario 5: 2 uploads, 0 downloads... 7503 KiB/s up, 4.11 Hz smoothness Scenario 6: 0 uploads, 3 downloads... 7298 KiB/s down, 2.08 Hz smoothness Scenario 7: 1 uploads, 2 downloads... 4323 KiB/s up, 4690 KiB/s down, 1.38 Hz smoothness Scenario 8: 2 uploads, 1 downloads... 4743 KiB/s up, 3422 KiB/s down, 0.02 Hz smoothness Scenario 9: 3 uploads, 0 downloads... 7558 KiB/s up, 1.39 Hz smoothness Scenario 10: 0 uploads, 4 downloads... 6719 KiB/s down, 2.08 Hz smoothness Scenario 11: 1 uploads, 3 downloads... 4372 KiB/s up, 4067 KiB/s down, 1.30 Hz smoothness Scenario 12: 2 uploads, 2 downloads... 4725 KiB/s up, 3645 KiB/s down, 0.81 Hz smoothness Scenario 13: 3 uploads, 1 downloads... 4917 KiB/s up, 3175 KiB/s down, 0.59 Hz smoothness Scenario 14: 4 uploads, 0 downloads... 7706 KiB/s up, 1.43 Hz smoothness Scenario 15: 0 uploads, 32 downloads... 6648 KiB/s down, 1.36 Hz smoothness Scenario 16: 1 uploads, 31 downloads... 4401 KiB/s up, 4111 KiB/s down, 0.59 Hz smoothness Scenario 17: 16 uploads, 16 downloads... 6108 KiB/s up, 2103 KiB/s down, 0.00 Hz smoothness Scenario 18: 31 uploads, 1 downloads... 8003 KiB/s up, 520 KiB/s down, 0.03 Hz smoothness Scenario 19: 32 uploads, 0 downloads... 7236 KiB/s up, 0.00 Hz smoothness OVERALL: Upload Capacity: 5486 KiB/s Download Capacity: 2836 KiB/s Link Responsiveness: 0 Hz Flow Smoothness: 0 Hz -- Dave Taht http://nex-6.taht.net ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Random idea in reaction to all the discussion of TCPflavours - timestamps?
On Mon, 21 Mar 2011, Jonathan Morton wrote: On 21 Mar, 2011, at 12:18 am, da...@lang.hm wrote: 0) Buffering more than 1 second of data is always unacceptable. what about satellite links? my understanding is that the four round trips to geosync orbit (request up, down, reply up down) result in approximatly 1 sec round trip. That is true, but it doesn't require more than a full second of buffering, just lots of FEC to avoid packet loss on the link. At those timescales, you want the flow to look smooth, not bursty. Bursty is normal at 100ms timescales. What I've heard is that most consumer satellite links use split-TCP anyway (proxy boxes at each end) thus relieving the Internet at large from coping with an unusual problem. However, it also seems likely that backbone satellite links exist which do not use this technique. I heard something about South America, maybe? I've heard that they do proxy boxes at each end for common protocols like HTTP, but they can't do so for other protocols (think ssh for example) Anyway, with a 1-second RTT, the formula comes out to max 1 second of buffering because of the clamping. and what if you have a 1 second satellite link plus 'normal internet latency', or worse, both ends are on a satellite link, giving you a 2-second+ round trip time? if you don't have large enough buffers to handle this, what happens? David Lang ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Random idea in reaction to all the discussion of TCPflavours - timestamps?
On 3/20/2011 9:28 PM, da...@lang.hm wrote: On Mon, 21 Mar 2011, Jonathan Morton wrote: On 21 Mar, 2011, at 12:18 am, da...@lang.hm wrote: 0) Buffering more than 1 second of data is always unacceptable. what about satellite links? my understanding is that the four round trips to geosync orbit (request up, down, reply up down) result in approximatly 1 sec round trip. That is true, but it doesn't require more than a full second of buffering, just lots of FEC to avoid packet loss on the link. At those timescales, you want the flow to look smooth, not bursty. Bursty is normal at 100ms timescales. What I've heard is that most consumer satellite links use split-TCP anyway (proxy boxes at each end) thus relieving the Internet at large from coping with an unusual problem. However, it also seems likely that backbone satellite links exist which do not use this technique. I heard something about South America, maybe? I've heard that they do proxy boxes at each end for common protocols like HTTP, but they can't do so for other protocols (think ssh for example) Anyway, with a 1-second RTT, the formula comes out to max 1 second of buffering because of the clamping. and what if you have a 1 second satellite link plus 'normal internet latency', or worse, both ends are on a satellite link, giving you a 2-second+ round trip time? if you don't have large enough buffers to handle this, what happens? Propagation delay to a geosynchronous relay is ~125 ms. Round-trip propagation delay contributes ~500 ms, so it isn't quite as bad as you think, though still long. Many tricks are played with accelerator gateways to improve bulk transfer throughput for TCP users (see e.g. RFC 3135). There may be a challenge in debloating the devices that support such links, as their buffers and the functions they serve optimize other metrics (e.g. low packet loss rate, bulk transfer throughput, etc.). -- Wes Eddy MTI Systems ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Progress with latency-under-load tool
On 21 Mar, 2011, at 6:09 am, Daniel Brooks wrote: > Oof. I guess the loopback driver could use some work as well :) Either that or there is some strange timing effect going on that I should be aware of. I'm going to try to run some more tests to see if I can get the smoothness and responsiveness values up, but if any of you notice something bizarre... Be aware however that both smoothness and responsiveness are taken as the absolute worst cases observed. If there is a bell curve, you're going to be looking at outliers. Perhaps I should calculate something like a 7-number summary to sanity-check it. - Jonathan ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
