Re: [Flent-users] [tohojo/flent] RRUL Upload plot - why such low granularity? (#185)
Rich Brown writes:
>> Basically, the problem is the way netperf determines its data output
>> intervals in demo mode:
>
> Ahah! That was the explanation, but I wasn't (yet) smart enough to
> comprehend it. [To further test out my understanding... If netperf
> were to use a smaller "data output interval" when it's transmitting,
> it would create more frequent updates and the granularity would be
> higher...]
>
> But I'm not sure that fully explains it unless netperf uses a (very)
> different data output interval for receiving... The download chart has
> hundreds of points, while the ratio between my download and upload is
> about 10:1 (7mbps vs 768kbps) so I'd expect the upload chart to have
> something like dozens of points...
This is because netperf only checks whether it should output a data
point after it has sent a certain number of bytes. 16k in your case
(it's in the series metdata). At 768kbps, 16k bytes takes ~2.6 seconds
to send, so you'll get at most one data point for every such interval.
Now that I'm looking at this again, it seems that you can actually set
the send size. However, lowering it naturally impacts CPU usage. A quick
test on my laptop indicates that lowering it to 512 bytes raises
netperf's CPU usage from ~2% to ~20% for a single flow at half a
gigabit.
So yeah, we could probably lower it somewhat, but to what? We can't
know ahead of time what connection speed we will be running on.
If you want to try a quick test with a smaller size, this patch (to
Flent) adds the option unconditionally; obviously not the right way to
go about it, but it should allow you to test the impact at least:
diff --git a/flent/runners.py b/flent/runners.py
index a75223d..3d5e373 100644
--- a/flent/runners.py
+++ b/flent/runners.py
@@ -1049,6 +1049,7 @@ class NetperfDemoRunner(ProcessRunner):
"{marking} -H {control_host} -p {control_port} " \
"-t {test} -l {length:d} {buffer} {format} " \
"{control_local_bind} {extra_args} -- " \
+ "-m 512,512 " \
"{socket_timeout} {local_bind} -H {host} -k
{output_vars} " \
"{cong_control} {extra_test_args}".format(**args)
--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/tohojo/flent/issues/185#issuecomment-533247772___
Flent-users mailing list
Flent-users@flent.org
http://flent.org/mailman/listinfo/flent-users_flent.org
Re: [Flent-users] [tohojo/flent] Problem with Flent Average for rrul & rrul_be? (#184)
Closed #184. -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/tohojo/flent/issues/184#event-2647685942___ Flent-users mailing list Flent-users@flent.org http://flent.org/mailman/listinfo/flent-users_flent.org
Re: [Flent-users] [tohojo/flent] Problem with Flent Average for rrul & rrul_be? (#184)
Better. Thanks! ``` Started Flent 1.9.9-git-fa47b0f using Python 3.7.3. Starting rrul test. Expected run time: 70 seconds. Data file written to ./rrul-2019-09-19T123807.165015.Rich_Test_-_Flent_1_9_9-git-fa47b0f.flent.gz Summary of rrul test run from 2019-09-19 16:38:07.165015 Title: 'Rich_Test_-_Flent_1.9.9-git-fa47b0f' avg median # data pts Ping (ms) ICMP : 107.00 105.00 ms 354 Ping (ms) ICMP 1.1.1.1 (extra) :30.0029.20 ms 354 Ping (ms) UDP BE : 106.84 107.41 ms 354 Ping (ms) UDP BK : 106.84 106.95 ms 354 Ping (ms) UDP EF : 106.38 107.07 ms 354 Ping (ms) avg :91.41 N/A ms 354 TCP download BE: 1.50 1.52 Mbits/s 354 TCP download BK: 1.54 1.56 Mbits/s 354 TCP download CS5 : 1.43 1.48 Mbits/s 354 TCP download EF: 1.59 1.68 Mbits/s 354 TCP download avg : 1.52 N/A Mbits/s 354 TCP download sum : 6.06 N/A Mbits/s 354 TCP totals : 6.50 N/A Mbits/s 354 TCP upload BE : 0.10 0.10 Mbits/s 354 TCP upload BK : 0.11 0.11 Mbits/s 354 TCP upload CS5 : 0.11 0.11 Mbits/s 354 TCP upload EF : 0.12 0.11 Mbits/s 354 TCP upload avg : 0.11 N/A Mbits/s 354 TCP upload sum : 0.44 N/A Mbits/s 354 Started Flent 1.9.9-git-fa47b0f using Python 3.7.3. Starting rrul_be test. Expected run time: 70 seconds. Data file written to ./rrul_be-2019-09-19T123928.602131.Rich_Test_-_Flent_1_9_9-git-fa47b0f.flent.gz Summary of rrul_be test run from 2019-09-19 16:39:28.602131 Title: 'Rich_Test_-_Flent_1.9.9-git-fa47b0f' avg median # data pts Ping (ms) ICMP : 106.00 105.00 ms 355 Ping (ms) ICMP 1.1.1.1 (extra) :30.7029.65 ms 355 Ping (ms) UDP BE1 : 105.26 105.71 ms 355 Ping (ms) UDP BE2 : 105.26 105.60 ms 355 Ping (ms) UDP BE3 : 105.60 105.54 ms 355 Ping (ms) avg :90.56 N/A ms 355 TCP download BE: 1.51 1.55 Mbits/s 355 TCP download BE2 : 1.67 1.67 Mbits/s 355 TCP download BE3 : 1.42 1.42 Mbits/s 355 TCP download BE4 : 1.40 1.45 Mbits/s 355 TCP download avg : 1.50 N/A Mbits/s 355 TCP download sum : 6.00 N/A Mbits/s 355 TCP totals : 6.44 N/A Mbits/s 355 TCP upload BE : 0.11 0.11 Mbits/s 355 TCP upload BE2 : 0.11 0.11 Mbits/s 355 TCP upload BE3 : 0.11 0.11 Mbits/s 355 TCP upload BE4 : 0.11 0.11 Mbits/s 355 TCP upload avg : 0.11 N/A Mbits/s 355 TCP upload sum : 0.44 N/A Mbits/s 355 ``` -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/tohojo/flent/issues/184#issuecomment-533215052___ Flent-users mailing list Flent-users@flent.org http://flent.org/mailman/listinfo/flent-users_flent.org
Re: [Flent-users] [tohojo/flent] RRUL Upload plot - why such low granularity? (#185)
> Basically, the problem is the way netperf determines its data output > intervals in demo mode: Ahah! That was the explanation, but I wasn't (yet) smart enough to comprehend it. [To further test out my understanding... If netperf were to use a smaller "data output interval" when it's transmitting, it would create more frequent updates and the granularity would be higher...] But I'm not sure that fully explains it unless netperf uses a (very) different data output interval for receiving... The download chart has hundreds of points, while the ratio between my download and upload is about 10:1 (7mbps vs 768kbps) so I'd expect the upload chart to have something like dozens of points... Wait... maybe the chart really does have more points: each of the four plots above might have many inflection points but since fq_codel controls their rates so carefully, there's little change to their values, thus no obvious change to the charted values... And yes - turning off SQM (see image below) shows many more data points for those upload plots. But are there enough points? In the download chart, I count around 30 samples for one plot in a 10-second period. So that's 3 per second. With an average data rate of 1.6mbps, it looks like 500 kbits/sample (a nice round number, arrived at with a SWAG - scientific wild ass guess.) If the upload has the same data output interval, those upload streams (averaging 0.11 mbps) would need about 4.5 seconds to transmit that 500kbits. And if you squint at the plot below, and take into account the enormous SWAG in the previous paragraph, it looks OK. All's right with the world. (But only after I remember to turn SQM back on...) Thanks. https://user-images.githubusercontent.com/1094930/65260618-a3f25380-dad4-11e9-880a-364522139d45.png";> -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/tohojo/flent/issues/185#issuecomment-533210171___ Flent-users mailing list Flent-users@flent.org http://flent.org/mailman/listinfo/flent-users_flent.org
Re: [Flent-users] [tohojo/flent] RRUL Upload plot - why such low granularity? (#185)
*answering the questions in the opposite order* > Any suggestions as for where to put this so it's easy to find? The bufferbloat.net site has an [RRUL Chart Explanation](https://www.bufferbloat.net/projects/bloat/wiki/RRUL_Chart_Explanation/) page. I have always felt the Flent site could have something like that as well, and this might be a good place to incorporate this information. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/tohojo/flent/issues/185#issuecomment-533208756___ Flent-users mailing list Flent-users@flent.org http://flent.org/mailman/listinfo/flent-users_flent.org
Re: [Flent-users] [tohojo/flent] Flent GUI - Two visual problems on macOS (#186)
Hmm, this probably has something to do with the GUI even processing on OSX. Not sure I can do much about this without an OSX box to test on, I'm afraid :( -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/tohojo/flent/issues/186#issuecomment-533178541___ Flent-users mailing list Flent-users@flent.org http://flent.org/mailman/listinfo/flent-users_flent.org
[Flent-users] [tohojo/flent] Flent GUI - Two visual problems on macOS (#186)
Using Flent 1.9.9-git-67ac0f7 on macOS 10.12.6: There are two visual problems with the Flent GUI on my Mac: 1. By default, Flent opens up using a large font. Choosing View / Refresh Plot changes the plot to use a smaller font, and the plots fill more of the space provided. 2. Moving the mouse over the plots (without clicking) causes the plots to disappear. View / Refresh Plot restores them. Here's the Youtube of what happens: https://youtu.be/C0xavMisVSM And here's the command line: ``` bash-3.2$ flent --gui rrul_be-2019-09-19T102325.892621.Rich_Test.flent.gz Started Flent 1.9.9-git-67ac0f7 using Python 3.7.3. Initialised matplotlib v3.1.1 on numpy v1.17.2. GUI loaded. Using Qt through pyqt5 v5.13.0. ... ``` -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/tohojo/flent/issues/186___ Flent-users mailing list Flent-users@flent.org http://flent.org/mailman/listinfo/flent-users_flent.org
Re: [Flent-users] [tohojo/flent] RRUL Upload plot - why such low granularity? (#185)
Rich Brown writes: > Here's an RRUL plot from my 7mbps/768kbps DSL circuit. The download > has good granularity, while the upload plot seems only to have a > half-dozen data points. This makes it seem faulty, or somehow > different/disconnected from the other two plots. > > I think I once saw a justification for the appearance of the upload > plot, but I didn't understand it when I read it, and can't find it > again to review. Well, it's not really by design, but it's the best we can do with current tools, unfortunately. Basically, the problem is the way netperf determines its data output intervals in demo mode: It sends a certain number of bytes, and waits for that to complete before outputting anything. And, well, if the link is really slow, this takes a while, so we don't get a data point until it's done... > If this behavior is expected, let's document the reason. Any suggestions as for where to put this so it's easy to find? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/tohojo/flent/issues/185#issuecomment-533171576___ Flent-users mailing list Flent-users@flent.org http://flent.org/mailman/listinfo/flent-users_flent.org
[Flent-users] [tohojo/flent] RRUL Upload plot - why such low granularity? (#185)
Here's an RRUL plot from my 7mbps/768kbps DSL circuit. The download has good granularity, while the upload plot seems only to have a half-dozen data points. This makes it seem faulty, or somehow different/disconnected from the other two plots. I think I once saw a justification for the appearance of the upload plot, but I didn't understand it when I read it, and can't find it again to review. If this behavior is expected, let's document the reason. The log and flent.gz files are at [Rich_Test_gz_&_logs.zip](https://github.com/tohojo/flent/files/3631723/Rich_Test_gz_._logs.zip) Thanks. https://user-images.githubusercontent.com/1094930/65253161-2c1e2c00-dac8-11e9-88cc-bdb23b625989.png";> -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/tohojo/flent/issues/185___ Flent-users mailing list Flent-users@flent.org http://flent.org/mailman/listinfo/flent-users_flent.org
Re: [Flent-users] [tohojo/flent] Problem with Flent Average for rrul & rrul_be? (#184)
Rich Brown writes: > I'm not sure I see any difference... I attach the flent.gz and log > files. Huh, no, seems I was wrong in my root cause analysis. The issue is that it's actually being output on stderr, not stdout. Should be fixed in fa47b0f6df7de6cc35eca5b6d04f77c2c59bdbf1... -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/tohojo/flent/issues/184#issuecomment-533164074___ Flent-users mailing list Flent-users@flent.org http://flent.org/mailman/listinfo/flent-users_flent.org
Re: [Flent-users] [tohojo/flent] Problem with Flent Average for rrul & rrul_be? (#184)
I'm not sure I see any difference... I attach the flent.gz and log files. [Rich_Test_gz_&_logs.zip](https://github.com/tohojo/flent/files/3631723/Rich_Test_gz_._logs.zip) https://user-images.githubusercontent.com/1094930/65253161-2c1e2c00-dac8-11e9-88cc-bdb23b625989.png";> ``` Started Flent 1.9.9-git-67ac0f7 using Python 3.7.3. Starting rrul test. Expected run time: 70 seconds. Data file written to ./rrul-2019-09-19T102204.337155.Rich_Test.flent.gz Summary of rrul test run from 2019-09-19 14:22:04.337155 Title: 'Rich_Test' avg median # data pts Ping (ms) ICMP : 107.87 107.00 ms 350 Ping (ms) ICMP 1.1.1.1 (extra) :30.4430.20 ms 350 Ping (ms) UDP BE : 103.84 104.17 ms 354 Ping (ms) UDP BK : 104.28 104.17 ms 354 Ping (ms) UDP EF : 104.28 104.49 ms 354 Ping (ms) avg : 104.13 N/A ms 354 TCP download BE: 1.35 1.39 Mbits/s 354 TCP download BK: 1.51 1.59 Mbits/s 354 TCP download CS5 : 1.47 1.49 Mbits/s 354 TCP download EF: 1.82 1.85 Mbits/s 354 TCP download avg : 1.54 N/A Mbits/s 354 TCP download sum : 6.15 N/A Mbits/s 354 TCP totals : 6.62 N/A Mbits/s 354 TCP upload BE : 0.12 0.11 Mbits/s 354 TCP upload BK : 0.12 0.11 Mbits/s 354 TCP upload CS5 : 0.12 0.11 Mbits/s 354 TCP upload EF : 0.11 0.11 Mbits/s 354 TCP upload avg : 0.12 N/A Mbits/s 354 TCP upload sum : 0.47 N/A Mbits/s 354 Started Flent 1.9.9-git-67ac0f7 using Python 3.7.3. Starting rrul_be test. Expected run time: 70 seconds. Data file written to ./rrul_be-2019-09-19T102325.892621.Rich_Test.flent.gz Summary of rrul_be test run from 2019-09-19 14:23:25.892621 Title: 'Rich_Test' avg median # data pts Ping (ms) ICMP : 106.41 105.00 ms 350 Ping (ms) ICMP 1.1.1.1 (extra) :30.6630.45 ms 350 Ping (ms) UDP BE1 : 104.60 104.38 ms 354 Ping (ms) UDP BE2 : 104.93 104.88 ms 354 Ping (ms) UDP BE3 : 103.73 103.84 ms 354 Ping (ms) avg : 104.42 N/A ms 354 TCP download BE: 1.64 1.70 Mbits/s 354 TCP download BE2 : 1.54 1.60 Mbits/s 354 TCP download BE3 : 1.43 1.48 Mbits/s 354 TCP download BE4 : 1.57 1.62 Mbits/s 354 TCP download avg : 1.54 N/A Mbits/s 354 TCP download sum : 6.18 N/A Mbits/s 354 TCP totals : 6.62 N/A Mbits/s 354 TCP upload BE : 0.11 0.11 Mbits/s 354 TCP upload BE2 : 0.11 0.11 Mbits/s 354 TCP upload BE3 : 0.11 0.11 Mbits/s 354 TCP upload BE4 : 0.11 0.11 Mbits/s 354 TCP upload avg : 0.11 N/A Mbits/s 354 TCP upload sum : 0.44 N/A Mbits/s 354 ``` -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/tohojo/flent/issues/184#issuecomment-533157455___ Flent-users mailing list Flent-users@flent.org http://flent.org/mailman/listinfo/flent-users_flent.org
