Re: [Cake] COBALT implementation in ns-3 with results under different traffic scenarios
Hello Jonathan, Thanks for your feedback. As suggested, we changed the NIC buffer size to 1 packet for the simulation and also tried these different buffer sizes: 10, 50 and 75. The default NIC buffer size in ns-3 is 100 packets. Additionally, we also enabled BQL and tried. We see that the link utilization gets significantly affected when we keep the NIC buffer size small. The results are put up on the following link: https://github.com/Daipu/COBALT/wiki/Link-Utilization-Graphs-with-Different-NetDeviceQueue-size Thanks and Regards, Jendaipou Palmei Shefali Gupta On Sun, Dec 9, 2018 at 6:51 PM Jonathan Morton wrote: > > On 9 Dec, 2018, at 10:37 am, Jendaipou Palmei > wrote: > > > > By hidden queues, do you mean the NIC buffers? ns-3 has a Linux-like > traffic control wherein the packets dequeued by a queue discipline are > enqueued into NIC buffer. > > That's right. Linux now uses BQL, which (given compatible NIC drivers) > limits the number of packets in the NIC buffers to a very small value - > much smaller than is evident from your data. If you were to measure the > end-to-end RTT of each, I'm certain you would see this effect dominating > the mere 50ms latency you're trying to model. > > Ideally, AQM would applied to the hardware queue anyway. For simulation > purposes, I recommend reducing the NIC buffer on the bottleneck interface > to 1 packet, so that the simulated AQM's effects are easiest to measure. > > - Jonathan Morton > > ___ Cake mailing list Cake@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cake
Re: [Cake] COBALT implementation in ns-3 with results under different traffic scenarios
Hi Jonathan, Yes, we are using TCP NewReno at the moment. There was a typo in labeling the Y-axis; instead of "Throughput" it should be "Link Utilization" in the following graphs (now corrected): https://github.com/Daipu/COBALT/wiki/Light-Traffic throughput graphs for the same scenario are here: https://github.com/Daipu/COBALT/wiki/Instantaneous-throughput:-Light-Traffic and cwnd graphs here: https://github.com/Daipu/COBALT/wiki/cwnd-plots:-Light-traffic So, now what we see is that although queue occupancy is under control and link remains fully utilized, the senders cwnd gets synchronized in one scenario (only when packet size is 1000 bytes and with COBALT). For all other cases, there is no synchronization of cwnd (including COBALT with packet size 1500 bytes). By hidden queues, do you mean the NIC buffers? ns-3 has a Linux-like traffic control wherein the packets dequeued by a queue discipline are enqueued into NIC buffer. The tasks that we're currently working on are listed here: https://github.com/Daipu/COBALT/issues/1 Thanks a lot for your help. We really appreciate it. Regards, Jendaipou Palmei Shefali Gupta On Thu, Dec 6, 2018 at 11:06 PM Jonathan Morton wrote: > >> We're currently working on the following: > >> > >> 1. plots for the actual number of marks/drops per time interval for > COBALT, CoDel, and PIE. > >> 2. zoomed in plots on small time intervals to show the dynamic behavior > of the algorithm. > >> 3. a file showing the timestamp of each drop. > > > > I await these with interest. > > I noticed that some progress has been made here already, in particular I > can now see cwnd graphs which make a very interesting datapoint when > directly compared with the throughput and queue-occupancy graphs. It's now > definitely clear that the senders are using TCP Reno or some close variant > thereof. > > In fact, the three graphs are mutually inconsistent. Aside from the sharp > cwnd reduction events, the cwnd of all the flows increases roughly linearly > over time, but the throughput remains flat while the queue is almost always > empty (for Codel and COBALT). This can only be explained if there's a > hidden queue at the bottleneck, perhaps associated with the simulated NIC > immediately downstream of the AQM. > > I would suggest checking the simulation setup carefully for hidden queues > of this sort. > > - Jonathan Morton > > ___ Cake mailing list Cake@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cake
Re: [Cake] COBALT implementation in ns-3 with results under different traffic scenarios
Hello Dave and Jonathan, Thanks for the feedback! We have uploaded the corresponding graphs for reference CoDel. Link: https://github.com/Daipu/COBALT/wiki/Drop-Count-Graph We have also plotted the instantaneous throughput for all flows in Light traffic scenario for COBALT and CoDel. These graphs are plotted for packet size with 1000 bytes and 1500 bytes. Link: https://github.com/Daipu/COBALT/wiki/Throughput-for-Separate-Flow We're currently working on the following: 1. plots for the actual number of marks/drops per time interval for COBALT, CoDel, and PIE. 2. zoomed in plots on small time intervals to show the dynamic behavior of the algorithm. 3. a file showing the timestamp of each drop. About collaborating for writing a paper on this work: we'd be glad to do so :) thanks for your guidance and help! Thanks and regards Jendaipou Palmei Shefali Gupta On Tue, Dec 4, 2018 at 8:51 PM Dave Taht wrote: > On Tue, Dec 4, 2018 at 7:02 AM Jonathan Morton > wrote: > > > > > On 4 Dec, 2018, at 12:31 pm, Jendaipou Palmei < > jendaipoupal...@gmail.com> wrote: > > > > > > We have uploaded the plots for the 'count' variable of COBALT (with a > segment size of 1500 and 1000 bytes). > > > > > > Link: https://github.com/Daipu/COBALT/wiki/Cobalt-Drop-Count > > > > > > We have not yet implemented ECN feature in COBALT, so packets are > currently dropped instead of being marked. > > > > > > Are these the plots that you were referring to? > > > > More-or-less, yes, though these actually show an internal state variable > of the Codel algorithm rather than the actual number of marks/drops per > time interval. I was hoping to see similar graphs for the reference-Codel > and PIE runs, since we can gain more insight from that, and PIE doesn't > have an internal "count" variable that corresponds with Codel. > Nevertheless, the view into "count" behaviour is interesting in itself, and > I'd like to see the corresponding graphs from reference Codel. > > > > An artefact visible in these graphs is an apparent lack of sampling > while not in the dropping state. Thus you seem to have a gradual ramp from > 0 to 1 count over the several seconds interval between activations, though > in fact the variable is discrete. It would be better to show that > transition more precisely. > > > > For study, it is also often helpful to zoom in on small time intervals > to see the dynamic behaviour of the algorithm, particularly during the > transition from slow-start to steady-state, where there is seemingly a big > difference between reference Codel and COBALT. > > I'm loving the slow start result. > > > > > Another interesting graph to produce for each algorithm and traffic type > is the instantaneous throughput of each flow. This offers insight into the > relative fairness of each algorithm, and might help to explain the anomaly > seen with 1000-byte packets and COBALT. Usually this graph also reveals, > through the shape of each throughput curve, which CC algorithm is in use - > currently I'm guessing NewReno. CUBIC and CTCP, which are also in common > use, would behave differently. > > a file showing the timestamp of each drop would be easier to post process. > > > > > - Jonathan Morton > > > > > -- > > Dave Täht > CTO, TekLibre, LLC > http://www.teklibre.com > Tel: 1-831-205-9740 > ___ Cake mailing list Cake@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cake
Re: [Cake] COBALT implementation in ns-3 with results under different traffic scenarios
Hello Jonathan, Thanks for the quick reply. We have uploaded the plots for the 'count' variable of COBALT (with a segment size of 1500 and 1000 bytes). Link: https://github.com/Daipu/COBALT/wiki/Cobalt-Drop-Count We have not yet implemented ECN feature in COBALT, so packets are currently dropped instead of being marked. Are these the plots that you were referring to? Thanks and Regards Jendaipou Palmei Shefali Gupta On Fri, Nov 30, 2018 at 5:28 PM Jonathan Morton wrote: > > On 30 Nov, 2018, at 1:53 pm, Jendaipou Palmei > wrote: > > > > However, we finally noticed that it was the packet size used in our > simulations that was affecting the throughput. It was earlier set to 1000 > bytes, and after making it 1500 bytes (including headers) we note that the > throughput is not affected. > > > > We have uploaded the new graphs on the same wiki link: > > https://github.com/Daipu/COBALT/wiki/Light-Traffic > > > > We're not sure why packet size is affecting the throughput so largely. > Is it the expected behavior? > > That particular effect is probably accidental. > > Is it possible to also show graphs of the packet marking (or dropping) > rate? That may be revealing. > > - Jonathan Morton > > ___ Cake mailing list Cake@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cake
Re: [Cake] COBALT implementation in ns-3 with results under different traffic scenarios
Hi Jonathan, Thanks a lot for the quick review and suggestions to improve the code readability. We incorporated all the changes but didn't see any improvement in the results. However, we finally noticed that it was the packet size used in our simulations that was affecting the throughput. It was earlier set to 1000 bytes, and after making it 1500 bytes (including headers) we note that the throughput is not affected. We have uploaded the new graphs on the same wiki link: https://github.com/Daipu/COBALT/wiki/Light-Traffic We're not sure why packet size is affecting the throughput so largely. Is it the expected behavior? Thanks and Regards Jendaipou Palmei Shefali Gupta On Tue, Nov 27, 2018 at 8:07 PM Jonathan Morton wrote: > > On 27 Nov, 2018, at 4:10 pm, Jendaipou Palmei > wrote: > > > > We have made the changes in the code as suggested. > > > > Here is the diff after making the changes: > > > > > https://github.com/Daipu/COBALT/commit/033db330287e2072bad94ac441f8aed774678a7d > > > > Additionally, we have also updated the values of 'Pincrement' and > 'Pdecrement' (parameters of BLUE) in ns-3 to match the ones used in the > Cake implementation in Linux. > > > > Does the diff look correct? > > It does look like an improvement. How's the behaviour? > > NB: the decimal values used in the constants are not quite precise > equivalents of the Linux values, since you have dropped some of the > trailing digits. You could use (1./256) and (1./4096) to both make them > precise and more human-readable. > > - Jonathan Morton > > ___ Cake mailing list Cake@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cake
Re: [Cake] COBALT implementation in ns-3 with results under different traffic scenarios
Hello Jonathan, We have made the changes in the code as suggested. Here is the diff after making the changes: https://github.com/Daipu/COBALT/commit/033db330287e2072bad94ac441f8aed774678a7d Additionally, we have also updated the values of 'Pincrement' and 'Pdecrement' (parameters of BLUE) in ns-3 to match the ones used in the Cake implementation in Linux. Does the diff look correct? Thanks and Regards, Jendaipou Palmei Shefali Gupta On Sun, Nov 25, 2018 at 11:52 AM Jendaipou Palmei wrote: > Thanks a lot for taking out time to review our code, Jonathan. > > We'll make the changes according to your suggestions and produce new plots. > > Thanks, Dave for the feedback. Yes, we will run the simulations with > fq-codel once we fix the code as suggested by Jonathan, and also run > simulations with higher bandwidths as you suggested. > > I'll upload the source code of the programs in the same repo, and give you > the link. > > Regards, > Jendaipou Palmei > Shefali Gupta > > On Sat, Nov 24, 2018 at 8:29 AM Jonathan Morton > wrote: > >> >> This is possibly a correct result in your simulation!! - the periodic >> >> throughput drop you are showing in cobalt at this bandwidth and rtt. >> >> I'm happy to see cobalt kick in early on slow start but not happy to >> >> see the periodic drop. Jon, do you have time for a code review? >> > >> > I looked at it briefly, but the code structure is different enough that >> I need to sit down and study it carefully to figure out whether there are >> any relevant differences. >> > >> > The throughput drops most likely occur because the TCPs become >> synchronised and remain so under AQM action. You can see that the >> frequency of the system is lower in the later part of the COBALT run than >> in the Codel run, but the same as Codel in the earlier part where >> throughput drops don't occur. But this shouldn't really occur with a >> Codel-based AQM (as COBALT is), because a single mark is sufficient to tell >> TCP to back off over one RTT. An explanation might be if this >> implementation of COBALT isn't running down correctly when deactivated, so >> the mark frequency only rises while being turned on and off. The run-down >> behaviour is a major intentional difference between COBALT and reference >> Codel. >> > >> > I'll look at the code more closely with that in mind. >> >> Okay, I've had a look - not quite line by line, but the parts I consider >> important for the behaviour seen so far. >> >> There are a couple of small behavioural differences between your code and >> mine, which should be corrected if the model is to accurately reflect the >> prototype. These are probably not relevant for the results shown so far, >> but are likely show up on more aggressive tests involving unresponsive >> traffic. >> >> - On queue overflow, a tail drop is used to resolve it. While not >> technically part of COBALT, Cake performs head-dropping on queue overflow, >> doing so from the longest queue, and I consider that to be best practice. >> This gets the message to the offending sender ASAP, without having to >> bubble up through the jammed queue first. If the packets currently at the >> head of the queue are smaller than the one being offered, you might need to >> drop more than one to maintain the size invariant. >> >> - The hard-drop flag for BLUE is set at the top of the control-law >> function, and tested in order to bypass the Codel logic if already set. >> This is not how the COBALT code operates; the BLUE logic should come last, >> and the Codel logic run unconditionally. >> >> Everything else looks reasonably correct at first glance (though the >> amount of boilerplate is epic). I would recommend verifying that >> CobaltQueueEmpty() actually gets called when appropriate though. Without >> it, I suspect that the run-down logic won't work as intended. >> >> - Jonathan Morton >> >> > > -- > Yours Faithfully, > Jendaipou Palmei > ___ Cake mailing list Cake@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cake
Re: [Cake] COBALT implementation in ns-3 with results under different traffic scenarios
Thanks a lot for taking out time to review our code, Jonathan. We'll make the changes according to your suggestions and produce new plots. Thanks, Dave for the feedback. Yes, we will run the simulations with fq-codel once we fix the code as suggested by Jonathan, and also run simulations with higher bandwidths as you suggested. I'll upload the source code of the programs in the same repo, and give you the link. Regards, Jendaipou Palmei Shefali Gupta On Sat, Nov 24, 2018 at 8:29 AM Jonathan Morton wrote: > >> This is possibly a correct result in your simulation!! - the periodic > >> throughput drop you are showing in cobalt at this bandwidth and rtt. > >> I'm happy to see cobalt kick in early on slow start but not happy to > >> see the periodic drop. Jon, do you have time for a code review? > > > > I looked at it briefly, but the code structure is different enough that > I need to sit down and study it carefully to figure out whether there are > any relevant differences. > > > > The throughput drops most likely occur because the TCPs become > synchronised and remain so under AQM action. You can see that the > frequency of the system is lower in the later part of the COBALT run than > in the Codel run, but the same as Codel in the earlier part where > throughput drops don't occur. But this shouldn't really occur with a > Codel-based AQM (as COBALT is), because a single mark is sufficient to tell > TCP to back off over one RTT. An explanation might be if this > implementation of COBALT isn't running down correctly when deactivated, so > the mark frequency only rises while being turned on and off. The run-down > behaviour is a major intentional difference between COBALT and reference > Codel. > > > > I'll look at the code more closely with that in mind. > > Okay, I've had a look - not quite line by line, but the parts I consider > important for the behaviour seen so far. > > There are a couple of small behavioural differences between your code and > mine, which should be corrected if the model is to accurately reflect the > prototype. These are probably not relevant for the results shown so far, > but are likely show up on more aggressive tests involving unresponsive > traffic. > > - On queue overflow, a tail drop is used to resolve it. While not > technically part of COBALT, Cake performs head-dropping on queue overflow, > doing so from the longest queue, and I consider that to be best practice. > This gets the message to the offending sender ASAP, without having to > bubble up through the jammed queue first. If the packets currently at the > head of the queue are smaller than the one being offered, you might need to > drop more than one to maintain the size invariant. > > - The hard-drop flag for BLUE is set at the top of the control-law > function, and tested in order to bypass the Codel logic if already set. > This is not how the COBALT code operates; the BLUE logic should come last, > and the Codel logic run unconditionally. > > Everything else looks reasonably correct at first glance (though the > amount of boilerplate is epic). I would recommend verifying that > CobaltQueueEmpty() actually gets called when appropriate though. Without > it, I suspect that the run-down logic won't work as intended. > > - Jonathan Morton > > -- Yours Faithfully, Jendaipou Palmei ___ Cake mailing list Cake@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cake
Re: [Cake] COBALT implementation in ns-3 with results under different traffic scenarios
Hi Dave, We have added the results for CoDel and PIE in the same setup. Link: https://github.com/Daipu/COBALT/wiki/Light-Traffic Thanks for your help. On Thu, Nov 22, 2018 at 9:02 PM Dave Taht wrote: > Your light traffic graph is puzzling, with the synchronized drops. Is > it possible for you to do another test run, same setup, with codel or > pie? > On Thu, Nov 22, 2018 at 5:57 AM Jendaipou Palmei > wrote: > > > > Hello Jonathan and Dave, > > > > Thanks for your timely inputs. > > > > We have implemented COBALT in ns-3. The main C++ file is here: > > > > > https://github.com/Daipu/COBALT/blob/master/src/traffic-control/model/cobalt-queue-disc.cc > > > > We have also added Queue length and Throughput graphs for COBALT in our > wiki: > > > > https://github.com/Daipu/COBALT/wiki > > > > Thanks and Regards, > > Jendaipou Palmei > > Shefali Gupta > > ___ > > Cake mailing list > > Cake@lists.bufferbloat.net > > https://lists.bufferbloat.net/listinfo/cake > > > > -- > > Dave Täht > CTO, TekLibre, LLC > http://www.teklibre.com > Tel: 1-831-205-9740 > -- Yours Faithfully, Jendaipou Palmei ___ Cake mailing list Cake@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cake
[Cake] COBALT implementation in ns-3 with results under different traffic scenarios
Hello Jonathan and Dave, Thanks for your timely inputs. We have implemented COBALT in ns-3. The main C++ file is here: https://github.com/Daipu/COBALT/blob/master/src/traffic-control/model/cobalt-queue-disc.cc We have also added Queue length and Throughput graphs for COBALT in our wiki: https://github.com/Daipu/COBALT/wiki Thanks and Regards, Jendaipou Palmei Shefali Gupta ___ Cake mailing list Cake@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cake
