Re: [Bloat] Fwd: [Bug 1436945] Re: devel: consider fq_codel as the default qdisc for networking
On Tue, 5 Jun 2018, Jonas Mårtensson wrote: What about PLPMTU? Do you think they might tweak that too? net.ipv4.tcp_mtu_probing=2 (despite name, applies to IPv6 too) Maybe, suggest it on their github. But I would maybe propose instead net.ipv4.tcp_mtu_probing=1. MTU probing would be awsome. I am great fan of PLPMTU and this should be default-on everywhere in all protocols. -- Mikael Abrahamssonemail: swm...@swm.pp.se___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Bug 1436945] Re: devel: consider fq_codel as the default qdisc for networking
> On 5 Jun, 2018, at 9:34 pm, Sebastian Moeller wrote: > > The rationale for that decision still is valid, at low bandwidth every > opportunity to send a packet matters… Yes, which is why the DRR++ algorithm is used to carefully choose which flow to send a packet from. > …and every packet being transferred will increase the queued packets delay by > its serialization delay. This is trivially true, but has no effect whatsoever on inter-flow induced latency, only intra-flow delay, which is already managed adequately well by an ECN-aware sender. May I remind you that Cake never drops the last packet in a flow subqueue due to AQM action, but may still apply an ECN mark to it. That's because dropping a tail packet carries a risk of incurring an RTO before retransmission occurs, rather than "only" an RTT delay. Both RTO and RTT are always greater than the serialisation delay of a single packet. Which is why ECN remains valuable even on very low-bandwidth links. - Jonathan Morton ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Bug 1436945] Re: devel: consider fq_codel as the default qdisc for networking
Hi Jonathan, > On Jun 5, 2018, at 17:10, Jonathan Foulkes wrote: > > Jonathan, in the past the recommendation was for NOECN on egress if capacity > <4Mbps. Is that still the case in light of this? > > Thanks, > > Jonathan Foulkes > >> On Jun 4, 2018, at 5:36 PM, Jonathan Morton wrote: >> >>> On 4 Jun, 2018, at 9:22 pm, Jonas Mårtensson >>> wrote: >>> >>> Speaking about systemd defaults, they just enabled ecn for outgoing >>> connections: >> >> That is also good news. With Apple *and* Ubuntu using it by default, we >> should finally get critical mass of ECN traffic and any remaining blackholes >> fixed, making it easy for everyone else to justify turning it on as well. The rationale for that decision still is valid, at low bandwidth every opportunity to send a packet matters and every packet being transferred will increase the queued packets delay by its serialization delay. The question IMHO is more is 4 Mbps a reasonable threshold to disable ECN or not. Here are the serialization delays for a few selected bandwidths: 1000*(1538*8)/(500*1000) = 24.61 ms 1000*(1538*8)/(1000*1000) = 12.30 ms 1000*(1538*8)/(2000*1000) = 6.15 ms 1000*(1538*8)/(4000*1000) = 3.08 ms 1000*(1538*8)/(8000*1000) = 1.54 ms 1000*(1538*8)/(1*1000) = 1.23 ms 1000*(1538*8)/(12000*1000) = 1.03 ms Personally, I guess I sort of agree with the <= 4Mbps threshold, maybe 2Mbps, but at <=1Mbps the serialization delay gets painful. In sqm-scripts we currently unconditionally default to egress(ECN) off, which might be to pessimistic about the usual egress bandwidths. Best Regards >> >> - Jonathan Morton >> >> ___ >> Bloat mailing list >> Bloat@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/bloat > > ___ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Bug 1436945] Re: devel: consider fq_codel as the default qdisc for networking
> On 5 Jun, 2018, at 6:10 pm, Jonathan Foulkes wrote: > > Jonathan, in the past the recommendation was for NOECN on egress if capacity > <4Mbps. Is that still the case in light of this? I would always use ECN, no exceptions - unless the sender is using a TCP congestion control algorithm that doesn't support it (eg. BBR currently). That's true for both fq_codel and Cake. With ECN, codel's action doesn't drop packets, only resizes the congestion window. - Jonathan Morton ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Fwd: [Bug 1436945] Re: devel: consider fq_codel as the default qdisc for networking
Am 05.06.2018 um 09:49 schrieb Jonathan Morton: On 5 Jun, 2018, at 10:44 am, Mario Hock wrote: Just to make sure that I got your answer correctly. The benefit for endsystems comes from the "fq" (flow queuing) part, not from the "codel" part of fq_codel? That's a fair characterisation, yes. In fact, even for middleboxes, the "flow isolation" semantics of FQ have the most impact on reducing inter-flow induced latency. The "codel" part (AQM) helps with intra-flow latency, which is usually much less important once flow isolation is in place, but is still worth having. Thanks for the confirmation. A potential drawback of using the codel part (of fq_codel) in the endsystems is that it can cause packet drops already at the sender. I could actually confirm this assumption with a very simple experiment consisting of two servers connected over a 1Gbit/s link and 100 parallel flows (iperf3). With fq_codel I had 5,000-10,000 retransmissions within 60s. With fq (or pfifo_fast) no packets are dropped. (I presume either "TCP small queues" or backpressure keeps the queues from overflowing.) Also, ping times (delays for short flows) were similar with fq and fq_codel (mostly <= 1ms). Is there any advantage of using fq_codel over fq at the endsystems? Mario Hock ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Fwd: [Bug 1436945] Re: devel: consider fq_codel as the default qdisc for networking
> On 5 Jun, 2018, at 10:44 am, Mario Hock wrote: > > Just to make sure that I got your answer correctly. The benefit for > endsystems comes from the "fq" (flow queuing) part, not from the "codel" part > of fq_codel? That's a fair characterisation, yes. In fact, even for middleboxes, the "flow isolation" semantics of FQ have the most impact on reducing inter-flow induced latency. The "codel" part (AQM) helps with intra-flow latency, which is usually much less important once flow isolation is in place, but is still worth having. - Jonathan Morton ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Fwd: [Bug 1436945] Re: devel: consider fq_codel as the default qdisc for networking
Am 05.06.2018 um 01:00 schrieb David Lang: On Mon, 4 Jun 2018, Bless, Roland (TM) wrote: Hi, Am 24.05.2018 um 17:38 schrieb Jan Ceuleers: Took 3 years after Dave approached them, but Ubuntu is finally adopting fq_codel as the default qdisc. Yes, if the Linux kernel is forwarding packets it makes a lot of sense, but I don't understand why it make sense for ordinary end-systems. Didn't Byte Queue Limits (BQL) suffice? Just curious... no, BQL makes things much better (and make it possible for more advanced quueing to take place), but you can still run into problems where a bulk stream can flood the output queue so that other traffic suffers badly. with fq_codel, the available bandwidth is distributed in a way that ends up being much more functional. It turns out that the behavior to prioritize new and sparse connections significantly improves perceived performance (no more long delays in DNS lookups before you start doing any real work for example) without BQL, you can't even see the rest of the problems, but BQL doesn't solve everything. Just to make sure that I got your answer correctly. The benefit for endsystems comes from the "fq" (flow queuing) part, not from the "codel" part of fq_codel? Mario Hock ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Fwd: [Bug 1436945] Re: devel: consider fq_codel as the default qdisc for networking
On Tue, Jun 5, 2018 at 2:22 AM, Michael Richardson wrote: > > Jonas Mårtensson wrote: > > Speaking about systemd defaults, they just enabled ecn for outgoing > > connections: > > > https://github.com/systemd/systemd/pull/9143 > > What about PLPMTU? Do you think they might tweak that too? > > net.ipv4.tcp_mtu_probing=2 > (despite name, applies to IPv6 too) Maybe, suggest it on their github. But I would maybe propose instead net.ipv4.tcp_mtu_probing=1. /Jonas ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
