On Mon, Nov 12, 2018 at 7:38 AM Mikael Abrahamsson <swm...@swm.pp.se> wrote: > > On Mon, 12 Nov 2018, Dave Taht wrote: > > > tc -s qdisc show dev your_device? > > tc -s qdisc show dev your_ifbdevice? > > I haven't restarted in 40 days and I don't remember restarting cake, so > this should be several weeks of data. > > qdisc cake 8031: dev eth1.2 root refcnt 2 bandwidth 100Mbit besteffort > triple-isolate nonat nowash no-ack-filter split-gso rtt 100.0ms raw
Pretty nice. There is not much need for it, but it would be interesting to know how much cpu overhead a full set of uploads with and without the ack-filter enabled at 100Mbit with this arch takes. > overhead 0 > Sent 70822286277 bytes 202513660 pkt (dropped 13984, overlimits 25350421 > requeues 0) > backlog 0b 0p requeues 0 > memory used: 5156288b of 5000000b > capacity estimate: 100Mbit > min/max network layer size: 42 / 1514 > min/max overhead-adjusted size: 42 / 1514 > average network hdr offset: 14 > > Tin 0 > thresh 100Mbit > target 5.0ms > interval 100.0ms > pk_delay 4us > av_delay 1us > sp_delay 1us > backlog 0b > pkts 202527644 > bytes 70842325936 Don't use this connection much, do you? :) > way_inds 4939006 > way_miss 11834545 > way_cols 0 > drops 13984 It's nice to know AQM is still needed at these speeds. > marks 512 and that you have at least one device with ecn enabled. Would this be OSX or IOS perhaps? > ack_drop 0 And how many ack-drops you get. > sp_flows 2 > bk_flows 1 > un_flows 0 > max_len 28766 I really hate GRO. At 100Mbit, without splitting, that's 2.5ms of jitter. I've had a long held dream of being able to do the "jamophone" with less than 2.5ms latency end-2-end across town, with multichannel 48khz 24 bit audio... at 130us, that leaves some room for the encoder. I don't suppose you have someone else "across town" you could run some benchmarks against? I did a extremely successful test of sonic's fiber network in SF where I got well under 2.5ms jitter for an 8 channel stream... then I left SF. :( I was kind of expecting 64k, though, here. > quantum 1514 > > qdisc ingress ffff: dev eth1.2 parent ffff:fff1 ---------------- > Sent 807912654344 bytes 631652827 pkt (dropped 0, overlimits 0 requeues > 0) > backlog 0b 0p requeues 0 > qdisc cake 8032: dev ifb4eth1.2 root refcnt 2 bandwidth 250Mbit besteffort > triple-isolate nonat nowash no-ack-filter split-gso rtt 100.0ms raw > overhead 0 Similarly, a cpu number under load. I note here, that splitting GSO has a big cost, (primarily in routing table lookup) and you can at these speeds, probably disable it. ack filtering will not help. > Sent 829571211610 bytes 631641016 pkt (dropped 11811, overlimits > 790055554 requeues 0) > backlog 0b 0p requeues 0 > memory used: 4540528b of 12500000b > capacity estimate: 250Mbit > min/max network layer size: 60 / 1514 > min/max overhead-adjusted size: 60 / 1514 > average network hdr offset: 14 > > Tin 0 > thresh 250Mbit > target 5.0ms > interval 100.0ms > pk_delay 1.2ms > av_delay 559us > sp_delay 1us > backlog 0b > pkts 631652827 > bytes 829588333230 > way_inds 12061686 > way_miss 12913211 > way_cols 1 > drops 11811 > marks 3589 > ack_drop 0 > sp_flows 1 > bk_flows 1 > un_flows 0 > max_len 38444 I was also expecting 64k here. I imagine you are using modern linuxes that don't overuse TSO anymore, and osx and windows never got into it to the extreme that linux did. > quantum 1514 > > > > > -- > Mikael Abrahamsson email: swm...@swm.pp.se > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat -- Dave Täht CTO, TekLibre, LLC http://www.teklibre.com Tel: 1-831-205-9740 _______________________________________________ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat