On Mon, Apr 13, 2015 at 5:39 PM, Sebastian Moeller <[email protected]> wrote:
> This looks reasonable.
>
> >
> > ~ # ifconfig
> > ge00 Link encap:Ethernet HWaddr 20:4E:7F:91:9F:5D
> > collisions:0 txqueuelen:1000
> >
> > gw00 Link encap:Ethernet HWaddr 22:4E:7F:91:9F:5C
> > collisions:0 txqueuelen:1000
> >
> >
> > ifb4ge00 Link encap:Ethernet HWaddr C2:00:73:A7:1E:30
> > collisions:0 txqueuelen:32
> >
> >
> > ifb4gw00 Link encap:Ethernet HWaddr 9A:BA:17:27:79:A5
> > collisions:0 txqueuelen:32
> >
> > lo Link encap:Local Loopback
> > collisions:0 txqueuelen:0
> >
> > se00 Link encap:Ethernet HWaddr 22:4E:7F:91:9F:5C
> > collisions:0 txqueuelen:1000
> >
> > sw00 Link encap:Ethernet HWaddr 20:4E:7F:91:9F:5C
> > collisions:0 txqueuelen:1000
> >
> > sw10 Link encap:Ethernet HWaddr 20:4E:7F:91:9F:5E
> > collisions:0 txqueuelen:1000
>
> I had a look at my cerowrt router and I also see txqueuelen at 1000,
> but IIRC that does not matter much anymore, since the wndr37/800 support BQL:
> cat /sys/class/net/ge00/queues/tx-0/byte_queue_limits/limit_max
> 3000
> So even with txqueuelen = 1000 the tx queue will only hold 3000 bytes. For
> fib and friends it does not really matter as far as I can tell.
Interesting, I had been reading about BQL but didn't fully understand
it until now. This clears up a lot of confusion, thank you.
I assume tweaking ring parameters from default RX:128 and TX:32
doesn't matter anymore thenr?
> > and after rebooting and running my [ethtool/txqueuelen]commands (still
> > testing different values. should the IFBs txqueuelen be left alone? I left
> > at default):
> >
> > ~ # ifconfig
> > ge00 Link encap:Ethernet HWaddr 20:4E:7F:91:9F:5D
> > collisions:0 txqueuelen:8
> >
> > gw00 Link encap:Ethernet HWaddr 22:4E:7F:91:9F:5C
> > collisions:0 txqueuelen:4
> >
> >
> > ifb4ge00 Link encap:Ethernet HWaddr C2:00:73:A7:1E:30
> > collisions:0 txqueuelen:32
> >
> >
> > ifb4gw00 Link encap:Ethernet HWaddr 9A:BA:17:27:79:A5
> > collisions:0 txqueuelen:32
> >
> > lo Link encap:Local Loopback
> > collisions:0 txqueuelen:0
> >
> > se00 Link encap:Ethernet HWaddr 22:4E:7F:91:9F:5C
> > collisions:0 txqueuelen: 8
> >
> > sw00 Link encap:Ethernet HWaddr 20:4E:7F:91:9F:5C
> > collisions:0 txqueuelen: 4
> >
> > sw10 Link encap:Ethernet HWaddr 20:4E:7F:91:9F:5E
> > collisions:0 txqueuelen: 4
> >
> >
>
> If you have time and netperf-wrapper it would be good to convince yourself
> and us again, that txqueuelen really does not matter for BQL’d interfaces by
> running RRUL tests with and without your modifications….
Will do if I can get a friend to set up a netperf server in a node at
his datacenter he works at, though I believe he's using Debian Jessy
without any kind of QoS, but his datacenter routers might screw with
the packets. Maybe I can hook a spare box up to the WAN port
temporarily to test with.
############################################################################
Off topic a bit, if I might pick some of your brains - does anyone
have experience with Intel gigE NICs and any advice on how they might
be tweaked for lowest latency possible when connected to a debloated
router? Most of the websites on this topic are inexperienced,
outdated, or have no evidence to back their assumptions.
Throughput is not a concern, as the machine in question is for gaming
only, on Windows 8.1 (though in linux it has similarly named
settings), connected via gigE to CeroWRT/WNDR3800 gateway which is
directly connected via gigE to fiber ONT providing 32/25 (Cero's SQM
capped to 28.8/22.5). ((bittorrent traffic still has latency issues
though even with really low bandwidth caps, measured from a linux box,
perhaps ISP related))
Intel Network Interface is forced into MSI mode (message signaled interrupts)
My adapter settings settings below, those with an asterisk have been modified
*Adaptive Inter-Frame Spacing: Disabled
*Flow Control: Disabled (Options: Disabled, Rx & Tx Enabled, Rx
Enabled, Tx Enabled)
## Default is Rx Enabled. I really don't see a benefit to pause frames
when we have QoS algorithms.
Receive Buffers: 256 (value: 80-2048, this is the equivalent of Ring
parameters in Linux)
Transmit Buffers: 512 (value: 80-2048, this is the equivalent of Ring
parameters in Linux)
## Buffer thoughts: maybe a lower ratio such as 128 and 256 or 64 and
128? Documentation says:
## each descriptor comes with a 2048 byte buffer.
## Why more TX than RX? Our architecture favors the
nondeterministic RX side for priority so there is more turnover of
descriptors
## than the TX side. Plus the O/S can sometimes not return TX
assets back to the driver in a timely fashion.
IPv4 Checksum Offload: Rx & Tx Enabled
Protocol ARP Offload: Enabled
Protocol NS Offload: Enabled
TCP Checksum Offload (IPv4): Rx & Tx Enabled (Options: Disabled, Rx
Enabled, Tx Enabled)
TCP Checksum Offload (IPv6): Rx & Tx Enabled (Options: Disabled, Rx
Enabled, Tx Enabled)
UDP Checksum Offload (IPv4): Rx & Tx Enabled (Options: Disabled, Rx
Enabled, Tx Enabled)
UDP Checksum Offload (IPv6): Rx & Tx Enabled (Options: Disabled, Rx
Enabled, Tx Enabled)
Large Send Offload V2 (IPv4): Enabled (this is called TSO, TCP
Segmentation Offload, in linux)
Large Send Offload V2 (IPv6): Enabled (this is called TSO, TCP
Segmentation Offload, in linux)
## Offload thoughts: Should any offloads be disabled if the host
machine is fast enough? do these cause delays?
*Jumbo Packet: Disabled
## Probably don't want anything potentially latency sensitive being
clumped up into a huge packet,
especially when the destination is WAN which the router would
have to segment anyways
*Interrupt Moderation: Disabled (This is probably similar to large
receive offload (LRO), description says it reduces interrupts.)
*Interrupt Moderation Rate: Off (Options: Adaptive, Extreme, High,
Low, Medium, Minimal, Off)
## Presumably Interrupt Moderation might cause latency by aggregating
packets into a buffer?
Receive Side Scaling: Enabled (Documentation says this makes
processing scale to multiple CPU cores)
Maximum Number of RSS Queues: 2 Queue (Options: 1 Queue, 2 Queue)
## Scaling up to 2 cpu cores? unsure if this would cause latency
Gigabit Master Slave Mode: Auto Detect (Options: Auto Detect, Force
Master Mode, Force Slave Mode)
Enable PME: Disabled
Energy Efficient Ethernet: Disabled
System Idle Power Saver: Disabled
_______________________________________________
Cerowrt-devel mailing list
[email protected]
https://lists.bufferbloat.net/listinfo/cerowrt-devel
