Hi all,
while testing mvs@ npppd diffs I've noticed that if TSO is enabled on
pppoe server, forwarding from/to pppoe client from/to some host outside
pppoe server is very slow. Interesting thing is that iperf between pppoe
clients behaves ok with or without TSO on pppoe server. Enabling or
disabling TSO on pppoe clients doesn't have any impact on network
performance.
Setup:
pf is disabled
pppoe server - 192.168.100.205 (ix0) <- this is where I'm disabling or
enabling TSO
- serves 10.53.253/24 network to pppoe clients
pppoe client1 - r620-2 - ix0 without address - ppppe0 10.53.253.12/24
pppoe client2 - smc24 - ix0 without address - pppoe0 10.53.253.13/24
pppoe client3 - x3550m4- ix0 without address - pppoe0 10.53.253.14/14
test2 box - 192.168.100.219
- route 10.53.253.0/24 via 192.168.100.205 dev p2p1
pppoe client iperf server - test2 box iperf client - TSO disabled
[root@test2 ~]# iperf3 -c 10.53.253.12
Connecting to host 10.53.253.12, port 5201
[ 4] local 192.168.100.219 port 40488 connected to 10.53.253.12 port 5201
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 79.4 MBytes 666 Mbits/sec 0 52.0 KBytes
[ 4] 1.00-2.00 sec 134 MBytes 1.12 Gbits/sec 0 87.2 KBytes
[ 4] 2.00-3.00 sec 160 MBytes 1.34 Gbits/sec 0 121 KBytes
[ 4] 3.00-4.00 sec 193 MBytes 1.62 Gbits/sec 0 155 KBytes
[ 4] 4.00-5.00 sec 239 MBytes 2.01 Gbits/sec 0 187 KBytes
[ 4] 5.00-6.00 sec 255 MBytes 2.14 Gbits/sec 0 226 KBytes
[ 4] 6.00-7.00 sec 262 MBytes 2.20 Gbits/sec 0 262 KBytes
[ 4] 7.00-8.00 sec 286 MBytes 2.39 Gbits/sec 0 300 KBytes
[ 4] 8.00-9.00 sec 304 MBytes 2.55 Gbits/sec 0 360 KBytes
[ 4] 9.00-10.00 sec 322 MBytes 2.70 Gbits/sec 0 360 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
pppoe client iperf client - test2 box iperf server - TSO disabled
r620-2# iperf3 -c 192.168.100.219
Connecting to host 192.168.100.219, port 5201
[ 5] local 10.53.253.12 port 20950 connected to 192.168.100.219 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 612 MBytes 5.13 Gbits/sec
[ 5] 1.00-2.00 sec 608 MBytes 5.11 Gbits/sec
[ 5] 2.00-3.00 sec 608 MBytes 5.10 Gbits/sec
[ 5] 3.00-4.00 sec 609 MBytes 5.11 Gbits/sec
[ 5] 4.00-5.00 sec 599 MBytes 5.02 Gbits/sec
[ 5] 5.00-6.00 sec 595 MBytes 4.99 Gbits/sec
[ 5] 6.00-7.00 sec 600 MBytes 5.03 Gbits/sec
[ 5] 7.00-8.00 sec 552 MBytes 4.63 Gbits/sec
[ 5] 8.00-9.00 sec 612 MBytes 5.13 Gbits/sec
[ 5] 9.00-10.00 sec 615 MBytes 5.16 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
pppoe client iperf server - test2 box iperf client - TSO enabled
[root@test2 ~]# iperf3 -c 10.53.253.12
Connecting to host 10.53.253.12, port 5201
[ 4] local 192.168.100.219 port 40496 connected to 10.53.253.12 port 5201
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 40.8 KBytes 334 Kbits/sec 15 2.81 KBytes
[ 4] 1.00-2.00 sec 136 KBytes 1.12 Mbits/sec 37 2.81 KBytes
[ 4] 2.00-3.00 sec 498 KBytes 4.08 Mbits/sec 97 2.81 KBytes
[ 4] 3.00-4.00 sec 658 KBytes 5.39 Mbits/sec 128 4.22 KBytes
[ 4] 4.00-5.00 sec 776 KBytes 6.36 Mbits/sec 148 4.22 KBytes
[ 4] 5.00-6.00 sec 523 KBytes 4.29 Mbits/sec 101 4.22 KBytes
[ 4] 6.00-7.00 sec 1012 KBytes 8.29 Mbits/sec 194 2.81 KBytes
[ 4] 7.00-8.00 sec 582 KBytes 4.77 Mbits/sec 116 4.22 KBytes
[ 4] 8.00-9.00 sec 599 KBytes 4.91 Mbits/sec 121 2.81 KBytes
[ 4] 9.00-10.00 sec 1.07 MBytes 8.99 Mbits/sec 209 4.22 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
pppoe client iperf client - test2 box iperf server - TSO enabled
r620-2# iperf3 -c 192.168.100.219
Connecting to host 192.168.100.219, port 5201
[ 5] local 10.53.253.12 port 36462 connected to 192.168.100.219 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.01 sec 33.8 KBytes 274 Kbits/sec
[ 5] 1.01-2.01 sec 14.1 KBytes 115 Kbits/sec
[ 5] 2.01-3.01 sec 2.81 KBytes 23.0 Kbits/sec
[ 5] 3.01-4.01 sec 9.84 KBytes 80.6 Kbits/sec
[ 5] 4.01-5.01 sec 0.00 Bytes 0.00 bits/sec
[ 5] 5.01-6.00 sec 4.22 KBytes 34.9 Kbits/sec
[ 5] 6.00-7.01 sec 7.03 KBytes 57.0 Kbits/sec
[ 5] 7.01-8.01 sec 0.00 Bytes 0.00 bits/sec
[ 5] 8.01-9.00 sec 4.22 KBytes 34.9 Kbits/sec
[ 5] 9.00-10.01 sec 0.00 Bytes 0.00 bits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
iperf between pppoe clients without TSO
x3550m4# iperf3 -c 10.53.253.13
Connecting to host 10.53.253.13, port 5201
[ 5] local 10.53.253.14 port 23985 connected to 10.53.253.13 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 65.1 MBytes 546 Mbits/sec
[ 5] 1.00-2.00 sec 114 MBytes 953 Mbits/sec
[ 5] 2.00-3.00 sec 182 MBytes 1.53 Gbits/sec
[ 5] 3.00-4.00 sec 185 MBytes 1.55 Gbits/sec
[ 5] 4.00-5.00 sec 209 MBytes 1.76 Gbits/sec
[ 5] 5.00-6.00 sec 233 MBytes 1.95 Gbits/sec
[ 5] 6.00-7.00 sec 261 MBytes 2.19 Gbits/sec
[ 5] 7.00-8.00 sec 298 MBytes 2.50 Gbits/sec
[ 5] 8.00-9.00 sec 330 MBytes 2.77 Gbits/sec
[ 5] 9.00-10.00 sec 350 MBytes 2.94 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
iperf between pppoe clients with TSO on pppoe server
x3550m4# iperf3 -c 10.53.253.13
Connecting to host 10.53.253.13, port 5201
[ 5] local 10.53.253.14 port 26763 connected to 10.53.253.13 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 79.4 MBytes 666 Mbits/sec
[ 5] 1.00-2.00 sec 122 MBytes 1.02 Gbits/sec
[ 5] 2.00-3.00 sec 186 MBytes 1.56 Gbits/sec
[ 5] 3.00-4.00 sec 195 MBytes 1.63 Gbits/sec
[ 5] 4.00-5.00 sec 207 MBytes 1.74 Gbits/sec
[ 5] 5.00-6.00 sec 236 MBytes 1.98 Gbits/sec
[ 5] 6.00-7.00 sec 267 MBytes 2.24 Gbits/sec
[ 5] 7.00-8.00 sec 336 MBytes 2.82 Gbits/sec
[ 5] 8.00-9.00 sec 336 MBytes 2.82 Gbits/sec
[ 5] 9.00-10.00 sec 351 MBytes 2.94 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
