On Dec 7, 2011, at 8:08 PM, Luigi Rizzo wrote: > Summary: > > - with default interrupt mitigation, the fastest configuration > is with checksums enabled on both sender and receiver, lro > enabled on the receiver. This gets about 8.0 Gbit/s
I do not observe this. With defaults: # nuttcp -t -T 5 -w 128 -v 10.2.101.11 nuttcp-t: v6.1.2: socket nuttcp-t: buflen=65536, nstream=1, port=5001 tcp -> 10.2.101.11 nuttcp-t: time limit = 5.00 seconds nuttcp-t: connect to 10.2.101.11 with mss=1448, RTT=0.053 ms nuttcp-t: send window size = 131768, receive window size = 66608 nuttcp-t: 1857.4978 MB in 5.02 real seconds = 378856.02 KB/sec = 3103.5885 Mbps nuttcp-t: host-retrans = 0 nuttcp-t: 29720 I/O calls, msec/call = 0.17, calls/sec = 5919.63 nuttcp-t: 0.0user 2.5sys 0:05real 52% 115i+1544d 630maxrss 0+2pf 107264+1csw nuttcp-r: v6.1.2: socket nuttcp-r: buflen=65536, nstream=1, port=5001 tcp nuttcp-r: accept from 10.2.101.12 nuttcp-r: send window size = 33304, receive window size = 131768 nuttcp-r: 1857.4978 MB in 5.15 real seconds = 369617.39 KB/sec = 3027.9057 Mbps nuttcp-r: 543991 I/O calls, msec/call = 0.01, calls/sec = 105709.95 nuttcp-r: 0.1user 4.1sys 0:05real 83% 110i+1482d 618maxrss 0+15pf 158432+0csw On receiver: ifconfig ix1 lro # nuttcp -t -T 5 -w 128 -v 10.2.101.11 nuttcp-t: v6.1.2: socket nuttcp-t: buflen=65536, nstream=1, port=5001 tcp -> 10.2.101.11 nuttcp-t: time limit = 5.00 seconds nuttcp-t: connect to 10.2.101.11 with mss=1448, RTT=0.068 ms nuttcp-t: send window size = 131768, receive window size = 66608 nuttcp-t: 1673.3125 MB in 5.02 real seconds = 341312.36 KB/sec = 2796.0308 Mbps nuttcp-t: host-retrans = 1 nuttcp-t: 26773 I/O calls, msec/call = 0.19, calls/sec = 5333.01 nuttcp-t: 0.0user 1.0sys 0:05real 21% 113i+1518d 630maxrss 0+2pf 12772+1csw nuttcp-r: v6.1.2: socket nuttcp-r: buflen=65536, nstream=1, port=5001 tcp nuttcp-r: accept from 10.2.101.12 nuttcp-r: send window size = 33304, receive window size = 131768 nuttcp-r: 1673.3125 MB in 5.15 real seconds = 332975.19 KB/sec = 2727.7327 Mbps nuttcp-r: 106268 I/O calls, msec/call = 0.05, calls/sec = 20650.82 nuttcp-r: 0.0user 1.3sys 0:05real 28% 101i+1354d 618maxrss 0+15pf 64567+0csw On sender: ifconfig ix1 lro (now both receiver and sender have LRO enabled) # nuttcp -t -T 5 -w 128 -v 10.2.101.11 nuttcp-t: v6.1.2: socket nuttcp-t: buflen=65536, nstream=1, port=5001 tcp -> 10.2.101.11 nuttcp-t: time limit = 5.00 seconds nuttcp-t: connect to 10.2.101.11 with mss=1448, RTT=0.063 ms nuttcp-t: send window size = 131768, receive window size = 66608 nuttcp-t: 1611.7805 MB in 5.02 real seconds = 328716.18 KB/sec = 2692.8430 Mbps nuttcp-t: host-retrans = 1 nuttcp-t: 25789 I/O calls, msec/call = 0.20, calls/sec = 5136.29 nuttcp-t: 0.0user 1.0sys 0:05real 21% 109i+1465d 630maxrss 0+2pf 12697+0csw nuttcp-r: v6.1.2: socket nuttcp-r: buflen=65536, nstream=1, port=5001 tcp nuttcp-r: accept from 10.2.101.12 nuttcp-r: send window size = 33304, receive window size = 131768 nuttcp-r: 1611.7805 MB in 5.15 real seconds = 320694.82 KB/sec = 2627.1319 Mbps nuttcp-r: 104346 I/O calls, msec/call = 0.05, calls/sec = 20275.05 nuttcp-r: 0.0user 1.3sys 0:05real 27% 113i+1516d 618maxrss 0+15pf 63510+0csw remove LRO from receiver (only sender has LRO): # nuttcp -t -T 5 -w 128 -v 10.2.101.11 nuttcp-t: v6.1.2: socket nuttcp-t: buflen=65536, nstream=1, port=5001 tcp -> 10.2.101.11 nuttcp-t: time limit = 5.00 seconds nuttcp-t: connect to 10.2.101.11 with mss=1448, RTT=0.065 ms nuttcp-t: send window size = 131768, receive window size = 66608 nuttcp-t: 1884.8702 MB in 5.02 real seconds = 384464.57 KB/sec = 3149.5338 Mbps nuttcp-t: host-retrans = 0 nuttcp-t: 30158 I/O calls, msec/call = 0.17, calls/sec = 6007.27 nuttcp-t: 0.0user 2.7sys 0:05real 55% 104i+1403d 630maxrss 0+2pf 106046+0csw nuttcp-r: v6.1.2: socket nuttcp-r: buflen=65536, nstream=1, port=5001 tcp nuttcp-r: accept from 10.2.101.12 nuttcp-r: send window size = 33304, receive window size = 131768 nuttcp-r: 1884.8702 MB in 5.15 real seconds = 375093.52 KB/sec = 3072.7661 Mbps nuttcp-r: 540237 I/O calls, msec/call = 0.01, calls/sec = 104988.68 nuttcp-r: 0.1user 4.2sys 0:05real 84% 110i+1483d 618maxrss 0+15pf 156340+0csw Strange enough, setting hw.ixgbe.max_interrupt_rate=0 does not have any effect.. Daniel _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
