Set the storm threshold to 0, that will disable it, its going to throttle your performance when it happens.
Jack On Tue, Dec 6, 2011 at 6:24 AM, Daniel Kalchev <dan...@digsys.bg> wrote: > Some tests with updated FreeBSD to 8-stable as of today, compared with the > previous run > > > > On 06.12.11 13:18, Daniel Kalchev wrote: > >> >> FreeBSD 8.2-STABLE #0: Wed Sep 28 11:23:59 EEST 2011 >> CPU: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz (2403.58-MHz >> K8-class CPU) >> real memory = 51539607552 (49152 MB) >> blade 1: >> >> # nuttcp -S >> # nuttcp -t -T 5 -w 128 -v localhost >> nuttcp-t: v6.1.2: socket >> nuttcp-t: buflen=65536, nstream=1, port=5001 tcp -> localhost >> nuttcp-t: time limit = 5.00 seconds >> nuttcp-t: connect to 127.0.0.1 with mss=14336, RTT=0.044 ms >> nuttcp-t: send window size = 143360, receive window size = 71680 >> nuttcp-t: 8959.8750 MB in 5.02 real seconds = 1827635.67 KB/sec = >> 14971.9914 Mbps >> nuttcp-t: host-retrans = 0 >> nuttcp-t: 143358 I/O calls, msec/call = 0.04, calls/sec = 28556.81 >> nuttcp-t: 0.0user 4.9sys 0:05real 99% 106i+1428d 602maxrss 0+5pf 16+46csw >> >> nuttcp-r: v6.1.2: socket >> nuttcp-r: buflen=65536, nstream=1, port=5001 tcp >> nuttcp-r: accept from 127.0.0.1 >> nuttcp-r: send window size = 43008, receive window size = 143360 >> nuttcp-r: 8959.8750 MB in 5.17 real seconds = 1773171.07 KB/sec = >> 14525.8174 Mbps >> nuttcp-r: 219708 I/O calls, msec/call = 0.02, calls/sec = 42461.43 >> nuttcp-r: 0.0user 3.8sys 0:05real 76% 105i+1407d 614maxrss 1+17pf >> 95059+22csw >> > > New results: > > FreeBSD 8.2-STABLE #1: Tue Dec 6 13:51:01 EET 2011 > > > > # nuttcp -t -T 5 -w 128 -v localhost > nuttcp-t: v6.1.2: socket > nuttcp-t: buflen=65536, nstream=1, port=5001 tcp -> localhost > nuttcp-t: time limit = 5.00 seconds > nuttcp-t: connect to 127.0.0.1 with mss=14336, RTT=0.030 ms > > nuttcp-t: send window size = 143360, receive window size = 71680 > nuttcp-t: 12748.0625 MB in 5.02 real seconds = 2599947.38 KB/sec = > 21298.7689 Mbps > nuttcp-t: host-retrans = 0 > nuttcp-t: 203969 I/O calls, msec/call = 0.03, calls/sec = 40624.18 > nuttcp-t: 0.0user 4.9sys 0:05real 99% 106i+1428d 620maxrss 0+2pf 1+82csw > > > nuttcp-r: v6.1.2: socket > nuttcp-r: buflen=65536, nstream=1, port=5001 tcp > nuttcp-r: accept from 127.0.0.1 > nuttcp-r: send window size = 43008, receive window size = 143360 > nuttcp-r: 12748.0625 MB in 5.15 real seconds = 2536511.81 KB/sec = > 20779.1048 Mbps > nuttcp-r: 297000 I/O calls, msec/call = 0.02, calls/sec = 57709.75 > nuttcp-r: 0.1user 4.0sys 0:05real 81% 109i+1469d 626maxrss 0+15pf > 121136+34csw > > Noticeable improvement. > > > > >> blade 2: >> >> # nuttcp -t -T 5 -w 128 -v 10.2.101.12 >> nuttcp-t: v6.1.2: socket >> nuttcp-t: buflen=65536, nstream=1, port=5001 tcp -> 10.2.101.12 >> nuttcp-t: time limit = 5.00 seconds >> nuttcp-t: connect to 10.2.101.12 with mss=1448, RTT=0.059 ms >> nuttcp-t: send window size = 131768, receive window size = 66608 >> nuttcp-t: 1340.6469 MB in 5.02 real seconds = 273449.90 KB/sec = >> 2240.1016 Mbps >> nuttcp-t: host-retrans = 171 >> nuttcp-t: 21451 I/O calls, msec/call = 0.24, calls/sec = 4272.78 >> nuttcp-t: 0.0user 1.9sys 0:05real 39% 120i+1610d 600maxrss 2+3pf >> 75658+0csw >> >> nuttcp-r: v6.1.2: socket >> nuttcp-r: buflen=65536, nstream=1, port=5001 tcp >> nuttcp-r: accept from 10.2.101.11 >> nuttcp-r: send window size = 33304, receive window size = 131768 >> nuttcp-r: 1340.6469 MB in 5.17 real seconds = 265292.92 KB/sec = >> 2173.2796 Mbps >> nuttcp-r: 408764 I/O calls, msec/call = 0.01, calls/sec = 78992.15 >> nuttcp-r: 0.0user 3.3sys 0:05real 64% 105i+1413d 620maxrss 0+15pf >> 105104+102csw >> > > # nuttcp -t -T 5 -w 128 -v 10.2.101.11 > nuttcp-t: v6.1.2: socket > nuttcp-t: buflen=65536, nstream=1, port=5001 tcp -> 10.2.101.11 > > nuttcp-t: time limit = 5.00 seconds > nuttcp-t: connect to 10.2.101.11 with mss=1448, RTT=0.055 ms > > nuttcp-t: send window size = 131768, receive window size = 66608 > nuttcp-t: 1964.8640 MB in 5.02 real seconds = 400757.59 KB/sec = 3283.0062 > Mbps > nuttcp-t: host-retrans = 0 > nuttcp-t: 31438 I/O calls, msec/call = 0.16, calls/sec = 6261.87 > nuttcp-t: 0.0user 2.7sys 0:05real 55% 112i+1501d 1124maxrss 1+2pf > 111165+112csw > > > nuttcp-r: v6.1.2: socket > nuttcp-r: buflen=65536, nstream=1, port=5001 tcp > nuttcp-r: accept from 10.2.101.12 > > nuttcp-r: send window size = 33304, receive window size = 131768 > nuttcp-r: 1964.8640 MB in 5.15 real seconds = 390972.20 KB/sec = 3202.8442 > Mbps > nuttcp-r: 560718 I/O calls, msec/call = 0.01, calls/sec = 108957.70 > nuttcp-r: 0.1user 4.2sys 0:05real 84% 111i+1494d 626maxrss 0+15pf > 151930+16csw > > Again, improvement. > > > >> >> Another pari of blades: >> >> FreeBSD 8.2-STABLE #0: Tue Aug 9 12:37:55 EEST 2011 >> CPU: AMD Opteron(tm) Processor 6134 (2300.04-MHz K8-class CPU) >> real memory = 68719476736 (65536 MB) >> >> first blade: >> >> # nuttcp -S >> # nuttcp -t -T 5 -w 128 -v localhost >> nuttcp-t: v6.1.2: socket >> nuttcp-t: buflen=65536, nstream=1, port=5001 tcp -> localhost >> nuttcp-t: time limit = 5.00 seconds >> nuttcp-t: connect to 127.0.0.1 with mss=14336, RTT=0.090 ms >> nuttcp-t: send window size = 143360, receive window size = 71680 >> nuttcp-t: 2695.0625 MB in 5.00 real seconds = 551756.90 KB/sec = >> 4519.9925 Mbps >> nuttcp-t: host-retrans = 0 >> nuttcp-t: 43121 I/O calls, msec/call = 0.12, calls/sec = 8621.20 >> nuttcp-t: 0.0user 4.9sys 0:05real 99% 106i+1428d 620maxrss 0+4pf 2+71csw >> >> nuttcp-r: v6.1.2: socket >> nuttcp-r: buflen=65536, nstream=1, port=5001 tcp >> nuttcp-r: accept from 127.0.0.1 >> nuttcp-r: send window size = 43008, receive window size = 143360 >> nuttcp-r: 2695.0625 MB in 5.14 real seconds = 536509.66 KB/sec = >> 4395.0871 Mbps >> nuttcp-r: 43126 I/O calls, msec/call = 0.12, calls/sec = 8383.94 >> nuttcp-r: 0.0user 3.1sys 0:05real 61% 94i+1264d 624maxrss 1+16pf >> 43019+0csw >> > > New result: > > FreeBSD 8.2-STABLE #1: Tue Dec 6 15:06:22 EET 2011 > > > > # nuttcp -t -T 5 -w 128 -v localhost > nuttcp-t: v6.1.2: socket > nuttcp-t: buflen=65536, nstream=1, port=5001 tcp -> localhost > nuttcp-t: time limit = 5.00 seconds > nuttcp-t: connect to 127.0.0.1 with mss=14336, RTT=0.094 ms > > nuttcp-t: send window size = 143360, receive window size = 71680 > nuttcp-t: 2659.1250 MB in 5.00 real seconds = 544493.51 KB/sec = 4460.4909 > Mbps > nuttcp-t: host-retrans = 0 > nuttcp-t: 42546 I/O calls, msec/call = 0.12, calls/sec = 8507.71 > nuttcp-t: 0.0user 4.9sys 0:05real 99% 106i+1428d 614maxrss 0+3pf 1+71csw > > > nuttcp-r: v6.1.2: socket > nuttcp-r: buflen=65536, nstream=1, port=5001 tcp > nuttcp-r: accept from 127.0.0.1 > nuttcp-r: send window size = 43008, receive window size = 143360 > nuttcp-r: 2659.1250 MB in 5.14 real seconds = 529484.10 KB/sec = 4337.5338 > Mbps > nuttcp-r: 42560 I/O calls, msec/call = 0.12, calls/sec = 8275.91 > nuttcp-r: 0.0user 3.1sys 0:05real 62% 103i+1391d 618maxrss 0+16pf > 42454+0csw > > Same result.. something is wrong with this Opteron system. > > > >> second blade: >> >> # nuttcp -t -T 5 -w 128 -v 10.2.101.13 >> nuttcp-t: v6.1.2: socket >> nuttcp-t: buflen=65536, nstream=1, port=5001 tcp -> 10.2.101.13 >> nuttcp-t: time limit = 5.00 seconds >> nuttcp-t: connect to 10.2.101.13 with mss=1448, RTT=0.164 ms >> nuttcp-t: send window size = 131768, receive window size = 66608 >> nuttcp-t: 1290.3750 MB in 5.00 real seconds = 264173.96 KB/sec = >> 2164.1131 Mbps >> nuttcp-t: host-retrans = 0 >> nuttcp-t: 20646 I/O calls, msec/call = 0.25, calls/sec = 4127.72 >> nuttcp-t: 0.0user 3.8sys 0:05real 77% 96i+1299d 616maxrss 0+3pf 27389+0csw >> >> nuttcp-r: v6.1.2: socket >> nuttcp-r: buflen=65536, nstream=1, port=5001 tcp >> nuttcp-r: accept from 10.2.101.14 >> nuttcp-r: send window size = 33304, receive window size = 131768 >> nuttcp-r: 1290.3750 MB in 5.14 real seconds = 256835.92 KB/sec = >> 2103.9998 Mbps >> nuttcp-r: 85668 I/O calls, msec/call = 0.06, calls/sec = 16651.70 >> nuttcp-r: 0.0user 4.8sys 0:05real 94% 107i+1437d 624maxrss 0+15pf >> 11848+0csw >> >> > # nuttcp -t -T 5 -w 128 -v 10.2.101.13 > nuttcp-t: v6.1.2: socket > nuttcp-t: buflen=65536, nstream=1, port=5001 tcp -> 10.2.101.13 > nuttcp-t: time limit = 5.00 seconds > nuttcp-t: connect to 10.2.101.13 with mss=1448, RTT=0.130 ms > > nuttcp-t: send window size = 131768, receive window size = 66608 > nuttcp-t: 1317.6875 MB in 5.00 real seconds = 269767.01 KB/sec = 2209.9313 > Mbps > nuttcp-t: host-retrans = 0 > nuttcp-t: 21083 I/O calls, msec/call = 0.24, calls/sec = 4215.11 > nuttcp-t: 0.0user 3.7sys 0:05real 75% 105i+1409d 616maxrss 0+3pf > 39629+11csw > > > nuttcp-r: v6.1.2: socket > nuttcp-r: buflen=65536, nstream=1, port=5001 tcp > nuttcp-r: accept from 10.2.101.14 > nuttcp-r: send window size = 33304, receive window size = 131768 > nuttcp-r: 1317.6875 MB in 5.14 real seconds = 262306.61 KB/sec = 2148.8157 > Mbps > nuttcp-r: 85980 I/O calls, msec/call = 0.06, calls/sec = 16714.53 > nuttcp-r: 0.0user 4.8sys 0:05real 94% 107i+1446d 618maxrss 0+15pf > 10783+0csw > > Same,but.. again interrupt storm: > > interrupt storm detected on "irq269:"; throttling interrupt source > > Also interesting, Intel (server) - AMD (client) > > # nuttcp -t -T 5 -w 128 -v 10.2.101.11 > nuttcp-t: v6.1.2: socket > nuttcp-t: buflen=65536, nstream=1, port=5001 tcp -> 10.2.101.11 > > nuttcp-t: time limit = 5.00 seconds > nuttcp-t: connect to 10.2.101.11 with mss=1448, RTT=0.087 ms > > nuttcp-t: send window size = 131768, receive window size = 66608 > nuttcp-t: 1301.1125 MB in 5.00 real seconds = 266388.35 KB/sec = 2182.2534 > Mbps > nuttcp-t: host-retrans = 1 > nuttcp-t: 20818 I/O calls, msec/call = 0.25, calls/sec = 4162.36 > nuttcp-t: 0.0user 4.3sys 0:05real 87% 106i+1430d 612maxrss 0+3pf > 88509+11csw > > > nuttcp-r: v6.1.2: socket > nuttcp-r: buflen=65536, nstream=1, port=5001 tcp > nuttcp-r: accept from 10.2.101.13 > > nuttcp-r: send window size = 33304, receive window size = 131768 > nuttcp-r: 1301.1125 MB in 5.14 real seconds = 259062.19 KB/sec = 2122.2374 > Mbps > nuttcp-r: 537176 I/O calls, msec/call = 0.01, calls/sec = 104449.37 > nuttcp-r: 0.0user 3.2sys 0:05real 64% 122i+1643d 626maxrss 0+15pf > 222593+0csw > > AMD (server) - Intel (client) > > # nuttcp -t -T 5 -w 128 -v 10.2.101.14 > nuttcp-t: v6.1.2: socket > nuttcp-t: buflen=65536, nstream=1, port=5001 tcp -> 10.2.101.14 > > nuttcp-t: time limit = 5.00 seconds > nuttcp-t: connect to 10.2.101.14 with mss=1448, RTT=0.099 ms > > nuttcp-t: send window size = 131768, receive window size = 66608 > nuttcp-t: 750.8897 MB in 5.02 real seconds = 153161.09 KB/sec = 1254.6957 > Mbps > nuttcp-t: host-retrans = 25 > nuttcp-t: 12015 I/O calls, msec/call = 0.43, calls/sec = 2393.29 > nuttcp-t: 0.0user 0.9sys 0:05real 19% 112i+1507d 618maxrss 0+2pf 97668+4csw > > > nuttcp-r: v6.1.2: socket > nuttcp-r: buflen=65536, nstream=1, port=5001 tcp > nuttcp-r: accept from 10.2.101.11 > nuttcp-r: send window size = 33304, receive window size = 131768 > nuttcp-r: 750.8897 MB in 5.15 real seconds = 149424.38 KB/sec = 1224.0845 > Mbps > nuttcp-r: 213009 I/O calls, msec/call = 0.02, calls/sec = 41394.56 > nuttcp-r: 0.0user 4.6sys 0:05real 91% 107i+1445d 618maxrss 0+16pf > 18273+0csw > > It seems performance measurements are more dependent on the server (nuttcp > -S) machine. > We will have to rule out the interrupt storms first of course, any advice? > > > Daniel > > ______________________________**_________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/**mailman/listinfo/freebsd-**current<http://lists.freebsd.org/mailman/listinfo/freebsd-current> > To unsubscribe, send any mail to "freebsd-current-unsubscribe@** > freebsd.org <freebsd-current-unsubscr...@freebsd.org>" > _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
