Moshe Kazir wrote:
While running netperf and iperf over SDP I get unstable Performance
results.
On iperf I get more then 25 % difference between minimum and maximum.
On netperf I get the following amazing result. ->
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
# while LD_PRELOAD=/usr/lib64/libsdp.so ./netperf -H 192.168.7.172
-- -m 512 -M 1047 ; do echo . ; done
TCP STREAM TEST to 192.168.7.172
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec
126976 126976 512 10.00 3247.39
.
TCP STREAM TEST to 192.168.7.172
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec
126976 126976 512 10.00 1222.48
.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Can some one spar me a hint ?
What I'm doing wrong ?
I would start by telling netperf you want CPU utilization measured. That way we
can see if there is a correlation between the CPU utilization and the throughput.
Also, if this were a "pure" TCP test, you would have a race between the nagle
algorithm and the speed at which ACK's come-back from the receiver affecting the
distribution of TCP segment sizes being transmitted since your send size is so
much smaller than the MTU of the link. IIRC for IPoIB in 1.2.mumble the MTU is
65520 or something like that. You might consider taking snapshots of the
link-level statistics (does ethtool -S work for an IB interface?) from before
and after each netperf test and run them through beforeafter:
ftp://ftp.cup.hp.com/dist/networking/tools/
You might also experiment with setting TCP_NODELAY - although since this is
LD_PRELOADED SDP I'm not sure what that really means/does.
Any particular reason you are telling the netperf side to post 1047 byte
receives when you are making 512 byte calls to send()?
The test run on x86_64 , sles 10 sp1 , OFED-1.2.5
I'm guessing you have multiple cores - how do interrutps from the HCA (?) get
distributed? What happens when you use the -T option of netperf to vary the CPU
binding of either netperf or netserver:
netperf -T N,M #bind netperf to CPU N, netserver to CPU M
netperf -T N, #just bind netperf to CPU N, netserver unbound
netperf -T ,M #netperf unbound, netserver bound to CPU M
relative to where the interrupts from the HCA go?
Finally, well for now :), there are "direct" SDP tests in netperf. Make sure
you are on say 2.4.4:
http://www.netperf.org/svn/netperf2/tags/netperf-2.4.4/ or
ftp://ftp.netperf.org/netperf
and add --enable-sdb to the ./configure command.
happy benchmarking,
rick jones
_______________________________________________
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
