Hi, 64K is borderline for seeing bzcopy effect. Using an AMD 6000+ (3 Ghz dual core) in Asus M2A-VM motherboard with ConnectX running 2.3 firmware and OFED 1.3-rc3 stack running on 2.6.23.8 kernel.org kernel, I ran the test for 128K: 5546 sdp_zcopy_thresh=0 (off) 8709 sdp_zcopy_thresh=65536
For these tests, I just have LD_PRELOAD set in my environment. ======================= I see that TCP_MAXSEG is not being handled by libsdp and will look into it. [EMAIL PROTECTED] ~]# modprobe ib_sdp [EMAIL PROTECTED] ~]# netperf -v2 -4 -H 193.168.10.198 -l 30 -t TCP_STREAM -c -C -- -m 128K TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 193.168.10.198 (193.168.10.198) port 0 AF_INET netperf: get_tcp_info: getsockopt TCP_MAXSEG: errno 92 Recv Send Send Utilization Service Demand Socket Socket Message Elapsed Send Recv Send Recv Size Size Size Time Throughput local remote local remote bytes bytes bytes secs. 10^6bits/s % S % S us/KB us/KB 87380 16384 131072 30.01 5545.69 51.47 14.43 1.521 1.706 Alignment Offset Bytes Bytes Sends Bytes Recvs Local Remote Local Remote Xfered Per Per Send Recv Send Recv Send (avg) Recv (avg) 8 8 0 0 2.08e+10 131072.00 158690 33135.60 627718 Maximum Segment Size (bytes) -1 [EMAIL PROTECTED] ~]# echo 65536 >/sys/module/ib_sdp/parameters/sdp_zcopy_thresh [EMAIL PROTECTED] ~]# netperf -v2 -4 -H 193.168.10.198 -l 30 -t TCP_STREAM -c -C -- -m 128K TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 193.168.10.198 (193.168.10.198) port 0 AF_INET netperf: get_tcp_info: getsockopt TCP_MAXSEG: errno 92 Recv Send Send Utilization Service Demand Socket Socket Message Elapsed Send Recv Send Recv Size Size Size Time Throughput local remote local remote bytes bytes bytes secs. 10^6bits/s % S % S us/KB us/KB 87380 16384 131072 30.01 8708.58 50.63 14.55 0.953 1.095 Alignment Offset Bytes Bytes Sends Bytes Recvs Local Remote Local Remote Xfered Per Per Send Recv Send Recv Send (avg) Recv (avg) 8 8 0 0 3.267e+10 131072.00 249228 26348.30 1239807 Maximum Segment Size (bytes) -1 Thanks, JIm Jim Mott Mellanox Technologies Ltd. mail: [EMAIL PROTECTED] Phone: 512-294-5481 -----Original Message----- From: Weikuan Yu [mailto:[EMAIL PROTECTED] Sent: Thursday, January 24, 2008 9:09 AM To: Scott Weitzenkamp (sweitzen) Cc: Jim Mott; ewg@lists.openfabrics.org; [EMAIL PROTECTED] Subject: Re: [ofa-general] RE: [ewg] Not seeing any SDP performance changes inOFED 1.3 beta, and I get Oops when enabling sdp_zcopy_thresh Hi, Scott, I have been running SDP tests across two woodcrest nodes with 4x DDR cards using OFED-1.2.5.4. The card/firmware info is below. CA 'mthca0' CA type: MT25208 Number of ports: 2 Firmware version: 5.1.400 Hardware version: a0 Node GUID: 0x0002c90200228e0c System image GUID: 0x0002c90200228e0f I could not get a bandwidth more than 5Gbps like you have shown here. Wonder if I need to upgrade to the latest software or firmware? Any suggestions? Thanks, --Weikuan TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.225.77 (192.168 .225.77) port 0 AF_INET Recv Send Send Utilization Service Demand Socket Socket Message Elapsed Send Recv Send Recv Size Size Size Time Throughput local remote local remote bytes bytes bytes secs. 10^6bits/s % S % S us/KB us/KB 131072 131072 131072 10.00 4918.95 21.29 24.99 1.418 1.665 Scott Weitzenkamp (sweitzen) wrote: > Jim, > > I am trying OFED-1.3-20071231-0600 and RHEL4 x86_64 on a dual CPU > (single core each CPU) Xeon system. I do not see any performance > improvement (either throughput or CPU utilization) using netperf when I > set /sys/module/ib_sdp/sdp_zcopy_thresh to 16384. Can you elaborate on > your HCA type, and performance improvement you see? > > Here's an example netperf command line when using a Cheetah DDR HCA and > 1.2.917 firmware (I have also tried ConnectX and 2.3.000 firmware too): > > [EMAIL PROTECTED] ~]$ LD_PRELOAD=libsdp.so netperf241 -v2 -4 -H > 192.168.1.201 -l 30 -t TCP_STREAM -c -C -- -m 65536 > TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.1.201 > (192.168.1.201) port 0 AF_INET : histogram : demo > > Recv Send Send Utilization Service > Demand > Socket Socket Message Elapsed Send Recv Send > Recv > Size Size Size Time Throughput local remote local > remote > bytes bytes bytes secs. 10^6bits/s % S % S us/KB > us/KB > > 87380 16384 65536 30.01 7267.70 55.06 61.27 1.241 > 1.381 > > Alignment Offset Bytes Bytes Sends Bytes > Recvs > Local Remote Local Remote Xfered Per Per > Send Recv Send Recv Send (avg) Recv (avg) > 8 8 0 0 2.726e+10 65536.00 415942 48106.01 > 566648 > _______________________________________________ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg