Regarding the BW numbers: they look reasonable. You can try to improve the BW up to 3.2GB/s by increasing PCIe MTU from 128 to 256 byte. It usually requires BIOS configuration changes.
Boris Shpolyansky Sr. Member of Technical Staff, Applications Mellanox Technologies Inc. 350 Oakmead Parkway, Suite 100 Sunnyvale, CA 94085 Tel.: (408) 916 0014 Fax: (408) 585 0314 Cell: (408) 834 9365 www.mellanox.com Mellanox on Twitter and Facebook -----Original Message----- From: linux-rdma-ow...@vger.kernel.org [mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of Jason Gunthorpe Sent: Wednesday, July 14, 2010 2:17 PM To: Tom Ammon Cc: linux-rdma@vger.kernel.org Subject: Re: IB perf test questions On Wed, Jul 14, 2010 at 02:28:15PM -0600, Tom Ammon wrote: > Also, whether I use ib_read_bw or ib_write_bw, the machine I initiate > the test from (in this case "taildrop") shows one of its CPU cores > pegged at 100% for the duration of the test, but I see no CPU > utilization at all on the receiving node. Can someone explain to me > what's going on under the hood, here? I would think that read_bw would > load up the sending host but that write_bw would load up the receiving > host (or maybe vice versa), so this seems counterintuitive to me. when I > use the -b flag to do a bidirectional test, a single CPU core on both > machines pegs at 100%. In all cases the master machine sits in a CPU bound loop waiting for completions so it can issue more RDMA operations. The difference between write and read is simply the RDMA op that is issued. The slave side just sits there and the NIC does all the work. Bi-directional mode runs a master operation on both sides.. Jason -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
