Interestingly enough on the SPARC platform the Solaris memcpy's actually
use non-temporal stores for copies >= 64KB. By default some of the mca
parameters to the sm BTL stop at 32KB. I've done experimentations of
bumping the sm segment sizes to above 64K and seen incredible speedup on
our M9000 platforms. I am looking for some nice way to integrate a
memcpy that lowers this boundary to 32KB or lower into Open MPI.
I have not looked into whether Solaris x86/x64 memcpy's use the
non-temporal stores or not.
--td
Message: 1
Date: Thu, 14 Aug 2008 09:28:59 -0400
From: Jeff Squyres <jsquy...@cisco.com>
Subject: Re: [OMPI users] SM btl slows down bandwidth?
To: rbbr...@sandia.gov, Open MPI Users <us...@open-mpi.org>
Message-ID: <562557eb-857c-4ca8-97ad-f294c7fed...@cisco.com>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
At this time, we are not using non-temporal stores for shared memory
operations.
On Aug 13, 2008, at 11:46 AM, Ron Brightwell wrote:
>> [...]
>>
>> MPICH2 manages to get about 5GB/s in shared memory performance on the
>> Xeon 5420 system.
>
> Does the sm btl use a memcpy with non-temporal stores like MPICH2?
> This can be a big win for bandwidth benchmarks that don't actually
> touch their receive buffers at all...
>
> -Ron
>
>
> _______________________________________________
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users
-- Jeff Squyres Cisco Systems
