On 9/20/2011 7:25 AM, Reuti wrote:
Hi,
Am 20.09.2011 um 00:41 schrieb Blosch, Edwin L:
I am observing differences in floating-point results from an application
program that appear to be related to whether I link with OpenMPI 1.4.3 or
MVAPICH 1.2.0. Both packages were built with the same installation of Intel
11.1, as well as the application program; identical flags passed to the
compiler in each case.
I’ve tracked down some differences in a compute-only routine where I’ve printed
out the inputs to the routine (to 18 digits) ; the inputs are identical. The
output numbers are different in the 16th place (perhaps a few in the 15th
place). These differences only show up for optimized code, not for –O0.
My assumption is that some optimized math intrinsic is being replaced
dynamically, but I do not know how to confirm this. Anyone have guidance to
offer? Or similar experience?
yes, I face it often but always at a magnitude where it's not of any concern
(and not related to any MPI). Due to the limited precision in computers, a
simple reordering of operation (although being equivalent in a mathematical
sense) can lead to different results. Removing the anomalies with -O0 could
proof that.
The other point I heard especially for the x86 instruction set is, that the
internal FPU has still 80 bits, while the presentation in memory is only 64
bit. Hence when all can be done in the registers, the result can be different
compared to the case when some interim results need to be stored to RAM. For
the Portland compiler there is a switch -Kieee -pc64 to force it to stay always
in 64 bit, and a similar one for Intel is -mp (now -fltconsistency) and -mp1.
Diagnostics below indicate that ifort 11.1 64-bit is in use. The
options aren't the same as Reuti's "now" version (a 32-bit compiler
which hasn't been supported for 3 years or more?).
With ifort 10.1 and more recent, you would set at least
-assume protect_parens -prec-div -prec-sqrt
if you are interested in numerical consistency. If you don't want
auto-vectorization of sum reductions, you would use instead
-fp-model source -ftz
(ftz sets underflow mode back to abrupt, while "source" sets gradual).
It may be possible to expose 80-bit x87 by setting the ancient -mp
option, but such a course can't be recommended without additional cautions.
Quoted comment from OP seem to show a somewhat different question: Does
OpenMPI implement any operations in a different way from MVAPICH? I
would think it probable that the answer could be affirmative for
operations such as allreduce, but this leads well outside my expertise
with respect to specific MPI implementations. It isn't out of the
question to suspect that such differences might be aggravated when using
excessively aggressive ifort options such as -fast.
libifport.so.5 =>
/opt/intel/Compiler/11.1/072/lib/intel64/libifport.so.5 (0x00002b6e7e081000)
libifcoremt.so.5 =>
/opt/intel/Compiler/11.1/072/lib/intel64/libifcoremt.so.5 (0x00002b6e7e1ba000)
libimf.so => /opt/intel/Compiler/11.1/072/lib/intel64/libimf.so
(0x00002b6e7e45f000)
libsvml.so => /opt/intel/Compiler/11.1/072/lib/intel64/libsvml.so
(0x00002b6e7e7f4000)
libintlc.so.5 =>
/opt/intel/Compiler/11.1/072/lib/intel64/libintlc.so.5 (0x00002b6e7ea0a000)
--
Tim Prince
