I'm building Open MPI 2.1.0 with PGI 17.3, and now I'm getting
'illegal instruction' errors during 'make check':
../../config/test-driver: line 107: 65169 Illegal instruction
"$@" > $log_file 2>&1
FAIL: atomic_math
- 1 threads: Passed
That's just one example of the error output. See all relevant
error output below.
Usually, I see these errors when trying to run an executable on
a processor that doesn't support the instruction set of the
executable. I used to see this all the time when I supported an
IBM Blue Gene/P system. I don't think I've ever seen it on an
x86 system.
I'm passing the argument '-tp=x64' to pgcc to build a unified
binary, so that might be part of the problem, but I've used this
exact same process to build 2.1.0 with PGI 16.5 just a couple
hours ago. I also built 1.10.3 with the same compiler flags with
PGI 16.5 and 17.3 without this error.
Any ideas?
The relevant output from 'make check':
make check-TESTS
make[3]: Entering directory `/local/pbisbal/openmpi-2.1.0/test/asm'
make[4]: Entering directory `/local/pbisbal/openmpi-2.1.0/test/asm'
basename: extra operand `--test-name'
Try `basename --help' for more information.
--> Testing
PASS: atomic_barrier
- 1 threads: Passed
PASS: atomic_barrier
- 2 threads: Passed
PASS: atomic_barrier
- 4 threads: Passed
PASS: atomic_barrier
- 5 threads: Passed
PASS: atomic_barrier
- 8 threads: Passed
basename: extra operand `--test-name'
Try `basename --help' for more information.
--> Testing
PASS: atomic_barrier_noinline
- 1 threads: Passed
PASS: atomic_barrier_noinline
- 2 threads: Passed
PASS: atomic_barrier_noinline
- 4 threads: Passed
PASS: atomic_barrier_noinline
- 5 threads: Passed
PASS: atomic_barrier_noinline
- 8 threads: Passed
basename: extra operand `--test-name'
Try `basename --help' for more information.
--> Testing
PASS: atomic_spinlock
- 1 threads: Passed
PASS: atomic_spinlock
- 2 threads: Passed
PASS: atomic_spinlock
- 4 threads: Passed
PASS: atomic_spinlock
- 5 threads: Passed
PASS: atomic_spinlock
- 8 threads: Passed
basename: extra operand `--test-name'
Try `basename --help' for more information.
--> Testing
PASS: atomic_spinlock_noinline
- 1 threads: Passed
PASS: atomic_spinlock_noinline
- 2 threads: Passed
PASS: atomic_spinlock_noinline
- 4 threads: Passed
PASS: atomic_spinlock_noinline
- 5 threads: Passed
PASS: atomic_spinlock_noinline
- 8 threads: Passed
basename: extra operand `--test-name'
Try `basename --help' for more information.
--> Testing
../../config/test-driver: line 107: 65169 Illegal instruction
"$@" > $log_file 2>&1
FAIL: atomic_math
- 1 threads: Passed
../../config/test-driver: line 107: 65172 Illegal instruction
"$@" > $log_file 2>&1
FAIL: atomic_math
- 2 threads: Passed
../../config/test-driver: line 107: 65176 Illegal instruction
"$@" > $log_file 2>&1
FAIL: atomic_math
- 4 threads: Passed
../../config/test-driver: line 107: 65180 Illegal instruction
"$@" > $log_file 2>&1
FAIL: atomic_math
- 5 threads: Passed
../../config/test-driver: line 107: 65185 Illegal instruction
"$@" > $log_file 2>&1
FAIL: atomic_math
- 8 threads: Passed
basename: extra operand `--test-name'
Try `basename --help' for more information.
--> Testing
../../config/test-driver: line 107: 65195 Illegal instruction
"$@" > $log_file 2>&1
FAIL: atomic_math_noinline
- 1 threads: Passed
../../config/test-driver: line 107: 65198 Illegal instruction
"$@" > $log_file 2>&1
FAIL: atomic_math_noinline
- 2 threads: Passed
../../config/test-driver: line 107: 65202 Illegal instruction
"$@" > $log_file 2>&1
FAIL: atomic_math_noinline
- 4 threads: Passed
../../config/test-driver: line 107: 65206 Illegal instruction
"$@" > $log_file 2>&1
FAIL: atomic_math_noinline
- 5 threads: Passed
../../config/test-driver: line 107: 65210 Illegal instruction
"$@" > $log_file 2>&1
FAIL: atomic_math_noinline
- 8 threads: Passed
basename: extra operand `--test-name'
Try `basename --help' for more information.
--> Testing
../../config/test-driver: line 107: 65220 Illegal instruction
"$@" > $log_file 2>&1
FAIL: atomic_cmpset
- 1 threads: Passed
../../config/test-driver: line 107: 65223 Illegal instruction
"$@" > $log_file 2>&1
FAIL: atomic_cmpset
- 2 threads: Passed
../../config/test-driver: line 107: 65227 Illegal instruction
"$@" > $log_file 2>&1
FAIL: atomic_cmpset
- 4 threads: Passed
../../config/test-driver: line 107: 65231 Illegal instruction
"$@" > $log_file 2>&1
FAIL: atomic_cmpset
- 5 threads: Passed
../../config/test-driver: line 107: 65235 Illegal instruction
"$@" > $log_file 2>&1
FAIL: atomic_cmpset
- 8 threads: Passed
basename: extra operand `--test-name'
Try `basename --help' for more information.
--> Testing
../../config/test-driver: line 107: 65245 Illegal instruction
"$@" > $log_file 2>&1
FAIL: atomic_cmpset_noinline
- 1 threads: Passed
../../config/test-driver: line 107: 65248 Illegal instruction
"$@" > $log_file 2>&1
FAIL: atomic_cmpset_noinline
- 2 threads: Passed
../../config/test-driver: line 107: 65252 Illegal instruction
"$@" > $log_file 2>&1
FAIL: atomic_cmpset_noinline
- 4 threads: Passed
../../config/test-driver: line 107: 65256 Illegal instruction
"$@" > $log_file 2>&1
FAIL: atomic_cmpset_noinline
- 5 threads: Passed
../../config/test-driver: line 107: 65260 Illegal instruction
"$@" > $log_file 2>&1
FAIL: atomic_cmpset_noinline
- 8 threads: Passed
============================================================================
Testsuite summary for Open MPI 2.1.0
============================================================================
# TOTAL: 8
# PASS: 4
# SKIP: 0
# XFAIL: 0
# FAIL: 4
# XPASS: 0
# ERROR: 0
============================================================================
See test/asm/test-suite.log
Please report to http://www.open-mpi.org/community/help/
============================================================================
make[4]: *** [test-suite.log] Error 1
make[4]: Leaving directory `/local/pbisbal/openmpi-2.1.0/test/asm'
make[3]: *** [check-TESTS] Error 2
make[3]: Leaving directory `/local/pbisbal/openmpi-2.1.0/test/asm'
make[2]: *** [check-am] Error 2
make[2]: Leaving directory `/local/pbisbal/openmpi-2.1.0/test/asm'
make[1]: *** [check-recursive] Error 1
make[1]: Leaving directory `/local/pbisbal/openmpi-2.1.0/test'
make: *** [check-recursive] Error 1
The contents of test/asm/test-suite.log:
$ cat test/asm/test-suite.log
=============================================
Open MPI 2.1.0: test/asm/test-suite.log
=============================================
# TOTAL: 8
# PASS: 4
# SKIP: 0
# XFAIL: 0
# FAIL: 4
# XPASS: 0
# ERROR: 0
.. contents:: :depth: 2
FAIL: atomic_math
=================
FAIL atomic_math (exit status: 132)
FAIL: atomic_math_noinline
==========================
FAIL atomic_math_noinline (exit status: 132)
FAIL: atomic_cmpset
===================
FAIL atomic_cmpset (exit status: 132)
FAIL: atomic_cmpset_noinline
============================
FAIL atomic_cmpset_noinline (exit status: 132)
And my configure command:
./configure \
--prefix=/usr/pppl/pgi/17.3-pkgs/openmpi-2.1.0 \
--disable-silent-rules \
--enable-mpi-fortran \
--enable-mpi-cxx \
--enable-shared \
--enable-static \
--enable-mpi-thread-multiple \
--with-cuda=/usr/pppl/cuda/cudatoolkit/6.5.14 \
--with-pmix \
--with-verbs \
--with-hwloc \
--with-pmi=/usr/pppl/slurm/15.08.8 \
--with-slurm \
--with-psm \
CPP=cpp \
CC=pgcc \
CFLAGS="-fpic -tp=x64 -fast" \
CXX=pgc++ \
CXXFLAGS="-fpic -tp=x64 -fast" \
FC=pgfortran \
FCFLAGS="-fpic -tp=x64 -fast" \
2>&1 | tee configure.log
