I tried a few builds with: '--with-64-bit-indices=1', '--with-memalign=64', '--with-blaslapack-dir=/home/intel/18/compilers_and_libraries_2018.0.128/linux/mkl', '--with-cc=icc', '--with-fc=ifort', '--with-cxx=0', '--with-debugging=0', '--with-mpi=0',
And then changed the OPTFLAGS: 1. 'basic -g' - works fine 'COPTFLAGS=-g', 'FOPTFLAGS=-g', 2. 'avx512' - works fine 'COPTFLAGS=-g -xMIC-AVX512 -O3', 'FOPTFLAGS=-g -xMIC-AVX512 -O3', 3. 'avx2' - breaks. '--COPTFLAGS=-g -axcore-avx2', '--FOPTFLAGS=-g -axcore-avx2', with a breakpoint at dmdavecrestorearrayf903_() in gdb - I see - the stack is fine during the first call to dmdavecrestorearrayf903_() - but is corrupted when it goes to the second call to dmdavecrestorearrayf903_() i.e ierr=0x7fffffffb4a0 changes to ierr=0x0] >>>>>>>>>> Breakpoint 1, dmdavecrestorearrayf903_ (da=0x603098 <test_$DA1.0.1>, v=0x6030c0 <test_$VEC2.0.1>, a=0x401abd <test+2301>, ierr=0x7fffffffb4a0) at /home/petsc/petsc.barry-test/src/dm/impls/da/f90-custom/zda1f90.c:153 153 { (gdb) where #0 dmdavecrestorearrayf903_ (da=0x603098 <test_$DA1.0.1>, v=0x6030c0 <test_$VEC2.0.1>, a=0x401abd <test+2301>, ierr=0x7fffffffb4a0) at /home/petsc/petsc.barry-test/src/dm/impls/da/f90-custom/zda1f90.c:153 #1 0x0000000000401abd in test () at ex1f.F90:80 #2 0x00000000004011ae in main () #3 0x00007fffef1c3c05 in __libc_start_main () from /lib64/libc.so.6 #4 0x00000000004010b9 in _start () (gdb) c Continuing. Breakpoint 1, dmdavecrestorearrayf903_ (da=0x603098 <test_$DA1.0.1>, v=0x6030b8 <test_$VEC1.0.1>, a=0x401ada <test+2330>, ierr=0x0) at /home/petsc/petsc.barry-test/src/dm/impls/da/f90-custom/zda1f90.c:153 153 { (gdb) where #0 dmdavecrestorearrayf903_ (da=0x603098 <test_$DA1.0.1>, v=0x6030b8 <test_$VEC1.0.1>, a=0x401ada <test+2330>, ierr=0x0) at /home/petsc/petsc.barry-test/src/dm/impls/da/f90-custom/zda1f90.c:153 #1 0x0000000000401ada in test () at ex1f.F90:81 #2 0x00000000004011ae in main () #3 0x00007fffef1c3c05 in __libc_start_main () from /lib64/libc.so.6 #4 0x00000000004010b9 in _start () (gdb) >>>>>>>>> Its not clear to me why this happens. [and why it would work with -xMIC-AVX512 but breaks with -axcore-avx2]. Perhaps Richard, Jeff have better insight on this. BTW: The above run is with: bash-4.2$ icc --version icc (ICC) 18.0.0 20170811 Satish On Mon, 9 Apr 2018, Satish Balay wrote: > I'm able to reproduce this problem on knl box [with the attached test code]. > But it goes away if I rebuild without the option --with-64-bit-indices. > > Will have to check further.. > > Satish > > > On Thu, 5 Apr 2018, Randall Mackie wrote: > > > Dear PETSc users, > > > > I’m curious if anyone else experiences problems using DMDAVecGetArrayF90 in > > conjunction with Intel compilers? > > We have had many problems (typically 11 SEGV segmentation violations) when > > PETSc is compiled in optimize mode (with various combinations of options). > > These same codes run valgrind clean with gfortran, so I assume this is an > > Intel bug, but before we submit a bug report I wanted to see if anyone else > > had similar experiences? > > We have basically gone back and replaced our calls to DMDAVecGetArrayF90 > > with calls to VecGetArrayF90 and pass those pointers into a “local” > > subroutine that works fine. > > > > In case anyone is curious, the attached test code shows this behavior when > > PETSc is compiled with the following options: > > > > ./configure \ > > --with-clean=1 \ > > --with-debugging=0 \ > > --with-fortran=1 \ > > --with-64-bit-indices \ > > --download-mpich=../mpich-3.3a2.tar.gz \ > > --with-blas-lapack-dir=/opt/intel/mkl \ > > --with-cc=icc \ > > --with-fc=ifort \ > > --with-cxx=icc \ > > --FOPTFLAGS='-O2 -xSSSE3 -axcore-avx2' \ > > --COPTFLAGS='-O2 -xSSSE3 -axcore-avx2' \ > > --CXXOPTFLAGS='-O2 -xSSSE3 -axcore-avx2’ \ > > > > > > > > Thanks, Randy M. > > > > >