On 01/14/14 16:41, Karl Rupp wrote:
Hi Pierre,
> I try running in parallel the ex19 test case on CPU and GPU:
OPT="_opt"
dir=/ccc/cont002/home/den/triou/Version_test_airain-hybrid/Trio_U/lib/src/LIBPETSC/petsc/linux$OPT/src/snes/examples/tutorials
option="-pc_type none -ksp_type fgmres -snes_monitor_short -snes_rtol
1.e-5 -log_summary -ksp_view -cuda_show_devices"
mpirun -np 2 $dir/ex19 $option 1>cpu$OPT.log 2>&1
mpirun -np 2 $dir/ex19 -dm_vec_type cusp -dm_mat_type aijcusp $option
1>gpu$OPT.log 2>&1
With OPT="", PETSc optimized library is used, parallel calculation runs
well on CPU and GPU.
With OPT="_opt", PETSc non optimized library is used, parallel
calculation crashes on GPU (it is OK on CPU).
I join the log files. The only difference seems that PETSc-dev is built
with -O3 intead of -g...
I could try to rebuild PETSc with -O2 but do you have any idea of the
problem ?
I could reproduce the problem and also get some uninitialized variable
warnings in Valgrind. The debug version detects these errors, hence
you only see the errors in the debug build. For the optimized build,
chances are good that the computed values are either wrong or may
become wrong in other environments. I'll see what I can do when I'm
again at GPU machine tomorrow (parallel GPU debugging via SSH is not
great...)
Sorry, I mean:
Parallel calculation on CPU or GPU run well with PETSc non optimized library
Parallel calculation on GPU crashes with PETSc optimized library (on CPU
it is OK)
I could add that the "mpirun -np 1 ex19" runs well for all builds on CPU
and GPU.
Pierre
Best regards,
Karli
--
*Trio_U support team*
Marthe ROUX (Saclay)
Pierre LEDAC (Grenoble)
