Hi all,
I have compiled the Siesta following the LeRoux mini-howto and the software
run perfect when isn't running with "mpiexec".
When i use the commands: "mpd --ncpus=4& ; mpiexec -n 2 siesta <INFILE.FDF>
OUTFILE.OUT" the process are executed but i receive errors messages like
this:
#mpiexec -n 2 siesta <cnt_6-6.fdf>saida.out
[cli_0]: aborting job:
Fatal error in MPI_Reduce: Invalid MPI_Op, error stack:
MPI_Reduce(850).............: MPI_Reduce(sbuf=0x7fff9516c9b8,
rbuf=0x7fff9516c9c8, count=1, MPI_2DOUBLE_PRECISION, MPI_MAXLOC, root=0,
MPI_COMM_WORLD) failed
MPIR_MAXLOC_check_dtype(217): MPI_Op MPI_MAXLOC operation not defined for this
datatype
[cli_1]: aborting job:
Fatal error in MPI_Reduce: Invalid MPI_Op, error stack:
MPI_Reduce(850).............: MPI_Reduce(sbuf=0x7ffffe5b5df8,
rbuf=0x7ffffe5b5e08, count=1, MPI_2DOUBLE_PRECISION, MPI_MAXLOC, root=0,
MPI_COMM_WORLD) failed
MPIR_MAXLOC_check_dtype(217): MPI_Op MPI_MAXLOC operation not defined for this
datatype
I'm confuse with that errors messages, because i search on Google and i can
found a answer on how to get ride od this errors.
To complete the information, i have compiled MPICH2 with ifort/icc and this
configure comand: /configure --prefix=/opt/cluster/mpich2 FC=ifort F90=ifort
F77=ifort CC=icc CXX=icc FFLAGS="-O3 -axT -openmp"
CFLAGS="-O3 -axT -openmp" --enable-mpe --with-device=ch3:ssm
The blas and lapack libs using:
FORTRAN = mpif90
OPTS = -O3 -axT
DRVOPTS = $(OPTS)
NOOPT = -O0
LOADER = mpif90
The BLACS lib:
F77 = mpif77
F77NO_OPTFLAGS = -O0
F77FLAGS = -O3 -axT
F77LOADER = $(F77)
F77LOADFLAGS =
CC = mpicc
CCFLAGS = -O3 -axT
CCLOADER = $(CC)
CCLOADFLAGS =
The Scalapack Lib:
USEMPI = -DUsingMpiBlacs
SMPLIB = /opt/cluster/mpich2/lib/libmpich.a
BLACSFINIT = $(BLACSdir)/blacsF77init_MPI-LINUX-0.a
BLACSCINIT = $(BLACSdir)/blacsCinit_MPI-LINUX-0.a
BLACSLIB = $(BLACSdir)/blacs_MPI-LINUX-0.a
TESTINGdir = $(home)/TESTING
F77 = mpif77
CC = mpicc
NOOPT = -O0
F77FLAGS = -O3 -axT
DRVOPTS = $(F77FLAGS)
CCFLAGS = -O3 -axT
SRCFLAG =
F77LOADER = $(F77)
CCLOADER = $(CC)
F77LOADFLAGS =
CCLOADFLAGS =
CDEFS = -DAdd_ -DNO_IEEE $(USEMPI)
SCALAPACKLIB = $(home)/libscalapack.a
BLASLIB = /opt/cluster/libs/blas_LINUX.a
LAPACKLIB = /opt/cluster/libs/lapack_LINUX.a
And finally Siesta-2.0.1:
.SUFFIXES :
.SUFFIXES : .f .F .o .a .f90 .F90
SIESTA_ARCH=x86_64-pc-linux-gnu-Intel
FPP=
FPP_OUTPUT=
FC=/opt/cluster/mpich2/bin/mpif90
F90=/opt/cluster/mpich2/bin/mpif90
F75=/opt/cluster/mpich2/bin/mpif90
CC=/opt/cluster/mpich2/bin/mpicc
CXX=/opt/cluster/mpich2/bin/mpicxx
RANLIB=ranlib
#RANLIB=echo
SYS=nag
SP_KIND=4
DP_KIND=8
KINDS=$(SP_KIND) $(DP_KIND)
FFLAGS=-g -w -O3 -axT -openmp -parallel
CCFLAGS=-g -w -O3 -axT -openmp -parallel
FPPFLAGS= -DMPI -DFC_HAVE_FLUSH -DFC_HAVE_ABORT -DGRID_DP
LDFLAGS= -Vaxlib -static -ipo
ARFLAGS_EXTRA=
FCFLAGS_fixed_f=
FCFLAGS_free_f90=
FPPFLAGS_fixed_F=
FPPFLAGS_free_F90=
FFLAGS_DEBUG= -g
HOME_LIB=/opt/cluster/libs
BLAS_LIBS=$(HOME_LIB)/libblas.a
LAPACK_LIBS=$(HOME_LIB)/liblapack.a
BLACS_LIBS=$(HOME_LIB)/libblacsCinit.a $(HOME_LIB)/libblacsF77init.a
$(HOME_LIB)/libblacs.a
SCALAPACK_LIBS=$(HOME_LIB)/libscalapack.a
NETCDF_LIBS=
LIBS= $(GUIDE) $(SCALAPACK_LIBS) $(BLACS_LIBS) $(LAPACK_LIBS) $(BLAS_LIBS)
$(NETCDF_LIBS) -lpthread -lmpich -openmp -parallel
#SIESTA needs an F90 interface to MPI
#This will give you SIESTA's own implementation
#If your compiler vendor offers an alternative, you may change
#to it here.
MPI_HOME=/opt/cluster/mpich2/
MPI_INTERFACE=libmpi_f90.a
MPI_INCLUDE=$(MPI_HOME)/include
MPI_LIB=$(MPI_HOME)/lib
DEFS_MPI=-DMPI
#Dependency rules are created by autoconf according to whether
#discrete preprocessing is necessary or not.
.F.o :
$(FC) -c $(FFLAGS) $(INCFLAGS) $(FPPFLAGS) $(FPPFLAGS_fixed_F) -lmpich
$<
.F90.o :
$(FC) -c $(FFLAGS) $(INCFLAGS) $(FPPFLAGS)
$(FPPFLAGS_free_F90) -lmpich $<
.f.o :
$(FC) -c $(FFLAGS) $(INCFLAGS) $(FCFLAGS_fixed_f) -lmpich $<
.f90.o :
$(FC) -c $(FFLAGS) $(INCFLAGS) $(FCFLAGS_free_f90) -lmpich $<
Any help?
Thank you very much,
Rod
