Hello, I'm currently using CISS via SLEPc. I would like to use parallelization by partitioning over quadrature points, using the option -eps_ciss_partitions.
I have done the following: 1. created a matrix on each MPI rank MatCreateDense(MPI_COMM_SELF, matrix_size, matrix_size, PETSC_DECIDE,PETSC_DECIDE,NULL,&matrix_petsc); 2. Created EPS object EPSCreate(MPI_COMM_WORLD,&eps); EPSSetOperators( eps,matrix_petsc,NULL); EPSSetType(eps,EPSCISS); EPSSetProblemType(eps, EPS_NHEP); EPSSetFromOptions(eps); EPSGetRG(eps,&rg); RGSetType(rg,RGRING); RGRingSetParameters(rg,center,radius,vscale,start_ang,end_ang,width); EPSSolve(eps); 3. Then I run the code: mpiexec -n 2 ./a.out -eps_ciss_partitions 2 The code gives an error: [1]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- [1]PETSC ERROR: Arguments are incompatible [1]PETSC ERROR: MatMatMultSymbolic requires A, seqdense, to be compatible with B, mpidense [1]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. [1]PETSC ERROR: Petsc Release Version 3.8.4, Mar, 24, 2018 [1]PETSC ERROR: ../lib/a.out on a linux-complex named brown-a337.rcac.purdue.edu by mpovolot Tue Sep 10 23:43:44 2019 [1]PETSC ERROR: Configure options --with-scalar-type=complex --with-x=0 --with-hdf5 --download-hdf5=1 --with-single-library=1 --with-pic=1 --with-shared-libraries=0 --with-log=0 --with-clanguage=C++ --CXXFLAGS="-fopenmp -fPIC" --CFLAGS="-fopenmp -fPIC" --with-fortran=0 --FFLAGS="-fopenmp -fPIC" --with-debugging=0 --with-cc=mpicc --with-fc=mpif90 --with-cxx=mpicxx COPTFLAGS= CXXOPTFLAGS= FOPTFLAGS= --download-metis=1 --download-parmetis=1 --with-valgrind-dir=/apps/brown/valgrind/3.13.0_gcc-4.8.5 --download-mumps=1 --with-fortran-kernels=0 --download-superlu_dist=1 --with-blaslapack-lib="-L/apps/cent7/intel/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64 -lmkl_intel_lp64 -lmkl_gnu_thread -lmkl_core " --with-blacs-lib=/apps/cent7/intel/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64/libmkl_blacs_intelmpi_lp64.so --with-blacs-include=/apps/cent7/intel/compilers_and_libraries_2017.1.132/linux/mkl/include --with-scalapack-lib="-Wl,-rpath,/apps/cent7/intel/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64 -L/apps/cent7/intel/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64 -lmkl_gf_lp64 -lmkl_gnu_thread -lmkl_core -lpthread -L/apps/cent7/intel/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64 -lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_lp64" --with-scalapack-include=/apps/cent7/intel/compilers_and_libraries_2017.1.132/linux/mkl/include [1]PETSC ERROR: #463 MatMatMultSymbolic() line 9692 in /depot/kildisha/apps/brown/nemo5/libs/petsc/build-cplx/src/mat/interface/matrix.c [1]PETSC ERROR: #464 BVMatMult_Svec() line 229 in /depot/kildisha/apps/brown/nemo5/libs/slepc/build-cplx/src/sys/classes/bv/impls/svec/svec.c [1]PETSC ERROR: #465 BVMatMult() line 589 in /depot/kildisha/apps/brown/nemo5/libs/slepc/build-cplx/src/sys/classes/bv/interface/bvops.c [1]PETSC ERROR: #466 BVMatProject_MatMult() line 903 in /depot/kildisha/apps/brown/nemo5/libs/slepc/build-cplx/src/sys/classes/bv/interface/bvglobal.c [1]PETSC ERROR: #467 BVMatProject() line 1151 in /depot/kildisha/apps/brown/nemo5/libs/slepc/build-cplx/src/sys/classes/bv/interface/bvglobal.c [1]PETSC ERROR: #468 EPSSolve_CISS() line 1066 in /depot/kildisha/apps/brown/nemo5/libs/slepc/build-cplx/src/eps/impls/ciss/ciss.c [1]PETSC ERROR: #469 EPSSolve() line 147 in /depot/kildisha/apps/brown/nemo5/libs/slepc/build-cplx/src/eps/interface/epssolve.c Could you, please, tell me what am I doing wrong?
