I don't know much about OpenMP, but do you need to disable Open MPI's default bind-to-core functionality (I'm assuming you're using Open MPI 1.8.x)?
You can try "mpirun --bind-to none ...", which will have Open MPI not bind MPI processes to cores, which might allow OpenMP to think that it can use all the cores, and therefore it will spawn num_cores threads...? On Aug 14, 2014, at 9:50 AM, Oscar Mojica <o_moji...@hotmail.com> wrote: > > Hello everybody > > I am trying to run a hybrid mpi + openmp program in a cluster. I created a > queue with 14 machines, each one with 16 cores. The program divides the work > among the 14 processors with MPI and within each processor a loop is also > divided into 8 threads for example, using openmp. The problem is that when I > submit the job to the queue the MPI processes don't divide the work into > threads and the program prints the number of threads that are working within > each process as one. > > I made a simple test program that uses openmp and I logged in one machine of > the fourteen. I compiled it using gfortran -fopenmp program.f -o exe, set > the OMP_NUM_THREADS environment variable equal to 8 and when I ran directly > in the terminal the loop was effectively divided among the cores and for > example in this case the program printed the number of threads equal to 8 > > This is my Makefile > > # Start of the makefile > # Defining variables > objects = inv_grav3d.o funcpdf.o gr3dprm.o fdjac.o dsvd.o > #f90comp = /opt/openmpi/bin/mpif90 > f90comp = /usr/bin/mpif90 > #switch = -O3 > executable = inverse.exe > # Makefile > all : $(executable) > $(executable) : $(objects) > $(f90comp) -fopenmp -g -O -o $(executable) $(objects) > rm $(objects) > %.o: %.f > $(f90comp) -c $< > # Cleaning everything > clean: > rm $(executable) > # rm $(objects) > # End of the makefile > > and the script that i am using is > > #!/bin/bash > #$ -cwd > #$ -j y > #$ -S /bin/bash > #$ -pe orte 14 > #$ -N job > #$ -q new.q > > export OMP_NUM_THREADS=8 > /usr/bin/time -f "%E" /opt/openmpi/bin/mpirun -v -np $NSLOTS ./inverse.exe > > am I forgetting something? > > Thanks, > > Oscar Fabian Mojica Ladino > Geologist M.S. in Geophysics > _______________________________________________ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2014/08/25016.php -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
