Well it is confusing!! As you can see, I added four nodes to the host file (the same nodes are used by PBS). The --map-by ppr:1:node works well. However, the PBS directive doesn't work
mahmood@cluster:mpitest$ /share/apps/computer/openmpi-2.0.1/bin/mpirun -hostfile hosts --map-by ppr:1:node a.out **************************************************************************** * hwloc 1.11.2 has encountered what looks like an error from the operating system. * * Package (P#1 cpuset 0xffff0000) intersects with NUMANode (P#1 cpuset 0xff00ffff) without inclusion! * Error occurred in topology.c line 1048 * * The following FAQ entry in the hwloc documentation may help: * What should I do when hwloc reports "operating system" warnings? * Otherwise please report this error message to the hwloc user's mailing list, * along with the output+tarball generated by the hwloc-gather-topology script. **************************************************************************** Hello world from processor cluster.hpc.org, rank 0 out of 4 processors Hello world from processor compute-0-0.local, rank 1 out of 4 processors Hello world from processor compute-0-1.local, rank 2 out of 4 processors Hello world from processor compute-0-2.local, rank 3 out of 4 processors mahmood@cluster:mpitest$ cat mmt.sh #!/bin/bash #PBS -V #PBS -q default #PBS -j oe #PBS -l nodes=4:ppn=1 #PBS -N job1 #PBS -o . cd $PBS_O_WORKDIR /share/apps/computer/openmpi-2.0.1/bin/mpirun a.out mahmood@cluster:mpitest$ qsub mmt.sh 6428.cluster.hpc.org mahmood@cluster:mpitest$ cat job1.o6428 Hello world from processor compute-0-1.local, rank 0 out of 32 processors Hello world from processor compute-0-1.local, rank 2 out of 32 processors Hello world from processor compute-0-1.local, rank 3 out of 32 processors Hello world from processor compute-0-1.local, rank 4 out of 32 processors Hello world from processor compute-0-1.local, rank 5 out of 32 processors Hello world from processor compute-0-1.local, rank 6 out of 32 processors Hello world from processor compute-0-1.local, rank 8 out of 32 processors Hello world from processor compute-0-1.local, rank 9 out of 32 processors Hello world from processor compute-0-1.local, rank 12 out of 32 processors Hello world from processor compute-0-1.local, rank 15 out of 32 processors Hello world from processor compute-0-1.local, rank 16 out of 32 processors Hello world from processor compute-0-1.local, rank 18 out of 32 processors Hello world from processor compute-0-1.local, rank 19 out of 32 processors Hello world from processor compute-0-1.local, rank 20 out of 32 processors Hello world from processor compute-0-1.local, rank 21 out of 32 processors Hello world from processor compute-0-1.local, rank 22 out of 32 processors Hello world from processor compute-0-1.local, rank 24 out of 32 processors Hello world from processor compute-0-1.local, rank 26 out of 32 processors Hello world from processor compute-0-1.local, rank 27 out of 32 processors Hello world from processor compute-0-1.local, rank 28 out of 32 processors Hello world from processor compute-0-1.local, rank 29 out of 32 processors Hello world from processor compute-0-1.local, rank 30 out of 32 processors Hello world from processor compute-0-1.local, rank 31 out of 32 processors Hello world from processor compute-0-1.local, rank 7 out of 32 processors Hello world from processor compute-0-1.local, rank 10 out of 32 processors Hello world from processor compute-0-1.local, rank 14 out of 32 processors Hello world from processor compute-0-1.local, rank 1 out of 32 processors Hello world from processor compute-0-1.local, rank 11 out of 32 processors Hello world from processor compute-0-1.local, rank 13 out of 32 processors Hello world from processor compute-0-1.local, rank 17 out of 32 processors Hello world from processor compute-0-1.local, rank 23 out of 32 processors Hello world from processor compute-0-1.local, rank 25 out of 32 processors Any idea? Regards, Mahmood
_______________________________________________ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
