Re: [OMPI devel] sm BTL performace of the openmpi-2.0.0

Hi Gilles, I confirmed the vader is used when I don't specify any BTL as you pointed out! Regards, Tetsuya Mishima [mishima@manage OMB-3.1.1-openmpi2.0.0]$ mpirun -np 2 --mca btl_base_verbose 10 -bind-to core -report-bindings osu_bw [manage.cluster:20006] MCW rank 0 bound to socket 0[core 0[hwt

Re: [OMPI devel] sm BTL performace of the openmpi-2.0.0

Hi, Thanks. I will try it and report later. Tetsuya Mishima 2016/07/27 9:20:28、"devel"さんは「Re: [OMPI devel] sm BTL performace of the openmpi-2.0.0」で書きました > sm is deprecated in 2.0.0 and will likely be removed in favor of vader in 2.1.0. > > This issue is probably this known issue:

Re: [OMPI devel] sm BTL performace of the openmpi-2.0.0

Also, btl/vader has a higher exclusivity than btl/sm, so if you do not manually specify any btl, vader should be used. you can run with --mca btl_base_verbose 10 to confirm which btl is used Cheers, Gilles On 7/27/2016 9:20 AM, Nathan Hjelm wrote: sm is deprecated in 2.0.0 and will

Re: [OMPI devel] sm BTL performace of the openmpi-2.0.0

sm is deprecated in 2.0.0 and will likely be removed in favor of vader in 2.1.0. This issue is probably this known issue: https://github.com/open-mpi/ompi-release/pull/1250 Please apply those commits and see if it fixes the issue for you. -Nathan > On Jul 26, 2016, at 6:17 PM,

Re: [OMPI devel] sm BTL performace of the openmpi-2.0.0

Hi Gilles, Thanks. I ran again with --mca pml ob1 but I've got the same results as below: [mishima@manage OMB-3.1.1-openmpi2.0.0]$ mpirun -np 2 -mca pml ob1 -bind-to core -report-bindings osu_bw [manage.cluster:18142] MCW rank 0 bound to socket 0[core 0[hwt 0]]: [B/././././.][./././././.]

Re: [OMPI devel] sm BTL performace of the openmpi-2.0.0

Hi, can you please run again with --mca pml ob1 if Open MPI was built with mxm support, pml/cm and mtl/mxm are used instead of pml/ob1 and btl/openib Cheers, Gilles On 7/27/2016 8:56 AM, tmish...@jcity.maeda.co.jp wrote: Hi folks, I saw a performance degradation of openmpi-2.0.0

[OMPI devel] sm BTL performace of the openmpi-2.0.0

Hi folks, I saw a performance degradation of openmpi-2.0.0 when I ran our application on a node (12cores). So I did 4 tests using osu_bw as below: 1: mpirun –np 2 osu_bw bad(30% of test2) 2: mpirun –np 2 –mca btl self,sm osu_bw good(same as openmpi1.10.3) 3:

Re: [OMPI devel] PGI built Open MPI vs GNU built slurm

Gilles, With the additional information you provided about "dependency_libs", I agree that that either of the fixes you propose sound safe. -Paul On Mon, Jul 25, 2016 at 6:26 PM, Gilles Gouaillardet wrote: > Paul, > > in my environment, libslurm.la contains > > # Linker

Re: [OMPI devel] tcp btl rendezvous performance question

Hi Sreenidhi Only partial resolution. By pushing out the eager path to 4 MB we were able to get around 2GB/sec per socket connection with osu bw test. The kernel is quite old though - 2.6.x - and being a summer student project with a focus on IB vs rout able ROCE we've moved on. Howard

Re: [OMPI devel] OpenMPI 2.0 and Petsc 3.7.2

Hi Gilles, On 25/07/16 10:38 PM, Gilles Gouaillardet wrote: Eric, where can your test case be downloaded ? how many nodes and tasks do you need to reproduce the bug ? Sadly, it is in our in-house code and it requires to whole source code which isn't public... :/ I have this bug with 20

Re: [OMPI devel] tcp btl rendezvous performance question

hi Howard, Was this issue resolved ? If so, what is the solution ? Please let me know. Curious to know , since we are also experimenting with these limits. Thanks, - Sreenidhi. On Tue, Jul 19, 2016 at 10:50 AM, Gilles Gouaillardet wrote: > Howard, > > > did you bump both

Re: [OMPI devel] OpenMPI 2.0 and Petsc 3.7.2

Nathan and Eric, there is a know issue of libnbc not correctly retaining datatypes, for example, if you start a non blocking collective operation (MPI_Ibcast for example) and they MPI_Type_destroy() the datatype *before* the non blocking collective completes, then the datatype is free'd by