Re: [OMPI users] Open MPI vs IBM MPI performance help
I'm curious if that resolved the issue. David Singleton wrote: http://www.open-mpi.org/faq/?category=running#oversubscribing On 12/03/2010 06:25 AM, Price, Brian M (N-KCI) wrote: Additional testing seems to show that the problem is related to barriers and how often they poll to determine whether or not it's time to leave. Is there some MCA parameter or environment variable that allows me to control the frequency of polling while in barriers? From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On Behalf Of Price, Brian M (N-KCI) Sent: Wednesday, December 01, 2010 11:29 AM To: Open MPI Users Cc: Stern, Craig J Subject: EXTERNAL: [OMPI users] Open MPI vs IBM MPI performance help OpenMPI version: 1.4.3 Platform: IBM P5, 32 processors, 256 GB memory, Symmetric Multi-Threading (SMT) enabled Application: starts up 48 processes and does MPI using MPI_Barrier, MPI_Get, MPI_Put (lots of transfers, large amounts of data) Issue: When implemented using Open MPI vs. IBM's MPI ('poe' from HPC Toolkit), the application runs 3-5 times slower. I suspect that IBM's MPI implementation must take advantage of some knowledge that it has about data transfers that Open MPI is not taking advantage of. Any suggestions?
Re: [OMPI users] Open MPI vs IBM MPI performance help
Collecting MPI Profile information might help narrow down the issue. You could use some of the tools mentioned here - http://www.open-mpi.org/faq/?category=perftools --Nysal On Wed, Dec 1, 2010 at 11:59 PM, Price, Brian M (N-KCI) < brian.m.pr...@lmco.com> wrote: > OpenMPI version: 1.4.3 > > Platform: IBM P5, 32 processors, 256 GB memory, Symmetric Multi-Threading > (SMT) enabled > > Application: starts up 48 processes and does MPI using MPI_Barrier, > MPI_Get, MPI_Put (lots of transfers, large amounts of data) > > Issue: When implemented using Open MPI vs. IBM’s MPI (‘poe’ from HPC > Toolkit), the application runs 3-5 times slower. > > I suspect that IBM’s MPI implementation must take advantage of some > knowledge that it has about data transfers that Open MPI is not taking > advantage of. > > Any suggestions? > > Thanks, > > Brian Price > > > > ___ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users >
Re: [OMPI users] Open MPI vs IBM MPI performance help
http://www.open-mpi.org/faq/?category=running#oversubscribing On 12/03/2010 06:25 AM, Price, Brian M (N-KCI) wrote: Additional testing seems to show that the problem is related to barriers and how often they poll to determine whether or not it's time to leave. Is there some MCA parameter or environment variable that allows me to control the frequency of polling while in barriers? Thanks, Brian Price From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On Behalf Of Price, Brian M (N-KCI) Sent: Wednesday, December 01, 2010 11:29 AM To: Open MPI Users Cc: Stern, Craig J Subject: EXTERNAL: [OMPI users] Open MPI vs IBM MPI performance help OpenMPI version: 1.4.3 Platform: IBM P5, 32 processors, 256 GB memory, Symmetric Multi-Threading (SMT) enabled Application: starts up 48 processes and does MPI using MPI_Barrier, MPI_Get, MPI_Put (lots of transfers, large amounts of data) Issue: When implemented using Open MPI vs. IBM's MPI ('poe' from HPC Toolkit), the application runs 3-5 times slower. I suspect that IBM's MPI implementation must take advantage of some knowledge that it has about data transfers that Open MPI is not taking advantage of. Any suggestions? Thanks, Brian Price
Re: [OMPI users] Open MPI vs IBM MPI performance help
Additional testing seems to show that the problem is related to barriers and how often they poll to determine whether or not it's time to leave. Is there some MCA parameter or environment variable that allows me to control the frequency of polling while in barriers? Thanks, Brian Price From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On Behalf Of Price, Brian M (N-KCI) Sent: Wednesday, December 01, 2010 11:29 AM To: Open MPI Users Cc: Stern, Craig J Subject: EXTERNAL: [OMPI users] Open MPI vs IBM MPI performance help OpenMPI version: 1.4.3 Platform: IBM P5, 32 processors, 256 GB memory, Symmetric Multi-Threading (SMT) enabled Application: starts up 48 processes and does MPI using MPI_Barrier, MPI_Get, MPI_Put (lots of transfers, large amounts of data) Issue: When implemented using Open MPI vs. IBM's MPI ('poe' from HPC Toolkit), the application runs 3-5 times slower. I suspect that IBM's MPI implementation must take advantage of some knowledge that it has about data transfers that Open MPI is not taking advantage of. Any suggestions? Thanks, Brian Price
[OMPI users] Open MPI vs IBM MPI performance help
OpenMPI version: 1.4.3 Platform: IBM P5, 32 processors, 256 GB memory, Symmetric Multi-Threading (SMT) enabled Application: starts up 48 processes and does MPI using MPI_Barrier, MPI_Get, MPI_Put (lots of transfers, large amounts of data) Issue: When implemented using Open MPI vs. IBM's MPI ('poe' from HPC Toolkit), the application runs 3-5 times slower. I suspect that IBM's MPI implementation must take advantage of some knowledge that it has about data transfers that Open MPI is not taking advantage of. Any suggestions? Thanks, Brian Price