Re: [OMPI users] How is the rank determined (Open MPI and Podman)

2019-07-25 Thread Adrian Reber via users
On Wed, Jul 24, 2019 at 09:46:13PM +, Jeff Squyres (jsquyres) wrote: > On Jul 24, 2019, at 5:16 PM, Ralph Castain via users > wrote: > > > > It doesn't work that way, as you discovered. You need to add this > > information at the same place where vader currently calls modex send, and > > t

Re: [OMPI users] How is the rank determined (Open MPI and Podman)

2019-07-24 Thread Jeff Squyres (jsquyres) via users
On Jul 24, 2019, at 5:16 PM, Ralph Castain via users wrote: > > It doesn't work that way, as you discovered. You need to add this information > at the same place where vader currently calls modex send, and then retrieve > it at the same place vader currently calls modex recv. Those macros don'

Re: [OMPI users] How is the rank determined (Open MPI and Podman)

2019-07-24 Thread Ralph Castain via users
It doesn't work that way, as you discovered. You need to add this information at the same place where vader currently calls modex send, and then retrieve it at the same place vader currently calls modex recv. Those macros don't do an immediate send/recv like you are thinking - the send simply ad

Re: [OMPI users] How is the rank determined (Open MPI and Podman)

2019-07-22 Thread Nathan Hjelm via users
Just add it to the existing modex. -Nathan > On Jul 22, 2019, at 12:20 PM, Adrian Reber via users > wrote: > > I have most of the code ready, but I still have troubles doing > OPAL_MODEX_RECV. I am using the following lines, based on the code from > orte/test/mpi/pmix.c: > > OPAL_MODEX_SEND_V

Re: [OMPI users] How is the rank determined (Open MPI and Podman)

2019-07-22 Thread Adrian Reber via users
I have most of the code ready, but I still have troubles doing OPAL_MODEX_RECV. I am using the following lines, based on the code from orte/test/mpi/pmix.c: OPAL_MODEX_SEND_VALUE(rc, OPAL_PMIX_LOCAL, "user_ns_id", &value, OPAL_INT); This sets rc to 0. For receiving: OPAL_MODEX_RECV_VALUE(rc, "us

Re: [OMPI users] How is the rank determined (Open MPI and Podman)

2019-07-22 Thread Ralph Castain via users
If that works, then it might be possible to include the namespace ID in the job-info provided by PMIx at startup - would have to investigate, so please confirm that the modex option works first. > On Jul 22, 2019, at 1:22 AM, Gilles Gouaillardet via users > wrote: > > Adrian, > > > An optio

Re: [OMPI users] How is the rank determined (Open MPI and Podman)

2019-07-22 Thread Gilles Gouaillardet via users
Adrian, An option is to involve the modex. each task would OPAL_MODEX_SEND() its own namespace ID, and then OPAL_MODEX_RECV() the one from its peers and decide whether CMA support can be enabled. Cheers, Gilles On 7/22/2019 4:53 PM, Adrian Reber via users wrote: I had a look at it and

Re: [OMPI users] How is the rank determined (Open MPI and Podman)

2019-07-22 Thread Adrian Reber via users
I had a look at it and not sure if it really makes sense. In btl_vader_{put,get}.c it would be easy to check for the user namespace ID of the other process, but the function would then just return OPAL_ERROR a bit earlier instead of as a result of process_vm_{read,write}v(). Nothing would really c