Re: [OMPI users] Enforcing specific interface and subnet usage

2018-07-01 Thread Maksym Planeta
Sorry for late response. But I just wanted to inform you that I found another workaround, unrelated to the method we discussed here. On 19/06/18 15:26, r...@open-mpi.org wrote: The OMPI cmd line converts "--mca ptl_tcp_remote_connections 1” to OMPI_MCA_ ptl_tcp_remote_connections, which is

Re: [OMPI users] Enforcing specific interface and subnet usage

2018-06-19 Thread r...@open-mpi.org
The OMPI cmd line converts "--mca ptl_tcp_remote_connections 1” to OMPI_MCA_ ptl_tcp_remote_connections, which is not recognized by PMIx. PMIx is looking for PMIX_MCA_ptl_tcp_remote_connections. The only way to set PMIx MCA params for the code embedded in OMPI is to put them in your environment

Re: [OMPI users] Enforcing specific interface and subnet usage

2018-06-19 Thread Gilles Gouaillardet
What is exactly the issue you are facing ? You also need to force the subnet used by oob/tcp mpirun —mca oob_tcp_if_include 10.233.0.0/19 ... iirc, Open MPI might discard addresses from a bridge interface, but I do not exactly remember if it affects both btl/tcp and/or oob/tcp and/or none by

Re: [OMPI users] Enforcing specific interface and subnet usage

2018-06-19 Thread Maksym Planeta
But what about remote connections parameter? Why is it not set? On 19/06/18 00:58, r...@open-mpi.org wrote: I’m not entirely sure I understand what you are trying to do. The PMIX_SERVER_URI2 envar tells local clients how to connect to their local PMIx server (i.e., the OMPI daemon on that

Re: [OMPI users] Enforcing specific interface and subnet usage

2018-06-18 Thread r...@open-mpi.org
I’m not entirely sure I understand what you are trying to do. The PMIX_SERVER_URI2 envar tells local clients how to connect to their local PMIx server (i.e., the OMPI daemon on that node). This is always done over the loopback device since it is a purely local connection that is never used for

[OMPI users] Enforcing specific interface and subnet usage

2018-06-18 Thread Maksym Planeta
Hello, I want to force OpenMPI to use TCP and in particular use a particular subnet. Unfortunately, I can't manage to do that. Here is what I try: $BIN/mpirun --mca pml ob1 --mca btl tcp,self --mca ptl_tcp_remote_connections 1 --mca btl_tcp_if_include '10.233.0.0/19' -np 4 --oversubscribe -H