Can you add

-mca oob_tcp_if_include eth0 ...

to your mpirun command line and see if it helps ?


if the error is still there, please follow-up in a new thread (since this error is unlikely related to openib and multi threading)


Cheers,


Gilles



On 3/23/2017 8:05 AM, Emin Nuriyev wrote:
Hi,

I reserved nodes in grid5000. I use latest Open MPI code from git. I am trying to execute MPI code. But There is error message:


ORTE does not know how to route a message to the specified daemon
located on the indicated node:

  my node:   graphite-1
  target node:  graphite-2

This is usually an internal programming error that should be
reported to the developers. In the meantime, a workaround may
be to set the MCA param routed=direct on the command line or
in your environment. We apologize for the problem.
Command line is

mpirun -mca btl_tcp_if_include eth0 -mca routed direct -hostfile graphite_nodes -n 4 mpi_executable

I thought that the problem is because of different networks. Reserved nodes are from grisou, griffon, graphite. From different clusters. Therefor, I created host files contains nodes only from grisou, graphite, griffon. I selected one node from each cluster and executed MPI code using nodes only from this cluster. Again I had the same problem. I selected the only eth0, but again there is no result. In error message is mentioned that it should be reported to the developer.

Best regards,
Emin Nuriyev

On 22 March 2017 at 15:45, Gilles Gouaillardet <gilles.gouaillar...@gmail.com <mailto:gilles.gouaillar...@gmail.com>> wrote:

    Enrico,

    this is fixed in Open MPI 2.1.0
    fwiw, you only need MPI_THREAD_MULTIPLE if you invoke MPI
    subroutines within an OpenMP parallel. if MPI is only used outside
    of OpenMP parallel, then MPI_THREAD_SINGLE is very likely enough.

    Cheers,

    Gilles


    On Thursday, March 23, 2017, Enrico Calore
    <enrico.cal...@fe.infn.it <mailto:enrico.cal...@fe.infn.it>> wrote:

        Hi all,
        when trying to run an OpenMPI + OpenMP application, using OpenMPI
        v1.10.2, compiled with multi-threading support, and the OpenIB
        BTL, the
        latter disable itself with this message:

        btl:openib: MPI_THREAD_MULTIPLE not suppported; skipping this
        component
        select: init of component openib returned failure
        mca: base: close: component openib closed
        mca: base: close: unloading component openib

        In an old mailing-list thread I read that multi-threading
        support for
        openib was expected in 1.9 series:

        
http://users.open-mpi.narkive.com/B2UJ86Wf/ompi-users-multi-threading-support-for-openib
        
<http://users.open-mpi.narkive.com/B2UJ86Wf/ompi-users-multi-threading-support-for-openib>

        But, looking in the changelogs I could not find any mention of
        this.

        May I ask which is the actual status? Is there some newer OpenMPI
        versions supporting it and/or are available some compile time
        flags to
        enable some kind of experimental support for it?


        Thanks and Best Regards,

        Enrico


    _______________________________________________
    devel mailing list
    devel@lists.open-mpi.org <mailto:devel@lists.open-mpi.org>
    https://rfd.newmexicoconsortium.org/mailman/listinfo/devel
    <https://rfd.newmexicoconsortium.org/mailman/listinfo/devel>




_______________________________________________
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel

_______________________________________________
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel

Reply via email to