On Wed, Apr 13, 2016 at 1:59 AM, Gilles Gouaillardet <gil...@rist.or.jp> wrote:
> George, > > about the process binding part > > On 4/13/2016 7:32 AM, George Bosilca wrote: > >> Also my processes, despite the fact that I asked for 1 per node, are not >> bound to the first core. Shouldn’t we release the process binding when we >> know there is a single process per node (as in the above case) ? >> > did you expect the tasks are bound to the first *core* on each node ? > > i would expect the tasks are bound to the first *socket* on each node. > In this particular instance, where it has been explicitly requested to have a single process per node, I would have expected the process to be unbound (we know there is only one per node). It is the responsibility of the application to bound itself or its thread if necessary. Why are we enforcing a particular binding policy? (since we do not know how many (OpenMP or other) threads will be used by > the application, --bind-to socket is a good policy imho. in this case (one task per node), > no binding at all would mean > the task can migrate from one socket to the other, and/or OpenMP threads > are bound accross sockets. > That would trigger some NUMA effects (better bandwidth if memory is > locally accessed, but worst performance > is memory is allocated only on one socket). > so imho, --bind-to socket is still my preferred policy, even if there is > only one MPI task per node. > Open MPI is about MPI ranks/processes. I don't think it is our job to try to figure out how the user handle do with it's own threads. Your justification make sense if the application only uses a single socket. It also make sense if one starts multiple ranks per node, and the internal threads of each MPI process inherit the MPI process binding. However, in the case where there is a single process per node, because there is a mismatch between the number of resources available (hardware threads) and the binding of the parent process, all the threads of the MPI application are [by default] bound on a single socket. George. PS: That being said I think I'll need to implement the binding code anyway in order to deal with the wide variety of behaviors in the different MPI implementations. > > Cheers, > > Gilles > _______________________________________________ > devel mailing list > de...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel > Link to this post: > http://www.open-mpi.org/community/lists/devel/2016/04/18758.php
