Re: [OMPI devel] problem when binding to socket on a single socket node
On Mon, 2010-04-12 at 10:07 -0600, Ralph Castain wrote: > By definition, if you bind to all available cpus in the OS, you are > bound to nothing (i.e., "unbound") as your process runs on any > available cpu. > > > PLPA doesn't care, and I personally don't care. I was just explaining > why it generates an error in the odls. > > > A user app would detect its binding by (a) getting the affiinity mask > from the OS, and then (b) seeing if the bits are set to '1' for all > available processors. If it is, then you are not bound - there is no > mechanism available for checking "are the bits set only for the > processors I asked to be bound to". The OS doesn't track what you > asked for, it only tracks where you are bound - and a mask with all > '1's is defined as "unbound". > > > So the reason for my question was simple: a user asked us to "bind" > their process. If their process checks to see if it is bound, it will > return "no". The user would therefore be led to believe that OMPI had > failed to execute their request, when in fact we did execute it - but > the result was (as Nadia says) a "no-op". > > > After talking with Jeff, I think he has the right answer. It is a > method we have used elsewhere, so it isn't unexpected behavior. > Basically, he proposed that we use an mca param to control this > behavior: > > > * default: generate an error message as the "bind" results in a no-op, > and this is our current behavior > > > * warn: generate a warning that the binding wound up being a "no-op", > but continue working > > > * quiet: just ignore it and keep going Excellent, I completely agree (though I would have put the 2nd star as the default behavior, but never mind, I don't want to restart the discussion ;-) ) Also this is a good opportunity to fix the other issue I talked about in the first message in this thread: the tag "odls-default:could-not-bind-to-socket" does not exist in orte/mca/odls/default/help-odls-default.txt Regards, Nadia > > > Fairly trivial to implement, and Bull could set the default mca param > file to "quiet" to get what they want. I'm not sure if that's what the > community wants or not - like I said, it makes no diff to me so long > as the code logic is understandable. > > > > On Apr 12, 2010, at 8:27 AM, Terry Dontje wrote: > > > Ralph, I guess I am curious why is it that if there is only one > > socket we cannot bind to it? Does plpa actually error on this or is > > this a condition we decided was an error at odls? > > > > I am somewhat torn on whether this makes sense. On the one hand it > > is definitely useless as to the result if you allow it. However if > > you don't allow it and you have a script or running tests on > > multiple systems it would be nice to have this run because you are > > not really running into a resource starvation issue. > > > > At a minimum I think the error condition/message needs to be spelled > > out (defined).As to whether we allow binding when only one > > socket exist I could go either way slightly leaning towards allowing > > such a specification to work. > > > > --td > > > > > > Ralph Castain wrote: > > > Guess I'll jump in here as I finally had a few minutes to look at the > > > code and think about your original note. In fact, I believe your original > > > statement is the source of contention. > > > > > > If someone tells us -bind-to-socket, but there is only one socket, then > > > we really cannot bind them to anything. Any check by their code would > > > reveal that they had not, in fact, been bound - raising questions as to > > > whether or not OMPI is performing the request. Our operating standard has > > > been to error out if the user specifies something we cannot do to avoid > > > that kind of confusion. This is what generated the code in the system > > > today. > > > > > > Now I can see an argument that -bind-to-socket with one socket maybe > > > shouldn't generate an error, but that decision then has to get reflected > > > in other code areas as well. > > > > > > As for the test you cite - it actually performs a valuable function and > > > was added to catch specific scenarios. In particular, if you follow the > > > code flow up just a little, you will see that it is possible to complete > > > the loop without ever actually setting a bit in the mask. This happens > > > when none of the cpus in that socket have been assigned to us via an > > > external bind. People actually use that as a means of suballocating > > > nodes, so the test needs to be there. Again, if the user said "bind to > > > socket", but none of that socket's cores are assigned for our use, that > > > is an error. > > > > > > I haven't looked at your specific fix, but I agree with Terry's question. > > > It seems to me that whether or not we were externally bound is > > > irrelevant. Even if the overall result is what you want, I think a more > > > logically understandable test would help others reading the co
Re: [OMPI devel] problem when binding to socket on a single socket node
On Apr 13, 2010, at 1:02 AM, Nadia Derbey wrote: > On Mon, 2010-04-12 at 10:07 -0600, Ralph Castain wrote: >> By definition, if you bind to all available cpus in the OS, you are >> bound to nothing (i.e., "unbound") as your process runs on any >> available cpu. >> >> >> PLPA doesn't care, and I personally don't care. I was just explaining >> why it generates an error in the odls. >> >> >> A user app would detect its binding by (a) getting the affiinity mask >> from the OS, and then (b) seeing if the bits are set to '1' for all >> available processors. If it is, then you are not bound - there is no >> mechanism available for checking "are the bits set only for the >> processors I asked to be bound to". The OS doesn't track what you >> asked for, it only tracks where you are bound - and a mask with all >> '1's is defined as "unbound". >> >> >> So the reason for my question was simple: a user asked us to "bind" >> their process. If their process checks to see if it is bound, it will >> return "no". The user would therefore be led to believe that OMPI had >> failed to execute their request, when in fact we did execute it - but >> the result was (as Nadia says) a "no-op". >> >> >> After talking with Jeff, I think he has the right answer. It is a >> method we have used elsewhere, so it isn't unexpected behavior. >> Basically, he proposed that we use an mca param to control this >> behavior: >> >> >> * default: generate an error message as the "bind" results in a no-op, >> and this is our current behavior >> >> >> * warn: generate a warning that the binding wound up being a "no-op", >> but continue working >> >> >> * quiet: just ignore it and keep going > > Excellent, I completely agree (though I would have put the 2nd star as > the default behavior, but never mind, I don't want to restart the > discussion ;-) ) I actually went back/forth on that as well - I personally think it might be better to just have warn and quiet, with warn being the default. The warning could be generated with orte_show_help so the messages would be consolidated across nodes. Given that the enhanced paffinity behavior is fairly new, and that no-one has previously raised this issue, I don't think the prior behavior is relevant. Would that make sense? If so, we could extend that to the other binding options for consistency. > > Also this is a good opportunity to fix the other issue I talked about in > the first message in this thread: the tag > "odls-default:could-not-bind-to-socket" does not exist in > orte/mca/odls/default/help-odls-default.txt I'll take that one - my fault for missing it. I'll cross-check the other messages as well. Thanks for catching it! As for your other change: let me think on it. I -think- I understand your logic, but honestly haven't had time to really walk through it properly. Got an ORCM deadline to meet, but hope to break free towards the end of this week. > > Regards, > Nadia >> >> >> Fairly trivial to implement, and Bull could set the default mca param >> file to "quiet" to get what they want. I'm not sure if that's what the >> community wants or not - like I said, it makes no diff to me so long >> as the code logic is understandable. >> >> >> >> On Apr 12, 2010, at 8:27 AM, Terry Dontje wrote: >> >>> Ralph, I guess I am curious why is it that if there is only one >>> socket we cannot bind to it? Does plpa actually error on this or is >>> this a condition we decided was an error at odls? >>> >>> I am somewhat torn on whether this makes sense. On the one hand it >>> is definitely useless as to the result if you allow it. However if >>> you don't allow it and you have a script or running tests on >>> multiple systems it would be nice to have this run because you are >>> not really running into a resource starvation issue. >>> >>> At a minimum I think the error condition/message needs to be spelled >>> out (defined).As to whether we allow binding when only one >>> socket exist I could go either way slightly leaning towards allowing >>> such a specification to work. >>> >>> --td >>> >>> >>> Ralph Castain wrote: Guess I'll jump in here as I finally had a few minutes to look at the code and think about your original note. In fact, I believe your original statement is the source of contention. If someone tells us -bind-to-socket, but there is only one socket, then we really cannot bind them to anything. Any check by their code would reveal that they had not, in fact, been bound - raising questions as to whether or not OMPI is performing the request. Our operating standard has been to error out if the user specifies something we cannot do to avoid that kind of confusion. This is what generated the code in the system today. Now I can see an argument that -bind-to-socket with one socket maybe shouldn't generate an error, but that decision then has to get reflected in other code areas as
Re: [OMPI devel] problem when binding to socket on a single socket node
On Tue, 2010-04-13 at 01:27 -0600, Ralph Castain wrote: > On Apr 13, 2010, at 1:02 AM, Nadia Derbey wrote: > > > On Mon, 2010-04-12 at 10:07 -0600, Ralph Castain wrote: > >> By definition, if you bind to all available cpus in the OS, you are > >> bound to nothing (i.e., "unbound") as your process runs on any > >> available cpu. > >> > >> > >> PLPA doesn't care, and I personally don't care. I was just explaining > >> why it generates an error in the odls. > >> > >> > >> A user app would detect its binding by (a) getting the affiinity mask > >> from the OS, and then (b) seeing if the bits are set to '1' for all > >> available processors. If it is, then you are not bound - there is no > >> mechanism available for checking "are the bits set only for the > >> processors I asked to be bound to". The OS doesn't track what you > >> asked for, it only tracks where you are bound - and a mask with all > >> '1's is defined as "unbound". > >> > >> > >> So the reason for my question was simple: a user asked us to "bind" > >> their process. If their process checks to see if it is bound, it will > >> return "no". The user would therefore be led to believe that OMPI had > >> failed to execute their request, when in fact we did execute it - but > >> the result was (as Nadia says) a "no-op". > >> > >> > >> After talking with Jeff, I think he has the right answer. It is a > >> method we have used elsewhere, so it isn't unexpected behavior. > >> Basically, he proposed that we use an mca param to control this > >> behavior: > >> > >> > >> * default: generate an error message as the "bind" results in a no-op, > >> and this is our current behavior > >> > >> > >> * warn: generate a warning that the binding wound up being a "no-op", > >> but continue working > >> > >> > >> * quiet: just ignore it and keep going > > > > Excellent, I completely agree (though I would have put the 2nd star as > > the default behavior, but never mind, I don't want to restart the > > discussion ;-) ) > > I actually went back/forth on that as well - I personally think it might be > better to just have warn and quiet, with warn being the default. The warning > could be generated with orte_show_help so the messages would be consolidated > across nodes. Given that the enhanced paffinity behavior is fairly new, and > that no-one has previously raised this issue, I don't think the prior > behavior is relevant. > > Would that make sense? If so, we could extend that to the other binding > options for consistency. Sure! Patch proposal attached. Regards, Nadia > > > > > Also this is a good opportunity to fix the other issue I talked about in > > the first message in this thread: the tag > > "odls-default:could-not-bind-to-socket" does not exist in > > orte/mca/odls/default/help-odls-default.txt > > I'll take that one - my fault for missing it. I'll cross-check the other > messages as well. Thanks for catching it! > > As for your other change: let me think on it. I -think- I understand your > logic, but honestly haven't had time to really walk through it properly. Got > an ORCM deadline to meet, but hope to break free towards the end of this week. > > > > > > Regards, > > Nadia > >> > >> > >> Fairly trivial to implement, and Bull could set the default mca param > >> file to "quiet" to get what they want. I'm not sure if that's what the > >> community wants or not - like I said, it makes no diff to me so long > >> as the code logic is understandable. > >> > >> > >> > >> On Apr 12, 2010, at 8:27 AM, Terry Dontje wrote: > >> > >>> Ralph, I guess I am curious why is it that if there is only one > >>> socket we cannot bind to it? Does plpa actually error on this or is > >>> this a condition we decided was an error at odls? > >>> > >>> I am somewhat torn on whether this makes sense. On the one hand it > >>> is definitely useless as to the result if you allow it. However if > >>> you don't allow it and you have a script or running tests on > >>> multiple systems it would be nice to have this run because you are > >>> not really running into a resource starvation issue. > >>> > >>> At a minimum I think the error condition/message needs to be spelled > >>> out (defined).As to whether we allow binding when only one > >>> socket exist I could go either way slightly leaning towards allowing > >>> such a specification to work. > >>> > >>> --td > >>> > >>> > >>> Ralph Castain wrote: > Guess I'll jump in here as I finally had a few minutes to look at the > code and think about your original note. In fact, I believe your > original statement is the source of contention. > > If someone tells us -bind-to-socket, but there is only one socket, then > we really cannot bind them to anything. Any check by their code would > reveal that they had not, in fact, been bound - raising questions as to > whether or not OMPI is performing the request. Our operating standard > has been to er
Re: [OMPI devel] problem when binding to socket on a singlesocket node
On Apr 13, 2010, at 3:27 AM, Ralph Castain wrote: > I actually went back/forth on that as well - I personally think it might be > better to just have warn and quiet, with warn being the default. The warning > could be generated with orte_show_help so the messages would be consolidated > across nodes. Given that the enhanced paffinity behavior is fairly new, and > that no-one has previously raised this issue, I don't think the prior > behavior is relevant. Works for me! -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
[OMPI devel] inquiring about openmpi
dear teachers:
LAM/MPI constructs communication channel via process location identified by
type struct _gps in LAM universe. struct _gps{
int4 gps_node;
int4 gps_pid;
int4 gps_idx;
int4 gps_granks;
}. In order to know of other processes, a mpi process in openmpi must possess
data structure similar to struct _gps. My motivation is to achieve process
migration, so I want to find this structure.
thanks
luyang dong
4.13
2010
Re: [OMPI devel] inquiring about openmpi
This is the list for Open MPI developers, not LAM/MPI. We ex-LAM/MPI
developers are on this list, but we're not inclined to answer LAM questions on
this list. :-) Heck, we're not inclined to answer LAM questions at all, if we
can avoid it. :-) :-) :-)
Specifically: LAM is pretty dead. We haven't worked on LAM in *years*
(literally). Indeed, it's probably not useful to try to implement process
migration with LAM/MPI.
On Apr 13, 2010, at 10:23 AM, luyang dong wrote:
> dear teachers:
>
> LAM/MPI constructs communication channel via process location identified by
> type struct _gps in LAM universe. struct _gps{
> int4 gps_node;
> int4 gps_pid;
> int4 gps_idx;
> int4 gps_granks;
> }. In order to know of other processes, a mpi process in openmpi must possess
> data structure similar to struct _gps. My motivation is to achieve process
> migration, so I want to find this structure.
>
>
>
> thanks
>
> luyang dong
>
> 4.13 2010
>
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel
--
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/
Re: [OMPI devel] inquiring about openmpi
Yikes! I totally mis-read your email as an inquiry about LAM/MPI. Looking at
it closer, I see that you are asking about Open MPI, but using LAM as a
reference. Sorry about that!
You probably want to look in ompi/proc/proc.h.
On Apr 13, 2010, at 10:23 AM, luyang dong wrote:
> dear teachers:
>
> LAM/MPI constructs communication channel via process location identified by
> type struct _gps in LAM universe. struct _gps{
> int4 gps_node;
> int4 gps_pid;
> int4 gps_idx;
> int4 gps_granks;
> }. In order to know of other processes, a mpi process in openmpi must possess
> data structure similar to struct _gps. My motivation is to achieve process
> migration, so I want to find this structure.
>
>
>
> thanks
>
> luyang dong
>
> 4.13 2010
>
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel
--
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/
[OMPI devel] 1.4.2rc1 available for test
Hi folks I rolled the 1.4.2 release candidate 1 tarball today. It is available on the web site: http://www.open-mpi.org/software/ompi/v1.4/ There is no rpm version available - just tarballs. Please give it a whirl and let us know of any problems. Thanks Ralph
