Re: [OMPI devel] Using ompi_proc_t's proc_name.vpid as Universal rank
Hi, Thanks for the clarification. So, now I am wondering how rank information regarding processes in MPI_COMM_WORLD are assigned. Is there a table that stores unique integer values for processess or is rank assignment done in some other manner? Thanks, Sajjad Tabib Tim Prins Sent by: devel-boun...@open-mpi.org 11/30/07 07:22 AM Please respond to Open MPI Developers To Open MPI Developers cc Subject Re: [OMPI devel] Using ompi_proc_t's proc_name.vpid as Universal rank Hi Sajjad, The vpid is not unique. If you do a comm_spawn then the newly launched processes will have a new jobid, and their vpids will start at 0. So the whole process name is unique. However, there is talk now of being able to join 2 jobs that were started completely independently. This may lead to the point where a process name is no longer unique, however this work appears to be a ways out and as far as I know no decisions have been made on it yet. Hope this helps, Tim Sajjad Tabib wrote: > > Hello, > > I have a proprietary transport/messaging layer that sits below an MTL > component. This transport layer requires OpenMPI to assign it a rank > that is unique and specific to that process and will not change from > execution to termination. In a way, I am trying to find a one-one > correspondence between a process's universal rank in OpenMPI and this > transport layer. I began looking at ompi_proc_t from different processes > and seemingly found a unique identifier, proc_name.vpid. Consequently, I > assigned the ranks to each process in my transport layer based on the > value of the local vpid of each process. > I have not tested this thoroughly, but it has been working so far. > Although, I would like to make sure that this is a good approach, or > know, at least, whether if there are other ways to do this. I would > appreciate it if you could leave me feedback or give suggestions on how > to assign universal ranks to a proprietary transport software. > > Thanks for your help, > > Sajjad Tabib > > > > > ___ > devel mailing list > de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/devel ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel
[OMPI devel] Using ompi_proc_t's proc_name.vpid as Universal rank
Hello, I have a proprietary transport/messaging layer that sits below an MTL component. This transport layer requires OpenMPI to assign it a rank that is unique and specific to that process and will not change from execution to termination. In a way, I am trying to find a one-one correspondence between a process's universal rank in OpenMPI and this transport layer. I began looking at ompi_proc_t from different processes and seemingly found a unique identifier, proc_name.vpid. Consequently, I assigned the ranks to each process in my transport layer based on the value of the local vpid of each process. I have not tested this thoroughly, but it has been working so far. Although, I would like to make sure that this is a good approach, or know, at least, whether if there are other ways to do this. I would appreciate it if you could leave me feedback or give suggestions on how to assign universal ranks to a proprietary transport software. Thanks for your help, Sajjad Tabib
Re: [OMPI devel] Question regarding MCA_PML_CM_SEND_REQUEST_INIT_COMMON
Hi Jeff, Now that you mention it, I believe you are right. In fact, I did not know that I needed to set the req_status.MPI_ERROR in my MTL. I looked at the mx mtl and realized that req_status.MPI_ERROR is getting set in their progress function. So, in general, when do you set the req_status.MPI_ERROR? After a send/recv has completed successfully? Thank You, Sajjad Tabib Jeff Squyres Sent by: devel-boun...@open-mpi.org 10/31/07 07:29 AM Please respond to Open MPI Developers To Open MPI Developers cc Subject Re: [OMPI devel] Question regarding MCA_PML_CM_SEND_REQUEST_INIT_COMMON I haven't done any work in the cm pml so I can't definitively answer your question, but wouldn't you set req_status.MPI_ERROR in your MTL depending on the result of the request? On Oct 29, 2007, at 9:20 AM, Sajjad Tabib wrote: > > Hi, > > I was issuing an MPI_Bcast in a sample program and was hitting an > unknown error; at least that was what MPI was telling me. I traced > through the code to find my error and came upon > MCA_PML_CM_REQUEST_INIT_COMMON macro function in pml_cm_sendreq.h. > I looked at the function and noticed that in this function the > elements of req_status were getting initialized; however, > req_status.MPI_ERROR was not. I thought that maybe MPI_ERROR must > also require initialization because if the value of MPI_ERROR was > some arbitrary value not equal to MPI_SUCCESS then my program will > definitely die. Unless, MPI_ERROR is propragating from upper layers > to signify an error, but I wasn't sure. Anyway, I assumed that > MPI_ERROR was not propagating from upper layers, so then I set > req_status.MPI_ERROR to MPI_SUCCUSS and reran my test program. My > program worked. Now, having gotten my program to work, I thought I > should run this by you to make sure that MPI_ERROR was not > propagating from upper layers. Is it ok that I did a: > "(req_send)->req_base.req_ompi.req_status.MPI_ERROR = MPI_SUCCESS;" > in MCA_PML_CM_REQUEST_INIT_COMMON? > > Thank You, > > Sajjad Tabib > ___ > devel mailing list > de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/devel -- Jeff Squyres Cisco Systems ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel
[OMPI devel] Question regarding MCA_PML_CM_SEND_REQUEST_INIT_COMMON
Hi, I was issuing an MPI_Bcast in a sample program and was hitting an unknown error; at least that was what MPI was telling me. I traced through the code to find my error and came upon MCA_PML_CM_REQUEST_INIT_COMMON macro function in pml_cm_sendreq.h. I looked at the function and noticed that in this function the elements of req_status were getting initialized; however, req_status.MPI_ERROR was not. I thought that maybe MPI_ERROR must also require initialization because if the value of MPI_ERROR was some arbitrary value not equal to MPI_SUCCESS then my program will definitely die. Unless, MPI_ERROR is propragating from upper layers to signify an error, but I wasn't sure. Anyway, I assumed that MPI_ERROR was not propagating from upper layers, so then I set req_status.MPI_ERROR to MPI_SUCCUSS and reran my test program. My program worked. Now, having gotten my program to work, I thought I should run this by you to make sure that MPI_ERROR was not propagating from upper layers. Is it ok that I did a: "(req_send)->req_base.req_ompi.req_status.MPI_ERROR = MPI_SUCCESS;" in MCA_PML_CM_REQUEST_INIT_COMMON? Thank You, Sajjad Tabib
Re: [OMPI devel] PML cm and heterogeneous support
Hi Brian,
I have actually created a new MTL, in which I have added heterogeneous
support. To experiment whether CM worked in this environment, I took out
the safeguards that prevented one to use CM in a heterogeneous
environment. Miraculously, things have been working so far. I haven't
examined data integrity to an extent that I could say everything works
perfectly, but with MPI_INTS, I do not have any endian problems. Now,
based on my initial tests, I have came to the understanding that the PML
CM safeguard against heterogeneous environments was a mechanism to prevent
users from using existing MTLs. But, if an MTL supports heterogeneous
communication, then it is possible to use the CM component. What is your
take on this?
Anyways, going back to the datatype usage. When you say that: "it's known
the datatype usage in the CM PML won't support heterogeneous operation"
could you please breifly explain this in more detail? I have been using
ompi_mtl_datatype_pack and ompi_mtl_datatype_unpack, which use
ompi_convertor_pack and ompi_convertor_unpack, for data packing. Do you
mean that these functions will not work correctly?
Thank You,
Sajjad Tabib
Brian Barrett
Sent by: devel-boun...@open-mpi.org
10/24/07 10:04 PM
Please respond to
Open MPI Developers
To
Open MPI Developers
cc
Subject
Re: [OMPI devel] PML cm and heterogeneous support
No, it's because the CM PML was never designed to be used in a
heterogeneous environment :). While the MX BTL does support
heterogeneous operations (at one point, I believe I even had it
working), none of the MTLs have ever been tested in heterogeneous
environments and it's known the datatype usage in the CM PML won't
support heterogeneous operation.
Brian
On Oct 24, 2007, at 6:21 PM, Jeff Squyres wrote:
> George / Patrick / Rich / Christian --
>
> Any idea why that's there? Is that because portals, MX, and PSM all
> require homogeneous environments?
>
>
> On Oct 18, 2007, at 3:59 PM, Sajjad Tabib wrote:
>
>>
>> Hi,
>>
>> I am tried to run an MPI program in a heterogeneous environment
>> using the pml cm component. However, open mpi returned with an
>> error message indicating that PML add procs returned "Not
>> supported". I dived into the cm code to see what was wrong and I
>> came upon the code below, which basically shows that if the
>> processes are running on different architectures, then return "not
>> supported". Now, I'm wondering whether my interpretation is correct
>> or not. Is it true that the cm component does not support a
>> heterogeneous environment? If so, will the developers support this
>> in the future? How could I get around this while still using the cm
>> component? What will happen if I rebuilt openmpi without these
>> statements?
>>
>> I would appreciate your help.
>>
>> Code:
>>
>> mca_pml_cm_add_procs(){
>>
>> #if OMPI_ENABLE_HETEROGENEOUS_SUPPORT
>> 107 for (i = 0 ; i < nprocs ; ++i) {
>> 108 if (procs[i]->proc_arch != ompi_proc_local()-
>> >proc_arch) {
>> 109 return OMPI_ERR_NOT_SUPPORTED;
>> 110 }
>> 111 }
>> 112 #endif
>> .
>> .
>> .
>> }
>>
>> Sajjad Tabib
>> ___
>> devel mailing list
>> de...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>
>
> --
> Jeff Squyres
> Cisco Systems
>
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel
___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel
[OMPI devel] PML cm and heterogeneous support
Hi,
I am tried to run an MPI program in a heterogeneous environment using the
pml cm component. However, open mpi returned with an error message
indicating that PML add procs returned "Not supported". I dived into the
cm code to see what was wrong and I came upon the code below, which
basically shows that if the processes are running on different
architectures, then return "not supported". Now, I'm wondering whether my
interpretation is correct or not. Is it true that the cm component does
not support a heterogeneous environment? If so, will the developers
support this in the future? How could I get around this while still using
the cm component? What will happen if I rebuilt openmpi without these
statements?
I would appreciate your help.
Code:
mca_pml_cm_add_procs(){
#if OMPI_ENABLE_HETEROGENEOUS_SUPPORT
107 for (i = 0 ; i < nprocs ; ++i) {
108 if (procs[i]->proc_arch != ompi_proc_local()->proc_arch) {
109 return OMPI_ERR_NOT_SUPPORTED;
110 }
111 }
112 #endif
.
.
.
}
Sajjad Tabib
[OMPI devel] Module Design Concept
Hi, I'm implementing a new MTL component that uses message queues to keep track of posted and unexpected messages. I intended to do this by creating two global queues, one for posted and one for unexpected, until I found that the portals MTL uses a different approach in their queue implemenation. The portal code uses handles to the queues from inside their mca_mtl_portals_module_t to post messages. I couldn't help but wonder, why are the queue handles here? What are the design implications of defining these handle queues in this module struct rather than globally defining them? I'm an Open MPI newbie and sort of confused on the modular approach taken here and was hoping somebody could point out the pros and the cons of the two approaches. I guess my next question would be: In general, what would you put into a module struct and what wouldn't you? I will appreciate any pointers that you could give me to help me understand this concept. Thanks in advance, Sajjad Tabib
[OMPI devel] Sajjad Tabib is out of the office.
I will be out of the office starting 09/21/2007 and will not return until 10/01/2007. I will respond to your message when I return.
Re: [OMPI devel] Adding a new component
Hi Aurelien, Thank you for the pointers. I was able to plug in a component to an existing framework. Thanks again, Sajjad Aurelien Bouteiller Sent by: devel-boun...@open-mpi.org 09/08/07 01:34 PM Please respond to Open MPI Developers To Open MPI Developers cc Subject Re: [OMPI devel] Adding a new component Hi Sajjad, First it will depend wether you are writing a new component in an existing framework (let say you are writing a new BTL for a new type of interconnect) or a totally new framework (you want to have a family of component that can manage a totally new functionality in Open MPI). In each Framework there is a "base" which take care of the component selection process. If you are just adding a component, you will just need to provide a mca_mycomponent_init(bool enable_progress_threads, bool enable_mpi_threads) as described in the mca_component_t structure. The mca_framework_base_select will then take care of everything for you. If you want to add a new framework you'll have to create a selection function by yourself (all along with a full bunch of other functions to populate the base of the framework). I'll give you more details on this if it is relevant for you, just ask. Aurelien Le 7 sept. 07 à 17:21, Sajjad Tabib a écrit : Hi, I am a complete newbie to Open MPI internals and just began browsing the code and reading up on slides and papers. From what I have read, I learned that I have to create a new component. What I do not know is how to make MPI aware of it or should I say make MPI open and select my component. I found a set of slides that briefly went over adding components. For example, it briefly described that I must add PARAM_INIT_FILE and PARAM_CONFIG_FILES options in configure.params, but I'm not sure what these mean. Does anybody know of any tutorials/documents that could help me with this? Any help is greatly appreciated. S Tabib ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel
[OMPI devel] Adding a new component
Hi, I am a complete newbie to Open MPI internals and just began browsing the code and reading up on slides and papers. From what I have read, I learned that I have to create a new component. What I do not know is how to make MPI aware of it or should I say make MPI open and select my component. I found a set of slides that briefly went over adding components. For example, it briefly described that I must add PARAM_INIT_FILE and PARAM_CONFIG_FILES options in configure.params, but I'm not sure what these mean. Does anybody know of any tutorials/documents that could help me with this? Any help is greatly appreciated. S Tabib
