Re: [OMPI devel] Help regarding Openmpi source code

[Clement Foyer]

Hi,

The reason you cannot find it is because it actually is not define as such. The 
PMPI interface is defined as the profiling interface. The principle is you have 
the real symbol in your shared library that is PMPI_Send, with a weak alias 
being MPI_Send. So, if a third party want to intercept calls to MPI_Send (as an 
example, to count how many times this function is called), he just need to 
provide its own library that present the MPI_Send symbol, does its work (e.g., 
increment some counter to keep track of how many times this function was called 
by the user), and then call the PMPI_Send function provided by Open MPI. As the 
ompi symbol for MPI_Send is defined as weak, it won’t collide with the already 
defined symbol and thus simply be ignored instead of rising a warning at link 
time. 

Note: in order to have the third party library loaded in time such as its 
MPI_Send symbol is considered and not the Open MPI one, you need to set your 
LD_PRELOAD variable accordingly.

You would end up with user code calling MPI_Send provided by third party, 
calling PMPI_Send provided by the Open MPI library. Without this third party 
library, you have user code that call MPI_Send which is actually an alias for 
PMPI_Send provided by Open MPI library.

There might be some other implications, but I may let the people more 
proficient than myself explain them :)

Regards,
Clément

On 04/03/2019, 16:45, "devel on behalf of vishakha patil" 
 wrote:

Hi,

Greetings for the day! 

I am a MTech(Computer) student of Savitribai Phule Pune University, 
Maharashtra, India.
For my project purpose I have downloaded nightly snapshot tarball of openmpi 
v4.0.x series.
It got build successfully. But while traversing the code (manually as well as 
using cscope) I am not able to find the implementation of PMPI_* (PMPI_Send 
etc) which is getting called from MPI_Send function of send.c file.
I have gone through the readme and make files but not able to find it. Could 
you please help me with the same? 
It is required to get complete understanding of openmpi algorithms as it is 
part of my MTech project.

Please let me know if any other details required. Thank you!

Regards,
Vishakha

___ devel mailing list 
devel@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/devel


___
devel mailing list
devel@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/devel

Re: [OMPI devel] OSC module change

[Clement FOYER]

Hi devels,

In fact the communicator's group was already retained in the window 
structure. So everything was already in place. I pushed the last 
modifications, and everything seems ready to be merged in PR#4527.


Jeff, the fixup commits are squashed :)

Clément


On 11/30/2017 12:00 AM, Barrett, Brian via devel wrote:
The group is the easiest way to do the mapping from rank in window to 
ompi_proc_t, so it’s safe to say every window will have one (also, as 
a way of holding a reference to the ompi_proc_t).  So I think it’s 
safe to say that every OSC module has a group handle somewhere 
(directly or through the communicator).


Remember that in some implementations of the MTL, a communicator ID is 
a precious resource.  I don’t know where Portals 4 falls right now, 
but in various of the 64 bit tag matching implementations, it’s been 
as low as 4k communicators.  There’s no need for a cid if all you hold 
is a group reference.  Plus, a communicator has a bunch of other state 
(collective modules handles, etc.) that aren’t necessarily needed by a 
window.


Brian

On Nov 29, 2017, at 5:57 AM, Clement FOYER <mailto:clement.fo...@gmail.com>> wrote:


Hi Brian,

Even if I see your point, I don't think a user request de free the 
communicator should necesserily lead to the communicator being 
deleted, only released from one hold, and available to be disposed by 
the library. I don't see objection to have the library keep a grab on 
these communicators, as the user give a handle to the actual object.


I do agree the point of asking if we want to keep only information 
relevant to all OSC components. Nevertheless, what would the 
difference be between holding the complete communicator and holding 
the group only? Is group the smallest part common to every component?


Clément


On 11/28/2017 07:46 PM, Barrett, Brian via devel wrote:

The following is perfectly legal:

MPI_Comm_dup(some_comm, &tmp_comm);
MPI_Win_create(…., tmp_comm, &window);
MPI_Comm_free(tmp_comm);



So I don’t think stashing away a communicator is the solution.  Is a 
group sufficient?  I think any rational reading of the standard 
would lead to windows needing to hold a group reference for the life 
of the window.  I’d be ok putting a group pointer in the base 
window, if that would work?


Brian

On Nov 28, 2017, at 10:19 AM, George Bosilca <mailto:bosi...@icl.utk.edu>> wrote:


Hi Brian,

Let me first start with explaining why we need the communicator. We 
need to translate local to global rank (aka. rank in your 
MPI_COMM_WORLD), so that the communication map we provide make 
sense. The only way today is to go back to a communicator and then 
basically translate a rank between this communicator and 
MPI_COMM_WORLD. We could use the gid, but then we have a hash table 
lookup for every operation.


While a communicator is not needed internally by an OSC, in MPI 
world all windows start with a communicator. This is the reason why 
I was proposing the change, not to force a window to create or hold 
a communicator, but simply because the existence of a communicator 
linked to the window is more of less enforced by the MPI standard.


  George.



On Tue, Nov 28, 2017 at 1:02 PM, Barrett, Brian via devel 
mailto:devel@lists.open-mpi.org>> wrote:


The objection I have to this is that it forces an
implementation where every one-sided component is backed by a
communicator.  While that’s the case today, it’s certainly not
required.  If you look at Portal 4, for example, there’s one
collective call outside of initialization, and that’s a barrier
in MPI_FENCE.  The SM component is the same way and given some
of the use cases for shared memory allocation using the SM
component, it’s very possible that we’ll be faced with a
situation where creating a communicator per SM region is too
expensive in terms of overall communicator count.

I guess a different question would be what you need the
communicator for.  It shouldn’t have any useful semantic
meaning, so why isn’t a silent implementation detail for the
monitoring component?

Brian



On Nov 28, 2017, at 8:45 AM, George Bosilca
mailto:bosi...@icl.utk.edu>> wrote:

Devels,

We would like to change the definition of the OSC module to
move the communicator one level up from the different module
structures into the base OSC module. The reason for this, as
well as a lengthy discussion on other possible solutions can
be found in https://github.com/open-mpi/ompi/pull/4527
<https://github.com/open-mpi/ompi/pull/4527>.

We need to take a decision on this asap, to prepare the PR for
the 3.1. Please comment asap.

  George.

___
devel mailing list
devel@lists.open-mpi.org <mailto:devel@lists.open-mpi.org>
https://lists.open-mpi.org/mailman/list

Re: [OMPI devel] OSC module change

[Clement FOYER]

Hi Brian,

Even if I see your point, I don't think a user request de free the 
communicator should necesserily lead to the communicator being deleted, 
only released from one hold, and available to be disposed by the 
library. I don't see objection to have the library keep a grab on these 
communicators, as the user give a handle to the actual object.


I do agree the point of asking if we want to keep only information 
relevant to all OSC components. Nevertheless, what would the difference 
be between holding the complete communicator and holding the group only? 
Is group the smallest part common to every component?


Clément


On 11/28/2017 07:46 PM, Barrett, Brian via devel wrote:

The following is perfectly legal:

MPI_Comm_dup(some_comm, &tmp_comm);
MPI_Win_create(…., tmp_comm, &window);
MPI_Comm_free(tmp_comm);



So I don’t think stashing away a communicator is the solution.  Is a 
group sufficient?  I think any rational reading of the standard would 
lead to windows needing to hold a group reference for the life of the 
window.  I’d be ok putting a group pointer in the base window, if that 
would work?


Brian

On Nov 28, 2017, at 10:19 AM, George Bosilca > wrote:


Hi Brian,

Let me first start with explaining why we need the communicator. We 
need to translate local to global rank (aka. rank in your 
MPI_COMM_WORLD), so that the communication map we provide make sense. 
The only way today is to go back to a communicator and then basically 
translate a rank between this communicator and MPI_COMM_WORLD. We 
could use the gid, but then we have a hash table lookup for every 
operation.


While a communicator is not needed internally by an OSC, in MPI world 
all windows start with a communicator. This is the reason why I was 
proposing the change, not to force a window to create or hold a 
communicator, but simply because the existence of a communicator 
linked to the window is more of less enforced by the MPI standard.


  George.



On Tue, Nov 28, 2017 at 1:02 PM, Barrett, Brian via devel 
mailto:devel@lists.open-mpi.org>> wrote:


The objection I have to this is that it forces an implementation
where every one-sided component is backed by a communicator. 
While that’s the case today, it’s certainly not required.  If you
look at Portal 4, for example, there’s one collective call
outside of initialization, and that’s a barrier in MPI_FENCE. 
The SM component is the same way and given some of the use cases
for shared memory allocation using the SM component, it’s very
possible that we’ll be faced with a situation where creating a
communicator per SM region is too expensive in terms of overall
communicator count.

I guess a different question would be what you need the
communicator for.  It shouldn’t have any useful semantic meaning,
so why isn’t a silent implementation detail for the monitoring
component?

Brian



On Nov 28, 2017, at 8:45 AM, George Bosilca mailto:bosi...@icl.utk.edu>> wrote:

Devels,

We would like to change the definition of the OSC module to move
the communicator one level up from the different module
structures into the base OSC module. The reason for this, as
well as a lengthy discussion on other possible solutions can be
found in https://github.com/open-mpi/ompi/pull/4527
.

We need to take a decision on this asap, to prepare the PR for
the 3.1. Please comment asap.

  George.

___
devel mailing list
devel@lists.open-mpi.org 
https://lists.open-mpi.org/mailman/listinfo/devel




___
devel mailing list
devel@lists.open-mpi.org 
https://lists.open-mpi.org/mailman/listinfo/devel







___
devel mailing list
devel@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/devel


___
devel mailing list
devel@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/devel

Re: [OMPI devel] about ompi_datatype_is_valid

[Clement FOYER]

As George pointed out earlier, there is no construct in C able to tell 
you if a random number is a valid C pointer, i.e., is a valid memory 
address. -1, once casted to a pointer, would correspond to the adress 
0x...f .


It would probably be better to ajust the initial code in order to use 
MPI_DATATYPE_NULL, which would be more standard compliant, and less 
implementation dependant. Also, this stands as well for any MPI_*_NULL, 
as these handlers are opaque, and corresponding to differents types 
depending on the library you are using.


The standard does not define what these handlers are corresponding to, 
letting the implementor defining these symbols. So using them is the 
only way to ensure portability through implementations.


Clément


On 06/02/2017 04:46 PM, Dahai Guo wrote:
Is it easy to update *ompi_datatype_is_valid* to judge invalid 
datatypes, such as -1?


Dahai

On Fri, Jun 2, 2017 at 9:00 AM, Gilles Gouaillardet 
mailto:gilles.gouaillar...@gmail.com>> 
wrote:


MPI_Datatype is a opaque handler, and -1 is a (signed) integer, so
what you are trying is very likely to have an undefined behavior per
the standard.

if this seems to work with MPICH, this is not portable anyway, and
will very likely cause a crash with OpenMPI


Cheers,

Gilles

On Fri, Jun 2, 2017 at 10:41 PM, Dahai Guo mailto:dahai@gmail.com>> wrote:
> so you are saying that a user should NOT define send/recv data
type as -1,
> in openmpi?
>
> Dahai
>
> On Thu, Jun 1, 2017 at 6:59 PM, Gilles Gouaillardet
mailto:gil...@rist.or.jp>>
> wrote:
>>
>> +1
>>
>>
>> MPI_Datatype is an opaque handler, and in Open MPI, this is an
>> ompi_datatype_t *
>>
>> so we can only test for NULL pointers or MPI_DATATYPE_NULL that
cannot be
>> used per the standard.
>>
>>
>> fwiw, and iirc, MPICH made an other design choice and
MPI_Datatype is a
>> number, so the mpich equivalent of ompi_datatype_is_valid()
>>
>> might be able to handle random values without crashing.
>>
>>
>> Cheers,
>>
>>
>> Gilles
>>
>>
>> On 6/2/2017 7:36 AM, George Bosilca wrote:
>>>
>>> You have to pass it an allocated datatype, and it tells you if the
>>> pointer object is a valid MPI datatype for communications (aka
it has a
>>> corresponding type with a well defined size, extent and
alignment).
>>>
>>> There is no construct in C able to tell you if a random number
if a valid
>>> C "object".
>>>
>>>   George.
>>>
>>>
>>> On Thu, Jun 1, 2017 at 5:42 PM, Dahai Guo mailto:dahai@gmail.com>
>>> >> wrote:
>>>
>>> Hi,
>>>
>>> if I insert following lines somewhere openmpi, such
>>> as ompi/mpi/c/iscatter.c
>>>
>>>   printf(" --- in MPI_Iscatter\n");
>>> //MPI_Datatype dt00 = (MPI_Datatype) MPI_INT;
>>> *MPI_Datatype dt00 = (MPI_Datatype) -1;*
>>> if(*!ompi_datatype_is_valid(dt00)* ) {
>>>   printf(" --- dt00 is NOT valid \n");
>>> }
>>>
>>> The attached test code will give the errors:
>>>
>>> *** Process received signal ***
>>> Signal: Segmentation fault (11)
>>> Signal code: Address not mapped (1)
>>> Failing at address: 0xf
>>> [ 0] [0x3fff9d480478]
>>> ...
>>>
>>> Is it a bug in the function *ompi_datatype_is_valid(..) *?
or I
>>> miss something?
>>>
>>> Dahai
>>>
>>> ___
>>> devel mailing list
>>> devel@lists.open-mpi.org 
>
>>> https://rfd.newmexicoconsortium.org/mailman/listinfo/devel

>>>   
 >
>>>
>>>
>>>
>>>
>>> ___
>>> devel mailing list
>>> devel@lists.open-mpi.org 
>>> https://rfd.newmexicoconsortium.org/mailman/listinfo/devel

>>
>>
>> ___
>> devel mailing list
>> devel@lists.open-mpi.org 
>> https://rfd.newmexicoconsortium.org/mailman/listinfo/devel

>
>
>
> ___
> devel mailing list
> devel@lists.open-mpi.org 
> https://rfd.newmexicoconsortium.org/mailman/listinfo/devel

[OMPI devel] Stack Overflow with osc_pt2pt _frag_alloc

[Clement FOYER]

Hi everyone,

While testing localy on my computer code using one-sided operations, the 
selected module is the pt2pt OSC component, which always crashes with a 
segmentation fault when the stack overflows.


From what I've seen, it occurs when ompi_osc_pt2pt_frag_alloc is 
called. This function recursively call itself as first statement, 
without modifying any argument. This function is defined in 
ompi/mca/osc/pt2pt/osc_pt2pt_frag.h line 169.


Is it the normal behaviour? How is it supposed to stop from recursively 
call itself?


Cheers,

Clement FOYER

___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel

[OMPI devel] Stack Overflow with mca_osc_pt2pt _frag_alloc

[Clement FOYER]

Hi everyone,

While testing localy on my computer code using one-sided operations, the 
selected module is the pt2pt OSC component, which always crashes with a 
segmentation fault when the stack overflows.


From what I've seen, it occurs when ompi_osc_pt2pt_frag_alloc is 
called. This function recursively call itself as first statement, 
without modifying any argument. This function is defined in 
ompi/mca/osc/pt2pt/osc_pt2pt_frag.h line 169.


Is it the normal behaviour? How is it supposed to stop from recursively 
call itself?


Cheers,

Clement FOYER

___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel

[OMPI devel] PVAR definition - Variable type regarding the PVAR class

[Clement FOYER]

Hi everyone,

I'm facing an issue with the typing of the performance variables I try 
to register.


They are basically counters of type MCA_BASE_PVAR_CLASS_AGGREGATE, 
MCA_BASE_PVAR_CLASS_COUNTER, or MCA_BASE_PVAR_CLASS_SIZE depending on 
the case. In order to silence some warnings when using 
opal_atomic_add_*() functions they were switched from uint64_t to 
int64_t. But doing so, we loose some consistency in the code. It was 
then decided to move from int64_t to size_t, in order to keep 
consistency across every functions of our component's API.


The problem the following : I can't register properly my variables 
anymore, because of the restrictions over the types as defined in the 
MPI3.1 standard (Sec. 14.3.7, p.580), I can't define these variable as 
MCA_BASE_VAR_TYPE_SIZE_T, as they are not explicitly authorized. 
However, the SIZE_T type is not even allowed as type for performance 
variables, as it is not defined in the list given Section 14.3.5 p.571. 
So should this type never to be used for defining performance variables, 
and thus it should not be given as part of the enum mca_base_var_type_t 
type, or can it be considered as an alias for unsigned long or unsigned 
long long, and thus it should be possible to use them in the same cases 
unsigned long int and unsigned long long int are ?


Thenk you in advance for your answer,

Clement FOYER

___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel

Re: [OMPI devel] Minor typo in opal/class/opal_list.h

[Clement FOYER]

Thank you Gilles for your answer.

Pull request #3023 added.

Cheers,

Clément FOYER


On 02/23/2017 03:35 PM, gil...@rist.or.jp wrote:

Thanks Clement,

the best way is to make a pull request.
your commit must contain a
Signed-off-by: ...
line in order to be considered for inclusion.
and that can be achieved wirh
git commit -s

signing off a commit has some legal implications, so i encourage you to
read the FAQ
(and make sure your employer is ok with that if appropriate)

Cheers,

Gilles

- Original Message -

Hi everyone,

I found a typo in the comment for the opal_list_item_compare_fn_t
typedef (opal/class/opal_list.h:877). It says the functions should
return "11" instead of "-1" for a < b. Should I open a issue on the
github, or is there already a thread where to add minor typo mistakes,
or should a simple comment be enough?

Cheers,

Clément FOYER

___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel

___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel


___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel

[OMPI devel] Minor typo in opal/class/opal_list.h

[Clement FOYER]

Hi everyone,

I found a typo in the comment for the opal_list_item_compare_fn_t 
typedef (opal/class/opal_list.h:877). It says the functions should 
return "11" instead of "-1" for a < b. Should I open a issue on the 
github, or is there already a thread where to add minor typo mistakes, 
or should a simple comment be enough?


Cheers,

Clément FOYER

___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel

Re: [OMPI devel] Problem with BTL while allocating window

[Clement FOYER]

Update :

From what I've tracked, while initializing the osc_rdma module, there 
is a btl selected, whose endpoint can't be found back when calling 
ompi_osc_rdma_peer_btl_endpoint().


It seems like that for even peers, the available btl endpoints are tcp, 
even though we only find openib and ugni in ompi_osc_rdma_btl_names.


(gdb) p ompi_osc_rdma_btl_names
$29 = 0x7981e0 "openib,ugni"

(gdb) p 
((mca_btl_base_selected_module_t*)(mca_btl_base_modules_initialized.opal_list_sentinel.opal_list_next)).btl_module->btl_component->btl_version.mca_component_name
$39 = "self", '\000' 
(gdb) p 
((mca_btl_base_selected_module_t*)(mca_btl_base_modules_initialized.opal_list_sentinel.opal_list_next)).btl_module
$40 = (mca_btl_base_module_t *) 0x7fffed102100 

(gdb) p 
((mca_btl_base_selected_module_t*)(mca_btl_base_modules_initialized.opal_list_sentinel.opal_list_next.opal_list_next)).btl_module->btl_component->btl_version.mca_component_name
$41 = "openib", '\000' 
(gdb) p 
((mca_btl_base_selected_module_t*)(mca_btl_base_modules_initialized.opal_list_sentinel.opal_list_next.opal_list_next)).btl_module
$42 = (mca_btl_base_module_t *) 0x759e30

(gdb) p 
((mca_btl_base_selected_module_t*)(mca_btl_base_modules_initialized.opal_list_sentinel.opal_list_next.opal_list_next.opal_list_next)).btl_module->btl_component->btl_version.mca_component_name
$43 = "sm", '\000' 
(gdb) p 
((mca_btl_base_selected_module_t*)(mca_btl_base_modules_initialized.opal_list_sentinel.opal_list_next.opal_list_next.opal_list_next)).btl_module
$44 = (mca_btl_base_module_t *) 0x7fffec89e200 

(gdb) p 
((mca_btl_base_selected_module_t*)(mca_btl_base_modules_initialized.opal_list_sentinel.opal_list_next.opal_list_next.opal_list_next.opal_list_next)).btl_module->btl_component->btl_version.mca_component_name
$46 = "tcp", '\000' 
(gdb) p 
((mca_btl_base_selected_module_t*)(mca_btl_base_modules_initialized.opal_list_sentinel.opal_list_next.opal_list_next.opal_list_next.opal_list_next)).btl_module
$47 = (mca_btl_base_module_t *) 0x76cab0

(gdb) p 
((mca_btl_base_selected_module_t*)(mca_btl_base_modules_initialized.opal_list_sentinel.opal_list_next.opal_list_next.opal_list_next.opal_list_next.opal_list_next)).btl_module->btl_component->btl_version.mca_component_name
$48 = "tcp", '\000' 
(gdb) p 
((mca_btl_base_selected_module_t*)(mca_btl_base_modules_initialized.opal_list_sentinel.opal_list_next.opal_list_next.opal_list_next.opal_list_next.opal_list_next)).btl_module
$49 = (mca_btl_base_module_t *) 0x72a680

(gdb) p 
((mca_btl_base_selected_module_t*)(mca_btl_base_modules_initialized.opal_list_sentinel.opal_list_next.opal_list_next.opal_list_next.opal_list_next.opal_list_next.opal_list_next)).btl_module->btl_component->btl_version.mca_component_name
$50 = "vader", '\000' 
(gdb) p 
((mca_btl_base_selected_module_t*)(mca_btl_base_modules_initialized.opal_list_sentinel.opal_list_next.opal_list_next.opal_list_next.opal_list_next.opal_list_next.opal_list_next)).btl_module
$51 = (mca_btl_base_module_t *) 0x7fffec275200 

(gdb) p 
((mca_btl_base_selected_module_t*)(mca_btl_base_modules_initialized.opal_list_sentinel.opal_list_next.opal_list_next.opal_list_next.opal_list_next.opal_list_next.opal_list_next.opal_list_next)).btl_module
$52 = (mca_btl_base_module_t *) 0x0

Sorry for the noise in your mail boxes. I thought it may have been 
valuable informations to know where these pointers point to.


Clement FOYER

 


On 02/02/2017 11:17 AM, Clement FOYER wrote:

Hi everyone,

I've been facing issues with the creations of windows 
(MPI_Win_create). Maybe it's an already known issue, or maybe you will 
be able to tell me where to check to find the problem.


I've been developping some benchmark to evaluate the overhead of a 
monitoring module. Everything works fine for PML based operations 
(coll and point-to-point). But I have some errors while creating 
windows (even without the monitoring component : I launch my 
applications with --mca pml ^monitoring --mca osc ^monitoring --mca 
coll ^monitoring, so my components shouldn't be loaded).


From what I've tracked, while initializing the osc_rdma module, there 
is a btl selected, whose endpoint can't be found back when calling 
ompi_osc_rdma_peer_btl_endpoint().


Here are the traces of a problematic example (with 4 processes, curent 
process is 1). Every processes are on one node :


Breakpoint 11, ompi_osc_rdma_query_btls (comm=0x85f8b0, btl=0x85e850)
 at ../../../../../../ompi/mca/osc/rdma/osc_rdma_component.c:735
735 *btl = selected_btl;
(gdb) p selected_btl
$1 = (struct mca_btl_base_module_t *) 0x759e30
Breakpoint 10, ompi_osc_rdma_peer_btl_endpoint (module=0x85e360, peer_id=0)
 at ../../../../../../ompi/mca/osc/rdma/osc_rdma_peer.c:54
54  return NULL;
(gdb)  p mca_bml_base_btl_arr

[OMPI devel] Problem with BTL while allocating window

[Clement FOYER]

Hi everyone,

I've been facing issues with the creations of windows (MPI_Win_create). 
Maybe it's an already known issue, or maybe you will be able to tell me 
where to check to find the problem.


I've been developping some benchmark to evaluate the overhead of a 
monitoring module. Everything works fine for PML based operations (coll 
and point-to-point). But I have some errors while creating windows (even 
without the monitoring component : I launch my applications with --mca 
pml ^monitoring --mca osc ^monitoring --mca coll ^monitoring, so my 
components shouldn't be loaded).


From what I've tracked, while initializing the osc_rdma module, there 
is btl that's selected, and can't be found back when calling 
ompi_osc_rdma_peer_btl_endpoint().


Here are the traces of a problematic example (with 4 processes, curent 
process is 1). Every processes are on one node :


Breakpoint 11, ompi_osc_rdma_query_btls (comm=0x85f8b0, btl=0x85e850)
at ../../../../../../ompi/mca/osc/rdma/osc_rdma_component.c:735
735 *btl = selected_btl;
(gdb) p selected_btl
$1 = (struct mca_btl_base_module_t *) 0x759e30

Breakpoint 10, ompi_osc_rdma_peer_btl_endpoint (module=0x85e360, peer_id=0)
at ../../../../../../ompi/mca/osc/rdma/osc_rdma_peer.c:54
54  return NULL;
(gdb)  p mca_bml_base_btl_array_get_size (&mca_bml_base_get_endpoint ( 
ompi_comm_peer_lookup (module->comm, 0))->btl_rdma)
$17 = 2
(gdb) p mca_bml_base_get_endpoint ( ompi_comm_peer_lookup (module->comm, 
0))->btl_rdma.bml_btls[0].btl
$18 = (struct mca_btl_base_module_t *) 0x76cab0
(gdb) p mca_bml_base_get_endpoint ( ompi_comm_peer_lookup (module->comm, 
0))->btl_rdma.bml_btls[1].btl
$19 = (struct mca_btl_base_module_t *) 0x72a680

(gdb)  p mca_bml_base_btl_array_get_size (&mca_bml_base_get_endpoint ( 
ompi_comm_peer_lookup (module->comm, 1))->btl_rdma)
$20 = 2
(gdb) p mca_bml_base_get_endpoint ( ompi_comm_peer_lookup (module->comm, 
1))->btl_rdma.bml_btls[0].btl
$21 = (struct mca_btl_base_module_t *) 0x759e30
(gdb) p mca_bml_base_get_endpoint ( ompi_comm_peer_lookup (module->comm, 
1))->btl_rdma.bml_btls[1].btl
$22 = (struct mca_btl_base_module_t *) 0x7fffec275200 

(gdb)  p mca_bml_base_btl_array_get_size (&mca_bml_base_get_endpoint ( 
ompi_comm_peer_lookup (module->comm, 2))->btl_rdma)
$23 = 2
(gdb) p mca_bml_base_get_endpoint ( ompi_comm_peer_lookup (module->comm, 
2))->btl_rdma.bml_btls[0].btl
$24 = (struct mca_btl_base_module_t *) 0x76cab0
(gdb) p mca_bml_base_get_endpoint ( ompi_comm_peer_lookup (module->comm, 
2))->btl_rdma.bml_btls[1].btl
$25 = (struct mca_btl_base_module_t *) 0x72a680

(gdb)  p mca_bml_base_btl_array_get_size (&mca_bml_base_get_endpoint ( 
ompi_comm_peer_lookup (module->comm, 3))->btl_rdma)
$26 = 1
(gdb) p mca_bml_base_get_endpoint ( ompi_comm_peer_lookup (module->comm, 
3))->btl_rdma.bml_btls[0].btl
$27 = (struct mca_btl_base_module_t *) 0x759e30

It seems that for odd proc_id's, the corresponding selected btl can be 
retrieved, but not for the odd ones. I haven't check deeply into the 
library to explain this behavior yet. Do you have any idea of where to 
look this up?


Thank's you in advance,

Clément FOYER

___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel

Re: [OMPI devel] LD_PRELOAD a C-coded shared object with a FORTRAN application

[Clement FOYER]

Thank you all for your answers.

I stayed with the C version, with the FORTRAN symbols added as it worked 
with the tests I was willing to start. Nevertheless, in order to keep a 
more proper/portable solution, is it possible to use the same tools as 
in ompi/mpi/fortran/mpif-h/init_f.c in order to generate the mangled 
symbols (i.e. using #pragma weak or OMPI_GENERATE_F77_BINDINGS ) ?


Thank you.

Clément FOYER


On 12/12/2016 04:21 PM, Jeff Squyres (jsquyres) wrote:

If your Fortran compiler is new enough (and it *probably* is...?), you can use 
the BIND(C) notation to ease C / Fortran interoperability issues.



On Dec 12, 2016, at 5:37 AM, Gilles Gouaillardet 
 wrote:

Clement,

Ideally, your LD_PRELOAD'able library should be written in Fortran so you do 
not even run into this kind of issues (name mangling + parameter types)

If you really want to write it in C, you have to do it all manually

SUBROUTINE MPI_INIT(ierror)
INTEGER IERROR

can become

void mpi_init_(MPI_Fint * ierror)

Note mangling is compiler dependent.
For most compilers, this is the function name with all lower cases, followed by 
one or two underscores.

You will also have to convert all parameters
INTEGER comm
will be replaced (modulo the typos) with
MPI_Comm c_comm;
MPI_Fint *comm;
c_comm = MPI_Comm_f2c(*comm);

And so on, that is why Fortran wrapper is preferred,
plus there might be over caveats with Fortean 2008

Cheers,

Gilles

On Monday, December 12, 2016, Clement FOYER  wrote:
Hello everyone,

I have been trying to redirect MPI_Init and MPI_Finalize calls from a FORTRAN application 
(the CG benchmark from NAS Parallel Benchmarks). It appears that in the fortran 
application the MPI_Init function signature is "mpi_init_", whereas in my 
shared object it is MPI_Init. How is the f-to-c binding done in Open-MPI? How can I 
change the Makefile.am (or add a configure.m4) in order to check the way this name 
mapping is done by the compiler, and how to add the proper symbols so that my shared 
object could be used also with FORTRAN programs ?

Thank you in advance,

Clément FOYER

___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel
___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel




___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel

[OMPI devel] LD_PRELOAD a C-coded shared object with a FORTRAN application

[Clement FOYER]

Hello everyone,

I have been trying to redirect MPI_Init and MPI_Finalize calls from a 
FORTRAN application (the CG benchmark from NAS Parallel Benchmarks). It 
appears that in the fortran application the MPI_Init function signature 
is "mpi_init_", whereas in my shared object it is MPI_Init. How is the 
f-to-c binding done in Open-MPI? How can I change the Makefile.am (or 
add a configure.m4) in order to check the way this name mapping is done 
by the compiler, and how to add the proper symbols so that my shared 
object could be used also with FORTRAN programs ?


Thank you in advance,

Clément FOYER

___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel

[OMPI devel] Removing from opal_hashtable while iterating over the elements

[Clement FOYER]

Hi everyone,

I was wondering if it was possible to remove an element while iterating 
over the elements of a hashtable. As saw that it wasn't while using the 
OPAL_HASHTABLE_FOREACH macro, and I suppose it's because of the possible 
loss of the current next element. But how about if the element have 
already been iterated over? If I save a pointer to the previous element, 
it is safe to remove it from the hashtable?


Thank you,

Clément FOYER

___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel

Re: [OMPI devel] devel Digest, Vol 3372, Issue 1

[Clement FOYER]

Gilles,

In fact, I tried to create the Makefile.am based on the 
ompi/mca/common/ompio example, which is based on 
opal/mca/common/sm/Makefile.am. In fact it's now working, eventhough I 
had an error about linking a shared library into a static one, which has 
now disappeared.


Thank you for the tools/wrappers/Makefile.am example, it resolved my 
problem.


Cheers,

Clement


Date: Wed, 26 Oct 2016 20:52:12 +0900

   From: Gilles Gouaillardet
   To: Open MPI Developers
   Subject: Re: [OMPI devel] Makefile.am configuration
   Message-ID:

   Content-Type: text/plain; charset="utf-8"

   Clement,

   For your first question, you can search for inspiration in
   https://github.com/open-mpi/ompi/blob/master/opal/mca/btl/sm/Makefile.am
   btl/sm uses common/sm (and common/cuda too), so it looks pretty similar to
   what you are trying to achieve.

   For your second question, and even if I do not fully get what you want to
   achieve, you can have a look at
   https://github.com/open-mpi/ompi/blob/master/ompi/tools/wrappers/Makefile.am
   and see how ompi_wrapper_script is installed via make install.
   During make check, it might be enough to invoke the script in
   $(top_srcdir)/...

   If your project is not confidential, you can publish it on github so we can
   have a look and provide better guidance. You can also create a private
   repository on bitbucket and grant access to only a few developers (both
   options are free)


   Cheers,

   Gilles



   On Wednesday, October 26, 2016, Clement FOYER
   wrote:


Hi everyone,

I've got two Makefile configuration problems, and I thought maybe someone
here could give me hints to resolve them.

I've been working on a component that is part in ompi/mca/pml/comp, and
part in mca/common/comp. And I don't really know which variables/target to
set in order to have a proper linking when generating both .la and .so
files.

Also, on the test Makefile.am, I would like to distribute a script that
strat different benchmarks. How should I that up the makefile so the script
is copied to (or linked in) the build directory when just calling "make",
and to the installation directory when calling "make install" ?

Thank you in advance,

Cl?ment FOYER

___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel


   -- next part --
   An HTML attachment was scrubbed...
   
URL:<https://rfd.newmexicoconsortium.org/mailman/private/devel/attachments/20161026/7da954bb/attachment.html>




___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel

[OMPI devel] Makefile.am configuration

[Clement FOYER]

Hi everyone,

I've got two Makefile configuration problems, and I thought maybe 
someone here could give me hints to resolve them.


I've been working on a component that is part in ompi/mca/pml/comp, and 
part in mca/common/comp. And I don't really know which variables/target 
to set in order to have a proper linking when generating both .la and 
.so files.


Also, on the test Makefile.am, I would like to distribute a script that 
strat different benchmarks. How should I that up the makefile so the 
script is copied to (or linked in) the build directory when just calling 
"make", and to the installation directory when calling "make install" ?


Thank you in advance,

Clément FOYER

___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel