Re: [OMPI devel] Warning on fork() disappears if I use MPI threads!!

2010-11-29 Thread Jeff Squyres
On Nov 29, 2010, at 6:25 PM, George Bosilca wrote: > The main problem is that openib require to pin memory pages in order to take > advantage of RMA features. There is a major issues with these pinned pages > and fork, leading to segmentation fault in some specific cases. However, we > only pin

Re: [OMPI devel] Warning on fork() disappears if I use MPI threads!!

2010-11-29 Thread Ralph Castain
Here is what one IB vendor says about the issue on their web site (redacted to protect the innocent): "At the time of this release, the (redacted-openib) driver has issues with buffers sharing pages when fork( ) is used. Pinned (locked in memory) pages are normally marked copy-on-write during a

Re: [OMPI devel] Warning on fork() disappears if I use MPI threads!!

2010-11-29 Thread George Bosilca
On Nov 29, 2010, at 17:44 , wrote: > George > > Thanks for the explanation. I am trying to understand the following line in > your mail: > > “In fact, any fork done prior to the communication is a non-issue, but it is > difficult to identify. Therefore, we output the warning as soon as we

Re: [OMPI devel] Warning on fork() disappears if I use MPI threads!!

2010-11-29 Thread ananda.mudar
George Thanks for the explanation. I am trying to understand the following line in your mail: "In fact, any fork done prior to the communication is a non-issue, but it is difficult to identify. Therefore, we output the warning as soon as we detect a fork after MPI_Init." Does it mean that if I h

Re: [OMPI devel] Warning on fork() disappears if I use MPI threads!!

2010-11-29 Thread ananda.mudar
Jeff I am invoking MPI_Init_thread() with MPI_THREAD_MULTIPLE. If openib BTL is responsible for the warning and openib BTL is excluded in this case, then that explains the discrepancy. FYI, I invoked MPI_Init_thread() with MPI_THREAD_SERIALIZED and I got the warning message about the fork(). Tha

Re: [OMPI devel] Warning on fork() disappears if I use MPI threads!!

2010-11-29 Thread N.M. Maclaren
On Nov 29 2010, George Bosilca wrote: If your code doesn't exactly what is described in the code snippet attached to your previous email, then you can safely ignore the warning. In fact, any fork done prior to the communication is a non-issue, but it is difficult to identify. Therefore, we ou

Re: [OMPI devel] Warning on fork() disappears if I use MPI threads!!

2010-11-29 Thread Jeff Squyres
On Nov 23, 2010, at 3:23 PM, wrote: > However this error message goes away, if I initialize MPI with threads ie; > MPI_Init_thread(). Can anyone explain this discrepancy? What thread level are you invoking MPI_INIT_THREAD with? One possible reason this could be happening is that the openib B

Re: [OMPI devel] Warning on fork() disappears if I use MPI threads!!

2010-11-29 Thread George Bosilca
If your code doesn't exactly what is described in the code snippet attached to your previous email, then you can safely ignore the warning. In fact, any fork done prior to the communication is a non-issue, but it is difficult to identify. Therefore, we output the warning as soon as we detect a f

Re: [OMPI devel] Warning on fork() disappears if I use MPI threads!!

2010-11-29 Thread ananda.mudar
I am posting this question again as it was sent before the long weekend and didn't see any responses so far. Can anyone please explain the discrepancy I am observing with the scenario explained in the post below? Thanks Ananda Sent: Tuesday, November 23, 2010 2:24 PM To: de...@open-mpi.org Sub

Re: [OMPI devel] RFC: Bring the lastest ROMIO version from MPICH2-1.3 into the trunk

2010-11-29 Thread Jeff Squyres
On Nov 29, 2010, at 11:40 AM, Pascal Deveze wrote: > The last changes are not committed back in bitbucket. I thought that was not > necessary. Would you like that I update also bitbucket ? If yes, I will do it. Yes, that would be most convenient. Thanks! > Applying the diff on a local copy of

[OMPI devel] 1.5 plans

2010-11-29 Thread Jeff Squyres
I'm about 2 weeks late on this email; apologies. SC and Thanksgiving got in the way. Per a discussion on the devel teleconf nearly 3 weeks ago, we have decided what to do with the v1.5 series: - 1.5.1 will be a bug fix release. There's 2 blocker bugs right now that need to be reviewed; those

Re: [OMPI devel] RFC: Bring the lastest ROMIO version from MPICH2-1.3 into the trunk

2010-11-29 Thread Pascal Deveze
Jeff, The last changes are not committed back in bitbucket. I thought that was not necessary. Would you like that I update also bitbucket ? If yes, I will do it. Applying the diff on a local copy of the trunk, you should be able to generated a library with the new ROMIO. Pascal Jeff Squyr

Re: [OMPI devel] RFC: Bring the lastest ROMIO version from MPICH2-1.3 into the trunk

2010-11-29 Thread Jeff Squyres
Some questions about the patch: configure.in: @@ -2002,9 +1987,8 @@ # Turn off the building of the Fortran interface and the Info routines EXTRA_DIRS="" AC_DEFINE(HAVE_STATUS_SET_BYTES,1,[Define if status_set_bytes available]) - DEFINE_HAVE_MPI_GREQUEST="#define HAVE_MPI_GREQUEST" -

Re: [OMPI devel] RFC: Bring the lastest ROMIO version from MPICH2-1.3 into the trunk

2010-11-29 Thread Jeff Squyres
Great! Are those final changes committed back to the bitbucket? If so, I'll give it a whirl. On Nov 24, 2010, at 10:48 AM, Pascal Deveze wrote: > Hi Jeff, > > Here is the unified diff. > As only the romio subtree is modified, I made the following command: > diff -u -r -x .svn ompi-trunk/

[OMPI devel] Sending large messages over RDMA fails

2010-11-29 Thread Doron Shoham
Hi, The maximum message size of ConnectX HCAs is 1GB (older cards have a maximum of 2GB). Trying to send larger messages over RDMA direct protocol will fail. A reminder - RDMA direct will be used if RDMA writes or reads are allowed by |btl_openib_flags| and the sender's message is already