Here is what one IB vendor says about the issue on their web site (redacted to protect the innocent):
"At the time of this release, the (redacted-openib) driver has issues with buffers sharing pages when fork( ) is used. Pinned (locked in memory) pages are normally marked copy-on-write during a fork. If a page is pinned before a fork and subsequently written to while RDMA operations are being performed on the same page, silent data corruption can occur as RDMA operations continue to stream data to a page that has moved. To avoid this, the (redacted-openib) driver does not use copy-on-write behavior during a fork for pinned pages. Instead, access to these pages by the child process will result in a segmentation violation." While there is some variation, I believe you will find that all IB comm shares this problem. So it is wise to avoid using fork if you want to use the openib transport. Hence the warning. Ignoring it is purely a "user beware" situation, but we provide that mechanism for the truly adventurous...or IB developers who want to someday resolve the problem. On Nov 29, 2010, at 3:44 PM, <ananda.mu...@wipro.com> wrote: > George > > Thanks for the explanation. I am trying to understand the following line in > your mail: > > “In fact, any fork done prior to the communication is a non-issue, but it is > difficult to identify. Therefore, we output the warning as soon as we detect > a fork after MPI_Init.” > > Does it mean that if I have a fork() after the communication (ie; mpi_send or > mpi_receive etc), I may have to relook at a different implementation to be at > safe side? I don’t want to suppress the messages if they result in any > corruption later. > > How can one be sure that the disabling the warning is ok? Could you please > elaborate on what makes forks vulnerable? May be that will guide the > developers to make an informed decision on whether to disable them or find > another alternative. > > > > Thanks > > Ananda > > ------------------ PREVIOUS MESSAGE --------------------------------- > > Subject: Re: [OMPI devel] Warning on fork() disappears if I use MPI threads!! > From: George Bosilca (bosilca_at_[hidden]) > Date: 2010-11-29 12:22:15 > > Next message: Jeff Squyres: "Re: [OMPI devel] Warning on fork() disappears if > I use MPI threads!!" > Previous message: ananda.mudar_at_[hidden]: "Re: [OMPI devel] Warning on > fork() disappears if I use MPI threads!!" > In reply to: ananda.mudar_at_[hidden]: "Re: [OMPI devel] Warning on fork() > disappears if I use MPI threads!!" > Next in thread: N.M. Maclaren: "Re: [OMPI devel] Warning on fork() disappears > if I use MPI threads!!" > Reply: N.M. Maclaren: "Re: [OMPI devel] Warning on fork() disappears if I use > MPI threads!!" > If your code doesn't exactly what is described in the code snippet attached > to your previous email, then you can safely ignore the warning. In fact, any > fork done prior to the communication is a non-issue, but it is difficult to > identify. Therefore, we output the warning as soon as we detect a fork after > MPI_Init. > > You can find more information about the usage of fork in Open MPI at > http://www.open-mpi.de/faq/?category=tuning#fork-warning > > george. > > On Nov 29, 2010, at 12:12 , <ananda.mudar_at_[hidden]> wrote: > > > I am posting this question again as it was sent before the long weekend and > > didn’t see any responses so far. Can anyone please explain the discrepancy > > I am observing with the scenario explained in the post below? > > > > Thanks > > Ananda > > Sent: Tuesday, November 23, 2010 2:24 PM > > To: devel_at_[hidden] > > Subject: Warning on fork() disappears if I use MPI threads!! > > > > Hi > > > > I am running into a very wierd problem. > > > > If I initialize MPI normally ie; with MPI_Init(), and make one of the MPI > > process to do "popen()" call, I get the following warning/error message: > > > > == Message start === > > An MPI process has executed an operation involving a call to the > > "fork()" system call to create a child process. Open MPI is currently > > operating in a condition that could result in memory corruption or > > other system errors; your MPI job may hang, crash, or produce silent > > data corruption. The use of fork() (or system() or other calls that > > create child processes) is strongly discouraged. > > == Message end ==== > > > > However this error message goes away, if I initialize MPI with threads ie; > > MPI_Init_thread(). Can anyone explain this discrepancy? > > > > I am giving a snippet of the program that causes this problem: > > > > == Code snippet start ====== > > if ( rank == 0) { > > output = popen("ls -l", "r"); > > while((c=getc(output))!=EOF) > > printf("%c",c); > > pclose(output); > > } > > == Code snippet end ====== > > > > If this is a design constraint, how can I overcome this problem. > > > > Thanks > > Ananda > > > > Ananda B Mudar, PMP > > Senior Technical Architect > > Wipro Technologies > > Ph: 972 765 8093 972 765 8093 > > Please do not print this email unless it is absolutely necessary. > > > > The information contained in this electronic message and any attachments to > > this message are intended for the exclusive use of the addressee(s) and may > > contain proprietary, confidential or privileged information. If you are not > > the intended recipient, you should not disseminate, distribute or copy this > > e-mail. Please notify the sender immediately and destroy all copies of this > > message and any attachments. > > > > WARNING: Computer viruses can be transmitted via email. The recipient > > should check this email and any attachments for the presence of viruses. > > The company accepts no liability for any damage caused by any virus > > transmitted by this email. > > > > www.wipro.com > > > > _______________________________________________ > > devel mailing list > > devel_at_[hidden] > > http://www.open-mpi.org/mailman/listinfo.cgi/devel > > > > Ananda B Mudar, PMP > Senior Technical Architect > Wipro Technologies > Ph: 972 765 8093 > ananda.mu...@wipro.com > > Please do not print this email unless it is absolutely necessary. > > The information contained in this electronic message and any attachments to > this message are intended for the exclusive use of the addressee(s) and may > contain proprietary, confidential or privileged information. If you are not > the intended recipient, you should not disseminate, distribute or copy this > e-mail. Please notify the sender immediately and destroy all copies of this > message and any attachments. > > WARNING: Computer viruses can be transmitted via email. The recipient should > check this email and any attachments for the presence of viruses. The company > accepts no liability for any damage caused by any virus transmitted by this > email. > > www.wipro.com > > _______________________________________________ > devel mailing list > de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/devel
