Just to follow up on this thread... Reuti: I merged the PR on to the main docs branch. They're now live -- we changed the text:
* here: https://docs.open-mpi.org/en/main/installing-open-mpi/configure-cli-options/installation.html * * and here: https://docs.open-mpi.org/en/main/installing-open-mpi/configure-cli-options/rpath-and-runpath.html Here's the corresponding PR to update the v5.0.x docs: https://github.com/open-mpi/ompi/pull/10640 Specifically, the answer to your original question is twofold: 1. It's complicated. 🙂 2. It looks like you did the Right Thing for your environment, but you might want to check the output of "readelf -d ..." to be sure. Does that additional text help explain things? -- Jeff Squyres jsquy...@cisco.com ________________________________ From: Jeff Squyres (jsquyres) <jsquy...@cisco.com> Sent: Saturday, August 6, 2022 9:36 AM To: Open MPI Users <users@lists.open-mpi.org> Subject: Re: [OMPI users] RUNPATH vs. RPATH Reuti -- See my disclaimers on other posts about apologies for taking so long to reply! This code was written forever ago; I had to dig through it a bit, read the comments and commit messages, and try to remember why it was done this way. What I thought would be a 5-minute search turned into a few hours of digging through code, multiple conversations with Brian, and one pull request (so far). We don't have a definitive answer yet, but we think we're getting closer. The short version is that what you did appears to be correct: ./configure LDFLAGS=-Wl,--enable-new-dtags ... The longer answer is that whenever you think you understand the shared library and run-time linkers, you inevitably find out that you don't. The complicated cases come from the fact that the handling of rpath and runpath can be different on different platforms, and there are subtle differences in their behavior (beyond the initial "search before or after LD_LIBRARY_PATH, such as the handling of primary and secondary/transitive dependencies). The pull request I have so far is https://github.com/open-mpi/ompi/pull/10624 (rendered here: https://ompi--10624.org.readthedocs.build/en/10624/installing-open-mpi/configure-cli-options/installation.html). We're not 100% confident in that text yet, but I think we're close to at least documenting what the current behavior is. Once we nail that down, we can talk about whether we want to change that behavior. ________________________________________ From: users <users-boun...@lists.open-mpi.org> on behalf of Reuti via users <users@lists.open-mpi.org> Sent: Friday, July 22, 2022 9:48 AM To: Open MPI Users Cc: Reuti; zuelc...@staff.uni-marburg.de Subject: [OMPI users] RUNPATH vs. RPATH Hi, using Open MPI 4.1.4 $ mpicc --show … tells me, that the command line contains "… -Wl,--enable-new-dtags …" so that even older linkers will include RUNPATH instead of RPATH in the created dynamic binary. On the other hand, Open MPI itself doesn't use this option for its own libraries: ./liboshmem.so.40.30.2 ./libmpi_mpifh.so.40.30.0 ./libmpi.so.40.30.4 ./libmpi_usempi_ignore_tkr.so.40.30.0 ./libopen-rte.so.40.30.2 Is this intended? Setting LD_LIBRARY_PATH will instruct the created binary to look for libraries first in that location and resolve it, but the loaded library in turn will then use RPATH inside itself first to load additional libraries. (I compile Open MPI in my home directory and move it then to the final destination for the group; setting OPAL_PREFIX of course. I see a mix of library locations when I run the created binary on my own with `ldd`.) Looks like I can get the intended behavior while configuring Open MPI on this (older) system: $ ./configure … LDFLAGS=-Wl,--enable-new-dtags -- Reuti
