On Tue, Mar 29, 2016 at 2:51 PM, Nathaniel Smith <[email protected]> wrote: > On Tue, Mar 29, 2016 at 10:50 AM, Matthew Brett <[email protected]> > wrote: >> On Tue, Mar 29, 2016 at 4:29 AM, Olivier Grisel >> <[email protected]> wrote: >>>> You can replicate fairly simply with: >>>> >>>> pip install -f >>>> https://d9a97980b71d47cde94d-aae005c4999d7244ac63632f8b80e089.ssl.cf2.rackcdn.com >>>> numpy >>>> cd numpy/f2py/tests/src/kind >>>> f2py -c foo.f90 -m foo >>>> python -c 'import numpy; import foo' >>> >>> I can replicate. I can also force the use of the system fortran with >>> LD_PRELOAD (on ubuntu 14.04): >>> >>> LD_PRELOAD='/usr/lib/x86_64-linux-gnu/libgfortran.so.3' python -c >>> 'import numpy; import foo' >> >> This one is obviously a show-stopper - I believe that it means that >> any module compiled against gfortran (and presumably any shipped lib >> that numpy loads) will break after numpy has been imported. >> >> I don't understand the linux link rules well enough to know how to >> fix. I guess we could rename the libraries that we ship / vendor into >> libs? (numpy_libgfortran.so etc). > > Yeah, I'm not at all sure what's going on here and whether it's > intentional or a bug in glibc, and we should figure out what's > actually happening and why for our own understanding, but I'm pretty > confident that this is going to be either the solution or the > workaround :-). > > I guess the general approach is that auditwheel should assign a unique > name to every vendored library. Some possible naming strategies: > <wheel package name>-libgfortran.so > <truncated sha-256 of libgfortran.so>-libgfortran.so > <random uuid>-libgfortran.so > > I guess the truncated-sha256 strategy might be best because if there > are lots of manylinux wheels all vendoring the same copy of > libgfortran, they will (or at least might) end up sharing it in memory > instead of mmap'ing multiple different copies of the identical .so > file, which is good? And this is certainly safe if the different > copies of libgfortran all have the same sha256. > > And then auditwheel should use patchelf --replace-needed to modify the > DT_NEEDED in the various extension modules to point to the new name, > at the same time it's changing the RPATH. (--replace-needed doesn't > seem to be mentioned in most of the patchelf docs I can find, but it > is described here: https://github.com/NixOS/patchelf)
So generally speaking this workaround is definitely the right workaround, and I just implemented and tested it in this repository and it works: https://github.com/njsmith/rpath-weirdness But, in the process I discovered that the text described above is missing a crucial step -- we need to also patch the DT_SONAME of the renamed+vendored library, as described here: https://github.com/pypa/auditwheel/issues/24 -n -- Nathaniel J. Smith -- https://vorpus.org _______________________________________________ Wheel-builders mailing list [email protected] https://mail.python.org/mailman/listinfo/wheel-builders
