Re: Finding .so files without setting LD_LIBRARY_PATH

[Lawrence D’Oliveiro]

On Thursday, May 12, 2016 at 9:51:02 AM UTC+12, Paul Smith wrote:
> ... here's the problem: because LD_LIBRARY_PATH is in
> Python's environment it is also passed down to programs invoked by
> Python.  That means if I (for example) invoke subprocess.call(['ssh',
> ...]) then it fails because the system ssh is looking for the system
> libcrypto.so, and when it finds the Python libcrypto.so instead
> (because of LD_LIBRARY_PATH) it fails.

That’s easy enough to fix: you can pass a custom environment down to 
subprocesses by using the “env” argument to the various subprocess calls 
.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Finding .so files without setting LD_LIBRARY_PATH

[Paul Smith]

On Thu, 2016-05-12 at 07:55 +0300, Jussi Piitulainen wrote:
> eryk sun writes:
> 
> > On Wed, May 11, 2016 at 10:39 PM, Paul Smith wrote:
> > > Hi all.  I have a locally-built version of Python (2.7.11) that I'm
> > > copying around to different systems, running all different versions of
> > > GNU/Linux.
> > ...
> > > What I'd like to do is have a way of setting the library path that
> > > Python uses when it tries to load .so files (for example in the ssl
> > > module which loads lib-dynload/_ssl.so which links to libssl.so and
> > > libcrypto.so), WITHOUT setting LD_LIBRARY_PATH in the environment that
> > > Python passes to its children.
> > 
> > An ELF header can contain either an RPATH or a RUNPATH to extend the
> > library search path at load time.

Yes, I'm familiar with rpath.  However I don't know how to set it for
Python .so's that appear in the lib-dynload directory, including site
-packages, which is where it needs to be ... I tried to go through the
Modules/Setup etc. but to no avail :(.

> There's a tool (GPLv3+) called patchelf that can set these in an ELF
> binary:

Aha!  That's excellent!  The only one I was aware of was chrpath which
doesn't allow you to add an rpath if one doesn't exist, or add an rpath
which is longer than the one that already exists.

This will make things much simpler.

Cheers!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Finding .so files without setting LD_LIBRARY_PATH

[dieter]

Paul Smith  writes:
> ...
> That works fine, but here's the problem: because LD_LIBRARY_PATH is in
> Python's environment it is also passed down to programs invoked by
> Python.  That means if I (for example) invoke subprocess.call(['ssh',
> ...]) then it fails because the system ssh is looking for the system
> libcrypto.so, and when it finds the Python libcrypto.so instead
> (because of LD_LIBRARY_PATH) it fails.

I have had a similar problem - however with "PYTHONPATH" instead
of "LD_LIBRARY_PATH". I solved it by removing "PYTHONPATH" from
"os.environ" during Python startup. This way, the envvar
"PYTHONPATH" was effective to set up "sys.path" but
processes started from this python would not see it.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Finding .so files without setting LD_LIBRARY_PATH

[Jussi Piitulainen]

eryk sun writes:

> On Wed, May 11, 2016 at 10:39 PM, Paul Smith wrote:
>> Hi all.  I have a locally-built version of Python (2.7.11) that I'm
>> copying around to different systems, running all different versions of
>> GNU/Linux.
> ...
>> What I'd like to do is have a way of setting the library path that
>> Python uses when it tries to load .so files (for example in the ssl
>> module which loads lib-dynload/_ssl.so which links to libssl.so and
>> libcrypto.so), WITHOUT setting LD_LIBRARY_PATH in the environment that
>> Python passes to its children.
>
> An ELF header can contain either an RPATH or a RUNPATH to extend the
> library search path at load time. RPATH is searched before
> LD_LIBRARY_PATH, while RUNPATH is searched after LD_LIBRARY_PATH. Note
> that the Linux loader only uses an embedded RUNPATH when searching for
> the immediate dependencies of an object. Use an RPATH instead if you
> need to extend the path that dlopen searches at runtime (e.g. for
> loading shared libraries via ctypes).
>
> Paths can be defined relative to the object using ${ORIGIN}. For
> example, the following configures a Python build to add a relative
> RPATH:
>
> $ ./configure LDFLAGS='-Wl,-rpath,\$${ORIGIN}/lib'
> $ make
>
> Or a RUNPATH:
>
> $ ./configure LDFLAGS='-Wl,-rpath,\$${ORIGIN}/lib,--enable-new-dtags'
> $ make
>
> Here's the resulting RPATH in the ELF header:
>
> $ readelf -d python | grep RPATH
>  0x000f (RPATH)  Library rpath: [${ORIGIN}/lib]

There's a tool (GPLv3+) called patchelf that can set these in an ELF
binary:

https://nixos.org/patchelf.html
https://github.com/NixOS/patchelf
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Finding .so files without setting LD_LIBRARY_PATH

[eryk sun]

On Wed, May 11, 2016 at 10:39 PM, Paul Smith  wrote:
> Hi all.  I have a locally-built version of Python (2.7.11) that I'm
> copying around to different systems, running all different versions of
> GNU/Linux.
...
> What I'd like to do is have a way of setting the library path that
> Python uses when it tries to load .so files (for example in the ssl
> module which loads lib-dynload/_ssl.so which links to libssl.so and
> libcrypto.so), WITHOUT setting LD_LIBRARY_PATH in the environment that
> Python passes to its children.

An ELF header can contain either an RPATH or a RUNPATH to extend the
library search path at load time. RPATH is searched before
LD_LIBRARY_PATH, while RUNPATH is searched after LD_LIBRARY_PATH. Note
that the Linux loader only uses an embedded RUNPATH when searching for
the immediate dependencies of an object. Use an RPATH instead if you
need to extend the path that dlopen searches at runtime (e.g. for
loading shared libraries via ctypes).

Paths can be defined relative to the object using ${ORIGIN}. For
example, the following configures a Python build to add a relative
RPATH:

$ ./configure LDFLAGS='-Wl,-rpath,\$${ORIGIN}/lib'
$ make

Or a RUNPATH:

$ ./configure LDFLAGS='-Wl,-rpath,\$${ORIGIN}/lib,--enable-new-dtags'
$ make

Here's the resulting RPATH in the ELF header:

$ readelf -d python | grep RPATH
 0x000f (RPATH)  Library rpath: [${ORIGIN}/lib]
-- 
https://mail.python.org/mailman/listinfo/python-list