[issue9998] ctypes find_library should search LD_LIBRARY_PATH on Linux

[Roundup Robot]

Roundup Robot added the comment:

New changeset 385181e809bc by Vinay Sajip in branch 'default':
Closes #9998: Allowed find_library to search additional locations for libraries.
https://hg.python.org/cpython/rev/385181e809bc

--
nosy: +python-dev
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on Linux

[Daniel Blanchard]

Changes by Daniel Blanchard :


--
nosy:  -Daniel.Blanchard

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on Linux

[Vinay Sajip]

Vinay Sajip added the comment:

Updated the last patch with code for handling the case where ld is not 
available.

--
Added file: http://bugs.python.org/file44052/refresh-2016-08-08.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on Linux

[Vinay Sajip]

Changes by Vinay Sajip :


Added file: http://bugs.python.org/file44036/refresh-2016-08-07.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on Linux

[Vinay Sajip]

Changes by Vinay Sajip :


Added file: http://bugs.python.org/file43937/refresh-2016-07-29.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on Linux

[Michael Felt]

Michael Felt added the comment:

Vinay - I am in favor, but if the root cause of the holdup is unclear 
documentation then let's address the root cause.

Stronger, I am in agreement. The first patch I submitted for AIX included using 
LIBPATH (and could have added LD_LIBRARY_PATH) - but that was rejected.

> In my view, the best should not be the enemy of the good, and pragmatism 
> beats purity. I don't have the resources to test this functionality on all 
> platforms - just Windows and Linux - and am not familiar with other platforms 
> like AIX, Solaris or the BSDs. The documentation makes clear that the 
> behaviour of find_library is system-dependent and makes no promises of a 
> particular level of consistency, either with dlopen() or across platforms.

* Again, basically I agree with the principles. I do not have the resources for 
all linux platforms, Solaris, BSD, Windows, etc..
* As far as "other platforms" and dlopen() behavior - from the man pages I have 
referenced I see more similarity than difference.
* behaviour of find_library is system-dependent - exactly - but unfortunately, 
for now AIX is whatever is in the else: box. That it works at all is a 
statement of the flexibility of AIX - and the sweat and tears of AIX admins to 
get the pieces to fit.

> Nor can we guarantee in the documentation that find_library() will exactly 
> emulate dlopen(), since that may not hold on all platforms - and in fact, 
> since Windows is in the mix, there is little point in trying to tie 
> find_library() behaviour to that of dlopen() directly.

Good points. However, while it cannot be guaranteed - upfront - (although I 
would say that I could make them mimic each other on AIX given the leeway, 
better directive) - it can be stated as an objective for "posix".
:) Windows is windows - maybe someone who knows windows well enough could make 
something very close.

> The purpose of find_library as currently documented seems adequately 
> described, and the documentation update in my latest patch clarifies things 
> further.

Obviously, here is where we disagree. If it was 'adequately' described you 
could have used the documentation as a justification years ago. Your change 
touches on it, but still leaves a great deal to discuss - and especially - 
convince those who may actually commit.

> I'm not sure the agreement you seek will come any time soon, as it has not 
> come in the last five years; it doesn't seem possible to aim for e.g. exact 
> behaviour of dlopen(), because (a) it's not the same on Windows and (b) 
> potentially varies too widely across POSIX platforms. Can you propose some 
> change to the find_library() contract which you can assure will be 
> implementable across all platforms? Certainly, fidelity with dlopen() isn't 
> it.

As you mentioned earlier - the implementations are system-dependent - so yes, 
we could argue for find_library search behavior that emulates dlopen() (better 
perhaps "LoadLibrary()") - but not define any particular behavior - precisely 
because it may be different on different platforms. Again, the objective is to 
replicate that platform's LoadLibrary search behavior - whatever that may be.

Once we have established what is needed/desired as a goal, over time 
implementations will catch up. Linux will be leading the way because you have, 
presumably, already implemented it.

Michael

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on Linux

[Vinay Sajip]

Vinay Sajip added the comment:

Added belopolsky and meador.inge to nosy as they are listed as ctypes experts 
in the experts index (Amaury was already there).

--
nosy: +meador.inge

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on Linux

[Vinay Sajip]

Vinay Sajip added the comment:

In my view, the best should not be the enemy of the good, and pragmatism beats 
purity. I don't have the resources to test this functionality on all platforms 
- just Windows and Linux - and am not familiar with other platforms like AIX, 
Solaris or the BSDs. The documentation makes clear that the behaviour of 
find_library is system-dependent and makes no promises of a particular level of 
consistency, either with dlopen() or across platforms. Nor can we guarantee in 
the documentation that find_library() will exactly emulate dlopen(), since that 
may not hold on all platforms - and in fact, since Windows is in the mix, there 
is little point in trying to tie find_library() behaviour to that of dlopen() 
directly.

My documentation update makes clear that for Linux only, LD_LIBRARY_PATH will 
be searched if the existing mechanisms give no joy.

This request has been around since 2010, and in my view implementing this 
long-overdue patch will improve matters for Linux users and IIUC meet the goals 
of the issue creator and other commenters who concurred with his sentiment. 
This does not preclude improving the functionality on other platforms later, 
but I think we should implement this patch unless someone can point out that it 
makes things worse in some way. If anyone can improve it, that is also to be 
welcomed, of course.

> As such, I would oppose a patch that only addresses the specifics of one 
> platform.

Why would improving behaviour on one platform, without any API changes or 
needing additional work by users, be worthy of opposition?

> What is needed is a "design" clarification of the purpose of find_library.
> If we can agree on that "implementation" can follow.

The purpose of find_library as currently documented seems adequately described, 
and the documentation update in my latest patch clarifies things further. IMO 
this is an area where the underlying platform features which find_library() 
relies on, as well as the run-time dynamic linking facilities available, are 
quite different across platforms. 

I'm not sure the agreement you seek will come any time soon, as it has not come 
in the last five years; it doesn't seem possible to aim for e.g. exact 
behaviour of dlopen(), because (a) it's not the same on Windows and (b) 
potentially varies too widely across POSIX platforms. Can you propose some 
change to the find_library() contract which you can assure will be 
implementable across all platforms? Certainly, fidelity with dlopen() isn't it.
 

> IMHO - trying to get a patch in for the convenience of one platform is not an 
> architectural enhancement.

We're not trying for an architectural enhancement here, AFAIK.

I would welcome some input from others!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on Linux

[Michael Felt]

Michael Felt added the comment:

imho - it is not correct to only make a modification of this nature for a 
single platform.

To be realistic, if the "design" goal is to 'find' what dlopen() will find when 
given a argument without a pathname component - then gcc should not be used, 
and perhaps only your 'ld' based solution (which I have not read).

>From linux man pages: both the justification to consider *PATH* for all 
>platforms (i.e., LD_LIBRARY_PATH is a GNU/Linux construct, not the default for 
>all platforms.)

"""
dlopen()

The function dlopen() loads the dynamic library file named by the 
null-terminated string filename and returns an opaque "handle" for the dynamic 
library. If filename is NULL, then the returned handle is for the main program. 
If filename contains a slash ("/"), then it is interpreted as a (relative or 
absolute) pathname. Otherwise, the dynamic linker searches for the library as 
follows (see ld.so(8) for further details):
o

(ELF only) If the executable file for the calling program contains a DT_RPATH 
tag, and does not contain a DT_RUNPATH tag, then the directories listed in the 
DT_RPATH tag are searched.

o

If, at the time that the program was started, the environment variable 
LD_LIBRARY_PATH was defined to contain a colon-separated list of directories, 
then these are searched. (As a security measure this variable is ignored for 
set-user-ID and set-group-ID programs.) 
""" end of excerpt

In short, this is more than just LD_LIBRARY_PATH. And my preference is that 
*PATH*, if predefined, should be taken into consideration by find_library. In 
other words, the purpose of find_library is to resolve platform naming 
conventions of shared libraries given a 'generic' argument, e.g., 
find_library("c") on Linux returns libc.so.6 while for AIX it (should return) 
libc.a(shr4.o).

As such, I would oppose a patch that only addresses the specifics of one 
platform. What is needed is a "design" clarification of the purpose of 
find_library. If we can agree on that "implementation" can follow. IMHO - 
trying to get a patch in for the convenience of one platform is not an 
architectural enhancement.

Again, I would like to see the documented behavior to be that find_library 
returns what dlopen() would open - when given an argument is the correct 
platform syntax. A patch for Linux does not - formally - guarantee that 
documented change.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on Linux

[Vinay Sajip]

Changes by Vinay Sajip :


--
hgrepos:  -130

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on Linux

[Vinay Sajip]

Vinay Sajip added the comment:

I have updated the patch to apply against 3.6, and changed it to be more 
conservative: it only uses ld and LD_LIBRARY_PATH when trying to find a library 
if the existing gcc-based method fails.

Seeing that this issue has been around for so long, I would really like to get 
this patch committed in the short window we have before 3.6 goes into beta. 
Please review!

--
assignee: theller -> vinay.sajip
title: ctypes find_library should search LD_LIBRARY_PATH on linux -> ctypes 
find_library should search LD_LIBRARY_PATH on Linux
versions: +Python 3.6 -Python 3.5
Added file: http://bugs.python.org/file43929/refresh-2016.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on linux

[Michael Felt]

Michael Felt added the comment:

Getting back to this summary:
Windows: {name} and {name}.dll, via %PATH%
OS X: lib{name}.dylib, {name}.dylib and {name}.framework/{name}, via dyld_find()
BSD: lib{name}.* via ldconfig (choose highest ABI version) and cc
Solaris: lib{name}.so via crle, and lib{name}.* via cc
Gnu: lib{name}.* via ldconfig and cc

The "Gnu" selection is what is used for "None of the above", so also for AIX - 
which generally has neither ldconfig nor cc installed.

Having worked on a patch for AIX so that something that is (aka should) always 
there (dump) for examining archives for dlopen() openable objects.

In another issue I have had the benefit of lots of feedback from Martin Panter 
- most of which I agree with.

As this issue is about enhancement my feedback re: AIX behavior of dlopen() is 
that dlopen() takes LD_LIBRARY_PATH (when LIBPATH is not defined) into account.

In general find_library() is used as: CDLL(find_library("xxx")) - as in I do 
not care if the result is None or libxxx.so.y.z - just do it. However, if 
find_library() and CDLL() are written so that they "find" the same library 
object then it does become possible for a programmer to verify that a supported 
version is what will be loaded.

The return value from find_library() should be related to it's argument - just 
as dlopen() would 'react' (better, search *PATH*). If no path element ("./*" | 
"../*", or "/*") then no path element should be in the return value. The case 
for using path elements is because find_library() could be used to verity it's 
existence before calling CDLL() - what I did not know initially, but consider 
vital for proper behavior is that CDLL(NULL) just returns 'python' itself.

To underline that there are many issues that have been left unaddressed for 
years (this discussion here is nearly 6 years old) I mention:
"The case of python calling find_library("uuid") - on AIX this is probably 
NULL, as libuuid.so(.X) is not native to AIX - and even if it is present will 
not be found via a non-existent ldconfig or cc (only thing the "Gnu" option 
even considers).

IMHO: find_library should reflect dlopen() - with find_library being, as 
documented.

>From the Python2.6 documentation (when introduced?)
"""The purpose of the find_library() function is to locate a library in a way 
similar to what the compiler does (on platforms with several versions of a 
shared library the most recent should be loaded), while the ctypes library 
loaders act like when a program is run, and call the runtime loader directly.

The ctypes.util module provides a function which can help to determine the 
library to load.

ctypes.util.find_library(name)
Try to find a library and return a pathname. name is the library name 
without any prefix like lib, suffix like .so, .dylib or version number (this is 
the form used for the posix linker option -l). If no library can be found, 
returns None.

The exact functionality is system dependent."""

This pre-dates my experience with python, so if I inaccurate in assumptions - 
correct me, but please be patient with me.

What I miss is a PEP on this topic. From the limited reads of other PEPs I have 
read I think (rather hope) that inconsistencies in documentation could have 
been caught.

While (as Martin mentioned earlier in this discussion) find_library() behaves 
as "build" aka cc/gcc and CDLL follows "run-time" loader. imho, this is 
inconsistent - and the inconsistency is also in that short bit of documentation:

a) """The purpose of the find_library() function is to locate a library in a 
way similar to what the compiler does (on platforms with several versions of a 
shared library the most recent should be loaded) (i.e., always find the latest 
version)"""

and
b)"""The ctypes.util module provides a function which can help to determine the 
library to load.""" IMHO: how it is expected to be used because I (i.e., the 
python programmer) cannot provide additional specification to locate a specific 
version - find_library() is meant to find the latest always

As the Python3.5.2 documentation is nearly verbatim - this is still the 
documented condition.

So - I am very happy about Martin's (partial) comment:
maybe changing towards run-time would not be such a problem as I first thought.

CDLL() does a search, by definition. Maybe I do not care what it finds - but 
the argument to it is expected to platform dependent.
The "remaining" limitation of find_library(), even with searching *PATH* 
included is that it MUST also return the latest version (as a request for a 
specific version may not be made, or any other "extension"). An additional 
weakness is that what it "finds" must always be prefixed by "lib", while 
dlopen() has not such requirement. (FYI: I ran into these 'unusual' shared 
libraries while packaging sudo. While it is unlikely that python would ever 
load these sudo libraries it does show that there can be shared libraries that 
can accessed by not "found")

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on linux

[Michael Felt]

Michael Felt added the comment:

In https://bugs.python.org/issue26439 I have been working on this for AIX - as 
the default behavior was to depend on two things:
a) ldconfig -p (which generally does not exist on AIX, and I doubt it will know 
about the non-gnu libraries
b) that the objects are files rather than members of an archive

For python cdll() to work (as find_library generally did not) - the work-around 
was to extract archive members into standard directories and/or hard-code 
unique AIX names. But again, always as files, as CDLL did not modify the 
dlopen() mode to permit loading a shared library from an archive.

re: this issue and behavioral differences between find_library and cdll() 
(rather dlopen()) searches - a linker is different from a dlopen. When 
compiling an argument such as -lc -- on AIX -- tells the linker to look in 
libc.a for any member (of the right size) that satisfies an unknown symbol. 
That archive (base) and member (member) name is remembered in the "loader 
section" during linking. Note that static members have no loader section of 
their own, nor do they add one to the object (program or shared library) being 
linked.
As a compatibility feature - when a member is not found in an archive the AIX 
rtl (run-time-linker) also looks for a file named libFOO.so (e.g., libc.so) - 
this grace behavior is why CDLL() has worked somewhat using default behavior in 
Lib/ctypes.

As to LD_LIBRARY_PATH and LIBPATH. In AIX terms, these are environment 
variables to supplement the library search path - only when no PATH information 
is provided (i.e., cdll("./libc.so") will only look in the current directory of 
the process, and only for the file libc.so)
The default search path is a list of the directories used to link the 
executable (e.g., python). For me that list is: 
***Import File Strings***
INDEX  PATH  BASEMEMBER
0  /usr/vac/lib:/usr/lib:/lib

In other words, python, by default, is only going to look in very standard 
locations - even though it knows it's prefix is not /usr.

There are two solutions - and I will make changes as appropriate.
a) automate setting an environment variable if it is not already set to 
something like 
 _os.environ['LIBPATH'] = "%s/lib" % _sys.prefix
and not something "arbitrary" such as /usr/local/lib
b) modify the LDFLAGS when packaging to include
-L${prefix}/lib which will have the effect of changing the default search path 
via -W,-blibpath=/usr/vac/lib:/usr/lib:/lib:${prefix}/lib


In any case, in my patch for AIX in ctypes the two will work identical (cdll() 
even calls aix.find_library() to simplify portability between GNU/posix 
environments that expect or support only filenames to work with AIX archive 
packaging.

My apologies for the wall of text. This is not a simple subject.

--
nosy: +Michael.Felt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on linux

[Martin Panter]

Martin Panter added the comment:

Thanks Pau, for some reason I didn’t pick up the dylib OS X problem last time I 
looked at that link. This is a quick summary of the library names searched on 
different platforms:

Windows: {name} and {name}.dll, via %PATH%
OS X: lib{name}.dylib, {name}.dylib and {name}.framework/{name}, via dyld_find()
BSD: lib{name}.* via ldconfig (choose highest ABI version) and cc
Solaris: lib{name}.so via crle, and lib{name}.* via cc
Gnu: lib{name}.* via ldconfig and cc

Already, these cases seem to be half emulating the run-time linker and half the 
build-time linker. I don’t have a good answer about which is “better” (it would 
depend on the use case). But since we already have a mix, maybe changing 
towards run-time would not be such a problem as I first thought.

A half-cooked idea of mine is a simpler function that just accounts for common 
library naming conventions on various platforms, but does not extract sonames 
or spawn compilers. I think this could work with what seems to be a common use 
case of passing the result directly to CDLL(), which will do the real search. 
For the Windows and OS X cases, a loop might be required to attempt CDLL() with 
the different possibilities.

But I agree this is straying from the topic of this bug report. Perhaps I 
should accept that people want to find libraries by just the build-time name, 
but that are accessible at runtime.

More related reports about find_library() vs CDLL():
Issue 19317: Search Python’s RPATH
Issue 21622: Work without DT_SONAME

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on linux

[Vinay Sajip]

Vinay Sajip added the comment:

> Does anyone have any valid use cases where they want to use a shared library 
> on LD_LIBRARY_PATH or similar

I presume that would include this issue's creator and other people who have 
commented here about what they see as a drawback in find_library's current 
behaviour.

Pau Tallada's point about wanting to use a cross-platform invocation also seems 
reasonable. Remember, if you know the exact library you want to use, you don't 
*need* find_library: and this issue is about making find_library useful in a 
wider set of cases than it currently is.

> The problem I see with using find_library() to blindly load a library

Nobody is saying that the result of find_library() has to be used to blindly 
load a library. The point you make about the code in the uuid module is 
orthogonal to the proposal in this issue - even the behaviour of find_library 
never changed, that code could break for the reasons you give. For that, it's 
not unreasonable to raise a separate issue about possible fragility of the code 
in uuid.

I asked a question which I think is relevant to this enhancement request - "is 
emulating a build-time linker the most useful thing? In the context of Python 
binding to external libraries, why is build-time linking behaviour better than 
run-time linking behaviour?"

Do you have an answer to that?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on linux

[Pau Tallada]

Pau Tallada added the comment:

> Pau: What is wrong with the more direct CDLL("libspatialindex_c.so") proposal 
> that bypasses find_library()?

I don't know much about library loading, but from what I understand at 
https://github.com/Toblerity/rtree/issues/56 , if they hardcode 
"libspatialindex_c.so" in CDLL call, then it breaks in OSX, because its uses 
.dylib extension. And they don't want to reimplement find_library to choose the 
right extension for each platform.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on linux

[Martin Panter]

Martin Panter added the comment:

I also found Issue 18502 essentially about the same incompatibility between 
CDLL() and find_library().

The problem I see with using find_library() to blindly load a library is that 
you could be loading an incompatible version (wrong soname). That is why I 
asked Pau why they couldn’t give the proper library name to CDLL().

I noticed that Python’s own “uuid” module calls CDLL(find_library("uuid")); see 
. On my Linux 
computer, the full library name is libuuid.so.1, and according to online man 
pages, this is also the case on Free BSD. So I wonder if the “uuid” module 
should call CDLL("libuuid.so.1") directly. If somebody invented a new 
incompatible version libuuid.so.2, Python’s uuid module might crash, or become 
subtly broken, but programs that linked directly to libuuid.so.1 would continue 
to work.

Does anyone have any valid use cases where they want to use a shared library on 
LD_LIBRARY_PATH or similar, but cannot use CDLL() or LoadLibrary() directly? If 
people really want a way to load a library given its compile-time name, maybe 
changing find_library() is a valid way forward. But to me the use case does not 
seem robust or worth blessing in Python’s standard library.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on linux

[Vinay Sajip]

Vinay Sajip added the comment:

> find_library() is documented as emulating a build-time linker, not run-time

It may be documented as that, but is emulating a build-time linker the most 
useful thing? In the context of Python binding to external libraries, why is 
build-time linking behaviour better than run-time linking behaviour? This is an 
enhancement request, not a bug request: so if a change was to be applied, the 
documentation could be updated to indicate any change in behaviour.

The use case is that a shared library needed by a Python extension is available 
on LD_LIBRARY_PATH such that a non-ctypes linking operation would find it, but 
ctypes.util.find_library() won't, so a user of the extension (who may not be 
its developer) can't load the extension easily. See also my comment 
http://bugs.python.org/issue9998#msg165806

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on linux

[Martin Panter]

Martin Panter added the comment:

I tend to agree with the comment at 
. Find_library() is documented as 
emulating a build-time linker, not run-time. Build-time linking ignores 
LD_LIBRARY_PATH.

Maybe it would make more sense to pass in an optional list of -L directories to 
search, but it really depends on what the use cases are.

Pau: What is wrong with the more direct CDLL("libspatialindex_c.so") proposal 
that bypasses find_library()?

--
nosy: +martin.panter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on linux

[Pau Tallada]

Pau Tallada added the comment:

Any progress on this?
It also affects installation of rtree 
(https://github.com/Toblerity/rtree/issues/56)

--
nosy: +Pau Tallada

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on linux

[Vinay Sajip]

Changes by Vinay Sajip vinay_sa...@yahoo.co.uk:


--
versions: +Python 3.5 -Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9998
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on linux

[Vinay Sajip]

Vinay Sajip added the comment:

Since 3.4 entered beta last week, it is now in feature freeze, so only bug 
fixes are allowed, unfortunately.

Since I did the ld patch I didn't want to commit it before getting another 
developer to review it, and it looks as if it just dropped under everyone's 
radar.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9998
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on linux

[Daniel Blanchard]

Daniel Blanchard added the comment:

Any chance of this making it into 3.4? This is a rather annoying deficiency of 
find_library().

--
nosy: +Daniel.Blanchard

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9998
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on linux

[Vinay Sajip]

Changes by Vinay Sajip vinay_sa...@yahoo.co.uk:


--
versions: +Python 3.4 -Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9998
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on linux

[Łukasz Langa]

Łukasz Langa luk...@langa.pl added the comment:

What changed since http://bugs.python.org/issue2936?

--
nosy: +lukasz.langa

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9998
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on linux

[Vinay Sajip]

Vinay Sajip vinay_sa...@yahoo.co.uk added the comment:

 What changed since http://bugs.python.org/issue2936?

Well for one thing, #2936 was a bug report (behavior) but this is an 
enhancement request. It seems that if you don't know exactly what a library is 
called, you can't use find_library to locate it even if it happens to be on 
LD_LIBRARY_PATH - and if you don't have the name, you can't load it using 
cdll.LoadLibrary.

ISTM LoadLibrary wants the full library name (including extension, on Linux); 
find_library helps determine that in a cross-platform way (since you can just 
give it the stem of the library rather than the whole filename).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9998
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on linux

[Łukasz Langa]

Łukasz Langa luk...@langa.pl added the comment:

Fair enough. From what I've gathered though, it looks like the window for 
inclusion in 3.3 is already closed, right?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9998
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on linux

[Vinay Sajip]

Vinay Sajip vinay_sa...@yahoo.co.uk added the comment:

I added an updated patch which uses ld, and added a test (Linux/Unix only).

--
hgrepos: +130
stage:  - patch review

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9998
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on linux

[Vinay Sajip]

Changes by Vinay Sajip vinay_sa...@yahoo.co.uk:


Added file: http://bugs.python.org/file25822/ce3be1c1091a.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9998
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on linux

[Vinay Sajip]

Vinay Sajip vinay_sa...@yahoo.co.uk added the comment:

There is a problem in this area of the code, not especially with the patch but 
with how gcc works (or doesn't). To illustrate:

---
Version info
---
vinay@eta-hardy:/tmp$ uname -a
Linux eta-hardy 2.6.24-29-generic #1 SMP Wed Aug 10 16:34:32 UTC 2011 i686 
GNU/Linux
vinay@eta-hardy:/tmp$ gcc --version
gcc (GCC) 4.2.4 (Ubuntu 4.2.4-1ubuntu4)
Copyright (C) 2007 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
---
Create an empty file:
---
vinay@eta-hardy:/tmp$ touch dummy.c
---
Compile a shared library from it:
---
vinay@eta-hardy:/tmp$ gcc -shared -o libdummy.so dummy.c
---
Invoke gcc with patched command line:
---
vinay@eta-hardy:/tmp$ gcc -L /tmp -Wl,t -o /dev/null -ldummy
/usr/bin/ld: t: No such file: No such file or directory
collect2: ld returned 1 exit status
---
It's not patch related, here's what happens with the unpatched line
---
vinay@eta-hardy:/tmp$ gcc -Wl,t -o /dev/null -ldummy
/usr/bin/ld: t: No such file: No such file or directory
collect2: ld returned 1 exit status
---
Calling ld directly works as expected. With no search path:
---
vinay@eta-hardy:/tmp$ ld -t -o /dev/null -ldummy
ld: mode elf_i386
ld: cannot find -ldummy
---
With the search path supplied:
---
vinay@eta-hardy:/tmp$ ld -t -L /tmp -o /dev/null -ldummy
ld: mode elf_i386
-ldummy (/tmp/libdummy.so)
ld: warning: cannot find entry symbol _start; not setting start address
vinay@eta-hardy:/tmp$ 

So, ISTM that the find_library code needs changing to use ld, else it will not 
work on all platforms.

--
nosy: +vinay.sajip

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9998
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on linux

[Yaroslav Bulatov]

Changes by Yaroslav Bulatov yarosla...@gmail.com:


--
nosy: +yaroslavvb

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9998
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on linux

[Yaroslav Bulatov]

Yaroslav Bulatov yarosla...@gmail.com added the comment:

This causes problem for Freetype Python bindings on Linux

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9998
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on linux

[Jonathan Niehof]

Jonathan Niehof jnie...@lanl.gov added the comment:

Yaroslav: does the patch cause problems, or the original issue? If the former, 
could you be specific so I can try and fix it?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9998
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on linux

[Yaroslav Bulatov]

Yaroslav Bulatov yarosla...@gmail.com added the comment:

Sorry for confusion, I meant the original problem causes problems. I haven't 
tested the patch because my target machines don't have gcc

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9998
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9998] ctypes find_library should search LD_LIBRARY_PATH on linux

[R. David Murray]

Changes by R. David Murray rdmur...@bitdance.com:


--
nosy: +amaury.forgeotdarc, belopolsky -theller
title: find_library should search LD_LIBRARY_PATH on linux - ctypes 
find_library should search LD_LIBRARY_PATH on linux

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9998
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com