[issue4434] Embedding into a shared library fails
Björn Lindqvist added the comment: The bug is still present on the default build which many distros uses which produces an invalid build. That means ./configure make make install is broken! --enable-shared should be turned on by default because many (most?) distros build with the default settings. -- components: +Build -Library (Lib) nosy: +Björn.Lindqvist versions: +Python 2.7 -Python 2.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4434 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
Terry J. Reedy tjre...@udel.edu added the comment: It is not obvious that this issue is about a Python bug, rather than how to use on a specific system, or that it applies to any current version. If this really is a Python bug and applies to a current version, respond to re-open. -- nosy: +terry.reedy status: open - pending versions: +Python 2.7 -Python 2.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4434 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
John Levon movem...@users.sourceforge.net added the comment: Terry, what in the above discussion leads you to believe there is no bug? The original complaint Python cannot be embedded in shared library due to dependency problems with lib-dynload is still valid. -- status: pending - open ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4434 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
Terry J. Reedy tjre...@udel.edu added the comment: I carefully and specifically questioned the existence of current 'Python bug', which is what this tracker is for. For Linux, that would mean a bug in any of the source files in a current 2.7, 3.1.2, or 3.2.1a1 tarball release or the newer source repository versions thereof. But I do not see any claim that there is any problem with any of the files. There certainly is no bug report for any specific stdlib module, which is what is meant by the Library (Lib) component. This issue *is* about lib-dynload that I gather is supplied by Ubuntu. There may or may not be a bug in how Ubuntu packages, compiles, and loads Python 2.5.2. John, your msg76536, beginning If *Ubuntu*..., it [Ubuntu] also needs... suggests that. But unless you *know* that that results from a bug in the files downloaded by Ubuntu, you should take that up with them. So I will follow Ralf's suggestion and close this, and suggest in turn that this issue be taken to an Ubuntu support list where help is more likely. There might even be knowledgeable Ubuntu users on python-list, I don't know. Another possible suggestion to the OP is compile everything yourself with whatever compile flags you need to get all your stuff to work together. In particular, can you make your own lib-dynload that acts the way you want it to work? (But here is not the place to ask about how.) -- resolution: - invalid status: open - closed versions: +Python 2.5 -Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4434 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
rb rb-os...@justgohome.co.uk added the comment: Original reporter here. lib-dynload is part of Python's dynamic loading mechanism. Perhaps somewhere like Python/dynload_dl.c is the relevant code? Lib seemed like the right place to put the bug at the time, since it was the stdlib module files that were missing dependency declarations. Although perhaps Extension Modules would have been better? In any case, I've just tried to reproduce the bug with 2.7 (on Debian Squeeze pulling version 2.7-2 of python2.7 and python2.7-dev from experimental), and failed (both 32- and 64-bit). So it seems that this bug has been fixed at some point since 2.5. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4434 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
Terry J. Reedy tjre...@udel.edu added the comment: perhaps Extension Modules would have been better? For future reference (like if some 3.x is not ok), yes So it seems that this bug has been fixed at some point since 2.5. Great! -- resolution: invalid - out of date ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4434 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
Changes by Jan Kratochvil jan.kratoch...@redhat.com: -- nosy: +jankratochvil ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4434 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
Amaury Forgeot d'Arc amaur...@gmail.com added the comment: Does it help if you add the --whole-archive option when linking with the static library? As explained there: http://www.lysium.de/blog/index.php?/archives/222-Lost-static-objects-in-static-libraries-with-GNU-linker-ld.html This way your main program will contain the whole python interpreter, and all symbols used by time.so co will be resolved. -- nosy: +amaury.forgeotdarc ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4434 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
Changes by Thijs Triemstra li...@collab.nl: -- nosy: +thijs ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4434 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
Michael Gorven mich...@gorven.za.net added the comment: Note that on Ubuntu Hardy at least, the libpython2.5.so symlink only exists in the python2.5-dev package. This means that in order for the library to work without the dev package installed, it should dlopen libpython2.5.so.1 instead. -- nosy: +mgorven ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4434 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
Yonas yona...@gmail.com added the comment: Ralf, Linking against the static library (with '-Xlinker -export-dynamic') did _not_ solve the problem for me. gcc `python2.6-config --cflags` -c -fPIC mylib.c -o mylib.o -Xlinker -export-dynamic gcc -Xlinker -export-dynamic -lpthread -ldl -lutil -lm -lz -shared -Wl,-soname,mylib.so -o mylib.so mylib.o /usr/lib/python2.6/config/libpython2.6.a gcc `python2.6-config --cflags` -Xlinker -export-dynamic -ldl -lutil -lm -lz -o main main.c /usr/lib/python2.6/config/libpython2.6.a -- `python2.6-config --cflags` returns -I/usr/include/python2.6 -I/usr/include/python2.6 -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes Ubuntu Jaunty Linux yonas-laptop 2.6.28-11-generic #42-Ubuntu SMP Fri Apr 17 01:57:59 UTC 2009 i686 GNU/Linux -- nosy: +yonas ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4434 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
Yonas yona...@gmail.com added the comment: Same error message: import dl: ImportError: /usr/lib/python2.6/lib-dynload/dl.so: undefined symbol: PyExc_ValueError -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4434 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
Yonas yona...@gmail.com added the comment: Here's my log file (`LD_DEBUG=all ./main`) -- Added file: http://bugs.python.org/file14033/log-main.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4434 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
Changes by Yonas yona...@gmail.com: Removed file: http://bugs.python.org/file14033/log-main.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4434 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
Yonas yona...@gmail.com added the comment: Updated log file. -- Added file: http://bugs.python.org/file14034/log-main.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4434 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
Yonas yona...@gmail.com added the comment: rb's workaround works, though. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4434 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
Matthias Klose d...@debian.org added the comment: I'm sure it does work, but what is the point of linking statically to libpython.a but then having other dependencies, for example on lib-dynload/time.so? Why not just link to libpython2.5.so in the first place? speed. Using a python executable with a statically linked libpython is about 10% faster on ix86, plus with some compilers on linux profile based optimization can be used on static objects, which doesn't work with shared libraries (at least on linux using gcc). -- nosy: +doko ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4434 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
rb [EMAIL PROTECTED] added the comment: The problem, and the reason for the existence of this bug, is that I cannot build a shared object that links to libpython2.5.so.1 and works. Please don't mark this bug invalid until this problem is fixed. My proposal of adding dependencies to lib-dynload/*.so is just a suggestion on how to go about fixing this bug. If this is not a suitable solution, please suggest a solution that is. In response to your discussion on why the static library is currently useful, you haven't really answered the question. In what circumstance would someone actually want to build against the static library and then dynamically load other libraries? Because it works isn't an answer, as if this bug were fixed, then building against the shared library would validly have the same response. Why static over dynamic, if when static you have to link to dynamic anyway? ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue4434 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
John Levon [EMAIL PROTECTED] added the comment: If Ubuntu wants to deliver a static libpython, it also needs to deliver static versions of the Python modules, or accept that they're not usable from a static libpython. It makes no sense at all to mix libpython.a with these .so files. Many of them declare dynamic dependencies on stuff like libkrb5.so or libsqlite3.so. I can't conceive of a scenario where you need a static libpython AND you're OK with both dlopen() of the Python modules and any of its dynamic dependencies. What is this case? ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue4434 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
Ralf Schmitt [EMAIL PROTECTED] added the comment: no john, linking with the static library really works. the resulting executable does not depend on the shared library and it is possible to import the e.g. the time module. ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue4434 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
rb [EMAIL PROTECTED] added the comment: Ralf, I'm sure it does work, but what is the point of linking statically to libpython.a but then having other dependencies, for example on lib-dynload/time.so? Why not just link to libpython2.5.so in the first place? ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue4434 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
rb [EMAIL PROTECTED] added the comment: What is the purpose of linking to the static library if you're still going to depend on shared objects in lib-dynload? ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue4434 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
John Levon [EMAIL PROTECTED] added the comment: Besides, .so files should always declare their dependencies. ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue4434 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
Ralf Schmitt [EMAIL PROTECTED] added the comment: Yes, there is no purpose in linking with a static library when the dynamically loaded modules still depend on the shared library. And yes, the dynamically loaded libraries should declare their dependencies. My point is they do *not* depend on the shared library. That's why the static lib currently makes sense on ubuntu, but wouldn't make any sense when your proposal would be implemented. I recommend closing this issue as invalid. ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue4434 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
New submission from rb [EMAIL PROTECTED]: Python cannot be embedded in shared library due to dependency problems with lib-dynload. I am implementing a shared library (in C) that implements a specific API as a dynamically loadable plugin to an application. To implement this library, I am calling out to Python via the C API. When my code tries to load a Python module, it fails. This is because lib-dynload/time.so (for example) cannot resolve symbols that are in libpython2.5.so because it never looks there. In this test case I'm trying to import time, but it seems to fail on other modules too - presumably anything that is in lib-dynload. It also fails if I import a pure Python module that then tries to import time. The error produced is: ImportError: /usr/lib/python2.5/lib-dynload/time.so: undefined symbol: PyExc_ValueError From LD_DEBUG: 29682:file=/usr/lib/python2.5/lib-dynload/time.so [0]; needed by /usr/lib/libpython2.5.so.1.0 [0] ... 29682:relocation processing: /usr/lib/python2.5/lib-dynload/time.so 29682:symbol=PyExc_ValueError; lookup in file=./myprog [0] 29682:symbol=PyExc_ValueError; lookup in file=/lib/libdl.so.2 [0] 29682:symbol=PyExc_ValueError; lookup in file=/lib/libc.so.6 [0] 29682:symbol=PyExc_ValueError; lookup in file=/lib64/ld-linux-x86-64.so.2 [0] 29682:symbol=PyExc_ValueError; lookup in file=/usr/lib/python2.5/lib-dynload/time.so [0] 29682:symbol=PyExc_ValueError; lookup in file=/lib/libm.so.6 [0] 29682:symbol=PyExc_ValueError; lookup in file=/lib/libpthread.so.0 [0] 29682:symbol=PyExc_ValueError; lookup in file=/lib/libc.so.6 [0] 29682:symbol=PyExc_ValueError; lookup in file=/lib64/ld-linux-x86-64.so.2 [0] 29682:/usr/lib/python2.5/lib-dynload/time.so: error: symbol lookup error: undefined symbol: PyExc_ValueError (fatal) $ nm -D /usr/lib/libpython2.5.so.1|grep PyExc_ValueError 0033a7e0 D PyExc_ValueError $ ldd /usr/lib/python2.5/lib-dynload/time.so libm.so.6 = /lib/libm.so.6 (0x2afe014c9000) libpthread.so.0 = /lib/libpthread.so.0 (0x2afe0174a000) libc.so.6 = /lib/libc.so.6 (0x2afe01967000) /lib64/ld-linux-x86-64.so.2 (0x4000) I am told that the problem is that lib-dynload/*.so should depend on libpython2.5.so.1. Test case attached. mylib.c is the library that I'm implementing reduced to the problem case. myprog.c is a stub program that loads mylib.c to demonstrate the problem. The real myprog would be any third-party application that I have no control over that expects to be able to dlopen() mylib.so and call functions within it. I have been given the following workaround: in mylib.c, before PyInitialize() I can call dlopen(libpython2.5.so, RTLD_LAZY | RTLD_GLOBAL); This works, but I believe that lib-dynload/*.so should depend on libpython2.5.so.1 so this hack should not be necessary. I am using Ubuntu 8.04 with Python version 2.5.2-2ubuntu4.1. -- components: Library (Lib) files: mylib.c messages: 76454 nosy: rb severity: normal status: open title: Embedding into a shared library fails type: behavior versions: Python 2.5 Added file: http://bugs.python.org/file12134/mylib.c ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue4434 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
Changes by rb [EMAIL PROTECTED]: Added file: http://bugs.python.org/file12135/myprog.c ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue4434 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
Changes by John Levon [EMAIL PROTECTED]: -- nosy: +movement ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue4434 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
Ralf Schmitt [EMAIL PROTECTED] added the comment: ubuntu comes with a static libpython.a which you can also link against. Linking the c modules against the shared python library would make that static library pretty much useless... -- nosy: +schmir ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue4434 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
