[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

reimar added the comment: Sorry for my laziness. I opened Issue36753. -- ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

STINNER Victor added the comment: > In short, RTLD_LOCAL is not supported. reimar: "I am sorry, this is not a workable stance. This does not just affect loading libpython directly. (...)" This issue is now closed, as bpo-21536. Would you mind to open a new issuse to clearly explain your

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

reimar added the comment: > In short, RTLD_LOCAL is not supported. I am sorry, this is not a workable stance. This does not just affect loading libpython directly. It also affects dlopen of a library that links to a library ... that links to libpython. For a developer is impossible to know

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

STINNER Victor added the comment: FYI I modified Python 3.8 to never link C extensions to libpython on Unix (except on Android): bpo-21536, but with a different rationale (better reasons to do so). -- ___ Python tracker

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

STINNER Victor added the comment: > What do you mean, "no consensus could be found"? I don't see anyone objecting > the change. I propose a change to always link and a change to never link. I don't see any tracking towards one option. It seems like there are issues on Android. Anyway, this

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

Antoine Pitrou added the comment: What do you mean, "no consensus could be found"? I don't see anyone objecting the change. -- ___ Python tracker ___

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

STINNER Victor added the comment: > Downstream (RHEL) issue: > https://bugzilla.redhat.com/show_bug.cgi?id=1585201 This issue has been closed as "not a bug". -- Since this issue has been created, no consensus could be found. So I close the issue to keep the status quo. In short,

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

STINNER Victor added the comment: Another Fedora on Python2: https://bugzilla.redhat.com/show_bug.cgi?id=1667914 -- ___ Python tracker ___

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

Change by STINNER Victor : -- nosy: +petr.viktorin ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

Ned Deily added the comment: Perhaps you should bring up this proposed change in distutils-sig before committing. It's probably an OK change but it would be good to try to get some feedback from the downstream users who might be affected by it. -- nosy: +ned.deily

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

Change by Xavier de Gaye : -- nosy: -xdegaye ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

Change by Xavier de Gaye : -- Removed message: https://bugs.python.org/msg326509 ___ Python tracker ___ ___ Python-bugs-list

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

STINNER Victor added the comment: I wrote the PR 9912 to ensure that C extensions are never linked to libpython. I tested my change using: git clean -fdx ./configure --with-pydebug --enable-shared make for SO in build/lib.*/*.so Modules/*.so; do ldd $SO|grep libpython; done # grep must not

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

Change by STINNER Victor : -- pull_requests: +9270 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

serge-sans-paille added the comment: Not an expert of Python build, but I've been creating a few « reverse engineer challenge » where I had to ship modified version of the interpreter, so played with it a bit. I agree consistency is nice to reason about. It looks better to me to *not* link

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

Change by Charalampos Stratakis : -- nosy: +cstratak ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

Matthias Klose added the comment: Debian/Ubuntu doesn't link against the library because it would add dependencies on all supported Python versions. Normally this is just during transition times, but e.g. for the upcoming Ubuntu 18.10 release we didn't finish the transition and so ship two

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

Antoine Pitrou added the comment: But as you said, we need consistency: either *never* link to libpython, or *always* link to libpython. You are proposing to always link, I'm arguing we should never link. -- ___ Python tracker

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

STINNER Victor added the comment: > I'm not talking about the standard library obviously. I don't remember my > original use case exactly, but I must have been compiling a C extension on a > system and expected it to work on another. It seems like we are talking about two different things:

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

Antoine Pitrou added the comment: > Is it a real use case? Why would anyone use a RHEL binary on Debian? Debian > already provides the full standard library. I'm not talking about the standard library obviously. I don't remember my original use case exactly, but I must have been compiling

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

STINNER Victor added the comment: > It means a C extension compiled with a shared-library Python cannot be > imported on a monolithic Python (which doesn't have libpython.so). It's a > real problem when you want to redistribute compiled C extensions: if you > compile it on RedHat/CentOS,

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

STINNER Victor added the comment: > * FreeBSD 12 (alpha): Python 2.7 and Python 3.6 are linked to libpython > (--enable-shared) Note: _ctypes, _hashlib and _struct are all linked to libpython, on Python 2 and Python 3. Antoine: > Do you realize libpython.so doesn't exist on Debian /

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

STINNER Victor added the comment: Antoine: > Why do you call this a bug? "./configure --enable-shared && make" links C extensions to libpython. It's surprising that C extensions compiled by Makefile behave differently (not linked to libpython). We need consistency: either *never* link to

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

STINNER Victor added the comment: Extract of Antoine's comment on bpo-21536: > (AFAIK, systems notorious for providing shared library Pythons are > RedHat-alike systems, while Debian/Ubuntu provide statically linked Pythons) Oh. I didn't notice this major difference... * Ubuntu 16.04:

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

Antoine Pitrou added the comment: Le 27/09/2018 à 12:49, STINNER Victor a écrit : > > I search if C extensions of the Python standard libraries are always linked > or not to libpython... it's complicated. I tested _ctypes, _hashlib and > _struct modules: > > * Debian and Ubuntu: NOT linked

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

STINNER Victor added the comment: Ah, it seems like the bpo-832799 (reported in 2003) is similar to the RHEL bug: https://bugzilla.redhat.com/show_bug.cgi?id=1585201 Extract of the RHEL bug report: --- pythontest.c: #include int main(int argc, char *argv[]) { void *pylib =

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

STINNER Victor added the comment: I search if C extensions of the Python standard libraries are always linked or not to libpython... it's complicated. I tested _ctypes, _hashlib and _struct modules: * Debian and Ubuntu: NOT linked to libpython * Conda: LINKED to libpython * Mageia 7: LINKED

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

Xavier de Gaye added the comment: > Downstream (RHEL) issue: > https://bugzilla.redhat.com/show_bug.cgi?id=1585201 Not sure that this is a bug. As Petr Viktorin points out in Comment #4, the reproducer is not expected to work because of the semantics of RTLD_LOCAL. So the current bpo issue

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

Antoine Pitrou added the comment: Of course, one workaround to satisfy everyone would be to build a (empty) libpython.so even on static Python builds. But I'm not sure Debian/Ubuntu would package it. -- nosy: +doko ___ Python tracker

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

Antoine Pitrou added the comment: Why do you call this a bug? For me it's the reverse: it's linking to libpython.so which is a bug. It means a C extension compiled with a shared-library Python cannot be imported on a monolithic Python (which doesn't have libpython.so). It's a real problem

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

STINNER Victor added the comment: I copied the nosy list from bpo-32430: people who understand and care about the Modules/Setup file :-) -- nosy: +barry, eric.smith, koobs, martin.panter, mdk, nascheme, pitrou, twouters, xdegaye, yan12125 ___

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

STINNER Victor added the comment: Example of the bug: --- $ git apply ~/Setup.patch $ ./configure --with-pydebug --enable-shared $ make $ grep _contextvars Makefile (...) Modules/_contextvarsmodule.o: $(srcdir)/Modules/_contextvarsmodule.c; $(CC) $(CCSHARED) $(PY_CFLAGS) $(PY_CPPFLAGS) -c

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

STINNER Victor added the comment: Downstream (RHEL) issue: https://bugzilla.redhat.com/show_bug.cgi?id=1585201 -- ___ Python tracker ___

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

STINNER Victor added the comment: Setup.patch: Example of patch to modify Modules/Setup to compile _contextvars as a shared library, to test the fix. -- Added file: https://bugs.python.org/file47831/Setup.patch ___ Python tracker

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

Change by STINNER Victor : -- keywords: +patch pull_requests: +8991 stage: -> patch review ___ Python tracker ___ ___

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

New submission from STINNER Victor : Python can be compiled in "shared" mode: "./configure --enable-shared", Py_ENABLE_SHARED is defined in pyconfig.h. Most Linux distributions use this configuration. By default, Python builds most C extensions using setup.py which is based on distutils.