[issue38794] Setup: support linking openssl statically
Lukas Vacek added the comment: Thanks for the explanation of the secret workings behind the scenes. What a productive and polite response. -- ___ Python tracker <https://bugs.python.org/issue38794> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38794] Setup: support linking openssl statically
Lukas Vacek added the comment: For the record, this would have been solved more than a year ago already. When this change was proposed more than a year ago it was rejected with "There is no need to add more configure flags to build Python with a custom OpenSSL installation. " yet now it's ok to add a new option --with-openssl-rpath https://bugs.python.org/issue43466 ? And the first comment there, from python core dev nonetheless, is suggesting static linking as well. Emm... this would have been solved year and half ago. I would be happy to completely drop my proposed (and approved on gihub) changes and implement it in a different way. The maintainer's attitude as demonstrated here can be really harmful in open-source projects (many of us still remember eglibc fork back in the day) but fortunately this is the first time I noticed such attitude among python developers. Importantly the issue is resolved now (did it take a request from IBM's customer to get this implemented ;-) ?) and hopefully a lesson learnt and Christian will be more welcoming and less judgemental of outsiders' contributions. -- ___ Python tracker <https://bugs.python.org/issue38794> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38794] Setup: support linking openssl statically
Lukas Vacek added the comment:
Christian, I don't want to argue.
Feel free to close both this bug and the PR as "enhancement rejected". Adding
one ./configure option is not really worth arguing over, I can always compile
python as I need. I just tried to solve the same problem for others because
other people also want a simple way to install a new version of Python on old
distros and have a working ssl module.
If my contribution is not welcome it's fine.
However, I have a feeling the reason for the rejection is not technical because
after I stated I am happy to help solve the problem in other ways ("If you can
think of a better real world solution than static linking OpenSSL please share
and let's do it!") you literally twisted my words 180 degrees ("You are trying
to solve a problem by dictating the solution at the same time."). This is very
inappropriate.
--
___
Python tracker
<https://bugs.python.org/issue38794>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38794] Setup: support linking openssl statically
Lukas Vacek added the comment: In ideal world OpenSSL would provide stable ABI just like the other libraries Python depends on. That would be, unarguably, the best way to achieve that goal. Agreed. Aferall frequent OpenSSL ABI breakages are the reason why apple switched to their own cryptography ABI (https://developer.apple.com/library/archive/documentation/Security/Conceptual/cryptoservices/SecureNetworkCommunicationAPIs/SecureNetworkCommunicationAPIs.html): """ Although OpenSSL is commonly used in the open source community, it doesn’t provide a stable API from version to version. For this reason, the programmatic interface to OpenSSL is deprecated in macOS and is not provided in iOS. Use of the Apple-provided OpenSSL libraries by apps is strongly discouraged. To ensure compatibility, if your app depends on OpenSSL, you should compile it yourself and statically link a known version of OpenSSL into your app. Such use works on both iOS and macOS. """ However, OpenSSL does not provide stable ABI and Python should deal with that as nicely as possible. If you can think of a better real world solution than static linking OpenSSL please share and let's do it! -- ___ Python tracker <https://bugs.python.org/issue38794> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38794] Setup: support linking openssl statically
Lukas Vacek added the comment: However, I'm still convinced many would appreciate adding this ./configure option: https://github.com/pyenv/pyenv/issues/1184#issuecomment-403149437 https://joshspicer.com/python37-ssl-issue https://superuser.com/questions/1428109/install-python-3-7-3-from-source-with-ssl-failed https://raspberrypi.stackexchange.com/questions/88150/how-to-install-python-3-7-with-ssl https://stackoverflow.com/questions/53543477/building-python-3-7-1-ssl-module-failed https://help.dreamhost.com/hc/en-us/articles/115000702772-Installing-a-custom-version-of-Python-3#3.7.1 https://stackoverflow.com/questions/55928393/compile-python-3-6-statically-with-openssl https://readthisblog.net/2018/08/13/til-how-to-build-python-3-7-with-statically-linked-libssl-and-libcrypto/ https://gist.github.com/eddy-geek/9604982 If you look at the associated PR you will see it should work fine, and be useful, on other POSIX systems as well. I can test on *BSD if necessary. Only extra depedency of --with-openssl-static compared to --with-openssl is an assumption of existing files libssl.a libcrypto.a and OpenSSL has provided these files (when compiled static) since the very beginning despite somewhat frequent ABI changes. Option --with-openssl-static works with current libressl as well. -- ___ Python tracker <https://bugs.python.org/issue38794> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38794] Setup: support linking openssl statically
Lukas Vacek added the comment: Fair enough. Btw. I picked versions 3.7 and up to show this feature should eventually be backported to older CPython versions that depend on OpenSSL >= 1.0.2. This feature is to avoid relying on rpath and relying on a seperate installation to be installed in specific location in your target system. Also there is no Tools/ssl/multissl.py in current cpython master. I believe it's much preferably for administrators to use static linking in this case, but if you think it's not worth it, that's fine. Please close the related PR in the github with your explanation. Thank you. -- ___ Python tracker <https://bugs.python.org/issue38794> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38794] Setup: support linking openssl statically
Change by Lukas Vacek : -- title: Setup: support linking openssl staticly -> Setup: support linking openssl statically ___ Python tracker <https://bugs.python.org/issue38794> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38794] Setup: support linking openssl staticly
Change by Lukas Vacek : -- keywords: +patch pull_requests: +16663 stage: -> patch review pull_request: https://github.com/python/cpython/pull/17153 ___ Python tracker <https://bugs.python.org/issue38794> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38794] Setup: support linking openssl staticly
New submission from Lukas Vacek : Since 3.7 python depends on OpenSSL>=1.0.2 - this makes it hard to compile Python with SSL on older (yet still vendor supported) linux distributions. It's easy to compile CPython even on old distributions like RHEL5, RHEL6, Ubuntu14.04 etc. except for ssl module. Let's add an option to compile SSL staticly (--with-openssl-static) to make it easy to compile Python with SSL on systems with OpenSSL<1.0.2 as you usually don't want to install newer openssl as system libary nor mess with rpath/set LD_LIBRARY_PATH every time you run python. When --with-openssl-static is not passed to ./configure everything should behave like before. Installing CPython including ssl on system as old as RHEL5 with this option would be as easy as (after installing required build dependencies from rhel5 repositories and libffi(-devel) libraries): wget https://www.openssl.org/source/openssl-1.0.2t.tar.gz tar xf openssl-1.0.2t.tar.gz cd openssl-1.0.2t ./config --openssldir=/etc/pki/tls -fPIC make wget https://www.python.org/ftp/python/3.7.5/Python-3.7.5.tgz tar xf Python-3.7.5.tgz cd Python-3.7.5 ./configure --with-openssl-static=path_to_just_compiled_ssl --prefix=prefix_path make make install -- assignee: christian.heimes components: SSL messages: 356600 nosy: Lukas.Vacek, christian.heimes priority: normal severity: normal status: open title: Setup: support linking openssl staticly type: enhancement versions: Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue38794> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19186] expat symbols should be namespaced in pyexpat again
Lukas Vacek added the comment: Actually CFLAGS do indeed come before any include directories. I can reproduce your problem: mkdir /tmp/extra_include cp /usr/include/expat.h /usr/include/expat_external.h /tmp/extra_include/ make CFLAGS="-I/tmp/extra_include" You should use CPPFLAGS for extra include directories. Not an issue in CPython. -- ___ Python tracker <http://bugs.python.org/issue19186> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19186] expat symbols should be namespaced in pyexpat again
Lukas Vacek added the comment: Hi Peter, Thanks for taking the time to report your issue. I just tried on fresh up-to-date CentOS 6 and I could not reproduce the issue. However, I think I figured out what went wrong. I can see you are supplying custom -I and -L paths (-I/apps/prod/releases/3.0/include ...) I suspect that when compiling pyexpat.c (in Modules/) the header file expat.h is picked up from this location before the bundled expat.h (in Modules/expat/) is found. But when compiling xmlparse.c (in Modules/expat/) the bundled header file expat.h *is* used this time. So we end up with our bundled expat compiled using namespacing and so defining symbols like PyExpat_XML_SetCommentHandler while pyexpat.c is looking for XML_SetCommentHandler because it is compiled *not* using our bundled expat.h. I am not sure yet why your custom "-I" comes before "-I/ae/data/soft/opensource/build/python3/master/Python-3.4.0/Modules/expat" as it certainly should not. So far I tried building with CPPFLAGS and CFLAGS and in both cases my custom paths come after "-I.../Modules/expat" as it should. Can you please provide your ./configure and make command lines and relevant environment variables? Thanks, Lucas -- ___ Python tracker <http://bugs.python.org/issue19186> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21282] setup.py: More informative error msg for modules which built but failed import check
New submission from Lukas Vacek: Hey, Currently when a module builds successfully during cpython build but it can't be imported (import check around line 330 in setup.py) the module shows in "Failed to build these modules: " which can be misleading. Especially when linking against libraries in non-standard locations the user would set LD_FLAGS and CPPFLAGS and then ... wonder why the cpython build process is not picking the libraries up and the module is not built. I think the modules which *have built* but were removed because of a failed import check should be marked as such so the user knows what happens and can take appropriate actions (set LD_LIBRARY_PATH as well, for example). A patch attached - it's a very simple change (about 10 lines), created with "hg export". Thanks, Lucas -- components: Build files: setup_failed_import_hgexport messages: 216684 nosy: Lukas.Vacek priority: normal severity: normal status: open title: setup.py: More informative error msg for modules which built but failed import check type: enhancement versions: Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 3.4, Python 3.5 Added file: http://bugs.python.org/file34942/setup_failed_import_hgexport ___ Python tracker <http://bugs.python.org/issue21282> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19186] expat symbols should be namespaced in pyexpat again
Lukas Vacek added the comment: attaching patch -- keywords: +patch nosy: +Lukas.Vacek Added file: http://bugs.python.org/file33905/fix_expat_names.patch ___ Python tracker <http://bugs.python.org/issue19186> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
