[Distutils] Varying dependencies by python minor version (2.7.9)
Hi distutils-sig, Tornado depends on backports.ssl_match_hostname on python 2, but this is no longer needed with (cpython) 2.7.9. We have a PR to make this dependency conditional on the minor version of python ( https://github.com/tornadoweb/tornado/pull/1276/files). We could also make the test something like `hasattr(ssl, 'SSLContext')`, which would be more compatible with pypy and other implementations, but the fundamental issues remain. Is it a good idea to distribute packages that will install different dependencies on different minor versions of python? Is it even possible with binary distributions? I'm thinking it's probably best to just live with the unused extra dependency (until we can completely drop support for python versions without modern ssl libraries), but I wanted to check and see if there is a clean way to deal with this. Thanks, -Ben ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Varying dependencies by python minor version (2.7.9)
It is possible and supported but not very elegant since the marker
syntax can only compare versions as strings (you wouldn't want to use
and have the wrong dependencies under Python 2.7.10). One solution
would be to check that you are not running on any of the older
releases of Python 2.7, like so:
In setup.py:
extras_require={
':python_version == 2.7 and (python_full_version!=2.7.8
and python_full_version!=2.7.7 and ...)':
['backports.ssl_match_hostname'],
}
This means the extra named '' a.k.a. the default dependencies includes
your package, but only if the expression after the colon : evaluates
to True. bdist_wheel preserves the expression in .whl binary packages.
On Fri, Dec 12, 2014 at 11:09 AM, Ben Darnell b...@bendarnell.com wrote:
Hi distutils-sig,
Tornado depends on backports.ssl_match_hostname on python 2, but this is no
longer needed with (cpython) 2.7.9. We have a PR to make this dependency
conditional on the minor version of python
(https://github.com/tornadoweb/tornado/pull/1276/files). We could also make
the test something like `hasattr(ssl, 'SSLContext')`, which would be more
compatible with pypy and other implementations, but the fundamental issues
remain. Is it a good idea to distribute packages that will install different
dependencies on different minor versions of python? Is it even possible with
binary distributions? I'm thinking it's probably best to just live with the
unused extra dependency (until we can completely drop support for python
versions without modern ssl libraries), but I wanted to check and see if
there is a clean way to deal with this.
Thanks,
-Ben
___
Distutils-SIG maillist - Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig
___
Distutils-SIG maillist - Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] PYD/SO platform tags (#22980)
Hi distutils-sig, There's a bit of discussion going on at http://bugs.python.org/issue22980 about extending SO tags to include bitness values, and I took the opportunity to look into adding platform tags for .pyd files on Windows. There's a patch up there, but I'm interested in any thoughts or concerns people may have that I haven't considered. Distribution is likely to be most affected, and I figured distutils-sig is most likely to have people with strong opinions on the subject anyway, so I'm posting here :) Basically, importlib.machinery.EXTENSION_SUFFIXES is all that changes. It'll go from [.pyd] to [.cp35-win32.pyd, .cp3-win32.pyd, .pyd] (where win32 may also be win_amd64, win_ia64 or win_arm, which are the platforms that have #ifdefs in pyconfig.h). By default, distutils' build_ext will use the first extension, and so builds will get the .cp35-win32.pyd tag. You can only get the .cp3-win32.pyd tag (for stable ABI) by customizing the build process - I don't think that's an unreasonable expectation, especially on Windows. I can see potential for making more generic installable packages with this change, though I'm not pushing for that. The big win is when you're building and testing 32-bit and 64-bit pyds through an IDE - it greatly simplifies how the output directories and search paths are organized. Thoughts/concerns? Cheers, Steve ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PYD/SO platform tags (#22980)
On Dec 12, 2014, at 6:23 PM, Steve Dower steve.do...@microsoft.com wrote: Hi distutils-sig, There's a bit of discussion going on at http://bugs.python.org/issue22980 about extending SO tags to include bitness values, and I took the opportunity to look into adding platform tags for .pyd files on Windows. There's a patch up there, but I'm interested in any thoughts or concerns people may have that I haven't considered. Distribution is likely to be most affected, and I figured distutils-sig is most likely to have people with strong opinions on the subject anyway, so I'm posting here :) Basically, importlib.machinery.EXTENSION_SUFFIXES is all that changes. It'll go from [.pyd] to [.cp35-win32.pyd, .cp3-win32.pyd, .pyd] (where win32 may also be win_amd64, win_ia64 or win_arm, which are the platforms that have #ifdefs in pyconfig.h). By default, distutils' build_ext will use the first extension, and so builds will get the .cp35-win32.pyd tag. You can only get the .cp3-win32.pyd tag (for stable ABI) by customizing the build process - I don't think that's an unreasonable expectation, especially on Windows. I can see potential for making more generic installable packages with this change, though I'm not pushing for that. The big win is when you're building and testing 32-bit and 64-bit pyds through an IDE - it greatly simplifies how the output directories and search paths are organized. Thoughts/concerns? Well here’s the code that computes what tags pip will accept: https://github.com/pypa/pip/blob/develop/pip/pep425tags.py Obviously any additional information put into the tag will increase the number of builds someone has to do in order to get full coverage, however if all of the data in that are things that genuinely make things binary incompatible then that’s the right thing to do. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PYD/SO platform tags (#22980)
On Fri, 12 Dec 2014 23:23:03 + Steve Dower steve.do...@microsoft.com wrote: Hi distutils-sig, There's a bit of discussion going on at http://bugs.python.org/issue22980 about extending SO tags to include bitness values, and I took the opportunity to look into adding platform tags for .pyd files on Windows. There's a patch up there, but I'm interested in any thoughts or concerns people may have that I haven't considered. Distribution is likely to be most affected, and I figured distutils-sig is most likely to have people with strong opinions on the subject anyway, so I'm posting here :) Basically, importlib.machinery.EXTENSION_SUFFIXES is all that changes. It'll go from [.pyd] to [.cp35-win32.pyd, .cp3-win32.pyd, .pyd] (where win32 may also be win_amd64, win_ia64 or win_arm, which are the platforms that have #ifdefs in pyconfig.h). For the record, https://www.python.org/dev/peps/pep-3149/#pep-384 suggests abi3 as the ABI tag for the Python 3 stable ABI. Regards Antoine. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PYD/SO platform tags (#22980)
Antoine Pitrou wrote: Sent: Friday, December 12, 2014 1551 To: Distutils-Sig@Python.Org Subject: Re: [Distutils] PYD/SO platform tags (#22980) On Fri, 12 Dec 2014 23:23:03 + Steve Dower steve.do...@microsoft.com wrote: Hi distutils-sig, There's a bit of discussion going on at http://bugs.python.org/issue22980 about extending SO tags to include bitness values, and I took the opportunity to look into adding platform tags for .pyd files on Windows. There's a patch up there, but I'm interested in any thoughts or concerns people may have that I haven't considered. Distribution is likely to be most affected, and I figured distutils-sig is most likely to have people with strong opinions on the subject anyway, so I'm posting here :) Basically, importlib.machinery.EXTENSION_SUFFIXES is all that changes. It'll go from [.pyd] to [.cp35-win32.pyd, .cp3-win32.pyd, .pyd] (where win32 may also be win_amd64, win_ia64 or win_arm, which are the platforms that have #ifdefs in pyconfig.h). For the record, https://www.python.org/dev/peps/pep-3149/#pep-384 suggests abi3 as the ABI tag for the Python 3 stable ABI. My legalistic rationale for using cp3 is that it's actually the version tag, not the ABI tag. It seemed from my reading that you'd get tags like cp35-abi3-win32, which is not helpful because you've still specified the minor version in the tag. Either it'll work in any CPython 3.x (cp3) or it needs a specific minor version (cp35), so I didn't see the value in keeping the ABI tag. Of course, this only works because Windows doesn't use the ABI tag for anything else. Debug builds are encoded differently, and AFAICT the malloc and Unicode options are currently unnecessary and I'm not making any change to make them necessary. I could be swayed on this point fairly easily though, if say there are other interpreters out there not compatible enough with CPython to use the cp tag but that can still use stable ABI extensions that were build assuming CPython. Or I could just drop the second tag and only have cp35-win32.pyd or .pyd. We've made it this long without a specifier, so there's no real need to go overboard :) Cheers, Steve Regards Antoine. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PYD/SO platform tags (#22980)
Donald Stufft wrote: On Dec 12, 2014, at 6:23 PM, Steve Dower steve.do...@microsoft.com wrote: Hi distutils-sig, There's a bit of discussion going on at http://bugs.python.org/issue22980 about extending SO tags to include bitness values, and I took the opportunity to look into adding platform tags for .pyd files on Windows. There's a patch up there, but I'm interested in any thoughts or concerns people may have that I haven't considered. Distribution is likely to be most affected, and I figured distutils-sig is most likely to have people with strong opinions on the subject anyway, so I'm posting here :) Basically, importlib.machinery.EXTENSION_SUFFIXES is all that changes. It'll go from [.pyd] to [.cp35-win32.pyd, .cp3-win32.pyd, .pyd] (where win32 may also be win_amd64, win_ia64 or win_arm, which are the platforms that have #ifdefs in pyconfig.h). By default, distutils' build_ext will use the first extension, and so builds will get the .cp35-win32.pyd tag. You can only get the .cp3-win32.pyd tag (for stable ABI) by customizing the build process - I don't think that's an unreasonable expectation, especially on Windows. I can see potential for making more generic installable packages with this change, though I'm not pushing for that. The big win is when you're building and testing 32-bit and 64-bit pyds through an IDE - it greatly simplifies how the output directories and search paths are organized. Thoughts/concerns? Well here’s the code that computes what tags pip will accept: https://github.com/pypa/pip/blob/develop/pip/pep425tags.py Obviously any additional information put into the tag will increase the number of builds someone has to do in order to get full coverage, however if all of the data in that are things that genuinely make things binary incompatible then that’s the right thing to do. It could reduce the number of builds that someone has to do, as you could have a single wheel (for Windows) that includes all the platform-specific extensions rather than needing separate wheels. Whether this is an advantage or not deserves to be debated, but there are other installers that could benefit from having incompatible pyd's side-by-side in a way that Python will load the correct one. Oh, it's also essential for user site-packages on Windows, since the same directory is used for both 32-bit and 64-bit versions of Python. That's actually the most compelling use-case for me, since I'm still keen to see about moving to a secure install for Python 3.5 and in that case we'll have more people using it. Cheers, Steve ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PYD/SO platform tags (#22980)
Steve Dower wrote: Antoine Pitrou wrote: My legalistic rationale for using cp3 is that it's actually the version tag, not the ABI tag. It seemed from my reading that you'd get tags like cp35-abi3-win32, which is not helpful because you've still specified the minor version in the tag. Either it'll work in any CPython 3.x (cp3) or it needs a specific minor version (cp35), so I didn't see the value in keeping the ABI tag. Apologies, I got this understanding from PEP 425, not 3149/384. Using abi3-win32 rather than cp3-win32 would also make sense. I'm not strongly for or against either variant. Cheers, Steve ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
