On 4 September 2013 16:44, M.-A. Lemburg <m...@egenix.com> wrote: > We need to get rid off hacks like setuptools if we ever > want to see light at the end of the packaging tunnel.
Yes, the whole point of formally defining the requirements for the setup.py CLI (along with a hook system for metadata 2.0 to make providing setup.py itself completely optional) is to make it so that projects don't need to use distutils *or* setuptools. They just need to provide a setup.py that follows the spec or the appropriate static metadata to tell automated build tools how to run Python code from the source distribution to do the build. With the assumption that pip (or an equivalent tool) will be available to retrieve dependencies, developers can just declare the appropriate dependency metadata for their chosen build system (using install_requires in current setuptools defined metadata, or build_requires in metadata 2.0), and the build system will sort it out based on the dependency declarations. However, the lack of a viable *transition plan* is one of the big things that crippled the ultimate effectiveness of the distutils2 effort. The two most popular build systems for Python at this point are vanilla distutils and setuptools. setuptools is the one that people that actually need the dependency management and entry point features are using. Our feelings about its implementation details are completely irrelevant: those projects *have* to be able to migrate easily, or the transition to metadata 2.0 isn't going to work. Fortunately, Jason and PJE are both on-board with the current metadata 2.0 efforts, and metadata 2.0 has been redesigned so it can be shipped in parallel with metadata 1.1 and the supporting setuptools-defined metadata formats. This means it is likely that setuptools will gain the ability to emit those formats once they're standardised. So, the projects that explicitly use setuptools shouldn't be a problem. That then leaves the projects that use plain distutils. How do we get *them* into the shiny metadata 2.0 future, given that we have five legacy versions of distutils in widespread use that we need to get to support the new metadata 2.0 standards, when those standards *haven't even been defined yet*, and definitely won't be defined in time for Python 3.4 feature freeze in November (adding a *sixth* legacy version to support). Note that the distutils freeze has *already been lifted*. The problem that originally led to the freeze was a change to an internal API breaking setuptools in a CPython *maintenance* release. PJE has since indicated that he would have accepted that breakage as requiring a setuptools upgrade in a *feature* release - it was the fact it happened in a maintenance release that he had a problem with. So, no distutils freeze, but still a problem with supporting new metadata standards in previously released versions of Python. Once that was clarified, the freeze was lifted, but there was also the realisation around that time that any proposed changes to the packaging ecosystem are irrelevant if they aren't supported at least as far back as Python 2.6 (so they work with the system Python on RHEL/CentOS/ScientificLinux and other long term support Linux versions of similar vintage). Given where we are now, the *ideal* solution would be for Python 3.4 distutils to be brought up to feature parity with current setuptools and a "backports.distutils" module published on PyPI. When the metadata 2.0 standards are defined, support would be added to the Python 3.5 distutils, and published immediately on PyPI as backports.distutils. One potential problem with that approach is that distutils in trunk is currently Python 3 only, while a backport would need to be 2/3 compatible. That's likely more feasible now than it was when packaging/distutils2 were being worked on, but the initial set of changes to make it 2/3 compatible will probably still involve some pain. However, a more significant problem is that the whole idea is based on pure vapourware. That ideal "it's just like setuptools, but with a more elegant implementation that could be used to replace distutils in Python 3.4" library *doesn't exist*, and I have no desire to wait around in the (likely vain) hope of somebody stepping up to create it. Instead, I think the far more pragmatic option at this point is to just tell people "your setup.py must run correctly with setuptools imported in that process. If it doesn't, it is your setup.py is broken, not the build tool that imported setuptools, or setuptools itself for monkey-patching distutils". Most projects don't need to do anything special to handle that situation, and for the few that do "'setuptools' in sys.modules" is a reliable way to check for its presence. This approach is a useful hack until somebody actually *produces* a "setuptools-lite" which does everything important that setuptools does without monkey-patching distutils, or until we have successfully made the transition to a metadata 2.0 ecosystem where projects feel comfortable declaring a dependency on a custom build system and assuming that end users will be able to handle it (whether directly or by downloading a wheel file). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
