On 21 May 2015 at 16:37, Donald Stufft <[email protected]> wrote: > Also, one of the goals a few of us has in the PyPA is that we move to a > future where the build systems are pluggable. So one package could be > building using setuptools, another building using some SciPy specific build > tool, another using a whole other one. They will all ideally have some sort > of generic interface that they need to work with, but using pip means you > get the details of abstracting out to the different build tools handled for > you, for “free”. At least, in theory that’s how it’ll work :)
Note that this is a key to why wheel is important to this discussion. The "build interface" in pip is "pip wheel foo", which will (in the "pluggable build" future) run whatever build tool the project specifies, and produce as output a wheel. That wheel is then the input for any packaging systems that want to build their own formats. So, we have: 1. sdist: The source format for packages. 2. distutils/setuptools/bdist_wheel: The only major "build tool" currently in existence. Our goal is to seamlessly allow others to fit here. 3. pip wheel: The build tool interface designed to convert sdist->wheel using the appropriate build tool. 4. wheel: The built format for Python packages. Acts as a common target for build tools and as a common source for distribution package builders. Also directly installable via pip. 5. pip install <wheelfile>. The canonical Python installer, taking wheels as input. Pip can also combine this whole sequence, and install direct from sdist (via wheel in the next version, currently by direct install from sdist), but that;s not the important point for this discussion. In terms of tool interoperability, therefore, there are a couple of places things can hook in. 1. Anything that can take an sdist and build a wheel can be treated as a "build tool". You could run the appropriate build tool for your package manually, but it's a goal for pip to provide a unified interface to that process. 2. Any installer can use wheels as the source for building its install packages. This frees package build processes from needing to deal with compiling Python extensions, packaging up Python sources, etc. We (the PyPA) haven't really done a particularly good job of articulating this design, not least because a lot of it is still ideas in progress, rather than concrete plans. And as a result, it's hard for tools like conda to clearly understand how they could fit into this stack. And of course, the backward compatibility pressures on any change in Python packaging causes things to go pretty slowly, meaning that projects like conda have an additional pressure to come up with a solution *right now* rather than waiting for a standard solution that frankly is currently vapourware. Ideally, the scientific community's experiences with building complex Python packages can help us to improve the wheel spec to ensure that it can better act as that universal binary format (for repackaging or direct installation). But that does require ongoing effort to make sure we understand where the wheel format falls short, and how we can fix those issues. Doing this without getting sucked into trying to solve problems that the wheel format is *not* intended to cover (packaging and distribution of non-Python code) is hard - particularly where we need to express dependencies on such things. Paul. _______________________________________________ Distutils-SIG maillist - [email protected] https://mail.python.org/mailman/listinfo/distutils-sig
