On 23 August 2013 08:03, Chris Barker - NOAA Federal <chris.bar...@noaa.gov> wrote: > On Thu, Aug 22, 2013 at 9:37 AM, Paul Moore <p.f.mo...@gmail.com> wrote: > >> That is essentially possible now. >> >> 1. Go to Christoph Gohlke's website and download his bdist_wininst >> installers for numpy and scipy. > > .... > > Exactly. And when all this settles down, hopefully Christoph, and > others, will put up binary wheels and we'll have one stop installing > that supports virtualenv, and pypi discover of "good enough" binary > wheels.
Right - this is exactly my ambition. Have pip+wheel+virtualenv provide a "good enough" out-of-the-box experience, but not necessarily support a fully optimised experience for any given platform. There are too many different possible full stack integration technologies for it to make sense for us to try to supersede them all - instead, I'd like to provide an 80% cross-platform solution that "plays well enough with others" to cover the remaining 20%. The "others" then includes things like zc.buildout, conda, Linux distro package managers, Microsoft System Centre, automated configuration management tools, PaaS providers like OpenShift and Heroku, container technologies like Docker and the underpinnings of OpenShift. In this space, the goal of the pip+wheel ecosystem will be to make it possible to either use a command like "pip install zc.buildout" or "pip install conda" to bootstrap a cross-platform toolchain, or else to use a platform specific toolchain (like pyp2rpm) to *consume* the upstream packages and produce nice, policy compliant, packages automatically. Managing arbitrary binary dependencies adds a lot of complexity for a capability that I believe the majority of Python projects don't need. Even the ones that can use it (like scientific tools) can often provide a "good enough" fallback option that will fit within the constraints of the draft metadata 2.0 standards. > My point is that is may not be wise to try to support the more complex > builds -- they ARE complex, and trying to suppor it with an auto tool > is a bit much. This is where I think "pip install conda" shows a lot of promise as a good, cross-platform solution, at least in the scientific space. However, the trade-off it makes is that the hash-based packaging system means you *always* pin your dependencies when building a package, with all the downsides that entails (mainly in increasing the complexity of security updates). It's just that in the scientific space, easily reproducing a particular software stack will often trump the desire to make security updates easy to deploy with minimal impact on other components. That said, I'm considering the idea of adding a "variant" field to the compatibility tags for wheel 1.1, along the lines of what Oscar Benjamin suggested earlier. By default, installers would only find wheels without a variant defined, but users could opt in to looking for particular variants. The meaning of the variants field would be entirely distribution specific. Numpy, for example, could publish: numpy-1.7.1-cp27-cp22m-win32.whl numpy-1.7.1-cp27-cp22m-win32-sse.whl numpy-1.7.1-cp27-cp22m-win32-sse2.whl numpy-1.7.1-cp27-cp22m-win32-sse3.whl The only restrictions on the variant tags would be: 1. must be ASCII 2. must not contain '.' or '-' characters You could even go to the extent of using hashes as variant tags. > Oscar wrote: >> And actually 'pip wheel numpy' works fine for me on Windows with MinGW > installed. > > Good start, but the bigger issue is that 'pip install' finds that wheel... > > I'm still confused as to the state of all this -- are the tools ready > for project to start posting wheels so that pip can find them? As others have noted: - definite yes for building your own wheels for internal use (including simple build caching to speed up virtualenv creation) - qualified yes for publication on PyPI (i.e. "there may still be rough edges, so don't be too surprised if this still has flaws at this point, especially on OS X and Linux") Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig