Re: [Distutils] pip on windows experience
On 24 January 2014 00:17, Oscar Benjamin oscar.j.benja...@gmail.com wrote: You need to bear in mind that people currently have a variety of ways to install numpy on Windows that do work already without limitations on CPU instruction set. Most numpy users will not get any immediate benefit from the fact that it works using pip rather than it works using the .exe installer (or any of a number of other options). It's the unfortunate end users and the numpy folks who would have to pick up the pieces if/when the SSE2 assumption fails. The people who would benefit are those who (like me!) don't have a core requirement for numpy, but who just want to try it out casually, or for experimenting or one-off specialised scripts. These are the people who won't be using one of the curated distributions, and quite possibly will be using a virtualenv, so the exe installers won't work. Giving these people a means to try numpy could introduce a wider audience to it. Having said that, I can understand the reluctance to have to deal with non-specialist users hitting obscure your CPU is too old errors - that's *not* a good initial experience. And your point that it's just as reasonable for pip to adopt a partial solution in the short term is also fair - although it would be harder for pip to replace an API we added and which people are using, than it would be for numpy to switch to deploying better wheels when the facilities become available. So the comparison isn't entirely equal. Paul ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pip on windows experience
On 24 Jan 2014 19:41, Paul Moore p.f.mo...@gmail.com wrote: On 24 January 2014 00:17, Oscar Benjamin oscar.j.benja...@gmail.com wrote: You need to bear in mind that people currently have a variety of ways to install numpy on Windows that do work already without limitations on CPU instruction set. Most numpy users will not get any immediate benefit from the fact that it works using pip rather than it works using the .exe installer (or any of a number of other options). It's the unfortunate end users and the numpy folks who would have to pick up the pieces if/when the SSE2 assumption fails. The people who would benefit are those who (like me!) don't have a core requirement for numpy, but who just want to try it out casually, or for experimenting or one-off specialised scripts. These are the people who won't be using one of the curated distributions, and quite possibly will be using a virtualenv, so the exe installers won't work. Giving these people a means to try numpy could introduce a wider audience to it. Having said that, I can understand the reluctance to have to deal with non-specialist users hitting obscure your CPU is too old errors - that's *not* a good initial experience. And your point that it's just as reasonable for pip to adopt a partial solution in the short term is also fair - although it would be harder for pip to replace an API we added and which people are using, than it would be for numpy to switch to deploying better wheels when the facilities become available. So the comparison isn't entirely equal. There's also the fact that we're still trying to recover from the setup.py situation (which was a quick and easy alternative to a declarative build system), so quick hacks in the core metadata specs that will then be locked in for years by backwards compatibility requirements are definitely *not* acceptable. We already have more than enough of those in the legacy metadata we're aiming to replace :P All NumPy should need to reduce end user confusion to tolerable levels is an import time CPU check that raises an error that includes a link to stable URL explaining the limitations of the published wheel file, and alternative ways of obtaining NumPy (like Christophe's installers, or a science data analysis focused distribution like Anaconda or EPD, or bootstrapping conda). In return, as Paul points out, it becomes substantially easier for people that *aren't* wholly invested in the scientific Python stack to try it out with their regular tools, rather than having to completely change how they work with Python. Also consider that, given the status quo, any users that might see that new error instead get even *more* incomprehensible errors as pip attempts to build NumPy from source and fails at doing so. The choice given the current metadata standards isn't between confusing Windows users or not, it's between confusing 100% of those that try pip install numpy with cryptic errors from a failed build at install time and confusing a much smaller percentage of those with a CPU compatibility error at runtime. Is the latter a desirable *final* state? No, and metadata 2.0 will aim to address that. It is, however, substantially better than the status quo and doesn't run the risk of compromising interoperability standards we're going to have to live with indefinitely. Cheers, Nick. Paul ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pip on windows experience
Might I suggest you could upload some wheels (both windows and linux) to testpypi, which afaik is pretty much made for this purpose? https://wiki.python.org/moin/TestPyPI People can easily install then with e.g. `pip install --index-url https://testpypi.python.org/pypi numpy`, and see what tends to break or what doesn't. On 24 January 2014 05:52, Ralf Gommers ralf.gomm...@gmail.com wrote: On Thu, Jan 23, 2014 at 3:42 PM, Oscar Benjamin oscar.j.benja...@gmail.com wrote: On Thu, Jan 23, 2014 at 12:16:02PM +, Paul Moore wrote: The official numpy installer uses some complex magic to select the right binaries based on your CPU, and this means that the official numpy superpack wininst files don't convert (at least I don't think they do, it's a while since I tried). It's probably worth noting that numpy are toying around with wheels and have uploaded a number of them to PyPI for testing: http://sourceforge.net/projects/numpy/files/wheels_to_test/ Currently there are only OSX wheels there (excluding the puer Python ones) and they're not available on PyPI. I assume that they're waiting for a solution for the Windows installer (a post-install script for wheels). That would give a lot more impetus to put wheels up on PyPI. Indeed. We discussed just picking the SSE2 or SSE3 build and putting that up as a wheel, but that was deemed a not so great idea: http://article.gmane.org/gmane.comp.python.numeric.general/56072 The Sourceforge OSX wheels are presumably not getting that much use right now. The OSX-specific numpy wheel has been downloaded 4 times in the last week: twice on Windows and twice on Linux! Some feedback from the people who did try those wheels would help. I asked for that on the numpy list after creating them, but didn't get much. So I haven't been in a hurry to move them over to PyPi. Ralf ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pip on windows experience
On 24 January 2014 20:09, Matthew Iversen m...@notevencode.com wrote: Might I suggest you could upload some wheels (both windows and linux) to testpypi, which afaik is pretty much made for this purpose? Well, Windows and Mac OS X - we don't allow PyPI wheels for Linux at the moment (since it turns out the compatibility tagging needs to be updated to distinguish distros as well before that's a good idea), although Armin Ronacher pointed out our documentation of that limitation is rather lacking at this point in time :( https://wiki.python.org/moin/TestPyPI People can easily install then with e.g. `pip install --index-url https://testpypi.python.org/pypi numpy`, and see what tends to break or what doesn't. And yes, using testpypi to experiment with wheels is a good idea. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pip on windows experience
On Fri, Jan 24, 2014 at 2:18 AM, Nick Coghlan ncogh...@gmail.com wrote: In return, as Paul points out, it becomes substantially easier for people that *aren't* wholly invested in the scientific Python stack to try it out with their regular tools, rather than having to completely change how they work with Python. This is a really important constituency, actually. And one that has been neglected for a while. Also consider that, given the status quo, any users that might see that new error instead get even *more* incomprehensible errors as pip attempts to build NumPy from source and fails at doing so. well, numpy _should_ build out of the box with nothing special if you are set up to build regular extensions. I understand that a lto f Windows users are not set up to build extensions at all, but tehy ar presumably used to getting compiler not found errors (or whatever the message is). But you won't get an optimized numpy and much of the rest of the stack is harder to build: scipy, matplotlib. So a set of working binary wheels would be great. And while we in the numpy commmunity don't really want a lot of numpy is slower than MATLAB FUD out there, I still think it's better to get a sub-optimum, but working build out there. The should I use python instead of MATLAB? crowd would be better served by one of the other options anyway... So how rare are non-SSE2 systems? Any w ay to find out? Im guessing rare enough that we can a) not worry about it, and b) those users will know they have an old system and may expect issue,s particularly with something billed as being for high-performance computation. So I say SSE2 -- but if we do think there ar a lot of non-SSE2 users out there, then do SSE1-only , it would still work just fine for casual use. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pip on windows experience
On 24 January 2014 22:21, Chris Barker chris.bar...@noaa.gov wrote: well, numpy _should_ build out of the box with nothing special if you are set up to build regular extensions. I understand that a lto f Windows users are not set up to build extensions at all, but tehy ar presumably used to getting compiler not found errors (or whatever the message is). But you won't get an optimized numpy and much of the rest of the stack is harder to build: scipy, matplotlib. Seriously? If I have MSVC 2010 installed, pip install numpy will correctly build numpy from source? It's a *long* time since I tried this, but I really thought building numpy was harder than that. A quick test later: No BLAS/ATLAS/LAPACK causes a string of warnings, And ignoring the rest of the error stack (which I'm frankly not interested in investing the time to diagnose and fix) I get RuntimeError: Broken toolchain: cannot link a simple C program. Which is utter rubbish - I routinely build extensions with this installation. So no, numpy does not build out of the box. Ah well. Paul ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] pip on windows experience
On Fri, Jan 24, 2014 at 2:40 PM, Paul Moore p.f.mo...@gmail.com wrote: So no, numpy does not build out of the box. Ah well. Darn -- it used to, and it should. It has shipped for years with an LAPACK light, and shouldn't need any fortran. It used to not even look for LAPACK with a default configuration. But I haven't done if or years, so who know when this might have been broken? -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
