On Jun 29, 2016 2:49 AM, "Andrew Jaffe" <a.h.ja...@gmail.com> wrote: > > On 28/06/2016 18:50, Ralf Gommers wrote: >> >> >> On Tue, Jun 28, 2016 at 5:50 PM, Chris Barker <chris.bar...@noaa.gov >> <mailto:chris.bar...@noaa.gov>> wrote: >> >> > This doesn't really matter too much imho, we have to support Accelerate >> > either way. >> >> do we? -- so if we go OpenBlas, and someone want to do a simple >> build from source, what happens? Do they get accelerate? >> >> Indeed, unless they go through the effort of downloading a separate BLAS >> and LAPACK, and figuring out how to make that visible to >> numpy.distutils. Very few users will do that. >> >> or would we ship OpenBlas source itself? >> >> Definitely don't want to do that. >> >> or would they need to install OpenBlas some other way? >> >> Yes, or MKL, or ATLAS, or BLIS. We have support for all these, and >> that's a good thing. Making a uniform choice for our official binaries >> on various OSes doesn't reduce the need or effort for supporting those >> other options. >> >> >> >> Faster to fix bugs with good support from main developer. No >> >> >> multiprocessing crashes for Python 2.7. >> >> this seems to be the compelling one. >> >> How does the performance compare? >> >> For most routines performance seems to be comparable, and both are much >> better than ATLAS. When there's a significant difference, I have the >> impression that OpenBLAS is more often the slower one (example: >> https://github.com/xianyi/OpenBLAS/issues/533). > > > In that case: > > -1 > > (but this seems so obvious that I'm probably missing the point of the +1s)
Speed is important, but it's far from the only consideration, especially since differences between the top tier libraries are usually rather small. (And note that even though that bug is still listed as open, it has a link to a commit that appears to have fixed it by implementing the missing kernels.) The advantage of openblas is that it's open source, fixable, and we already focus energy on supporting it for Linux (and probably windows too soon). Accelerate is closed, so when we hit bugs then there's often nothing we can do except file a bug with apple and hope that it gets fixed within a year or two. This isn't hypothetical -- we've hit cases where accelerate gave wrong answers. Numpy actually carries some scary code right now to work around one of these bugs by monkeypatching (!) accelerate using dynamic linker trickiness. And, of course, there's the thing where accelerate totally breaks multiprocessing. Apple has said that they don't consider this a bug. Which is probably not much comfort to the new users who are getting obscure hangs when they try to use Python's most obvious and commonly recommended concurrency library. If you sum across our user base, I'm 99% sure that this means accelerate is slower than openblas on net, because you need a *lot* of code to get 10% speedups before it cancels out one person spending 3 days trying to figure out why their code is silently hanging for no reason. This probably makes me sound more negative about accelerate then I actually am -- it does work well most of the time, and obviously lots of people are using it successfully with numpy. But for our official binaries, my vote is we should switch to openblas, because these binaries are likely to be used by non-experts who are likely to hit the multiprocessing issue, and because when we're already struggling to do sufficient QA on our releases then it makes sense to focus our efforts on a single blas library. -n
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
