On Fri, Sep 6, 2019 at 4:51 PM Nathaniel Smith <n...@pobox.com> wrote:
> On Fri, Sep 6, 2019 at 2:45 PM Ralf Gommers <ralf.gomm...@gmail.com> > wrote: > > There may be another very concrete one (that's not yet in the NEP): > allowing other libraries that consume ndarrays to use overrides. An example > is numpy.fft: currently both mkl_fft and pyfftw monkeypatch NumPy, > something we don't like all that much (in particular for mkl_fft, because > it's the default in Anaconda). `__array_function__` isn't able to help > here, because it will always choose NumPy's own implementation for ndarray > input. With unumpy you can support multiple libraries that consume ndarrays. > > unumpy doesn't help with this either though, does it? unumpy is > double-opt-in: the code using np.fft has to switch to using unumpy.fft > instead, and then someone has to enable the backend. Very good point. It would make a lot of sense to at least make unumpy default on fft/linalg/random, even if we want to keep it opt-in for the functions in the main namespace. But MKL/pyfftw > started out as opt-in – you could `import mkl_fft` or `import pyfftw` > – and the whole reason they switched to monkeypatching is that they > decided that opt-in wasn't good enough for them. > No, that's not correct. The MKL team has asked for a proper backend system, so they can plug into numpy rather than monkeypatch it. Oleksey, Chuck and I discussed that two years ago already at the NumFOCUS Summit 2017. This has been explicitly on the NumPy roadmap for quite a while: "A backend system for numpy.fft (so that e.g. fft-mkl doesn’t need to monkeypatch numpy)" (see https://numpy.org/neps/roadmap.html#other-functionality) And if Anaconda would like to default to it, that's possible - because one registered backend needs to be chosen as the default, that could be mkl-fft. That is still a major improvement over the situation today. > >> The import numpy.overridable part is meant to help garner adoption, and > to prefer the unumpy module if it is available (which will continue to be > developed separately). That way it isn't so tightly coupled to the release > cycle. One alternative Sebastian Berg mentioned (and I am on board with) is > just moving unumpy into the NumPy organisation. What we fear keeping it > separate is that the simple act of a pip install unumpy will keep people > from using it or trying it out. > > > > Note that this is not the most critical aspect. I pushed for vendoring > as numpy.overridable because I want to not derail the comparison with NEP > 30 et al. with a "should we add a dependency" discussion. The interesting > part to decide on first is: do we need the unumpy override mechanism? > Vendoring opt-in vs. making it default vs. adding a dependency is of > secondary interest right now. > > Wait, but I thought the only reason we would have a dependency is if > we're exporting it as part of the numpy namespace. If we keep the > import as `import unumpy`, then it works just as well, without any > dependency *or* vendoring in numpy, right? > Vendoring means "include the code". So no dependency on an external package. If we don't vendor, it's going to be either unused, or end up as a dependency for the whole SciPy/PyData stack. Actually, now that we've discussed the fft issue, I'd suggest to change the NEP to: vendor, and make default for fft, random, and linalg. Cheers, Ralf
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
