>> There may be another very concrete one (that's not yet in the NEP): allowing
>> other libraries that consume ndarrays to use overrides. An example is
>> numpy.fft: currently both mkl_fft and pyfftw monkeypatch NumPy, something we
>> don't like all that much (in particular for mkl_fft, because it's the
>> default in Anaconda). `__array_function__` isn't able to help here, because
>> it will always choose NumPy's own implementation for ndarray input. With
>> unumpy you can support multiple libraries that consume ndarrays.
> unumpy doesn't help with this either though, does it? unumpy is
> double-opt-in: the code using np.fft has to switch to using unumpy.fft
> instead, and then someone has to enable the backend. But MKL/pyfftw started
> out as opt-in – you could `import mkl_fft` or `import pyfftw` – and the whole
> reason they switched to monkeypatching is that they decided that opt-in
> wasn't good enough for them.
Because numpy functions are used to write many library functions, the end user
isn't always able to opt-in by changing imports. So, for library functions,
monkey patching is not simply convenient but actually necessary. Take for
example scipy.signal.fftconvolve: SciPy can't change to pyfftw for licensing
reasons so with SciPy < 1.4 your only option is to monkey patch scipy.fftpack
and numpy.fft. However in SciPy >= 1.4, thanks to the uarray-based backend
support in scipy.fft, I can write
from scipy import fft, signal
import pyfftw.interfaces.scipy_fft as pyfftw_fft
x = np.random.randn(1024, 1024)
with fft.set_backend(pyfftw_fft):
y = signal.fftconvolve(x, x) # Calls pyfftw's rfft, irfft
Yes, we had to opt-in in the library function (signal moved from scipy.fftpack
to scipy.fft). But because there can be distance between the set_backend call
and the FFT calls, the library is now much more configurable. Generally
speaking, any library written to use unumpy would be configurable: (i) by the
user, (ii) at runtime, (iii) without changing library code and (iv) without
monkey patching.
In scipy.fft I actually did it slightly differently than unumpy: the scipy.fft
interface itself has the uarray dispatch and I set SciPy's version of pocketfft
as the default global backend. This means that normal users don't need to set a
backend, and thus don't need to opt-in in any way. For NumPy to follow this
pattern as well would require more change to NumPy's code base than the current
NEP's suggestion, mainly in separating the interface from the implementation
that would become the default backend.
- Peter
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion
