On Mon, 2019-09-09 at 20:32 -0700, Stephan Hoyer wrote: > On Mon, Sep 9, 2019 at 6:27 PM Ralf Gommers <ralf.gomm...@gmail.com> > wrote: > > I think we've chosen to try the former - dispatch on functions so > > we can reuse the NumPy API. It could work out well, it could give > > some long-term maintenance issues, time will tell. The question is > > now if and how to plug the gap that __array_function__ left. It's > > main limitation is "doesn't work for functions that don't have an > > array-like input" - that left out ~10-20% of functions. So now we > > have a proposal for a structural solution to that last 10-20%. It > > seems logical to want that gap plugged, rather than go back and say > > "we shouldn't have gone for the first 80%, so let's go no further". > > > > I'm excited about solving the remaining 10-20% of use cases for > flexible array dispatching, but the unumpy interface suggested here > (numpy.overridable) feels like a redundant redo of __array_function__ > and __array_ufunc__. > > I would much rather continue to develop specialized protocols for the > remaining usecases. Summarizing those I've seen in this thread, these > include: > 1. Overrides for customizing array creation and coercion. > 2. Overrides to implement operations for new dtypes. > 3. Overriding implementations of NumPy functions, e.g., FFT and > ufuncs with MKL. > > (1) could mostly be solved by adding np.duckarray() and another > function for duck array coercion. There is still the matter of > overriding np.zeros and the like, which perhaps justifies another new > protocol, but in my experience the use-cases for truly an array from > scratch are quite rare. >
There is an issue open about adding more functions for that. Made me wonder if giving a method of choosing the duck-array whose `__array_function__` is used, could not solve it reasonably. Similar to explicitly choosing a specific template version to call in templated code. In other words `np.arange<type(duck_array)>(100)` (but with a completely different syntax, probably hidden away only for libraries to use). Maybe it is indeed time to write up a list of options to plug that hole, and then see where it brings us. Best, Sebastian > (2) should be tackled as part of overhauling NumPy's dtype system to > better support user defined dtypes. But it should definitely be in > the form of specialized protocols, e.g., which pass in preallocated > arrays to into ufuncs for a new dtype. By design, new dtypes should > not be able to customize the semantics of array *structure*. > > (3) could potentially motivate a new solution, but it should exist > *inside* of select existing NumPy implementations, after checking for > overrides with __array_function__. If the only option NumPy provides > for overriding np.fft is to implement np.overrideable.fft, I doubt > that would suffice to convince MKL developers from monkey patching it > -- they already decided that a separate namespace is not good enough > for them. > > I also share Nathaniel's concern that the overrides in unumpy are too > powerful, by allowing for control from arbitrary function arguments > and even *non-local* control (i.e., global variables) from context > managers. This level of flexibility can make code very hard to debug, > especially in larger codebases. > > Best, > Stephan > > > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion
signature.asc
Description: This is a digitally signed message part
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion