On Tue, Jul 2, 2019 at 1:15 PM Ralf Gommers <ralf.gomm...@gmail.com> wrote:
> > > On Tue, Jul 2, 2019 at 8:38 AM Stephan Hoyer <sho...@gmail.com> wrote: > >> On Tue, Jul 2, 2019 at 8:16 AM Ralf Gommers <ralf.gomm...@gmail.com> >> wrote: >> >>> >>> >>> On Tue, Jul 2, 2019 at 1:45 AM Juan Nunez-Iglesias <j...@fastmail.com> >>> wrote: >>> >>>> On Tue, 2 Jul 2019, at 4:34 PM, Stephan Hoyer wrote: >>>> >>>> This is addressed in the NEP, see bullet 1 under "Partial >>>> implementation of NumPy's API": >>>> >>>> http://www.numpy.org/neps/nep-0018-array-function-protocol.html#partial-implementation-of-numpy-s-api >>>> >>>> My concern is that fallback coercion behavior makes it difficult to >>>> reliably implement "strict" overrides of NumPy's API. Fallback coercion is >>>> certainly useful for interactive use, but it isn't really appropriate for >>>> libraries. >>>> >>>> >>> Do you mean "fallback coercion in NumPy itself", or "at all"? Right now >>> there's lots of valid code around, e.g. `np.median(some_dask_array)`. Users >>> will keep wanting to do that. Forcing everyone to write >>> `np.median(np.array(some_dask_array))` serves no purpose. So the coercion >>> has to be somewhere. You're arguing that that's up to Dask et al I think? >>> >> >> Yes, I'm arguing this is up to dask to maintain backwards compatibility >> -- or not, as the maintainers see fit. >> >> NumPy adding dispatching with __array_function__ did not break any >> existing code, until the maintainers of other libraries started adding >> __array_function__ methods. I hope that the risks of implementing such >> experimental methods were self-evident. >> > > Yeah, that's a bit of a chicken-and-egg story though. We add something and > try to be "strict". Dask adds something because they like the idea and > generally are quick to adopt these types of things. If we make it too hard > to be backwards compatible, then neither NumPy nor Dask may try and it ends > up breaking scikit-image & co. I for one don't care where the fix lands, > but it's pretty to me that breaking scikit-image is the worst of all > options. > > >> >>> Putting it in Dask right now still doesn't address Juan's backwards >>> compat concern, but perhaps that could be bridged with a Dask bugfix >>> release and some short-lived pain. >>> >> >> I really think this is the best (only?) path forward. >> > > I think I agree (depending on how easy it is to get the Dask fix landed). > That's landed, and Dask is planning a bugfix release in 2 days, so before the NumPy 1.17.0 release. So this is not a release blocker anymore for us I think. Cheers, Ralf >> I'm not convinced that this shouldn't be fixed in NumPy though. Your >>> concern "reliably implement "strict" overrides of NumPy's API" is a bit >>> abstract. Overriding the _whole_ NumPy API is definitely undesirable. If >>> we'd have a reference list somewhere about every function that is handled >>> with __array_function__, then would that address your concern? Such a list >>> could be auto-generated fairly easily. >>> >> >> By "reliably implement strict overrides" I mean the ability to ensure >> that every operation either uses an override or raises an informative error >> -- making it very clear which operation needs to be implemented or avoided. >> > > That isn't necessarily a good goal in itself though. In many cases, an > `asarray` call still needs to go *somewhere*. If the "reliably implement > strict overrides" is to help library authors, then there may be other ways > to do that. For end users it can only hurt; those TypeErrors aren't exactly > easy to understand. > > >> It's true that we didn't really consider "always issuing warnings" as a >> long term solution in the NEP. I can see how this would simply a backwards >> compatibility story for libraries like dask, but in general, I really don't >> like warnings: >> > > I agree. > > Cheers, > Ralf > > Using them like exceptions can easily result in code that is partially >> broken or that fails later for non-obvious reasons. There's a reason why >> Python's errors stop execution flow, until errors in languages like PHP or >> JavaScript. >> >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion@python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> >
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
