Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

I meant whatever the state of the dance routine is, e.g., the way the arguments are enumerated by the decorator ​(this is partially why some example code for the dance routine is needed -- I am not 100% how this should work, just seems logical that if the dance routine can understand it, so can __a

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

(offlist) To clarify, by "where_i_am" you mean something like the name of the argument where it was found? On Fri, Jun 8, 2018 at 4:49 PM Marten van Kerkwijk < m.h.vankerkw...@gmail.com> wrote: > and in particular how the implementation finds out where its own instances >>> are located. >>> >> >

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

> and in particular how the implementation finds out where its own instances >> are located. >> > > I think we've discussed this before, but I don't think this is feasible to > solve in general given the diversity of wrapped APIs. If you want to find > the arguments in which a class' own instances

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

On Fri, Jun 8, 2018 at 8:58 AM Marten van Kerkwijk < m.h.vankerkw...@gmail.com> wrote: > I think we're getting to the stage where an updated text would be useful. > Yes, I plan to work on this over the weekend. Stay tuned! > For that, you may want to consider an actual implementation of, e.g.,

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

Hi Stephan, I think we're getting to the stage where an updated text would be useful. For that, you may want to consider an actual implementation of, e.g., a very simple function like `np.reshape` as well as a more complicated one like `np.concatenate`, and in particular how the implementation fin

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

On 6. Jun 2018 at 05:41, Nathan Goldbaum wrote: Oh wait, since the decorated version of the ufunc will be the one in the public numpy API it won't break. It would only break if the callable that was passed in *wasn't* the decorated version, so it kinda *has* to pass in the decorated function to

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

Oh wait, since the decorated version of the ufunc will be the one in the public numpy API it won't break. It would only break if the callable that was passed in *wasn't* the decorated version, so it kinda *has* to pass in the decorated function to preserve backward compatibility. Apologies for the

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

Hmm, does this mean the callable that gets passed into __array_ufunc__ will change? I'm pretty sure that will break the dispatch mechanism I'm using in my __array_ufunc__ implementation, which directly checks whether the callable is in one of several tuples of functions that have different behavior

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

Yes, the function should definitely be the same as what the user called - i.e., the decorated function. I'm only wondering if it would also be possible to have access to the undecorated one (via `coerce` or `ndarray.__array_function__` or otherwise). -- Marten __

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

On Tue, Jun 5, 2018 at 2:47 PM Matti Picus wrote: > What is the difference between the `func` provided as the first argument > to `__array_function__` and `__array_ufunc__` and the "non-overloaded > version of the provided function"? > The ""non-overloaded version of the provided function" is en

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

On 05/06/18 14:11, Stephan Hoyer wrote: On Tue, Jun 5, 2018 at 12:35 PM Marten van Kerkwijk mailto:m.h.vankerkw...@gmail.com>> wrote: Things would, I think, make much more sense if `ndarray.__array_ufunc__` (or `*_function__`) actually *were* the implementation for array-only. But

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

Hi Stephan, On `NotImplementedButCoercible`: don't forget that even a preliminary implementation of `__array_function__` has always the choice of coercing its own instances to ndarray and re-calling the function; that is really no different from (though probably a bit slower than) what would happe

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

On Tue, Jun 5, 2018 at 12:35 PM Marten van Kerkwijk < m.h.vankerkw...@gmail.com> wrote: > Things would, I think, make much more sense if `ndarray.__array_ufunc__` > (or `*_function__`) actually *were* the implementation for array-only. But > while that is something I'd like to eventually get to, i

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

Hi Stephan, Things would, I think, make much more sense if `ndarray.__array_ufunc__` (or `*_function__`) actually *were* the implementation for array-only. But while that is something I'd like to eventually get to, it seems out of scope for the current discussion. But we should be sure that the nd

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

On Mon, Jun 4, 2018 at 7:35 AM Marten van Kerkwijk < m.h.vankerkw...@gmail.com> wrote: > Hi Stephan, > > Another potential consideration in favor of NotImplementedButCoercible is >> for subclassing: we could use it to write the default implementations of >> ndarray.__array_ufunc__ and ndarray.__ar

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

On Mon, Jun 4, 2018 at 5:39 AM Matthew Harrigan wrote: > Should there be discussion of typing (pep-484) or abstract base classes in > this nep? Are there any requirements on the result returned by > __array_function__? > This is a good question that should be addressed in the NEP. Currently, we

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

I agree that second rounds of overloads have to be left to the implementers of `__array_function__` - obviously, though, we should be sure that these rounds are rarely necessary... The link posted by Stephan [1] has some decent discussion for `__array_ufunc__` about when an override should re-call

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

Hi Stephan, Another potential consideration in favor of NotImplementedButCoercible is > for subclassing: we could use it to write the default implementations of > ndarray.__array_ufunc__ and ndarray.__array_function__, e.g., > > class ndarray: > def __array_ufunc__(self, *args, **kwargs): >

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

Should there be discussion of typing (pep-484) or abstract base classes in this nep? Are there any requirements on the result returned by __array_function__? On Mon, Jun 4, 2018, 2:20 AM Stephan Hoyer wrote: > > On Sun, Jun 3, 2018 at 9:54 PM Hameer Abbasi > wrote: > >> Mixed return values of

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

On Sun, Jun 3, 2018 at 9:54 PM Hameer Abbasi wrote: > Mixed return values of NotImplementedButCoercible and NotImplemented would > still result in TypeError, and there would be no second chances for > overloads. > > > I would like to differ with you here: It can be quite useful to have > second c

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

Mixed return values of NotImplementedButCoercible and NotImplemented would still result in TypeError, and there would be no second chances for overloads. I would like to differ with you here: It can be quite useful to have second chances for overloads. Think ``np.func(list, custom_array))``: If s

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

On Sun, Jun 3, 2018 at 5:44 PM Marten van Kerkwijk < m.h.vankerkw...@gmail.com> wrote: > Although I'm still not 100% convinced by NotImplementedButCoercible, I do > like the idea that this is the default for items that do not implement > `__array_function__`. And it might help avoid trying to find

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

This in certainly true in general, but given the complete flexibility of __array_function__ there's no way we can make every check convenient. The best we can do is make it easy to handle the common cases, where the argument position does not matter. I think those cases may not be as common as you

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

Although I'm still not 100% convinced by NotImplementedButCoercible, I do like the idea that this is the default for items that do not implement `__array_function__`. And it might help avoid trying to find oneself in a possibly long list. -- Marten ___ N

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

On Sun, Jun 3, 2018 at 4:25 PM Marten van Kerkwijk < m.h.vankerkw...@gmail.com> wrote: > I think one might still want to know *where* the type occurs (e.g., as an > output or index would have different implications). > This in certainly true in general, but given the complete flexibility of __arr

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

On Sun, Jun 3, 2018 at 11:12 AM Marten van Kerkwijk < m.h.vankerkw...@gmail.com> wrote: > On Sun, Jun 3, 2018 at 2:00 PM, Hameer Abbasi > wrote: > >> >>- Objects that don’t implement ``__array_function__`` should be >>treated as having returned ``np.NotImplementedButCoercible``. >>

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

> > In most cases, I suspect that the overhead of a function call and checking > several arguments for "__array_function__" will be negligible, like the > situation for __array_ufunc__. I'm not strongly opposed to either of your > proposed solutions, but I do think it would be a little strange to i

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

On Sun, Jun 3, 2018 at 8:19 AM Marten van Kerkwijk < m.h.vankerkw...@gmail.com> wrote: > My more general comment is one of speed: for *normal* operation > performance should be impacted as minimally as possible. I think this is a > serious issue and feel strongly it *has* to be possible to avoid a

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

I also am not sure there is an actual problem: In the scheme as proposed, implementations could just coerce themselves to array and call the routine again. (Or, in the scheme I proposed, call the routine again but with `coerce=True`.) Ah, I didn’t think of the first solution. `coerce=True` may no

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

On Sun, Jun 3, 2018 at 2:00 PM, Hameer Abbasi wrote: > The rules for dispatch with ``__array_function__`` match those for > ``__array_ufunc__`` (see > `NEP-13 `_). > In particular: > > - NumPy will gather implementations of ``__array_funct

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

The rules for dispatch with ``__array_function__`` match those for ``__array_ufunc__`` (see `NEP-13 `_). In particular: - NumPy will gather implementations of ``__array_function__`` from all specified inputs and call them in order: subcl

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

Hi Stephan, Thanks for posting. Overall, this is great! My more general comment is one of speed: for *normal* operation performance should be impacted as minimally as possible. I think this is a serious issue and feel strongly it *has* to be possible to avoid all arguments being checked for the `

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

Perhaps I missed this but I didn’t see: what happens when both __array_ufunc__ and __array_function__ are defined? I might want to do this to for example add support for functions like concatenate or stack to a class that already has an __array_ufunc__ defines. This is mentioned in the section “N

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

Perhaps I missed this but I didn’t see: what happens when both __array_ufunc__ and __array_function__ are defined? I might want to do this to for example add support for functions like concatenate or stack to a class that already has an __array_ufunc__ defines. On Sat, Jun 2, 2018 at 5:56 PM Steph

[Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

Matthew Rocklin and I have written NEP-18, which proposes a new dispatch mechanism for NumPy's high level API: http://www.numpy.org/neps/nep-0018-array-function-protocol.html There has already been a little bit of scattered discussion on the pull request (https://github.com/numpy/numpy/pull/11189)