Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Wed, Oct 9, 2019 at 6:00 PM Juan Nunez-Iglesias wrote: > Hi all, and thank you for all your hard work with this. > > I wanted to provide more of an "end user" perspective than I think has > been present in this discussion so far. Over the past month, I've quickly > skimmed some emails on this

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

Hi all, and thank you for all your hard work with this. I wanted to provide more of an "end user" perspective than I think has been present in this discussion so far. Over the past month, I've quickly skimmed some emails on this thread and skipped others altogether. I am far from a NumPy novic

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Wed, Sep 11, 2019 at 4:18 PM Ralf Gommers wrote: > > > On Tue, Sep 10, 2019 at 10:59 AM Stephan Hoyer wrote: > >> On Tue, Sep 10, 2019 at 6:06 AM Hameer Abbasi >> wrote: >> >>> On 10.09.19 05:32, Stephan Hoyer wrote: >>> >>> On Mon, Sep 9, 2019 at 6:27 PM Ralf Gommers >>> wrote: >>> I

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Tue, Sep 10, 2019 at 10:59 AM Stephan Hoyer wrote: > On Tue, Sep 10, 2019 at 6:06 AM Hameer Abbasi > wrote: > >> On 10.09.19 05:32, Stephan Hoyer wrote: >> >> On Mon, Sep 9, 2019 at 6:27 PM Ralf Gommers >> wrote: >> >>> I think we've chosen to try the former - dispatch on functions so we can

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Tue, Sep 10, 2019 at 10:53 AM Sebastian Berg wrote: > On Tue, 2019-09-10 at 17:28 +0200, Hameer Abbasi wrote: > > On 07.09.19 22:06, Sebastian Berg wrote: > > > > > > Now for the end-users choosing one array-like over another, seems > > > nicer > > > as an implicit mechanism (why should I not

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Tue, Sep 10, 2019 at 6:06 AM Hameer Abbasi wrote: > On 10.09.19 05:32, Stephan Hoyer wrote: > > On Mon, Sep 9, 2019 at 6:27 PM Ralf Gommers > wrote: > >> I think we've chosen to try the former - dispatch on functions so we can >> reuse the NumPy API. It could work out well, it could give some

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Tue, 2019-09-10 at 17:28 +0200, Hameer Abbasi wrote: > On 07.09.19 22:06, Sebastian Berg wrote: > > On Fri, 2019-09-06 at 14:45 -0700, Ralf Gommers wrote: > > > > > > > > Let me try to move the discussion from the github issue here (this > > may > > not be the best place). ( > > https://githu

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On 07.09.19 22:06, Sebastian Berg wrote: On Fri, 2019-09-06 at 14:45 -0700, Ralf Gommers wrote: Let me try to move the discussion from the github issue here (this may not be the best place). (https://github.com/numpy/numpy/issues/14441 which asked for easier creation functions together with `_

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On 10.09.19 05:32, Stephan Hoyer wrote: On Mon, Sep 9, 2019 at 6:27 PM Ralf Gommers > wrote: I think we've chosen to try the former - dispatch on functions so we can reuse the NumPy API. It could work out well, it could give some long-term maintenance i

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On 09.09.19 03:26, Nathaniel Smith wrote: [snip] Generic in the sense that you can write __array_ufunc__ once and have it work for all ufuncs. You can do that too with __ua_function__, you get np.ufunc.__call__, with self=. The same holds for say, RandomState objects, once implemented. Most

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On 08.09.19 10:56, Nathaniel Smith wrote: On Sun, Sep 8, 2019 at 1:04 AM Hameer Abbasi wrote: On 08.09.19 09:53, Nathaniel Smith wrote: OTOH, __array_function__ doesn't allow this kind of simplification: if we were using __array_function__ for ufuncs, every library would have to special-case e

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Mon, 2019-09-09 at 22:26 -0700, Eric Wieser wrote: > > In other words `np.arange(100)` (but > with a completely different syntax, probably hidden away only for > libraries to use). > > It sounds an bit like you're describing factory classmethods there. > Is the solution to this problem to move

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

> In other words `np.arange(100)` (but with a completely different syntax, probably hidden away only for libraries to use). It sounds an bit like you're describing factory classmethods there. Is the solution to this problem to move (leaving behind aliases) `np.arange` to `ndarray.arange`, `np.zero

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Mon, 2019-09-09 at 20:32 -0700, Stephan Hoyer wrote: > On Mon, Sep 9, 2019 at 6:27 PM Ralf Gommers > wrote: > > I think we've chosen to try the former - dispatch on functions so > > we can reuse the NumPy API. It could work out well, it could give > > some long-term maintenance issues, time wil

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Mon, Sep 9, 2019 at 6:27 PM Ralf Gommers wrote: > I think we've chosen to try the former - dispatch on functions so we can > reuse the NumPy API. It could work out well, it could give some long-term > maintenance issues, time will tell. The question is now if and how to plug > the gap that __a

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Sun, Sep 8, 2019 at 6:27 PM Nathaniel Smith wrote: > On Sun, Sep 8, 2019 at 8:40 AM Ralf Gommers > wrote: > > > > > > > > On Sun, Sep 8, 2019 at 12:54 AM Nathaniel Smith wrote: > >> > >> On Fri, Sep 6, 2019 at 11:53 AM Ralf Gommers > wrote: > >> > On Fri, Sep 6, 2019 at 12:53 AM Nathaniel S

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Sun, Sep 8, 2019 at 7:27 PM Nathaniel Smith wrote: > On Sun, Sep 8, 2019 at 8:40 AM Ralf Gommers > wrote: > > > > > > > > On Sun, Sep 8, 2019 at 12:54 AM Nathaniel Smith wrote: > >> > >> On Fri, Sep 6, 2019 at 11:53 AM Ralf Gommers > wrote: > >> > On Fri, Sep 6, 2019 at 12:53 AM Nathaniel S

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Sun, Sep 8, 2019 at 8:40 AM Ralf Gommers wrote: > > > > On Sun, Sep 8, 2019 at 12:54 AM Nathaniel Smith wrote: >> >> On Fri, Sep 6, 2019 at 11:53 AM Ralf Gommers wrote: >> > On Fri, Sep 6, 2019 at 12:53 AM Nathaniel Smith wrote: >> >> On Tue, Sep 3, 2019 at 2:04 AM Hameer Abbasi >> >> wrot

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Sun, Sep 8, 2019 at 12:54 AM Nathaniel Smith wrote: > On Fri, Sep 6, 2019 at 11:53 AM Ralf Gommers > wrote: > > On Fri, Sep 6, 2019 at 12:53 AM Nathaniel Smith wrote: > >> On Tue, Sep 3, 2019 at 2:04 AM Hameer Abbasi > wrote: > >> > The fact that we're having to design more and more protoco

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On 08.09.19 10:56, Nathaniel Smith wrote: On Sun, Sep 8, 2019 at 1:04 AM Hameer Abbasi wrote: On 08.09.19 09:53, Nathaniel Smith wrote: OTOH, __array_function__ doesn't allow this kind of simplification: if we were using __array_function__ for ufuncs, every library would have to special-case e

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Sun, Sep 8, 2019 at 1:04 AM Hameer Abbasi wrote: > > On 08.09.19 09:53, Nathaniel Smith wrote: >> OTOH, __array_function__ doesn't allow this kind of simplification: if >> we were using __array_function__ for ufuncs, every library would have >> to special-case every individual ufunc, which lead

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On 08.09.19 09:53, Nathaniel Smith wrote: On Fri, Sep 6, 2019 at 11:53 AM Ralf Gommers wrote: On Fri, Sep 6, 2019 at 12:53 AM Nathaniel Smith wrote: On Tue, Sep 3, 2019 at 2:04 AM Hameer Abbasi wrote: The fact that we're having to design more and more protocols for a lot of very similar thi

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Fri, Sep 6, 2019 at 11:53 AM Ralf Gommers wrote: > On Fri, Sep 6, 2019 at 12:53 AM Nathaniel Smith wrote: >> On Tue, Sep 3, 2019 at 2:04 AM Hameer Abbasi >> wrote: >> > The fact that we're having to design more and more protocols for a lot >> > of very similar things is, to me, an indicator

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Sat, Sep 7, 2019 at 5:08 PM Ralf Gommers wrote: > > On Sat, Sep 7, 2019 at 4:16 PM Nathaniel Smith wrote: >> >> On Fri, Sep 6, 2019 at 11:04 PM Ralf Gommers wrote: >> > Vendoring means "include the code". So no dependency on an external >> > package. If we don't vendor, it's going to be eit

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Sat, Sep 7, 2019 at 4:16 PM Nathaniel Smith wrote: > On Fri, Sep 6, 2019 at 11:04 PM Ralf Gommers > wrote: > > Vendoring means "include the code". So no dependency on an external > package. If we don't vendor, it's going to be either unused, or end up as a > dependency for the whole SciPy/Py

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Fri, Sep 6, 2019 at 11:04 PM Ralf Gommers wrote: > Vendoring means "include the code". So no dependency on an external package. > If we don't vendor, it's going to be either unused, or end up as a dependency > for the whole SciPy/PyData stack. If we vendor it then it also ends up as a depen

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Sat, Sep 7, 2019 at 2:18 PM sebastian wrote: > On 2019-09-07 15:33, Ralf Gommers wrote: > > On Sat, Sep 7, 2019 at 1:07 PM Sebastian Berg > > wrote: > > > >> On Fri, 2019-09-06 at 14:45 -0700, Ralf Gommers wrote: > >>> > >>> > >> > >> > That's part of it. The concrete problems it's solv

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On 2019-09-07 15:33, Ralf Gommers wrote: On Sat, Sep 7, 2019 at 1:07 PM Sebastian Berg wrote: On Fri, 2019-09-06 at 14:45 -0700, Ralf Gommers wrote: That's part of it. The concrete problems it's solving are threefold: Array creation functions can be overridden. Array coercion is now cov

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Sat, Sep 7, 2019 at 1:07 PM Sebastian Berg wrote: > On Fri, 2019-09-06 at 14:45 -0700, Ralf Gommers wrote: > > > > > > > > > That's part of it. The concrete problems it's solving are > > > threefold: > > > Array creation functions can be overridden. > > > Array coercion is now covered. > > >

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Fri, 2019-09-06 at 14:45 -0700, Ralf Gommers wrote: > > > > That's part of it. The concrete problems it's solving are > > threefold: > > Array creation functions can be overridden. > > Array coercion is now covered. > > "Default implementations" will allow you to re-write your NumPy > > arra

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

>> There may be another very concrete one (that's not yet in the NEP): allowing >> other libraries that consume ndarrays to use overrides. An example is >> numpy.fft: currently both mkl_fft and pyfftw monkeypatch NumPy, something we >> don't like all that much (in particular for mkl_fft, because

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Fri, Sep 6, 2019 at 4:51 PM Nathaniel Smith wrote: > On Fri, Sep 6, 2019 at 2:45 PM Ralf Gommers > wrote: > > There may be another very concrete one (that's not yet in the NEP): > allowing other libraries that consume ndarrays to use overrides. An example > is numpy.fft: currently both mkl_ff

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Fri, Sep 6, 2019 at 5:16 PM Nathaniel Smith wrote: > On Fri, Sep 6, 2019 at 11:44 AM Ralf Gommers > wrote: > > > > > > > > On Fri, Sep 6, 2019 at 1:32 AM Hameer Abbasi > wrote: > >> > >> That's a lot of very good questions! Let me see if I can answer them > one-by-one. > >> > >> On 06.09.19

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Fri, Sep 6, 2019 at 11:44 AM Ralf Gommers wrote: > > > > On Fri, Sep 6, 2019 at 1:32 AM Hameer Abbasi > wrote: >> >> That's a lot of very good questions! Let me see if I can answer them >> one-by-one. >> >> On 06.09.19 09:49, Nathaniel Smith wrote: >> >> But even that could be accomplished b

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Fri, Sep 6, 2019 at 2:45 PM Ralf Gommers wrote: > There may be another very concrete one (that's not yet in the NEP): allowing > other libraries that consume ndarrays to use overrides. An example is > numpy.fft: currently both mkl_fft and pyfftw monkeypatch NumPy, something we > don't like a

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Fri, Sep 6, 2019 at 1:32 AM Hameer Abbasi wrote: > That's a lot of very good questions! Let me see if I can answer them > one-by-one. > > On 06.09.19 09:49, Nathaniel Smith wrote: > > But those are general questions about unumpy, and I'm guessing no-one > knows all the answers yet... and these

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Fri, Sep 6, 2019 at 12:53 AM Nathaniel Smith wrote: > On Mon, Sep 2, 2019 at 11:21 PM Ralf Gommers > wrote: > > On Mon, Sep 2, 2019 at 2:09 PM Nathaniel Smith wrote: > > On Tue, Sep 3, 2019 at 2:04 AM Hameer Abbasi > wrote: > > The fact that we're having to design more and more protocols fo

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Fri, Sep 6, 2019 at 1:32 AM Hameer Abbasi wrote: > That's a lot of very good questions! Let me see if I can answer them > one-by-one. > > On 06.09.19 09:49, Nathaniel Smith wrote: > > But even that could be accomplished by just > putting something in the docs. And adding the alias has substant

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

That's a lot of very good questions! Let me see if I can answer them one-by-one. On 06.09.19 09:49, Nathaniel Smith wrote: Ah, whoops, I definitely missed that :-). That does change things! So one of the major decision points for any duck-array API work, is whether to modify the numpy semantics

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Mon, Sep 2, 2019 at 11:21 PM Ralf Gommers wrote: > On Mon, Sep 2, 2019 at 2:09 PM Nathaniel Smith wrote: >> The reason this is challenging is that there's a lot of code written >> in Cython/C/C++ that calls np.asarray, > > Cython code only perhaps? It would surprise me if there's a lot of C/C+

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

Hello everyone; Thanks to all the feedback from the community, in particular Sebastian Berg, we have a new draft of NEP-31. Please find the full text quoted below for discussion and reference. Any feedback and discussion is welcome. =

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

Hi Nathaniel, On 02.09.19 23:09, Nathaniel Smith wrote: On Mon, Sep 2, 2019 at 2:15 AM Hameer Abbasi wrote: Me, Ralf Gommers and Peter Bell (both cc’d) have come up with a proposal on how to solve the array creation and duck array problems. The solution is outlined in NEP-31, currently in t

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Mon, Sep 2, 2019 at 2:09 PM Nathaniel Smith wrote: > On Mon, Sep 2, 2019 at 2:15 AM Hameer Abbasi > wrote: > > Me, Ralf Gommers and Peter Bell (both cc’d) have come up with a proposal > on how to solve the array creation and duck array problems. The solution is > outlined in NEP-31, currently

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Mon, Sep 2, 2019 at 2:15 AM Hameer Abbasi wrote: > Me, Ralf Gommers and Peter Bell (both cc’d) have come up with a proposal on > how to solve the array creation and duck array problems. The solution is > outlined in NEP-31, currently in the form of a PR, [1] Thanks for putting this together!

[Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

Hello all, It was recently brought to my attention that my mails to NumPy-discussion were probably going into the spam folder for many people, so here I am trying from another email. Probably Google trying to force people onto their products as usual. 😉 Me, Ralf Gommers and Peter Bell (both cc

Re: [Numpy-discussion] NEP 31: Context-local and global overrides of the NumPy API

The full-text of the NEP is quoted below, apologies for not including it earlier: NEP 31 — Context-local and global overrides of the NumPy API :Author: Hameer Abbasi