Re: [Numpy-discussion] NumPy-Discussion Digest, Vol 163, Issue 23
On Sat, Apr 25, 2020 at 4:50 PM wrote: > Send NumPy-Discussion mailing list submissions to > numpy-discussion@python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/numpy-discussion > or, via email, send a message with subject or body 'help' to > numpy-discussion-requ...@python.org > > You can reach the person managing the list at > numpy-discussion-ow...@python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of NumPy-Discussion digest..." > Today's Topics: > >1. Re: Feelings about type aliases in NumPy (Sebastian Berg) >2. beginner introduction to group (Tina Oberoi) >3. Re: beginner introduction to group (Robert Kern) >4. Re: Feelings about type aliases in NumPy (Stephan Hoyer) >5. Re: Feelings about type aliases in NumPy (Kevin Sheppard) > > > > -- Forwarded message -- > From: Sebastian Berg > To: numpy-discussion@python.org > Cc: > Bcc: > Date: Fri, 24 Apr 2020 13:29:59 -0500 > Subject: Re: [Numpy-discussion] Feelings about type aliases in NumPy > On Fri, 2020-04-24 at 11:10 -0700, Stefan van der Walt wrote: > > On Fri, Apr 24, 2020, at 08:45, Joshua Wilson wrote: > > > But, Stephan pointed out that it might be confusing to users for > > > objects to only exist at typing time, so we came around to the > > > question of whether people are open to the idea of including the > > > type > > > aliases in NumPy itself. Ralf's concrete proposal was to make a > > > module > > > numpy.types (or maybe numpy.typing) to hold the aliases so that > > > they > > > don't pollute the top-level namespace. The module would initially > > > contain the types > > > > That sounds very sensible. Having types available with NumPy should > > also encourage their use, especially if we can add some documentation > > around it. > > I agree, I might have a small tendency for `numpy.types` if we ever > find any usage other than direct typing that may be the better name? > > Out of curiousity, I guess `ArrayLike` would be an ABC that a > downstream project can register with? > > - Sebastian > > > > > > Stéfan > > ___ > > NumPy-Discussion mailing list > > NumPy-Discussion@python.org > > https://mail.python.org/mailman/listinfo/numpy-discussion > > > > > > > -- Forwarded message -- > From: Tina Oberoi > To: numpy-discussion@python.org > Cc: > Bcc: > Date: Sat, 25 Apr 2020 10:30:38 +0530 > Subject: [Numpy-discussion] beginner introduction to group > Hi Everyone, > I am new to contributing to numpy. I have read the contributors guide and > done with the set-up. Hope to make some good contributions and also to > connect with all you great people in the numpy community. > Any suggestions and tips are always welcome. > > Thanks and Regards > Welcome, Tina! -- Inessa Pawson > > -- Forwarded message -- > From: Robert Kern > To: Discussion of Numerical Python > Cc: > Bcc: > Date: Sat, 25 Apr 2020 01:59:57 -0400 > Subject: Re: [Numpy-discussion] beginner introduction to group > On Sat, Apr 25, 2020 at 1:02 AM Tina Oberoi > wrote: > >> Hi Everyone, >> I am new to contributing to numpy. I have read the contributors guide and >> done with the set-up. Hope to make some good contributions and also to >> connect with all you great people in the numpy community. >> Any suggestions and tips are always welcome. >> > > Welcome! Do you have an idea what you would like to work on? > > -- > Robert Kern > > > > -- Forwarded message -- > From: Stephan Hoyer > To: Discussion of Numerical Python > Cc: > Bcc: > Date: Fri, 24 Apr 2020 23:40:20 -0700 > Subject: Re: [Numpy-discussion] Feelings about type aliases in NumPy > > > On Fri, Apr 24, 2020 at 11:31 AM Sebastian Berg < > sebast...@sipsolutions.net> wrote: > >> On Fri, 2020-04-24 at 11:10 -0700, Stefan van der Walt wrote: >> > On Fri, Apr 24, 2020, at 08:45, Joshua Wilson wrote: >> > > But, Stephan pointed out that it might be confusing to users for >> > > objects to only exist at typing time, so we came around to the >> > > question of whether people are open to the idea of including the >> > > type >> > > aliases in NumPy itself. Ralf's concrete proposal was to make a >> > > module >> > > numpy.types (or maybe numpy.typing) to hold the aliases so that >> > > they >> > > don't pollute the top-level namespace. The module would initially >> > > contain the types >> > >> > That sounds very sensible. Having types available with NumPy should >> > also encourage their use, especially if we can add some documentation >> > around it. >> >> I agree, I might have a small tendency for `numpy.types` if we ever >> find any usage other than direct typing that may be the better name? > > > Unless we anticipate adding a long list of type aliases (more than the > three suggested so far), I would lean towards adding ArrayLike to the top > level N
Re: [Numpy-discussion] Proposal: add `force=` or `copy=` kwarg to `__array__` interface
On Fri, Apr 24, 2020 at 12:35 PM Eric Wieser wrote: > Perhaps worth mentioning that we've discussed this sort of API before, in > https://github.com/numpy/numpy/pull/11897. > > Under that proposal, the api would be something like: > > * `copy=True` - always copy, like it is today > * `copy=False` - copy if needed, like it is today > * `copy=np.never_copy` - never copy, throw an exception if not possible > There's a couple of issues I see with using `copy` for __array__: - copy is already weird (False doesn't mean no), and a [bool, some_obj_or_str] keyword isn't making that better - the behavior we're talking about can do more than copying, e.g. for PyTorch it would modify the autograd graph by adding detach(), and for sparse it's not just "make a copy" (which implies doubling memory use) but it densifies which can massively blow up the memory. - I'm -1 on adding things to the main namespace (never_copy) for something that can be handled differently (like a string, or a new keyword) tl;dr a new `force` keyword would be better Cheers, Ralf > I think the discussion stalled on the precise spelling of the third option. > > `__array__` was not discussed there, but it seems like adding the `copy` > argument to `__array__` would be a perfectly reasonable extension. > > Eric > > On Fri, 24 Apr 2020 at 03:00, Juan Nunez-Iglesias > wrote: > >> Hi everyone, >> >> One bit of expressivity we would miss is “copy if necessary, but >>> otherwise don’t bother”, but there are workarounds to this. >>> >> >> After a side discussion with Stéfan van der Walt, we came up with >> `allow_copy=True`, which would express to the downstream library that we >> don’t mind waiting, but that zero-copy would also be ok. >> >> This sounds like the sort of thing that is use case driven. If enough >> projects want to use it, then I have no objections to adding the keyword. >> OTOH, we need to be careful about adding too many interoperability tricks >> as they complicate the code and makes it hard for folks to determine the >> best solution. Interoperability is a hot topic and we need to be careful >> not put too leave behind too many experiments in the NumPy code. Do you >> have any other ideas of how to achieve the same effect? >> >> >> Personally, I don’t have any other ideas, but would be happy to hear some! >> >> My view regarding API/experiment creep is that `__array__` is the oldest >> and most basic of all the interop tricks and that this can be safely >> maintained for future generations. Currently it only takes `dtype=` as a >> keyword argument, so it is a very lean API. I think this particular use >> case is very natural and I’ve encountered the reluctance to implicitly copy >> twice, so I expect it is reasonably common. >> >> Regarding difficulty in determining the best solution, I would be happy >> to contribute to the dispatch basics guide together with the new kwarg. I >> agree that the protocols are getting quite numerous and I couldn’t find a >> single place that gathers all the best practices together. But, to >> reiterate my point: `__array__` is the simplest of these and I think this >> keyword is pretty safe to add. >> >> For ease of discussion, here are the API options discussed so far, as >> well as a few extra that I don’t like but might trigger other ideas: >> >> np.asarray(my_duck_array, allow_copy=True) # default is False, or None >> -> leave it to the duck array to decide >> np.asarray(my_duck_array, copy=True) # always copies, but, if supported >> by the duck array, defers to it for the copy >> np.asarray(my_duck_array, copy=‘allow’) # could take values ‘allow’, >> ‘force’, ’no’, True(=‘force’), False(=’no’) >> np.asarray(my_duck_array, force_copy=False, allow_copy=True) # separate >> concepts, but unclear what force_copy=True, allow_copy=False means! >> np.asarray(my_duck_array, force=True) >> >> Juan. >> ___ >> NumPy-Discussion mailing list >> NumPy-Discussion@python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal: add `force=` or `copy=` kwarg to `__array__` interface
On Sat, Apr 25, 2020 at 10:40 AM Ralf Gommers wrote: > > > On Fri, Apr 24, 2020 at 12:35 PM Eric Wieser > wrote: > >> Perhaps worth mentioning that we've discussed this sort of API before, in >> https://github.com/numpy/numpy/pull/11897. >> >> Under that proposal, the api would be something like: >> >> * `copy=True` - always copy, like it is today >> * `copy=False` - copy if needed, like it is today >> * `copy=np.never_copy` - never copy, throw an exception if not possible >> > > There's a couple of issues I see with using `copy` for __array__: > - copy is already weird (False doesn't mean no), and a [bool, > some_obj_or_str] keyword isn't making that better > - the behavior we're talking about can do more than copying, e.g. for > PyTorch it would modify the autograd graph by adding detach(), and for > sparse it's not just "make a copy" (which implies doubling memory use) but > it densifies which can massively blow up the memory. > - I'm -1 on adding things to the main namespace (never_copy) for something > that can be handled differently (like a string, or a new keyword) > > tl;dr a new `force` keyword would be better > I agree, “copy” is not a good description of this desired coercion behavior. A new keyword argument like “force” would be much clearer. > Cheers, > Ralf > > >> I think the discussion stalled on the precise spelling of the third >> option. >> >> `__array__` was not discussed there, but it seems like adding the `copy` >> argument to `__array__` would be a perfectly reasonable extension. >> >> Eric >> >> On Fri, 24 Apr 2020 at 03:00, Juan Nunez-Iglesias >> wrote: >> >>> Hi everyone, >>> >>> One bit of expressivity we would miss is “copy if necessary, but otherwise don’t bother”, but there are workarounds to this. >>> >>> After a side discussion with Stéfan van der Walt, we came up with >>> `allow_copy=True`, which would express to the downstream library that we >>> don’t mind waiting, but that zero-copy would also be ok. >>> >>> This sounds like the sort of thing that is use case driven. If enough >>> projects want to use it, then I have no objections to adding the keyword. >>> OTOH, we need to be careful about adding too many interoperability tricks >>> as they complicate the code and makes it hard for folks to determine the >>> best solution. Interoperability is a hot topic and we need to be careful >>> not put too leave behind too many experiments in the NumPy code. Do you >>> have any other ideas of how to achieve the same effect? >>> >>> >>> Personally, I don’t have any other ideas, but would be happy to hear >>> some! >>> >>> My view regarding API/experiment creep is that `__array__` is the oldest >>> and most basic of all the interop tricks and that this can be safely >>> maintained for future generations. Currently it only takes `dtype=` as a >>> keyword argument, so it is a very lean API. I think this particular use >>> case is very natural and I’ve encountered the reluctance to implicitly copy >>> twice, so I expect it is reasonably common. >>> >>> Regarding difficulty in determining the best solution, I would be happy >>> to contribute to the dispatch basics guide together with the new kwarg. I >>> agree that the protocols are getting quite numerous and I couldn’t find a >>> single place that gathers all the best practices together. But, to >>> reiterate my point: `__array__` is the simplest of these and I think this >>> keyword is pretty safe to add. >>> >>> For ease of discussion, here are the API options discussed so far, as >>> well as a few extra that I don’t like but might trigger other ideas: >>> >>> np.asarray(my_duck_array, allow_copy=True) # default is False, or None >>> -> leave it to the duck array to decide >>> np.asarray(my_duck_array, copy=True) # always copies, but, if supported >>> by the duck array, defers to it for the copy >>> np.asarray(my_duck_array, copy=‘allow’) # could take values ‘allow’, >>> ‘force’, ’no’, True(=‘force’), False(=’no’) >>> np.asarray(my_duck_array, force_copy=False, allow_copy=True) # separate >>> concepts, but unclear what force_copy=True, allow_copy=False means! >>> np.asarray(my_duck_array, force=True) >>> >>> Juan. >>> ___ >>> NumPy-Discussion mailing list >>> NumPy-Discussion@python.org >>> https://mail.python.org/mailman/listinfo/numpy-discussion >>> >> ___ >> NumPy-Discussion mailing list >> NumPy-Discussion@python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] beginner introduction to group
On Sat, Sat, 25 Apr 2020 01:59:57 Robert Kern wrote: Welcome! Do you have an idea what you would like to work on? Hi Robert, Nothing specific for now, But I am at present trying to work on Issue #15961. Titled "Einsum indexing very fragile, because it tests for int(and int64 is not int). Tina Oberoi ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
