On Thu, Mar 11, 2021 at 6:08 PM Sebastian Berg <sebast...@sipsolutions.net> wrote:
> On Thu, 2021-03-11 at 12:37 +0100, Ralf Gommers wrote: > > On Wed, Mar 10, 2021 at 6:41 PM Sebastian Berg < > > sebast...@sipsolutions.net> > > wrote: > > > > > Top Posting, to discuss post specific questions about NEP 47 and > > > partially the start on implementing it in: > > > > > > https://github.com/numpy/numpy/pull/18585 > > > > > > There are probably many more that will crop up. But for me, each of > > > these is a pretty major difficulty without a clear answer as of > > > now. > > > > > > > All great questions, that Sebastian. Let me reply to the questions > > that > > Aaron didn't reply to inline below. > > > > To be clear, I do not expect complete answers to these questions right > now. (Although being unsure about some of them does make me slightly > reluctant to merge the work-in-progress into NumPy proper as opposed to > a separate repo.) > > Also, yes, most/all questions are hopefully are just trivialities to > check of (or no more than seeds for thought). Or even just a starting > point for making NEP 47's "Usage and Impact" section more complete > including them as either "example usage patterns" or "limitations". > Yes, those are always good to have more of. > > My second takeaway from the questions is that I have doubts the > "minimal" version will pan out, it feels like many of the questions > might disappear if you drop that part. My impression is that a strictly compliant (or "minimal") version is *more* useful than something that's a mix between portable and non-portable functionality. The reason to add more than the minimum required functionality would be that it's too hard to hide the numpy-specific extras. E.g., if we'd do `np.array_api.int32 = np.int32` then that dtype would have methods and behavior that's NumPy-specific. But it'd be hard to hide, so we'd accept it. It's maybe easier to discuss in a call, I've put it on the community meeting agenda. > So, from my current thinking, > the minimal implementation may not be a good "NEP 47" implementation. > > That does _not_ mean that I think you should pause and reconsider or > even worry about pleasing me with good answers! Just continue under > whatever assumption you prefer and if it turns out that "minimal" won't > work for NEP 47: no harm done! We need a "minimal implementation" in > any case. > Yes, I agree. > Cheers, > > Sebastian > > > > [1] If SciPy needs an additional NumPy code path to keep support > `object` arrays or other dtypes – right now even complex –, then the > reader needs to be aware of that to make a decision if NEP 47 will > actually help for their library. > Clearly. This is why we'd like to have some WIP PRs for other libraries, actual code to review will be more helpful than only a proposal. > Will AstroPy have to reimplement `astropy.units.Quantity` to be > "standard conform" (is that even possible!?) before it can easily adopt > it any of its API that currently works with `astropy.units.Quantity`? > I'm not sure if the question is well-defined, so let me answer both cases: 1. If the APIs in question require units, then there's no other array/tensor types that have unit support, so those APIs accept *only* Quantity. Adopting the standard isn't possible. 2. If the units are unnecessary/optional, then Quantity is not special and can be treated exactly the same as a `numpy.ndarray`. We don't intend to make any changes to how ndarray subclasses work, so if ndarray works with that API after adoption of the standard then Quantity works too. Cheers, Ralf > > > > > > 1. I still need clarity how a library is supposed to use this > > > namespace > > > when the user passes in a NumPy array (mentioned before). The user > > > must get back a NumPy array after all. Maybe that is just a > > > decorator, > > > but it seems important. > > > > > > > I agree that it will be a common pattern that libraries will accept > > all > > standard-compliant array types plus numpy.ndarray. And the output > > array > > type should match the input type. In Aaron's implementation the new > > array > > object has a numpy.ndarray as private attribute, so that's the > > instance > > that should be returned. A decorator seems like a sensible way to > > handle > > that. Or a simple utility function, something like `return > > correct_arraytype(out)`. > > > > Either way, that pattern should be added to NEP 47. I don't see a > > fundamental problem here, we just need to find the nicest UX for it. > > > > 3. For all other function, the same problem applies. You don't > > actually > > > have anything to fix NumPy promotion rules. You could bake your > > > own > > > cake here for numeric types, but I am not sure, you might also need > > > NEP > > > 43 in all its promotion power to pull it off. > > > > > > > This is probably the single most difficult question implementation- > > wise. > > Note that there are only numerical dtypes (plus boolean), so dealing > > with > > string, datetime, object or third-party dtypes is a non-issue. > > > > 4. The PR makes no attempt at handling binary operators in any way > > > aside from greedily coercing the other operand. > > > > > > > Agreed. This is the same point as (3) I think - how to handle dtype > > promotion is the main open question. > > > > > > > 5. What happens with a mix of array-likes or even array subclasses > > > like > > > `astropy.quantity`? > > > > > > > Array-likes (e.g. list) should raise an exception, the NEP clearly > > says "do > > not accept array_like dtypes". This is what every other array/tensor > > library already does. > > > > Array subclasses should work as expected, assuming they're valid > > subclasses > > and not things like np.matrix. Using Mypy will help avoid writing > > more > > subclasses that break the Liskov substitution principle. More > > comments in > > > https://numpy.org/neps/nep-0047-array-api-standard.html#the-asarray-asanyarray-pattern > > > > Mixing two different types of arrays into a single function call > > should > > raise an exception. A design goal is: enable writing functions > > `somefunc(x1, x2)` that work for any type of array where `x1, x2` > > come from > > the same library = so they're either the same type, or two types for > > which > > the library itself knows how to mix them. If x1 and x2 are from > > different > > libraries, this will raise an exception. > > > > To be clear, it is not intended that `np.array_api.somefunc(x_cupy)` > > works > > - this will raise an exception. > > > > Cheers, > > Ralf > > > > > > > > > > > > I don't think we have to figure out everything up-front, but I do > > > think > > > there are a few very fundamental questions still open, at least for > > > me > > > personally. > > > > > > Cheers, > > > > > > Sebastian > > > > > > > > > > > _______________________________________________ > > NumPy-Discussion mailing list > > NumPy-Discussion@python.org > > https://mail.python.org/mailman/listinfo/numpy-discussion > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion >
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
