On Sat, Jun 1, 2019 at 11:32 AM Nathaniel Smith <n...@pobox.com> wrote:
> It's possible I'm not getting what you're thinking, but from what you > describe in your email I think it's a bad idea. > Hi Nathaniel, I think you are indeed not getting what I meant and are just responding to the word "standard". I'll give a concrete example. Here is the xtensor to numpy comparison: https://xtensor.readthedocs.io/en/latest/numpy.html. The xtensor authors clearly have made sane choices, but they did have to spend a lot of effort making those choices - what to include and what not. Now, the XND team is just starting to build out their Python API. Hameer is building out unumpy. There's all the other arrays libraries I mentioned. We can say "sort it out yourself, make your own choices", or we can provide some guidance. So far the authors of those libaries I have asked say they would appreciate the guidance. Cheers, Ralf > Standards take a tremendous amount of work (no really, an absurdly > massively huge amount of work, more than you can imagine if you > haven't done it). And they don't do what people usually hope they do. > Many many standards are written all the time that have zero effect on > reality, and the effort is wasted. They're really only useful when you > have to solve a coordination problem: lots of people want to do the > same thing as each other, whatever that is, but no-one knows what the > thing should be. That's not a problem at all for us, because numpy > already exists. > > If you want to improve compatibility between Python libraries, then I > don't think it will be relevant. Users aren't writing code against > "the numpy standard", they're not testing their libraries against "the > numpy standard", they're using/testing against numpy. If library > authors want to be compatible with numpy, they need to match what > numpy does, not what some document says. OTOH if they think they have > a better idea and its worth breaking compatibility, they're going to > do it regardless of what some document somewhere says. > > If you want to share the lessons learned from numpy in the hopes of > improving future libraries that don't care about numpy compatibility > per se, in python or other languages, then that seems like a great > idea! But that's not a standard, that's a journal article called > something like "NumPy: A retrospective". Other languages aren't going > to match numpy one-to-one anyway, because they'll be adapting things > to their language's idioms; they certainly don't care about whether > you decided 'newaxis MUST be defined to None' or merely 'SHOULD' be > defined to None. > > IMO if you try the most likely outcome will be that it will suck up a > lot of energy writing it, and then the only effect is that everyone > will keep doing what they would have done anyway but now with extra > self-righteousness and yelling depending on whether that turns out to > match the standard or not. > > -n > > On Sat, Jun 1, 2019 at 1:18 AM Ralf Gommers <ralf.gomm...@gmail.com> > wrote: > > > > Hi all, > > > > I have an idea that I've discussed with a few people in person, and the > feedback has generally been positive. So I'd like to bring it up here, to > get a sense of if this is going to fly. Note that this is NOT a proposal at > this point. > > > > Idea in five words: define a NumPy API standard > > > > Observations > > ------------ > > - Many libraries, both in Python and other languages, have APIs copied > from or inspired by NumPy. > > - All of those APIs are incomplete, and many deviate from NumPy either > by accident or on purpose. > > - The NumPy API is very large and ill-defined. > > > > Libraries with a NumPy-like API > > ------------------------------- > > In Python: > > - GPU: Tensorflow, PyTorch, CuPy, MXNet > > - distributed: Dask > > - sparse: pydata/sparse > > - other: tensorly, uarray/unumpy, ... > > > > In other languages: > > - JavaScript: numjs > > - Go: Gonum > > - Rust: rust-ndarray, rust-numpy > > - C++: xtensor > > - C: XND > > - Java: ND4J > > - C#: NumSharp, numpy.net > > - Ruby: Narray, xnd-ruby > > - R: Rray > > > > This is an incomplete list. Xtensor and XND aim for multi-language > support. These libraries are of varying completeness, size and quality - > everything from one-person efforts that have just started, to large code > bases that go beyond NumPy in features or performance. > > > > Idea > > ---- > > Define a standard for "the NumPy API" (or "NumPy core API", or .... - > it's just a name for now), that > > other libraries can use as a guide on what to implement and when to say > they are NumPy compatible. > > > > In scope: > > - Define a NumPy API standard, containing an N-dimensional array object > and a set of functions. > > - List of functions and ndarray methods to include. > > - Recommendations about where to deviate from NumPy (e.g. leave out > array scalars) > > > > Out of scope, or to be treated separately: > > - dtypes and casting > > - (g)ufuncs > > - function behavior (e.g. returning views vs. copies, which keyword > arguments to include) > > - indexing behavior > > - submodules (fft, random, linalg) > > > > Who cares and why? > > - Library authors: this saves them work and helps them make decisions. > > - End users: consistency between libraries/languages, helps transfer > knowledge and understand code > > - NumPy developers: gives them a vocabulary for "the NumPy API", > "compatible with NumPy", etc. > > > > Risks: > > - If not done well, we just add to the confusion rather than make things > better. > > - Opportunity for endless amount of bikeshedding > > - ? > > > > Some more rationale: > > We (NumPy devs) mostly have a shared understanding of what is "core > NumPy functionality", what we'd like to remove but are stuck with, what's > not used a whole lot, etc. Examples: financial functions don't belong, > array creation methods with weird names like np.r_ were a mistake. We are > not communicating this in any way though. Doing so would be helpful. > Perhaps this API standard could even have layers, to indicate what's really > core, what are secondary sets of functionality to include in other > libraries, etc. > > > > Discussion and next steps > > ------------------------- > > What I'd like to get a sense of is: > > - Is this a good idea to begin with? > > - What should the scope be? > > - What should the format be (a NEP, some other doc, defining in code)? > > > > If this idea is well-received, I can try to draft a proposal during the > next month (help/volunteers welcome!). It can then be discussed at SciPy'19 > - high-bandwidth communication may help to get a set of people on the same > page and hash out a lot of details. > > > > Thoughts? > > > > Ralf > > > > _______________________________________________ > > NumPy-Discussion mailing list > > NumPy-Discussion@python.org > > https://mail.python.org/mailman/listinfo/numpy-discussion > > > > -- > Nathaniel J. Smith -- https://vorpus.org > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion >
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
