It's possible I'm not getting what you're thinking, but from what you describe in your email I think it's a bad idea.
Standards take a tremendous amount of work (no really, an absurdly massively huge amount of work, more than you can imagine if you haven't done it). And they don't do what people usually hope they do. Many many standards are written all the time that have zero effect on reality, and the effort is wasted. They're really only useful when you have to solve a coordination problem: lots of people want to do the same thing as each other, whatever that is, but no-one knows what the thing should be. That's not a problem at all for us, because numpy already exists. If you want to improve compatibility between Python libraries, then I don't think it will be relevant. Users aren't writing code against "the numpy standard", they're not testing their libraries against "the numpy standard", they're using/testing against numpy. If library authors want to be compatible with numpy, they need to match what numpy does, not what some document says. OTOH if they think they have a better idea and its worth breaking compatibility, they're going to do it regardless of what some document somewhere says. If you want to share the lessons learned from numpy in the hopes of improving future libraries that don't care about numpy compatibility per se, in python or other languages, then that seems like a great idea! But that's not a standard, that's a journal article called something like "NumPy: A retrospective". Other languages aren't going to match numpy one-to-one anyway, because they'll be adapting things to their language's idioms; they certainly don't care about whether you decided 'newaxis MUST be defined to None' or merely 'SHOULD' be defined to None. IMO if you try the most likely outcome will be that it will suck up a lot of energy writing it, and then the only effect is that everyone will keep doing what they would have done anyway but now with extra self-righteousness and yelling depending on whether that turns out to match the standard or not. -n On Sat, Jun 1, 2019 at 1:18 AM Ralf Gommers <ralf.gomm...@gmail.com> wrote: > > Hi all, > > I have an idea that I've discussed with a few people in person, and the > feedback has generally been positive. So I'd like to bring it up here, to get > a sense of if this is going to fly. Note that this is NOT a proposal at this > point. > > Idea in five words: define a NumPy API standard > > Observations > ------------ > - Many libraries, both in Python and other languages, have APIs copied from > or inspired by NumPy. > - All of those APIs are incomplete, and many deviate from NumPy either by > accident or on purpose. > - The NumPy API is very large and ill-defined. > > Libraries with a NumPy-like API > ------------------------------- > In Python: > - GPU: Tensorflow, PyTorch, CuPy, MXNet > - distributed: Dask > - sparse: pydata/sparse > - other: tensorly, uarray/unumpy, ... > > In other languages: > - JavaScript: numjs > - Go: Gonum > - Rust: rust-ndarray, rust-numpy > - C++: xtensor > - C: XND > - Java: ND4J > - C#: NumSharp, numpy.net > - Ruby: Narray, xnd-ruby > - R: Rray > > This is an incomplete list. Xtensor and XND aim for multi-language support. > These libraries are of varying completeness, size and quality - everything > from one-person efforts that have just started, to large code bases that go > beyond NumPy in features or performance. > > Idea > ---- > Define a standard for "the NumPy API" (or "NumPy core API", or .... - it's > just a name for now), that > other libraries can use as a guide on what to implement and when to say they > are NumPy compatible. > > In scope: > - Define a NumPy API standard, containing an N-dimensional array object and a > set of functions. > - List of functions and ndarray methods to include. > - Recommendations about where to deviate from NumPy (e.g. leave out array > scalars) > > Out of scope, or to be treated separately: > - dtypes and casting > - (g)ufuncs > - function behavior (e.g. returning views vs. copies, which keyword arguments > to include) > - indexing behavior > - submodules (fft, random, linalg) > > Who cares and why? > - Library authors: this saves them work and helps them make decisions. > - End users: consistency between libraries/languages, helps transfer > knowledge and understand code > - NumPy developers: gives them a vocabulary for "the NumPy API", "compatible > with NumPy", etc. > > Risks: > - If not done well, we just add to the confusion rather than make things > better. > - Opportunity for endless amount of bikeshedding > - ? > > Some more rationale: > We (NumPy devs) mostly have a shared understanding of what is "core NumPy > functionality", what we'd like to remove but are stuck with, what's not used > a whole lot, etc. Examples: financial functions don't belong, array creation > methods with weird names like np.r_ were a mistake. We are not communicating > this in any way though. Doing so would be helpful. Perhaps this API standard > could even have layers, to indicate what's really core, what are secondary > sets of functionality to include in other libraries, etc. > > Discussion and next steps > ------------------------- > What I'd like to get a sense of is: > - Is this a good idea to begin with? > - What should the scope be? > - What should the format be (a NEP, some other doc, defining in code)? > > If this idea is well-received, I can try to draft a proposal during the next > month (help/volunteers welcome!). It can then be discussed at SciPy'19 - > high-bandwidth communication may help to get a set of people on the same page > and hash out a lot of details. > > Thoughts? > > Ralf > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion -- Nathaniel J. Smith -- https://vorpus.org _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
