Re: [Numpy-discussion] Type annotations for NumPy
(a) it would be good if NumPy type annotations could include an “array_like” type that allows lists, tuples, etc. I think that would be a sequence — already supported by the Typing system. (b) I’ve always thought (since PEP561) that it would be cool for type annotations to replace compiler type annotations for e.g. Cython and Numba. Is this in the realm of possibility for the future? Well, this was brought up early in the Typing discussion, and it was made clear that these kinds of truly static types, as needed by Cython, was a non-goal of the project. That being said, perhaps it could be made to work with a bunch of additional type objects. And we should lol lol to Cython for ideas about how to type numpy arrays. One note: in addition to shape (rank) and types, there is contiguous and C or F order. That may want to be considered. -CHB ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Type annotations for NumPy
On Nov 25, 2017, at 3:35 PM, Matthew Rocklin wrote: Thoughts on basing this on a more generic Array type rather than the np.ndarray? This would actually be more consistent with the current python typing approach. I can imagine other nd-array libraries (XArray, Tensorflow, Dask.array) wanting to reuse this work. It may be tough to come up with the right ABC though— see another recent thread on this list. -CHB ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Type annotations for NumPy
Here's the code: https://github.com/rmcgibbo/numpy-mypy. It's not 100% working yet, but it can do simple stuff, like inferring the shape of arrays created from np.zeros(literal_tuple), and fixing out the shape of the result of an indexing operation (i.e. https://github.com/rmcgibbo/numpy-mypy/blob/master/tests/test_indexing.py). To implement it, I have the beginnings of the stubs that you'd expect, borrowed from https://github.com/machinalis/mypy-data and then revised. Then, on top of that, I wrote some special type-level functions that are implemented inside of a mypy plugin. So, for example, the stub's signature for np.sum is def sum(a: ndarray[_S, _D], axis: AxesType=None, dtype: DtypeType=None, out: ndarray=None, keepdims: bool=False) -> ndarray[_InferDtypeWithDefault[_S], _InferNdimsReduction[_D]]: ... When the stub is applied, the resut's dtype is determined application of the _InferDtypeWithDefault type function, which defaults, as expected, to the dtype of the input array but checks of that was overridden dtype=None kwarg as well. And the _InferNdimsReduction type function has to check the axis and keepdims arguments as well. It's by no means ready for real users, but I hope this is a useful place to build from. Any feedback or contributions would be appreciated. -Robert On Tue, Nov 28, 2017 at 2:04 PM, Stephan Hoyer wrote: > On Tue, Nov 28, 2017 at 5:11 PM Robert T. McGibbon > wrote: > >> I'm strongly in support of this proposal. Type annotations have really >> helped me write more correct code. >> >> I started working on numpy type stubs a few months ago. I needed a mypy >> plugin to support shape-aware functions. Those whole thing is pretty >> tricky. Still very WIP, but I'll clean them up a little bit and opensource >> it shortly. >> > > Great to hear -- I'd love to see what this looks like, or hear any lessons > you learned from the experience! > > Actual experience using and writing such a type checker gives you a > valuable perspective to share, as opposed to my speculation. > > Cheers, > Stephan > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > > -- -Robert ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Type annotations for NumPy
On Tue, Nov 28, 2017 at 5:11 PM Robert T. McGibbon wrote: > I'm strongly in support of this proposal. Type annotations have really > helped me write more correct code. > > I started working on numpy type stubs a few months ago. I needed a mypy > plugin to support shape-aware functions. Those whole thing is pretty > tricky. Still very WIP, but I'll clean them up a little bit and opensource > it shortly. > Great to hear -- I'd love to see what this looks like, or hear any lessons you learned from the experience! Actual experience using and writing such a type checker gives you a valuable perspective to share, as opposed to my speculation. Cheers, Stephan ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Type annotations for NumPy
I'm strongly in support of this proposal. Type annotations have really helped me write more correct code. I started working on numpy type stubs a few months ago. I needed a mypy plugin to support shape-aware functions. Those whole thing is pretty tricky. Still very WIP, but I'll clean them up a little bit and opensource it shortly. -Robert On Sun, Nov 26, 2017 at 1:58 PM, Stephan Hoyer wrote: > On Sat, Nov 25, 2017 at 3:34 PM Matthew Rocklin > wrote: > >> Thoughts on basing this on a more generic Array type rather than the >> np.ndarray? I can imagine other nd-array libraries (XArray, Tensorflow, >> Dask.array) wanting to reuse this work. For dask.array in particular we >> would want to copy this entirely, but we probably can't specify that >> dask.arrays are np.ndarrays. It would be nice to ensure that the container >> type was swappable. >> > > Yes, absolutely. I do briefly mention this in my longer doc (see the > "Syntax" section). This is also one of my personal goals for this project. > > This will be most relevant when we start working on typing support for > array shapes and broadcasting: details like data types can be more library > specific, and can probably be expressed with the existing generics system > in the typing module. > > After we do some experimentation to figure out appropriate syntax and > semantics for array shape typing, I would like to standardize the rules for > typing multi-dimensional arrays in Python. This will probably entail > writing a PEP, so we can add appropriate base classes in the typing module. > I view this as the natural complement to existing standard library features > that make it easier to interchange between multiple multi-dimensional array > libraries, such as memory views and the buffer protocol. > >> > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > > -- -Robert ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
