[Numpy-discussion] Re: Types in pure Python sources

[Ralf Gommers]

On Sat, Apr 30, 2022 at 4:51 PM Sergei Lebedev 
wrote:

> Hi everyone,
>
> I've noticed that NumPy currently users separate type stub files for
> specifying types for both pure Python and native modules. For example the
> (untyped) implementation of np.core._asarray is in [1], but the types are
> in [2]. This works fine for type checkers, and if anything, improves type
> checking performance, because type stubs are much smaller than their pure
> Python counterparts. It does, however, create problems for IDEs which now
> have to either work with untyped APIs or manually merge them with type
> stubs to provide better coding assistance.
>
> Is there a reason for type stubs being separate (other than historical
> [3])?


I think that's on purpose, because the type annotations are quite complex.
For reasons of correctness/completeleness, they use protocols, mixins, and
overloads. Inside pure Python code, that would be harder to read and
maintain.

I agree that for projects with simple type annotations, it's best to have
them inline. But here that doesn't seem like a good idea.

IDEs should be able to deal with stub files anyway for compiled code (i.e.,
the majority of NumPy), so that shouldn't be a fundamental issue right? And
either way, IDE support for type annotations is primarily for users of the
NumPy API, in which case it doesn't matter where the annotations live.


> If not, how should I go about starting a conversation upstream about
> this?
>

You just did:)

Cheers,
Ralf


> Sergei
>
> [1]: https://github.com/numpy/numpy/blob/main/numpy/core/_asarray.py
> [2]: https://github.com/numpy/numpy/blob/main/numpy/core/_asarray.pyi
> [3]: https://github.com/numpy/numpy-stubs
> ___
> NumPy-Discussion mailing list -- numpy-discussion@python.org
> To unsubscribe send an email to numpy-discussion-le...@python.org
> https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
> Member address: ralf.gomm...@gmail.com
>
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] Re: Types in pure Python sources

[Sergei Lebedev]

> I think that's on purpose, because the type annotations are quite complex.
> For reasons of correctness/completeleness, they use protocols, mixins, and
> overloads. Inside pure Python code, that would be harder to read and
> maintain.

I agree that NumPy type annotations are quite complex. However, having them 
alongside the implementation might actually make maintenance easier, because 
there will be no need to update type stubs separately from the implementation.

> IDEs should be able to deal with stub files anyway for compiled code (i.e.,
> the majority of NumPy), so that shouldn't be a fundamental issue right? 

You are right, IDEs have to support type stubs anyway, but unlike type checkers 
IDEs cannot always prefer type stubs to Python sources (if they are available), 
because some IDE features need metadata which is *not* available in type stubs 
(e.g. docstring and source location). 

To make things a bit more specific, consider the following snippet

```
import numpy as np
np.array()
```

Here a user is in the middle of calling np.array and an IDE could render 

* the signature(s) of np.array (available in type stubs)
* the docstring of np.array (available in Python sources)
* (maybe) the exact module where np.array is defined (available in both)

"Go to definition" on np.array should take the user to the implementation of 
np.array or to both the implementation and the type stub.

Hovering over np.array would also need the signature and the docstring, but you 
could also imagine it rendering e.g. a snippet of the implementation etc.

Sergei
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com