Re: Technical Board statement on type hints for Django

[charettes]

Thanks for sharing your answer publicly Adam and Markus. Here are the 
points I brought up in the discussion

>From my limited experience with type annotation on a large monolithic code 
> base that was ported from Python 2 to 3 even gradual typing has been a lot 
> of trouble for the few benefits it brought us.
>
> This partially explains why I had reservations about the benefits of the 
> proposal .
>
> I also think that it's a bit too early to bundle type annotations in the 
> core repository and I'd favour an approach where we make adjustments to the 
> source to add entry points allowing third parties to implement and 
> maintains such type annotations. That's the approach we've taken in the 
> past when a new feature is suggested for inclusion in core; we add missing 
> entry points for the feature to be implemented externally instead. This 
> should allow different tools to compete organically in the ecosystem 
> without having to choose the best candidate right now.
>
> I'm -0 on allowing stdlib type annotations in the mean time since I feels 
> like it would open a can of worms that we want to keep sealed for now. What 
> if a function accepts/return a non-stdlib type (or an Union of a stdlib 
> and non-stdlib), do we type annotate it? What if we have to change this 
> function to a Union[non-stdlib-type, stdlib-type], should we completely 
> drop the type annotation on that function?
>

Simon

Le vendredi 17 avril 2020 06:07:23 UTC-4, Markus Holtermann a écrit :
>
> Thanks for pushing this public, Adam. In the discussion I brought up these 
> points. 
>
> I've been using static typing in Python for about 1.5 years now. Every now 
> and then it's neat, but often enough I get annoyed by it. Either because I 
> simply don't know how to use the tools at hand correctly or because there's 
> something that's just not expressible via type annotations. 
>
> The DEP is indeed very much draft. It covers so much, for a rather young 
> setup, that at this point, I would not accept it. Everything from it's 
> mention of `@overload` is too complex and I'm not sure any of that API is 
> stable enough to rely on it in Django. 
>
> What I could see Django doing is having type annotations that don't go 
> beyond the stdlib support: e.g. `def foo(bar: str) -> Optional[int]` or 
> `def foo(instance: models.Model) -> int:`. But far more complex constructs 
> will eventually cause breakage and displeasure during development of and 
> with Django. 
>
> Often enough at work there's situations where no one can figure out how to 
> properly type-annotate something. We then end up with `# type: ignore` or 
> not typing a function at all. That's technical dept that we'll hardly ever 
> lose. And because it doesn't show up in any CI, it won't cause any issue. 
> But that means, we'll eventually have everything type-ignored or heaps of 
> function not typed at all. I do not want to see that in Django. If 
> something can't be typed or the typing is wrong: `# type: ignore` shouldn't 
> ever be used. At this point, I'd instead drop the typing that causes it. 
>
> The DEP kind of singles out mypy. As Adam already mentioned, there's about 
> four type checkers out there. I wouldn't want Django to pick one and have 
> special support for that. Instead, everything that would be special to one 
> of them should be developed externally. Especially since these tools 
> iterate quickly and change quickly. If a plugin were to be bundled with 
> Django, we'd need to support it for 3 years. What when that tool changes 
> the entire API (again, young scenery, high velocity). 
>
> With regards to the PR Carlton asked about (
> https://github.com/django/django/pull/12405), if that helps people to 
> build tooling around Django, let's do it. 
>
> Cheers, 
>
> Markus 
>
> On Fri, Apr 17, 2020, at 11:41 AM, Adam Johnson wrote: 
> > So we in the technical board were a bit opaque and had our discussion 
> > in private before Carlton posted our summary. Apologies for this. We'll 
> > repeat the discussion in the open so you can see our reasoning. 
> > 
> > On 4 March Carlton prompted for our input. I replied: 
> > 
> > > My experience using types, so far: 
> > >  * I'm the author of flake8-no-types, the flake8 plugin for banning 
> type annotations 😂 I wrote it whilst reviewing a client's code base which 
> had type annotations, mostly added by PyCharm autocomplete, but no MyPy 
> running. Running MyPy produced >1k errors. Eventually I actually fixed the 
> errors rather than stripping the annotations and installing flake8-no-types 
> - there was thankfully a lot of repetition in the errors. 
> > >  * I have a vague memory writing one open source PR using types but 
> can't remember the project. 
> > >  * I'm a little confused keeping up with the various ways of declaring 
> types, 'best practices", and which type checkers are out there (last I 
> heard there are 4!?). 
> > > I agree t

Re: Technical Board statement on type hints for Django

[Markus Holtermann]

Thanks for pushing this public, Adam. In the discussion I brought up these 
points.

I've been using static typing in Python for about 1.5 years now. Every now and 
then it's neat, but often enough I get annoyed by it. Either because I simply 
don't know how to use the tools at hand correctly or because there's something 
that's just not expressible via type annotations.

The DEP is indeed very much draft. It covers so much, for a rather young setup, 
that at this point, I would not accept it. Everything from it's mention of 
`@overload` is too complex and I'm not sure any of that API is stable enough to 
rely on it in Django.

What I could see Django doing is having type annotations that don't go beyond 
the stdlib support: e.g. `def foo(bar: str) -> Optional[int]` or `def 
foo(instance: models.Model) -> int:`. But far more complex constructs will 
eventually cause breakage and displeasure during development of and with Django.

Often enough at work there's situations where no one can figure out how to 
properly type-annotate something. We then end up with `# type: ignore` or not 
typing a function at all. That's technical dept that we'll hardly ever lose. 
And because it doesn't show up in any CI, it won't cause any issue. But that 
means, we'll eventually have everything type-ignored or heaps of function not 
typed at all. I do not want to see that in Django. If something can't be typed 
or the typing is wrong: `# type: ignore` shouldn't ever be used. At this point, 
I'd instead drop the typing that causes it.

The DEP kind of singles out mypy. As Adam already mentioned, there's about four 
type checkers out there. I wouldn't want Django to pick one and have special 
support for that. Instead, everything that would be special to one of them 
should be developed externally. Especially since these tools iterate quickly 
and change quickly. If a plugin were to be bundled with Django, we'd need to 
support it for 3 years. What when that tool changes the entire API (again, 
young scenery, high velocity).

With regards to the PR Carlton asked about 
(https://github.com/django/django/pull/12405), if that helps people to build 
tooling around Django, let's do it.

Cheers,

Markus

On Fri, Apr 17, 2020, at 11:41 AM, Adam Johnson wrote:
> So we in the technical board were a bit opaque and had our discussion 
> in private before Carlton posted our summary. Apologies for this. We'll 
> repeat the discussion in the open so you can see our reasoning.
> 
> On 4 March Carlton prompted for our input. I replied:
> 
> > My experience using types, so far:
> >  * I'm the author of flake8-no-types, the flake8 plugin for banning type 
> > annotations 😂 I wrote it whilst reviewing a client's code base which had 
> > type annotations, mostly added by PyCharm autocomplete, but no MyPy 
> > running. Running MyPy produced >1k errors. Eventually I actually fixed the 
> > errors rather than stripping the annotations and installing flake8-no-types 
> > - there was thankfully a lot of repetition in the errors.
> >  * I have a vague memory writing one open source PR using types but can't 
> > remember the project.
> >  * I'm a little confused keeping up with the various ways of declaring 
> > types, 'best practices", and which type checkers are out there (last I 
> > heard there are 4!?).
> > I agree that the tools are young, and evolving relatively quickly - both 
> > MyPy and the standard library. I'm not sure there's quite enough stability 
> > yet to roll types out across the whole of Django.
> > 
> > There's the risk that we end up in a chicken/egg stalemate where if no 
> > major Python project commits to being typed, types don't get enough 
> > adoption to progress. But I think there are enough popular projects, like 
> > Starlette, that are using them.
> > 
> > At current it does seem to be a vocal niche of developers who want them.
> > 
> > 
> > Yes it will make PR's harder to review, and definitely adds a barrier to 
> > contribution. Even relatively Pythonic Django-ey constructs like "accept 
> > anything with an as_expression method, or castable to a string" require 
> > quite a lot of type system knowledge to write correctly. On top of this the 
> > best practices seem to still be changing.
> > 
> > 
> > One extra benefit would be a little more clarity around Django API's. For 
> > example, I think a past mistake was allowing database expressions' 'params' 
> > to be *either* a tuple or list, which means casting everywhere (a bug for 
> > years in django-mysql: 
> > https://github.com/adamchainz/django-mysql/pull/558/files ). That said this 
> > is relatively low impact - the documentation often doesn't mention types at 
> > all and Python developers are used to checking things in a REPL.
> > 
> > I think accepting the current PR is okay. [the one for `__class_getitem__` ]
> > 
> > A next step could potentially be experimentally by providing type 
> > annotations in more limited scope projects. One that could work well is 

Re: Technical Board statement on type hints for Django

[Adam Johnson]

So we in the technical board were a bit opaque and had our discussion in
private before Carlton posted our summary. Apologies for this. We'll repeat
the discussion in the open so you can see our reasoning.

On 4 March Carlton prompted for our input. I replied:

My experience using types, so far:
>
>- I'm the author of flake8-no-types, the flake8 plugin for banning type
>annotations 😂 I wrote it whilst reviewing a client's code base which had
>type annotations, mostly added by PyCharm autocomplete, but no MyPy
>running. Running MyPy produced >1k errors. Eventually I actually fixed the
>errors rather than stripping the annotations and installing flake8-no-
>types - there was thankfully a lot of repetition in the errors.
>- I have a vague memory writing one open source PR using types but
>can't remember the project.
>- I'm a little confused keeping up with the various ways of declaring
>types, 'best practices", and which type checkers are out there (last I
>heard there are 4!?).
>
> I agree that the tools are young, and evolving relatively quickly - both
> MyPy and the standard library. I'm not sure there's quite enough stability
> yet to roll types out across the whole of Django.
>
> There's the risk that we end up in a chicken/egg stalemate where if no
> major Python project commits to being typed, types don't get enough
> adoption to progress. But I think there are enough popular projects, like
> Starlette, that are using them.
>
> At current it does seem to be a vocal niche of developers who want them.
>
>
> Yes it will make PR's harder to review, and definitely adds a barrier to
> contribution. Even relatively Pythonic Django-ey constructs like "accept
> anything with an as_expression method, or castable to a string" require
> quite a lot of type system knowledge to write correctly. On top of this
> the best practices seem to still be changing.
>
>
> One extra benefit would be a little more clarity around Django API's. For
> example, I think a past mistake was allowing database expressions' 'params'
> to be *either* a tuple or list, which means casting everywhere (a bug for
> years in django-mysql:
> https://github.com/adamchainz/django-mysql/pull/558/files ). That said
> this is relatively low impact - the documentation often doesn't mention
> types at all and Python developers are used to checking things in a REPL.
>
> I think accepting the current PR is okay. [the one for __class_getitem__ ]
>
> A next step could potentially be experimentally by providing type
> annotations in more limited scope projects. One that could work well is
> asgiref, as it could also provide "the" ASGI type signatures - which
> Starlette already started internally:
> https://github.com/encode/starlette/blob/master/starlette/types.py
>

On Tue, 14 Apr 2020 at 10:46, Carlton Gibson 
wrote:

> Hi all.
>
> The question of using typing, or type hints, or type checking, in (or
> with) Django
> has come up several times. Whether we would add inline annotations, or use
> stub
> files, or what?
>
> Most recently, this resulted in a draft DEP[0] to try and formalize the
> situation,
> followed by a PR[1] adding the `__class_getitem__()` to `QuerySet` and
> `Manager`
> that would ease the job of working on external stub files.
>
> [0]: https://github.com/django/deps/pull/65
> [1]: https://github.com/django/django/pull/12405
>
> There was extended discussion on both the draft DEP and the PR, which I
> recommend for anyone particularly interested in this topic.
>
> Given the lack of consensus on the issue, I asked the Technical Board to
> review
> the situation in full, and make a judgement on the way forward.
>
> Having been charged to do so, I here post the Technical Board's response:
>
>
> It is the view of the Django Technical Board that inline type
> annotations
> should not be added to Django at the current time.
>
> A brief idea of the considerations are these:
>
> * Typing in Python is still young and in flux. As such it is not
>   sufficiently stable.
> * There are competing technologies, which may settle with time, but at
> this
>   moment Django is not in a position to favor any one.
> * Writing correct type hints is hard. And reviewing them to ensure
>   correctness isn't any easier. Even ostensibly simple examples often
> have
>   hidden complexity. Thus the barrier to contribution and maintenance
> is
>   raised.
> * Due to the very dynamic nature of much of Django, example type hints
> very
>   often do not pass an acceptable standard for readability. This might
> be
>   manageable with extensive aliasing but that effort itself involves
>   further overhead.
>
> The Technical Board acknowledges the type checking is gaining ground in
> Python and that many users are keen to employ it.
>
> The django-stubs project is notable for the impressive work that it has
> done adding external stub files for Django.
>