[Python-Dev] Re: Keeping Python a Duck Typed Language.
On Fri, Apr 23, 2021 at 10:33 AM Chris Angelico wrote: > > On Fri, Apr 23, 2021 at 11:22 AM Larry Hastings wrote: > > > > > > On 4/20/21 10:03 AM, Mark Shannon wrote: > > > > If you guarded your code with `isinstance(foo, Sequence)` then I could not > > use it with my `Foo` even if my `Foo` quacked like a sequence. I was forced > > to use nominal typing; inheriting from Sequence, or explicitly registering > > as a Sequence. > > > > > > If I'm reading the library correctly, this is correct--but, perhaps, it > > could be remedied by adding a __subclasshook__ to Sequence that looked for > > an __iter__ attribute. That technique might also apply to other ABCs in > > collections.abc, Mapping for example. Would that work, or am I missing an > > critical detail? > > > > How would you distinguish between a Sequence and a Mapping? Both have > __iter__ and __len__. Without actually calling those methods, how > would the subclass hook tell them apart? > > ChrisA We can add .keys() to Mapping to distinguish Mapping and Sequence. But it is breaking change, of course. We shouldn't change it. I think using ABC to distinguish sequence or mapping is a bad idea. There are three policies: a) Use duck-typing; just us it as sequence. No type check at all. b) Use strict type checking; isinstance(x, list) / isinstance(x, (list, tuple)). c) Use ABC. But (c) is broken by design. It is not fixable. IMHO, We should chose (a) or (b) and reject any idea relying on Sequence ABC. Regards, -- Inada Naoki ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ESLOPO4GLC2QZW4ZDBYEQDPPGB4ZYDWM/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Keeping Python a Duck Typed Language.
On Fri, Apr 23, 2021 at 11:22 AM Larry Hastings wrote: > > > On 4/20/21 10:03 AM, Mark Shannon wrote: > > If you guarded your code with `isinstance(foo, Sequence)` then I could not > use it with my `Foo` even if my `Foo` quacked like a sequence. I was forced > to use nominal typing; inheriting from Sequence, or explicitly registering as > a Sequence. > > > If I'm reading the library correctly, this is correct--but, perhaps, it could > be remedied by adding a __subclasshook__ to Sequence that looked for an > __iter__ attribute. That technique might also apply to other ABCs in > collections.abc, Mapping for example. Would that work, or am I missing an > critical detail? > How would you distinguish between a Sequence and a Mapping? Both have __iter__ and __len__. Without actually calling those methods, how would the subclass hook tell them apart? ChrisA ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/FJZAJTGCCKJD4MYC3OGRFCGJGBIP5EVT/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Keeping Python a Duck Typed Language.
On 4/20/21 10:03 AM, Mark Shannon wrote: If you guarded your code with `isinstance(foo, Sequence)` then I could not use it with my `Foo` even if my `Foo` quacked like a sequence. I was forced to use nominal typing; inheriting from Sequence, or explicitly registering as a Sequence. If I'm reading the library correctly, this is correct--but, perhaps, it could be remedied by adding a __subclasshook__ to Sequence that looked for an __iter__ attribute. That technique might also apply to other ABCs in collections.abc, Mapping for example. Would that work, or am I missing an critical detail? Cheers, //arry/ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/QRS7JWBP3KX3TYSYDVKILSYOXTLOFUY3/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Keeping Python a Duck Typed Language.
On Thu, 22 Apr 2021 at 21:40, Adrian Freund wrote: > If I understand correctly your concerns with inferring return types for > inferred protocols are that it might be to restrictive and prevent > gradual typing. Here are some examples to show how gradual typing would > still work. OK, I have no idea what's going on here any more. I have *no* concerns with inferring the return type. It was you who said that that inferring would be difficult - the exact quote is "You could statically type t as Union[Tuple[Literal['version'], int], Tuple[Literal['name'], str]], but inferring a Protocol for this would be either very hard or even impossible, especially with even more complex conditions." I don't know why you think I have a problem with inferring return types. All I've ever said is that I thought it might be an interesting idea if typing an argument as "DuckTyped" could result in type checkers automatically generated a suitable protocol type, based on the actual usage of the argument in the function (so that the programmer doesn't have to explicitly write and maintain a protocol class in parallel with the code). > If you have any concrete examples where inferring the return > type would break gradual typing let me know and I'll have a look at them. I don't, and I never have. As I say, it seemed to be you who was claiming that inferring would be too hard. I don't see much point in continuing this. You seem to be arguing against points I never made, or maybe I'm completely misunderstanding you. Either way, we're getting nowhere. Thanks for taking the time to try to explain, but I think all this has accomplished is to convince me that there's a "typing mindset" that embraces a level of strictness that I want nothing to do with. That's fine, we can agree to differ, but I'm a bit saddened at the thought that a certain proportion of the information available about typing might be hard for me to follow because its underlying assumptions are too different from mine. Paul ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/KADXCO56OFVLILMWWT5GDRGZERBJUQPB/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Keeping Python a Duck Typed Language.
On 4/22/21 5:00 PM, Paul Moore wrote: > On Thu, 22 Apr 2021 at 15:22, Adrian Freund wrote: >> On April 22, 2021 3:15:27 PM GMT+02:00, Paul Moore >> wrote: >>> but that's *absolutely* as far as I'd want to go. Note in particular >>> that I don't want to constrain the return value >> The problem is that this isn't enough to have a type safe program. You need to also constrain the return type to make sure the returned value can be safely passed to other functions. > But I don't want a type safe program. At least not in an absolute > sense. All I want is for mypy to catch the occasional error I make > where I pass the wrong parameter. For me, that's the "gradual" in > "gradual typing" - it's not a lifestyle, just a convenience. You seem > to be implying that it's "all or nothing". I don't think that inferring the required return type breaks gradual typing, but it is required for people who want type safety. If I understand correctly your concerns with inferring return types for inferred protocols are that it might be to restrictive and prevent gradual typing. Here are some examples to show how gradual typing would still work. If you have any concrete examples where inferring the return type would break gradual typing let me know and I'll have a look at them. def foo(x: DuckType): # x has to have a .bar(self) method. # The return type of which is inferred as Any, as it isn't used x.bar() def bar(x): x.bar() def foo(x: DuckType): # x has to have a .read(self) method. # The return type of which ist inferred as Any, as the parameter to bar isn't typed. bar(x.read()) Contrast that with def bar(x: DuckType): # x has to have a .bar(self) method. # The return type of which is inferred as Any. x.bar() def foo(x: DuckType): # x has to have a .read(self) method that returns something with a .bar(self) method. # If we don't infer the return type our call to bar() might be unsafe despite both foo and bar being typed. bar(x.read()) > I repeat, all I'm proposing is that > > def f(x: int): ... > def g(x: str): ... > > def main(t: DuckTyped) -> None: > if t[0] == 'version': > f(t[1]) > elif t[0] == 'name': > g(t[1]) > > gets interpreted *exactly* the same as if I'd written > > class TType(Protocol): > def __getitem__(self, int): ... > > def f(x: int): ... > def g(x: str): ... > > def main(t: TType) -> None: > if t[0] == 'version': > f(t[1]) > elif t[0] == 'name': > g(t[1]) > > How can you claim that the second example requires that " large parts > of your codebase will either need explicit annotations or will be > unchecked"? And if the second example doesn't require that, nor does > the first because it's equivalent. Both examples don't check the calls to f and g despite f and g both being typed functions and being called from typed functions. In a real codebase this will lead to a lot more instances of this happening. It would happen every time you do anything with something returned from a method on an inferred protocol > > Honestly, this conversation is just reinforcing my suspicion that > people invested in type annotations have a blind spot when it comes to > dealing with people and use cases that don't need to go "all in" with > typing :-( I don't think this is an all in or nothing. You can infer return types of inferred protocols and still use gradual typing. It's just that not inferring return types causes problems for both full and gradual typing. Adrian Freund ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/KVB33BV6T6H7PFXN574CMFZUJOVSVUBV/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Keeping Python a Duck Typed Language.
On Thu, Apr 22, 2021 at 5:43 AM Chris Angelico wrote: > File-like objects are used VERY frequently in the stdlib, and actual > open file objects have quite a large interface. The use-case is a > pretty real one: "if I were to create a simulant file object to pass > to json.load(), what methods do I need?". My experience with so-called "file-like objects" is that the interface required most of the time consists of a single method: read() > Maybe in some cases, the "smaller protocols" option is practical, but > it would need to have a useful name. The authors of the typing module already came up with an excellent convention. For the narrow protocol I mentioned, the conventional name would be "SupportsRead". Maybe "SupportsRead[str]" and "SupportsRead[bytes]". > For instance, if it needs to be > readable, iterable, closeable, and autocloseable via > __enter__/__exit__, that's ... uhh a readable, iterable, closeable > context manager? Not an improvement over "file-like object". Yes, file-like objects can and do have lots of methods. Often you don't need more than read() Cheers, Luciano On Thu, Apr 22, 2021 at 7:04 AM Chris Angelico wrote: > > On Thu, Apr 22, 2021 at 7:53 PM Paul Moore wrote: > > I wonder whether type checkers could handle a "magic" type (let's call > > it DuckTyped for now :-)) which basically means "infer a protocol > > based on usage in this function". So if I do: > > > > def my_fn(f: DuckTyped): > > with f: > > data = f.read() > > for line in f: > > print(line) > > f.close() > > > > then the type checker would automatically build a protocol type like > > the one I defined above and use that as the type of f? That would make > > it much easier to include duck typed arguments in function signatures > > while keeping the benefits of static type checking. > > > > Someone will likely correct me if this is inaccurate, but my > understanding is that that's exactly what you get if you just don't > give a type hint. The point of type hints is to give more information > to the type checker when it's unable to simply infer from usage and > context. > > ChrisA > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/RW5ACSLJP2RLBZWDGQRGBD6ZAVRUQWMG/ > Code of Conduct: http://python.org/psf/codeofconduct/ -- Luciano Ramalho | Author of Fluent Python (O'Reilly, 2015) | http://shop.oreilly.com/product/0636920032519.do | Technical Principal at ThoughtWorks | Twitter: @ramalhoorg ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HP25YCWUQVGPGWDFFFSNOLQQOHGRKEVG/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Keeping Python a Duck Typed Language.
On Thu, Apr 22, 2021 at 6:57 AM Paul Moore wrote: > > On Thu, 22 Apr 2021 at 09:46, Chris Angelico wrote: > > Maybe in some cases, the "smaller protocols" option is practical, but > > it would need to have a useful name. For instance, if it needs to be > > readable, iterable, closeable, and autocloseable via > > __enter__/__exit__, that's ... uhh a readable, iterable, closeable > > context manager? Not an improvement over "file-like object". > > Note: I've not used protocols myself, so this is speculation. > > Is the name of the protocol important? Specifically, if I do, in my code > > class X(Protocol): > def read(self): ... > def __iter__(self): ... > def close(self): ... > def __enter__(self): ... > def __exit__(self, exc_type, exc_val, exc_tb): ... > > def my_fn(fileobj: X) -> None: > # my stuff > That is not a very good example of a Protocol. If you google for best practices for interfaces in Go (#golang), you'll find they advocate for very narrow protocols—what they call "interfaces" we decided to call "protocols". Many (perhaps most) protocols in the Go standard library define a single method. I highly recommend reading up on how "interfaces" are used in Go to reason about how "protocols" should be used in Python (*) Cheers, Luciano (*) That reminded me of how I found Python. In 1998 I was using Perl, which had just started to support classes. So in the Perl mailing lists there were quite a few messages then about how classes were used in Python. After a few mentions, I read the Python tutorial and never looked back. On Thu, Apr 22, 2021 at 6:57 AM Paul Moore wrote: > > On Thu, 22 Apr 2021 at 09:46, Chris Angelico wrote: > > Maybe in some cases, the "smaller protocols" option is practical, but > > it would need to have a useful name. For instance, if it needs to be > > readable, iterable, closeable, and autocloseable via > > __enter__/__exit__, that's ... uhh a readable, iterable, closeable > > context manager? Not an improvement over "file-like object". > > Note: I've not used protocols myself, so this is speculation. > > Is the name of the protocol important? Specifically, if I do, in my code > > class X(Protocol): > def read(self): ... > def __iter__(self): ... > def close(self): ... > def __enter__(self): ... > def __exit__(self, exc_type, exc_val, exc_tb): ... > > def my_fn(fileobj: X) -> None: > # my stuff > > would that not work? An argument is checked to see if it conforms with > a protocol by confirming it has the right methods, not by name, > inheritance or registration. And if you want, you can just call X > "FileLike", it's only a local name so it won't clash with whatever > other people (or you, in a different module) want to say is > "file-like". Of course, that's incredibly verbose and messy, and it > would result in a huge proliferation of throw-away protocol classes, > which is probably not a good practice that we'd want to encourage, but > it works. > > IMO, the problem isn't that *technically* static typing excludes the > more traditional duck typing, but rather that the design, approach and > as a result the emerging "best practices" are focused around > inheritance based (is that what people mean by "nominal"?) models, to > the point where duck typing feels like an afterthought that you have > to work to include. > > I wonder whether type checkers could handle a "magic" type (let's call > it DuckTyped for now :-)) which basically means "infer a protocol > based on usage in this function". So if I do: > > def my_fn(f: DuckTyped): > with f: > data = f.read() > for line in f: > print(line) > f.close() > > then the type checker would automatically build a protocol type like > the one I defined above and use that as the type of f? That would make > it much easier to include duck typed arguments in function signatures > while keeping the benefits of static type checking. > > I will say that at the moment, this doesn't bother me much personally. > On the larger projects where I've used typing, we've been fine with > class-based typing and haven't really needed anything more complex > like protocols. On smaller projects, I just don't use typing at all. > Whether this will change if I decide to introduce typing in more > places, I don't know at the moment. I've also not really used typing > for public APIs, where an over-restrictive type would be more of an > issue. > > Paul > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/EQDLTZDXEE7RRFCAVCGLR5OTJOWFVXH5/ > Code of Conduct: http://python.org/psf/codeofconduct/ -- Luciano Ramalho | Author of Fluent Python (O'Reilly, 2015) |
[Python-Dev] Re: Keeping Python a Duck Typed Language.
On Wed, Apr 21, 2021 at 7:31 PM Paul Bryan wrote: > As demonstrated, protocols don't get us there because duck typing isn't a > matter of having an object exhibit all of the attributes of a duck, but > rather some subset of attributes to be used by the consumer. I want this duck > to quack; someone else will want it to waddle. A HUGE insight I learned studying Go is that Protocols should be defined near the code that CONSUMES it, and not near the code that PROVIDES it. That's exactly the opposite of how we use ABCs, or Java folks use interfaces (most of the time). Cheers, Luciano On Wed, Apr 21, 2021 at 7:31 PM Paul Bryan wrote: > > As demonstrated, protocols don't get us there because duck typing isn't a > matter of having an object exhibit all of the attributes of a duck, but > rather some subset of attributes to be used by the consumer. I want this duck > to quack; someone else will want it to waddle. I don't see how type hints > could reasonably support "file like object" in the duck type sense (unless > the consumer were to specify the exact attributes of the duck it's interested > in, which I fear would become a tedious type writing style). > > I too have sensed static typing driving the typing development agenda in > Python recently, causing other typing methods to take a back seat, so to > speak. I add my voice to those requesting Python handle other typing methods. > > Barring an innovation to allow a "subset" of a type to be declared in a type > hint, I would conclude that static typing and duck typing are diametrically > opposed. If we agree that both are valuable, developers could build consensus > on that point, and work to ensure that one does not move forward at the > expense of the other. > > Paul > > On Wed, 2021-04-21 at 12:36 -0700, Christopher Barker wrote: > > Thanks Mark for posting this. I know some of us are uneasy about the pace of > the typing train > > On Tue, Apr 20, 2021 at 11:20 AM Nathaniel Smith wrote: > > > If you guarded your code with `isinstance(foo, Sequence)` then I could > > not use it with my `Foo` even if my `Foo` quacked like a sequence. I was > > forced to use nominal typing; inheriting from Sequence, or explicitly > > registering as a Sequence. > > You say this like it's a bad thing, but how is this avoidable, even in > principle? Structural typing lets you check whether Foo is duck-shaped > -- has appropriate attribute names, etc. But quacking like a duck is > harder: you also have to implement the Sequence behavioral contract, > and realistically the only way to know that is if the author of Foo > tells you. > > > But that's not what duck typing is (at least to me :-) ) For a given > function, I need the passed in object to quack (and yes, I need that quack to > sound like a duck) -- but I usually don't care whether that object waddles > like a duck. > > So yes, isinstance(obj, Sequence) is really the only way to know that obj is > a Sequence in every important way -- but if you only need it to do one or two > things like a Sequence, then you don't care. > > And this is not uncommon -- I suspect it's very rare for a single function to > use most of the methods of a given ABC (or protocol, or whatever). > > And a lot of the standard library works exactly this way. Two examples > (chosen arbitrarily, I just happen to have thought about how they work): > > json.load() simply calls ``fp.read()``, and passes the result on down to > json.loads(). That's it -- no checking of anything. > > If fp does not have a read() method, you get an AttributeError. If fp has a > read() method, but it returns something other than a string, then you get > some other Exception. And if it returns a string, but that string isn't valid > JSON, you get yet another kind of error. > > In short, json.load(fp, ...) requires fp to have a read() method that returns > a valid JSON string. But it doesn't check, nor does it need to, if it's > getting an actual io.TextIOBase object. Is that the right one? I'm not > totally sure, which I kind of think makes my point -- I've been using > "file-like" objects for years (decades) without worrying about it. > > Example 2: > > The str.translate method takes: > > "a mapping of Unicode ordinals to Unicode ordinals, strings, or None" > > Ok, then you need to pass in a Mapping, yes? Well, no you don't. The docs go > on to say: > > The table must implement lookup/indexing via __getitem__, for instance a > dictionary or list. > > Ah -- so we don't need a Mapping -- we need anything indexable by an integer > that contains "ordinals, strings, or None". What the heck ABC could we use > for that? > > The ABCs do have an already complex hierarchy of containers, but there is no > "Indexable", (quacks) and certainly no "indexable and returns these > particular things. (quacks a certain way). (maybe there's something in the > typing module that would work for static typing -- I have no idea). > > I'm pretty sure this
[Python-Dev] Re: Keeping Python a Duck Typed Language.
On Wed, Apr 21, 2021 at 4:44 PM Christopher Barker wrote: >> You say this like it's a bad thing, but how is this avoidable, even in >> principle? Structural typing lets you check whether Foo is duck-shaped >> -- has appropriate attribute names, etc. But quacking like a duck is >> harder: you also have to implement the Sequence behavioral contract, >> and realistically the only way to know that is if the author of Foo >> tells you. > > > But that's not what duck typing is (at least to me :-) ) For a given > function, I need the passed in object to quack (and yes, I need that quack to > sound like a duck) -- but I usually don't care whether that object waddles > like a duck. I agree with Christopher here. For duck typing, all we care is that the method(s) we need are implemented in a sensible way. > And this is not uncommon -- I suspect it's very rare for a single function to > use most of the methods of a given ABC (or protocol, or whatever). Also agree, and very often the function cares only about one single method. > And a lot of the standard library works exactly this way. Totally. I found it astonishing that PEP 484 did not allow to correctly annotate LOTS of functions in the standard library, including built-ins people use every day like `sorted()`. Fortunately, that was corrected with PEP 544. > But there is the "culture" of Python -- and it has been very much shifting > toward more typing I disagree because Python is too widespread these days to talk about "the culture". We'll just need to learn to coexist as groups that use the language differently. -- A recent publication (sorry can't find it now -- my google fu is failing me) examined code on PyPi and found a lot of type hints -- many of which were apparently not being used with a static type checker. So why were they there? Maybe you mean this one: "Python 3 types in the wild: a tale of two type systems" Metadata: https://dl.acm.org/doi/10.1145/3426422.3426981 PDF: http://hirzels.com/martin/papers/dls20-python-types.pdf > And I've seen a lot more isinstance(Some_ABC) code lately as well. I believe a lot of that is people coming from statically typed languages and not feeling comfortable with duck typing, while for some other reason not adopting type hints. > From looking at the work of my beginning students, I can tell that they are > seeing examples out there that use more typing, to the point that they think > it's a best practice (or maybe even required?). It is considered best practice in some places. I hope we as a community never adopt it as the canonical way. > Maybe it is -- but if the community is moving that way, we should be honest > about it. Clearly parts of the community are moving that way. The paper I linked shows the use of type hints is still very limited in their sample. >> I'm not even sure that this *is* nominal typing. You could just as >> well argue that "the operation `isinstance(..., Sequence)` returns >> `True`" is just another of the behavioral constraints that are >> required to quack like a sequence. It is not just nominal typing because of the subclass hook. > I'm not sure of the definition of "nominal" typing -- but it absolutely is > NOT duck typing (As Luciano pointed out, Alex Martelli coined the term Goose > Typing for this). Yes. I wish Alex joined this conversation. Cheers, Luciano On Wed, Apr 21, 2021 at 8:28 PM Luciano Ramalho wrote: > > On Wed, Apr 21, 2021 at 4:44 PM Christopher Barker > wrote: > >> You say this like it's a bad thing, but how is this avoidable, even in > >> principle? Structural typing lets you check whether Foo is duck-shaped > >> -- has appropriate attribute names, etc. But quacking like a duck is > >> harder: you also have to implement the Sequence behavioral contract, > >> and realistically the only way to know that is if the author of Foo > >> tells you. > > > > > > But that's not what duck typing is (at least to me :-) ) For a given > > function, I need the passed in object to quack (and yes, I need that quack > > to sound like a duck) -- but I usually don't care whether that object > > waddles like a duck. > > I agree with Christopher here. For duck typing, all we care is that > the method(s) we need are implemented in a sensible way. > > > And this is not uncommon -- I suspect it's very rare for a single function > > to use most of the methods of a given ABC (or protocol, or whatever). > > Also agree, and very often the function cares only about one single method. > > > And a lot of the standard library works exactly this way. > > Totally. I found it astonishing that PEP 484 did not allow to > correctly annotate LOTS of functions in the standard library, > including built-ins people use every day like `sorted()`. Fortunately, > that was corrected with PEP 544. > > > But there is the "culture" of Python -- and it has been very much shifting > > toward more typing > > I disagree because Python is too widespread these days to talk about >
[Python-Dev] Re: Keeping Python a Duck Typed Language.
On 4/22/2021 9:15 AM, Paul Moore wrote: Absolutely, I see no problem with "use duck typing for this argument" being opt-in. As in x: 'duck'? or x: '!', where '!' means 'infer it!', or from typing import Infer ... x: Infer ? Ditto for -> ? -- Terry Jan Reedy ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/NSALGTJQIITQO3ZHI7E5G2J77JQERAKZ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Keeping Python a Duck Typed Language.
On Thu, Apr 22, 2021 at 10:47 AM Matthew Einhorn wrote: > In PyCharm, the above code will result in it highlighting the number `12` with the following warning: "Type 'int' doesn't have expected attribute 'close'" Which gives yet another use for type hints: helping out IDEs. > If instead you add an `elif isinstance(t, str):`, under that condition it'll auto-complete `t.` with all string properties/methods. now this makes me nervous -- if folks start adding isinstance checks to make their IDE more helpful , we are rally getting away from Duck Typing. However, you could have presumably typed it as Sequence[int] and gotten all the benefits of duck typing and IDE completion. However, if code were to really use a duck-typed "str-like" i would produce a failure in the type checker even if it was perfectly functional. NOTE: str is not a great example, as it's one type that we often do need to explicitly check -- to make the distinction between a Sequence of strings, which a str is, and a str itself. And str is so fundamental and complex a type it's rarely duck-typed anyway. -CHB ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/EGBSQALPGCTLAPM6FLIQLDV2YD2OLVAB/ > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/PTVPRVZDZLXGLOM5WREHWQX3ZWUKO5TS/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Keeping Python a Duck Typed Language.
On Thu, Apr 22, 2021, at 1:01 PM, Brett Cannon wrote: > > > On Thu, Apr 22, 2021 at 4:11 AM Paul Moore wrote: >> On Thu, 22 Apr 2021 at 11:21, Paul Moore wrote: >> > >> > On Thu, 22 Apr 2021 at 11:06, Chris Angelico wrote: >> > > >> > > Someone will likely correct me if this is inaccurate, but my >> > > understanding is that that's exactly what you get if you just don't >> > > give a type hint. The point of type hints is to give more information >> > > to the type checker when it's unable to simply infer from usage and >> > > context. >> > >> > Hmm, I sort of wondered about that as I wrote it. But in which case, >> > what's the problem here? My understanding was that people were >> > concerned that static typing was somehow in conflict with duck typing, >> > but if the static checkers enforce the inferred duck type on untyped >> > arguments, then that doesn't seem to be the case. Having said that, I >> > thought that untyped arguments were treated as if they had a type of >> > "Any", which means "don't type check". >> >> Looks like it doesn't: >> >> > cat .\test.py >> def example(f) -> None: >> f.close() >> >> import sys >> example(12) >> example(sys.stdin) >> PS 12:00 00:00.009 C:\Work\Scratch\typing >> > mypy .\test.py >> Success: no issues found in 1 source file >> >> What I was after was something that gave an error on the first call, >> but not on the second. Compare this: >> In PyCharm, the above code will result in it highlighting the number `12` with the following warning: "Type 'int' doesn't have expected attribute 'close'" Similarly, for: def f(x: int): ... def g(x: str): ... def main(t: DuckTyped) -> None: if t[0] == 'version': f(t[1]) elif t[0] == 'name': g(t[1]) If you replace `f(t[1])` or `g(t[1])` with just `t.` and activate auto-completion, it'll show `__getitem__` as an option (but it won't show any additional int/str methods). If instead you add an `elif isinstance(t, str):`, under that condition it'll auto-complete `t.` with all string properties/methods.___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/EGBSQALPGCTLAPM6FLIQLDV2YD2OLVAB/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Keeping Python a Duck Typed Language.
On Thu, Apr 22, 2021 at 4:11 AM Paul Moore wrote: > On Thu, 22 Apr 2021 at 11:21, Paul Moore wrote: > > > > On Thu, 22 Apr 2021 at 11:06, Chris Angelico wrote: > > > > > > Someone will likely correct me if this is inaccurate, but my > > > understanding is that that's exactly what you get if you just don't > > > give a type hint. The point of type hints is to give more information > > > to the type checker when it's unable to simply infer from usage and > > > context. > > > > Hmm, I sort of wondered about that as I wrote it. But in which case, > > what's the problem here? My understanding was that people were > > concerned that static typing was somehow in conflict with duck typing, > > but if the static checkers enforce the inferred duck type on untyped > > arguments, then that doesn't seem to be the case. Having said that, I > > thought that untyped arguments were treated as if they had a type of > > "Any", which means "don't type check". > > Looks like it doesn't: > > > cat .\test.py > def example(f) -> None: > f.close() > > import sys > example(12) > example(sys.stdin) > PS 12:00 00:00.009 C:\Work\Scratch\typing > > mypy .\test.py > Success: no issues found in 1 source file > > What I was after was something that gave an error on the first call, > but not on the second. Compare this: > > > cat .\test.py > from typing import Protocol > > class X(Protocol): > def close(self): ... > > def example(f: X) -> None: > f.close() > > import sys > example(12) > example(sys.stdin) > PS 12:03 00:00.015 C:\Work\Scratch\typing > > mypy .\test.py > test.py:10: error: Argument 1 to "example" has incompatible type > "int"; expected "X" > Found 1 error in 1 file (checked 1 source file) > Do note that this is only based on what mypy does, not necessarily what all Python type checkers do. I.e. it's quite possible pytype, pyre, or pyright infer more (especially https://pypi.org/project/pytype/ since they specifically say they infer types). ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/Y42GJAJH3BSW5ZDLZQKDZDWNAGRMXDQ3/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Keeping Python a Duck Typed Language.
On April 22, 2021 3:15:27 PM GMT+02:00, Paul Moore wrote: >On Thu, 22 Apr 2021 at 13:23, Adrian Freund wrote: >> >> According to PEP 484 all missing annotations in checked functions should be >> handled as Any. Any is compatible with all types. > >Yep, that's what I understood to be the case. > >> I think from a technical standpoint it should be possible to infer protocols >> for arguments for most functions, but there are some edge cases where this >> would not be possible, making it impractical to make this the default >> behavior. Having an annotation to make a type checker infer a protocol would >> be interesting though. > >Absolutely, I see no problem with "use duck typing for this argument" >being opt-in. > >> For example: >> >> def f(x: int): ... >> def g(x: str): ... >> >> def main(t): >> if t[0] == 'version': >> f(t[1]) >> elif t[0] == 'name': >> g(t[1]) >> >> >> You could statically type t as Union[Tuple[Literal['version'], int], >> Tuple[Literal['name'], str]], but inferring a Protocol for this would be >> either very hard or even impossible, especially with even more complex >> conditions. > >Yes, but that's inferred static typing which is *not* what I was >proposing. I think I understood what you were proposing, but my example might have been less than ideal. Sorry for that. I mixed some static types in there to simplify it. The union wasn't meant at what it should infer but was meant as a comparison to what we would to currently, with static, nominal typing. Let me try again without static types. def file(x): print(x.read()) # x has to have .read(): object def string(x): print(str(x)) # x has to have .__str__(self): object def main(t): if t[0] == 'file': file(t[1]) elif t[0] == 'string': string(t[1]) Here we can infer that t has to have a __getitem__(self, idx: int), but we can't infer it's return type >I was suggesting that the checker could easily infer that t >must have a __getitem__ method, and nothing more. So the protocol to >infer is > >class TypeOfT(Protocol): >def __getitem__(self, idx): ... > >It would be nice to go one step further and infer > >class TypeOfT(Protocol): >def __getitem__(self, idx: int): ... > >but that's *absolutely* as far as I'd want to go. Note in particular >that I don't want to constrain the return value The problem is that this isn't enough to have a type safe program. You need to also constrain the return type to make sure the returned value can be safely passed to other functions. If you don't do this large parts of your codebase will either need explicit annotations or will be unchecked. >- we've no way to know >what type it might have in the general case. IMO, inferring anything >else would over-constrain t - there's nothing in the available >information, for example, that says t must be a tuple, or a list, or >that t[3] should have any particular type, or anything like that. You can infer the return type of a function by looking at all the returns it contains, and inferring the types of the returned expressions. That isn't too hard and pytype for example already does it. You can infer the return type a protocol function should have by looking at all the places it's result are used. If you have inferred return types then constraining return types using inferred protocols would be practical in my opinion. > >My instinct is that working out that t needs to have a __getitem__ >that takes an int is pretty straightforward, as all you have to do is >look at where t is used in the function. Four places, all followed by >[] with a literal integer in the brackets. That's it. I fully >appreciate that writing *code* to do that can be a lot harder than it >looks, but that's an implementation question, not a matter of whether >it's reasonable as a proposal in theory. > >This feels like *precisely* where there seems to be a failure of >communication between the static typing and the duck typing worlds. I >have no idea what I said that would make you think that I wanted >anything like that Union type you quoted above. And yet obviously, you >somehow got that message from what I did say. Like I said above the Union. Was just meant as an example of that we would do with static, nominal typing, not what we want with duck typing. Sorry for the misunderstanding. > >Anyway, as I said this is just an interesting idea as far as I'm >concerned. I've no actual need for it right now, so I'm happy to leave >it to the mypy developers whether they want to do anything with it. > >Paul ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/AG6LWGMUNGP5O3PXJZUVDRRXO5VTJ7FW/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Keeping Python a Duck Typed Language.
According to PEP 484 all missing annotations in checked functions should be handled as Any. Any is compatible with all types. I think from a technical standpoint it should be possible to infer protocols for arguments for most functions, but there are some edge cases where this would not be possible, making it impractical to make this the default behavior. Having an annotation to make a type checker infer a protocol would be interesting though. For example: def f(x: int): ... def g(x: str): ... def main(t): if t[0] == 'version': f(t[1]) elif t[0] == 'name': g(t[1]) You could statically type t as Union[Tuple[Literal['version'], int], Tuple[Literal['name'], str]], but inferring a Protocol for this would be either very hard or even impossible, especially with even more complex conditions. Adrian Freund On April 22, 2021 1:04:11 PM GMT+02:00, Paul Moore wrote: >On Thu, 22 Apr 2021 at 11:21, Paul Moore wrote: >> >> On Thu, 22 Apr 2021 at 11:06, Chris Angelico wrote: >> > >> > Someone will likely correct me if this is inaccurate, but my >> > understanding is that that's exactly what you get if you just don't >> > give a type hint. The point of type hints is to give more information >> > to the type checker when it's unable to simply infer from usage and >> > context. >> >> Hmm, I sort of wondered about that as I wrote it. But in which case, >> what's the problem here? My understanding was that people were >> concerned that static typing was somehow in conflict with duck typing, >> but if the static checkers enforce the inferred duck type on untyped >> arguments, then that doesn't seem to be the case. Having said that, I >> thought that untyped arguments were treated as if they had a type of >> "Any", which means "don't type check". > >Looks like it doesn't: > >> cat .\test.py >def example(f) -> None: >f.close() > >import sys >example(12) >example(sys.stdin) >PS 12:00 00:00.009 C:\Work\Scratch\typing >> mypy .\test.py >Success: no issues found in 1 source file > >What I was after was something that gave an error on the first call, >but not on the second. Compare this: > >> cat .\test.py >from typing import Protocol > >class X(Protocol): >def close(self): ... > >def example(f: X) -> None: >f.close() > >import sys >example(12) >example(sys.stdin) >PS 12:03 00:00.015 C:\Work\Scratch\typing >> mypy .\test.py >test.py:10: error: Argument 1 to "example" has incompatible type >"int"; expected "X" >Found 1 error in 1 file (checked 1 source file) > >Paul >___ >Python-Dev mailing list -- python-dev@python.org >To unsubscribe send an email to python-dev-le...@python.org >https://mail.python.org/mailman3/lists/python-dev.python.org/ >Message archived at >https://mail.python.org/archives/list/python-dev@python.org/message/54C6G2JLYYD6B37J5KVKPCKSQDCGLRKA/ >Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/M4XDZUPWKPGRO3NF5VONG22YHOHAYZCM/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Keeping Python a Duck Typed Language.
Am 22.04.21 um 10:42 schrieb Chris Angelico: File-like objects are used VERY frequently in the stdlib, and actual open file objects have quite a large interface. The use-case is a pretty real one: "if I were to create a simulant file object to pass to json.load(), what methods do I need?". Maybe in some cases, the "smaller protocols" option is practical, but it would need to have a useful name. For instance, if it needs to be readable, iterable, closeable, and autocloseable via __enter__/__exit__, that's ... uhh a readable, iterable, closeable context manager? Not an improvement over "file-like object". Experience from typeshed shows that many functions in the stdlib and third-party libraries only use one or very few methods, very often just read() or write(). From a practical standpoint, small protocols seem quite feasible. These are quite an improvement over "file-like" objects, where no one knows what that actually means. - Sebastian ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/H33OH6GGIGHR7SVE6NL32I7OBO7B7F7E/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Keeping Python a Duck Typed Language.
Please let's not try to make Python a "typesafe" language. The success of Python owes a lot to the fact that duck typing is approachable, flexible and powerful. Even if you advocate static typing, I think it's a very good idea to limit the ambition of the type system if you want to keep most users happy—despite the opinion of language lawyers. Go is by far the most successful statically typed language created so far in the 21st century, and I believe a lot of that success is due to the simplicity of its type system. It's way more successful than Rust, possibly for this very reason. I find it admirable that Go was released without generics, which were seriously considered only after its runaway success. Generics will appear in Go 1.17, scheduled for August, 2021—9 years after Go 1.0. Please let us heed the wise words of Brian W. Kernighan and Alan A. A. Donovan in the introduction of "The Go Programming Language" book, a masterpiece (even if you don't like the language, the book is one of the best introduction to any language that I've ever read): """ Go has enough of a type system to avoid most of the careless mistakes that plague programmers in dynamic languages, but it has a simpler type system than comparable typed languages. This approach can sometimes lead to isolated pockets of ‘‘untyped’’ programming within a broader framework of types, and Go programmers do not go to the lengths that C++ or Haskell programmers do to express safety properties as type-based proofs. But in practice Go gives programmers much of the safety and run-time performance benefits of a relatively strong type system without the burden of a complex one. """ I also consider the support of static duck typing and goose typing (via runtime type assertions and type switches) key factors that make Go an excellent language to "get stuff done"—the best compliment Dave Beazley traditionally has for Python. Cheers, Luciano On Thu, Apr 22, 2021 at 12:05 PM Paul Moore wrote: > > On Thu, 22 Apr 2021 at 15:22, Adrian Freund wrote: > > > > On April 22, 2021 3:15:27 PM GMT+02:00, Paul Moore > > wrote: > > > >but that's *absolutely* as far as I'd want to go. Note in particular > > >that I don't want to constrain the return value > > The problem is that this isn't enough to have a type safe program. You need > > to also constrain the return type to make sure the returned value can be > > safely passed to other functions. > > But I don't want a type safe program. At least not in an absolute > sense. All I want is for mypy to catch the occasional error I make > where I pass the wrong parameter. For me, that's the "gradual" in > "gradual typing" - it's not a lifestyle, just a convenience. You seem > to be implying that it's "all or nothing". > > > If you don't do this large parts of your codebase will either need explicit > > annotations or will be unchecked. > > That's just not true. (And even if it were true, you're assuming that > I care - I've already said that my goal is much more relaxed than > complete type safety). > > I repeat, all I'm proposing is that > > def f(x: int): ... > def g(x: str): ... > > def main(t: DuckTyped) -> None: > if t[0] == 'version': > f(t[1]) > elif t[0] == 'name': > g(t[1]) > > gets interpreted *exactly* the same as if I'd written > > class TType(Protocol): > def __getitem__(self, int): ... > > def f(x: int): ... > def g(x: str): ... > > def main(t: TType) -> None: > if t[0] == 'version': > f(t[1]) > elif t[0] == 'name': > g(t[1]) > > How can you claim that the second example requires that " large parts > of your codebase will either need explicit annotations or will be > unchecked"? And if the second example doesn't require that, nor does > the first because it's equivalent. > > Honestly, this conversation is just reinforcing my suspicion that > people invested in type annotations have a blind spot when it comes to > dealing with people and use cases that don't need to go "all in" with > typing :-( > > Paul > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/CUZNKEA357N3672CCRA7DZ6C4WABN6FK/ > Code of Conduct: http://python.org/psf/codeofconduct/ -- Luciano Ramalho | Author of Fluent Python (O'Reilly, 2015) | http://shop.oreilly.com/product/0636920032519.do | Technical Principal at ThoughtWorks | Twitter: @ramalhoorg ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/STAULSS6WI7DLSHEXYXHFLHQFVWAIAKY/ Code of Conduct:
[Python-Dev] Re: Keeping Python a Duck Typed Language.
On Thu, 22 Apr 2021 at 15:22, Adrian Freund wrote: > > On April 22, 2021 3:15:27 PM GMT+02:00, Paul Moore > wrote: > >but that's *absolutely* as far as I'd want to go. Note in particular > >that I don't want to constrain the return value > The problem is that this isn't enough to have a type safe program. You need > to also constrain the return type to make sure the returned value can be > safely passed to other functions. But I don't want a type safe program. At least not in an absolute sense. All I want is for mypy to catch the occasional error I make where I pass the wrong parameter. For me, that's the "gradual" in "gradual typing" - it's not a lifestyle, just a convenience. You seem to be implying that it's "all or nothing". > If you don't do this large parts of your codebase will either need explicit > annotations or will be unchecked. That's just not true. (And even if it were true, you're assuming that I care - I've already said that my goal is much more relaxed than complete type safety). I repeat, all I'm proposing is that def f(x: int): ... def g(x: str): ... def main(t: DuckTyped) -> None: if t[0] == 'version': f(t[1]) elif t[0] == 'name': g(t[1]) gets interpreted *exactly* the same as if I'd written class TType(Protocol): def __getitem__(self, int): ... def f(x: int): ... def g(x: str): ... def main(t: TType) -> None: if t[0] == 'version': f(t[1]) elif t[0] == 'name': g(t[1]) How can you claim that the second example requires that " large parts of your codebase will either need explicit annotations or will be unchecked"? And if the second example doesn't require that, nor does the first because it's equivalent. Honestly, this conversation is just reinforcing my suspicion that people invested in type annotations have a blind spot when it comes to dealing with people and use cases that don't need to go "all in" with typing :-( Paul ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/CUZNKEA357N3672CCRA7DZ6C4WABN6FK/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Keeping Python a Duck Typed Language.
On Thu, 22 Apr 2021 at 13:23, Adrian Freund wrote: > > According to PEP 484 all missing annotations in checked functions should be > handled as Any. Any is compatible with all types. Yep, that's what I understood to be the case. > I think from a technical standpoint it should be possible to infer protocols > for arguments for most functions, but there are some edge cases where this > would not be possible, making it impractical to make this the default > behavior. Having an annotation to make a type checker infer a protocol would > be interesting though. Absolutely, I see no problem with "use duck typing for this argument" being opt-in. > For example: > > def f(x: int): ... > def g(x: str): ... > > def main(t): > if t[0] == 'version': > f(t[1]) > elif t[0] == 'name': > g(t[1]) > > > You could statically type t as Union[Tuple[Literal['version'], int], > Tuple[Literal['name'], str]], but inferring a Protocol for this would be > either very hard or even impossible, especially with even more complex > conditions. Yes, but that's inferred static typing which is *not* what I was proposing. I was suggesting that the checker could easily infer that t must have a __getitem__ method, and nothing more. So the protocol to infer is class TypeOfT(Protocol): def __getitem__(self, idx): ... It would be nice to go one step further and infer class TypeOfT(Protocol): def __getitem__(self, idx: int): ... but that's *absolutely* as far as I'd want to go. Note in particular that I don't want to constrain the return value - we've no way to know what type it might have in the general case. IMO, inferring anything else would over-constrain t - there's nothing in the available information, for example, that says t must be a tuple, or a list, or that t[3] should have any particular type, or anything like that. My instinct is that working out that t needs to have a __getitem__ that takes an int is pretty straightforward, as all you have to do is look at where t is used in the function. Four places, all followed by [] with a literal integer in the brackets. That's it. I fully appreciate that writing *code* to do that can be a lot harder than it looks, but that's an implementation question, not a matter of whether it's reasonable as a proposal in theory. This feels like *precisely* where there seems to be a failure of communication between the static typing and the duck typing worlds. I have no idea what I said that would make you think that I wanted anything like that Union type you quoted above. And yet obviously, you somehow got that message from what I did say. Anyway, as I said this is just an interesting idea as far as I'm concerned. I've no actual need for it right now, so I'm happy to leave it to the mypy developers whether they want to do anything with it. Paul ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/CU7OES4P3KUCQMD3U5766XVOESL43DRF/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Keeping Python a Duck Typed Language.
On Thu, 22 Apr 2021 at 11:21, Paul Moore wrote: > > On Thu, 22 Apr 2021 at 11:06, Chris Angelico wrote: > > > > Someone will likely correct me if this is inaccurate, but my > > understanding is that that's exactly what you get if you just don't > > give a type hint. The point of type hints is to give more information > > to the type checker when it's unable to simply infer from usage and > > context. > > Hmm, I sort of wondered about that as I wrote it. But in which case, > what's the problem here? My understanding was that people were > concerned that static typing was somehow in conflict with duck typing, > but if the static checkers enforce the inferred duck type on untyped > arguments, then that doesn't seem to be the case. Having said that, I > thought that untyped arguments were treated as if they had a type of > "Any", which means "don't type check". Looks like it doesn't: > cat .\test.py def example(f) -> None: f.close() import sys example(12) example(sys.stdin) PS 12:00 00:00.009 C:\Work\Scratch\typing > mypy .\test.py Success: no issues found in 1 source file What I was after was something that gave an error on the first call, but not on the second. Compare this: > cat .\test.py from typing import Protocol class X(Protocol): def close(self): ... def example(f: X) -> None: f.close() import sys example(12) example(sys.stdin) PS 12:03 00:00.015 C:\Work\Scratch\typing > mypy .\test.py test.py:10: error: Argument 1 to "example" has incompatible type "int"; expected "X" Found 1 error in 1 file (checked 1 source file) Paul ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/54C6G2JLYYD6B37J5KVKPCKSQDCGLRKA/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Keeping Python a Duck Typed Language.
On Thu, 22 Apr 2021 at 11:06, Chris Angelico wrote: > > On Thu, Apr 22, 2021 at 7:53 PM Paul Moore wrote: > > I wonder whether type checkers could handle a "magic" type (let's call > > it DuckTyped for now :-)) which basically means "infer a protocol > > based on usage in this function". So if I do: > > > > def my_fn(f: DuckTyped): > > with f: > > data = f.read() > > for line in f: > > print(line) > > f.close() > > > > then the type checker would automatically build a protocol type like > > the one I defined above and use that as the type of f? That would make > > it much easier to include duck typed arguments in function signatures > > while keeping the benefits of static type checking. > > > > Someone will likely correct me if this is inaccurate, but my > understanding is that that's exactly what you get if you just don't > give a type hint. The point of type hints is to give more information > to the type checker when it's unable to simply infer from usage and > context. Hmm, I sort of wondered about that as I wrote it. But in which case, what's the problem here? My understanding was that people were concerned that static typing was somehow in conflict with duck typing, but if the static checkers enforce the inferred duck type on untyped arguments, then that doesn't seem to be the case. Having said that, I thought that untyped arguments were treated as if they had a type of "Any", which means "don't type check". So I guess the point here is that either the typing community/documentation isn't doing a very good job of explaining how duck types and static types work together, or that people who like duck typed interfaces¹ aren't reading the available documentation in type checkers explaining how to do that with static typing :-) Paul ¹ And I include myself in that - maybe I need to go and read the mypy docs properly rather than just learning what I need by following examples in existing code... ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/IRSXXYHMYCRBBA5HKPNZ7OJDRTEGUAG7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Keeping Python a Duck Typed Language.
On Thu, Apr 22, 2021 at 7:53 PM Paul Moore wrote: > I wonder whether type checkers could handle a "magic" type (let's call > it DuckTyped for now :-)) which basically means "infer a protocol > based on usage in this function". So if I do: > > def my_fn(f: DuckTyped): > with f: > data = f.read() > for line in f: > print(line) > f.close() > > then the type checker would automatically build a protocol type like > the one I defined above and use that as the type of f? That would make > it much easier to include duck typed arguments in function signatures > while keeping the benefits of static type checking. > Someone will likely correct me if this is inaccurate, but my understanding is that that's exactly what you get if you just don't give a type hint. The point of type hints is to give more information to the type checker when it's unable to simply infer from usage and context. ChrisA ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/RW5ACSLJP2RLBZWDGQRGBD6ZAVRUQWMG/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Keeping Python a Duck Typed Language.
On Thu, 22 Apr 2021 at 09:46, Chris Angelico wrote: > Maybe in some cases, the "smaller protocols" option is practical, but > it would need to have a useful name. For instance, if it needs to be > readable, iterable, closeable, and autocloseable via > __enter__/__exit__, that's ... uhh a readable, iterable, closeable > context manager? Not an improvement over "file-like object". Note: I've not used protocols myself, so this is speculation. Is the name of the protocol important? Specifically, if I do, in my code class X(Protocol): def read(self): ... def __iter__(self): ... def close(self): ... def __enter__(self): ... def __exit__(self, exc_type, exc_val, exc_tb): ... def my_fn(fileobj: X) -> None: # my stuff would that not work? An argument is checked to see if it conforms with a protocol by confirming it has the right methods, not by name, inheritance or registration. And if you want, you can just call X "FileLike", it's only a local name so it won't clash with whatever other people (or you, in a different module) want to say is "file-like". Of course, that's incredibly verbose and messy, and it would result in a huge proliferation of throw-away protocol classes, which is probably not a good practice that we'd want to encourage, but it works. IMO, the problem isn't that *technically* static typing excludes the more traditional duck typing, but rather that the design, approach and as a result the emerging "best practices" are focused around inheritance based (is that what people mean by "nominal"?) models, to the point where duck typing feels like an afterthought that you have to work to include. I wonder whether type checkers could handle a "magic" type (let's call it DuckTyped for now :-)) which basically means "infer a protocol based on usage in this function". So if I do: def my_fn(f: DuckTyped): with f: data = f.read() for line in f: print(line) f.close() then the type checker would automatically build a protocol type like the one I defined above and use that as the type of f? That would make it much easier to include duck typed arguments in function signatures while keeping the benefits of static type checking. I will say that at the moment, this doesn't bother me much personally. On the larger projects where I've used typing, we've been fine with class-based typing and haven't really needed anything more complex like protocols. On smaller projects, I just don't use typing at all. Whether this will change if I decide to introduce typing in more places, I don't know at the moment. I've also not really used typing for public APIs, where an over-restrictive type would be more of an issue. Paul ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/EQDLTZDXEE7RRFCAVCGLR5OTJOWFVXH5/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Keeping Python a Duck Typed Language.
On Thu, Apr 22, 2021 at 5:03 PM Ryan Gonzalez wrote: > > On Apr 21, 2021, 5:29 PM -0500, Paul Bryan , wrote: > > As demonstrated, protocols don't get us there because duck typing isn't a > matter of having an object exhibit all of the attributes of a duck, but > rather some subset of attributes to be used by the consumer. I want this duck > to quack; someone else will want it to waddle. I don't see how type hints > could reasonably support "file like object" in the duck type sense (unless > the consumer were to specify the exact attributes of the duck it's interested > in, which I fear would become a tedious type writing style). > > > I'd argue that, if you frequently have cases where functions use a relatively > small subset of a much larger interface, it's simply that your interfaces are > too large. You're always free to define your own, smaller protocols that just > implement the subset of the interfaces you need. > File-like objects are used VERY frequently in the stdlib, and actual open file objects have quite a large interface. The use-case is a pretty real one: "if I were to create a simulant file object to pass to json.load(), what methods do I need?". Maybe in some cases, the "smaller protocols" option is practical, but it would need to have a useful name. For instance, if it needs to be readable, iterable, closeable, and autocloseable via __enter__/__exit__, that's ... uhh a readable, iterable, closeable context manager? Not an improvement over "file-like object". ChrisA ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2LQQXLETT6TFB4UATJBEN5HDI47KAZME/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Keeping Python a Duck Typed Language.
On Apr 21, 2021, 5:29 PM -0500, Paul Bryan , wrote: > As demonstrated, protocols don't get us there because duck typing isn't a > matter of having an object exhibit all of the attributes of a duck, but > rather some subset of attributes to be used by the consumer. I want this duck > to quack; someone else will want it to waddle. I don't see how type hints > could reasonably support "file like object" in the duck type sense (unless > the consumer were to specify the exact attributes of the duck it's interested > in, which I fear would become a tedious type writing style). I'd argue that, if you frequently have cases where functions use a relatively small subset of a much larger interface, it's simply that your interfaces are too large. You're always free to define your own, smaller protocols that just implement the subset of the interfaces you need. > > I too have sensed static typing driving the typing development agenda in > Python recently, causing other typing methods to take a back seat, so to > speak. I add my voice to those requesting Python handle other typing methods. > > Barring an innovation to allow a "subset" of a type to be declared in a type > hint, I mentioned above that defining smaller protocols is an option. If the main concern is due to wanting to define them in the type signature, or wanting to ensure the types match the base interface, then that's not new: TypeScript does them both (well, at least you could accomplish the latter roughly using intersection types). > > Paul > > On Wed, 2021-04-21 at 12:36 -0700, Christopher Barker wrote: > > Thanks Mark for posting this. I know some of us are uneasy about the pace > > of the typing train > > > > On Tue, Apr 20, 2021 at 11:20 AM Nathaniel Smith wrote: > > > > If you guarded your code with `isinstance(foo, Sequence)` then I could > > > > not use it with my `Foo` even if my `Foo` quacked like a sequence. I was > > > > forced to use nominal typing; inheriting from Sequence, or explicitly > > > > registering as a Sequence. > > > > > > You say this like it's a bad thing, but how is this avoidable, even in > > > principle? Structural typing lets you check whether Foo is duck-shaped > > > -- has appropriate attribute names, etc. But quacking like a duck is > > > harder: you also have to implement the Sequence behavioral contract, > > > and realistically the only way to know that is if the author of Foo > > > tells you. > > > > > > > But that's not what duck typing is (at least to me :-) ) For a given > > function, I need the passed in object to quack (and yes, I need that quack > > to sound like a duck) -- but I usually don't care whether that object > > waddles like a duck. > > > > So yes, isinstance(obj, Sequence) is really the only way to know that obj > > is a Sequence in every important way -- but if you only need it to do one > > or two things like a Sequence, then you don't care. > > > > And this is not uncommon -- I suspect it's very rare for a single function > > to use most of the methods of a given ABC (or protocol, or whatever). > > > > And a lot of the standard library works exactly this way. Two examples > > (chosen arbitrarily, I just happen to have thought about how they work): > > > > json.load() simply calls ``fp.read()``, and passes the result on down to > > json.loads(). That's it -- no checking of anything. > > > > If fp does not have a read() method, you get an AttributeError. If fp has a > > read() method, but it returns something other than a string, then you get > > some other Exception. And if it returns a string, but that string isn't > > valid JSON, you get yet another kind of error. > > > > In short, json.load(fp, ...) requires fp to have a read() method that > > returns a valid JSON string. But it doesn't check, nor does it need to, if > > it's getting an actual io.TextIOBase object. Is that the right one? I'm not > > totally sure, which I kind of think makes my point -- I've been using > > "file-like" objects for years (decades) without worrying about it. > > > > Example 2: > > > > The str.translate method takes: > > > > "a mapping of Unicode ordinals to Unicode ordinals, strings, or None" > > > > Ok, then you need to pass in a Mapping, yes? Well, no you don't. The docs > > go on to say: > > > > The table must implement lookup/indexing via __getitem__, for instance a > > dictionary or list. > > > > Ah -- so we don't need a Mapping -- we need anything indexable by an > > integer that contains "ordinals, strings, or None". What the heck ABC could > > we use for that? > > > > The ABCs do have an already complex hierarchy of containers, but there is > > no "Indexable", (quacks) and certainly no "indexable and returns these > > particular things. (quacks a certain way). (maybe there's something in the > > typing module that would work for static typing -- I have no idea). > > > > I'm pretty sure this particular API was designed to accommodate the old py2 > > str.translate,
