Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Sun, Apr 26, 2015 at 2:01 AM, Ronan Lamy ronan.l...@gmail.com wrote: * PEP484 hints are too high-level. Replacing an 'int' object with a single machine word would be useful, but an 'int' annotation gives no guarantee that it's correct (because Python 3 ints can have arbitrary size and because subclasses of 'int' can override any operation to invoke arbitrary code). Then create your own int16, uint64 etc types. The PEP doesn't explain how to do that. And even if it's possible, such types wouldn't be very useful as they're not stable under any arithmetic operation (e.g. int16 + int16 doesn't necessarily fit in int16). If you define a function that adds two integers and want it to be optimized, it's going to have to check for overflow anyway. That said, though, I'd much rather a machine-word optimization be a feature of int than a special adornment that you give to some of your data. CPython 3.x doesn't do it, CPython 2.x kinda did it (an int would turn into a long if it got too big, but a long would never turn into an int), but MicroPython would be most welcome to do the job fully. (I would guess that PyPy already does this kind of thing.) Yes, that means integer addition can't become a single machine language instruction, but on the other hand, it means the Python code doesn't need to concern itself with any boundaries. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Le 25/04/15 04:07, Steven D'Aprano a écrit : On Sat, Apr 25, 2015 at 02:05:15AM +0100, Ronan Lamy wrote: * Hints have no run-time effect. The interpreter cannot assume that they are obeyed. I know what you mean, but just for the record, annotations are runtime inspectable, so people can (and probably have already started) to write runtime argument checking decorators or frameworks which rely on the type hints. * PEP484 hints are too high-level. Replacing an 'int' object with a single machine word would be useful, but an 'int' annotation gives no guarantee that it's correct (because Python 3 ints can have arbitrary size and because subclasses of 'int' can override any operation to invoke arbitrary code). Then create your own int16, uint64 etc types. The PEP doesn't explain how to do that. And even if it's possible, such types wouldn't be very useful as they're not stable under any arithmetic operation (e.g. int16 + int16 doesn't necessarily fit in int16). * A lot more information is needed to produce good code (e.g. “this f() called here really means this function there, and will never be monkey-patched” – same with len() or list(), btw). * Most of this information cannot easily be expressed as a type * If the interpreter gathers all that information, it'll probably have gathered a superset of what PEP484 can provide anyway. All this is a red herring. If type hints are useful to PyPy, that's a bonus. Cython uses its own system of type hints, a future version may be able to use PEP 484 hints instead. But any performance benefit is a bonus. PEP 484 is for increasing correctness, not speed. Yes, talking about performance in the context of PEP 484 is a red herring, that's what I'm saying. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Le 24/04/15 19:45, Paul Sokolovsky a écrit : Hello, On Fri, 24 Apr 2015 18:27:29 +0100 Ronan Lamy ronan.l...@gmail.com wrote: PyPy's FAQ has an explanation of why type hints are not for performance. http://pypy.readthedocs.org/en/latest/faq.html#would-type-annotations-help-pypy-s-performance You probably intended to write why type hints are not for *PyPy's* performance. There're many other language implementations and modules for which it may be useful, please don't limit your imagination by a single case. Those points apply to basically any compliant implementation of Python relying on speculative optimisation. Python is simply too dynamic for PEP484-style hints to provide any useful performance improvement targets. What's your point - saying that type annotations alone not enough to achieve the best (C-like) performance, which is true, or saying that if they are alone not enough, then they are not needed at all, which is ... strange ? My point is that the arguments in the PyPy FAQ aren't actually specific to PyPy, and therefore that the conclusion, that hints are almost entirely useless if you’re looking at performance, holds in general. So let me restate these arguments in terms of a generic, performance-minded implementation of the full Python language spec: * Hints have no run-time effect. The interpreter cannot assume that they are obeyed. * PEP484 hints are too high-level. Replacing an 'int' object with a single machine word would be useful, but an 'int' annotation gives no guarantee that it's correct (because Python 3 ints can have arbitrary size and because subclasses of 'int' can override any operation to invoke arbitrary code). * A lot more information is needed to produce good code (e.g. “this f() called here really means this function there, and will never be monkey-patched” – same with len() or list(), btw). * Most of this information cannot easily be expressed as a type * If the interpreter gathers all that information, it'll probably have gathered a superset of what PEP484 can provide anyway. And speaking of PyPy, it really should think how to improve its performance - not of generated programs, but of generation itself. If compilation of a trivial program on a pumpy hardware takes 5 minutes and gigabytes of RAM and diskspace, few people will use it for other purposes beyond curiosity. There's something very un-Pythonic in waiting 5 mins just to run 10-line script. Type hints can help here too ;-) (by not wasting resources propagating types thru the same old standard library for example). Sorry, but that's nonsense. PyPy would be a seriously useless interpreter if running a 10-line script required such a lengthy compilation, so, obviously, that's not what happens. You seem to misunderstand what PyPy is: it's an interpreter with a just-in-time compiler, not a static compiler. It doesn't generate programs in any meaningful sense. Instead, it interprets the program, and when it detects a hot code path, it compiles it to machine code based on the precise types it sees. No resources are wasted on code that isn't actually executed. Regardless of whether I understood that meta-meta stuff, I just followed couple of tutorials, each of them warning of memory and disk space issues, and both running long to get results. Everyone else following tutorials will get the same message I did - PyPy is a slow-to-work-with bloat. Ah, I suppose you're talking about the RPython tool chain, which is used to build PyPy. Though it's an interesting topic in itself (and is pretty much comparable to Cython wrt. type hints), it has about as much relevance to PyPy users as the inner workings of GCC have to CPython users. Well, the thing is that people don't seem to want to write PyPy tutorials, because it's boring. However, I can give you the definitive 3-line version: 1. Download and install PyPy [http://pypy.org/download.html] 2. Launch the 'pypy' executable. 3. Go read https://docs.python.org/2/tutorial/ As for uber-meta stuff PyPy offers - I'm glad that's all done in my favorite language, leaving all other languages behind. I'm saddened there's no mundane JIT or static compiler usable and accepted by all community - many other languages have that. This all goes pretty offtopic wrt to the original discussion, so again, what's your point - you say that all these things can't be done in Python, or there's no need for it to be done? That people should look somewhere else? I submitted a bug to jinja2 project and posted message on its mailing list - I didn't get reply for 3 months. Why? Because its maintainer went hacking another language, how was it called, alGOl, or something. You want me and other folks to go too? Sorry, I'm staying so far, and keep dreaming of better Python's future (where for example if I need to get more performance from existing app, I can gradually optimize it based on need, not rewrite it in another language or be hitting not implemented in
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Fri, Apr 24, 2015 at 6:05 PM, Ronan Lamy ronan.l...@gmail.com wrote: Le 24/04/15 19:45, Paul Sokolovsky a écrit : Hello, On Fri, 24 Apr 2015 18:27:29 +0100 Ronan Lamy ronan.l...@gmail.com wrote: PyPy's FAQ has an explanation of why type hints are not for performance. http://pypy.readthedocs.org/en/latest/faq.html#would-type-annotations-help-pypy-s-performance You probably intended to write why type hints are not for *PyPy's* performance. There're many other language implementations and modules for which it may be useful, please don't limit your imagination by a single case. Those points apply to basically any compliant implementation of Python relying on speculative optimisation. Python is simply too dynamic for PEP484-style hints to provide any useful performance improvement targets. What's your point - saying that type annotations alone not enough to achieve the best (C-like) performance, which is true, or saying that if they are alone not enough, then they are not needed at all, which is ... strange ? My point is that the arguments in the PyPy FAQ aren't actually specific to PyPy, and therefore that the conclusion, that hints are almost entirely useless if you’re looking at performance, holds in general. So let me restate these arguments in terms of a generic, performance-minded implementation of the full Python language spec: * Hints have no run-time effect. The interpreter cannot assume that they are obeyed. * PEP484 hints are too high-level. Replacing an 'int' object with a single machine word would be useful, but an 'int' annotation gives no guarantee that it's correct (because Python 3 ints can have arbitrary size and because subclasses of 'int' can override any operation to invoke arbitrary code). * A lot more information is needed to produce good code (e.g. “this f() called here really means this function there, and will never be monkey-patched” – same with len() or list(), btw). * Most of this information cannot easily be expressed as a type * If the interpreter gathers all that information, it'll probably have gathered a superset of what PEP484 can provide anyway. I'm with the PyPy folks here -- I don't see any use for PEP 484 type hints from a code generation perspective. Even if the hints were guaranteed to be correct, the PEP 484 type system doesn't follow substitutability. I don't mean that as a critique, I think it's a decision that makes it more useful by keeping it in line with the majority of type usage in Python, but it means that even if the hints are correct they don't really end up providing any guarantees to the JIT. And speaking of PyPy, it really should think how to improve its performance - not of generated programs, but of generation itself. If compilation of a trivial program on a pumpy hardware takes 5 minutes and gigabytes of RAM and diskspace, few people will use it for other purposes beyond curiosity. There's something very un-Pythonic in waiting 5 mins just to run 10-line script. Type hints can help here too ;-) (by not wasting resources propagating types thru the same old standard library for example). Sorry, but that's nonsense. PyPy would be a seriously useless interpreter if running a 10-line script required such a lengthy compilation, so, obviously, that's not what happens. You seem to misunderstand what PyPy is: it's an interpreter with a just-in-time compiler, not a static compiler. It doesn't generate programs in any meaningful sense. Instead, it interprets the program, and when it detects a hot code path, it compiles it to machine code based on the precise types it sees. No resources are wasted on code that isn't actually executed. Regardless of whether I understood that meta-meta stuff, I just followed couple of tutorials, each of them warning of memory and disk space issues, and both running long to get results. Everyone else following tutorials will get the same message I did - PyPy is a slow-to-work-with bloat. Ah, I suppose you're talking about the RPython tool chain, which is used to build PyPy. Though it's an interesting topic in itself (and is pretty much comparable to Cython wrt. type hints), it has about as much relevance to PyPy users as the inner workings of GCC have to CPython users. Well, the thing is that people don't seem to want to write PyPy tutorials, because it's boring. However, I can give you the definitive 3-line version: 1. Download and install PyPy [http://pypy.org/download.html] 2. Launch the 'pypy' executable. 3. Go read https://docs.python.org/2/tutorial/ As for uber-meta stuff PyPy offers - I'm glad that's all done in my favorite language, leaving all other languages behind. I'm saddened there's no mundane JIT or static compiler usable and accepted by all community - many other languages have that. This all goes pretty offtopic wrt to the original discussion, so again, what's your point - you say that all these things can't
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Sat, Apr 25, 2015 at 02:05:15AM +0100, Ronan Lamy wrote: * Hints have no run-time effect. The interpreter cannot assume that they are obeyed. I know what you mean, but just for the record, annotations are runtime inspectable, so people can (and probably have already started) to write runtime argument checking decorators or frameworks which rely on the type hints. * PEP484 hints are too high-level. Replacing an 'int' object with a single machine word would be useful, but an 'int' annotation gives no guarantee that it's correct (because Python 3 ints can have arbitrary size and because subclasses of 'int' can override any operation to invoke arbitrary code). Then create your own int16, uint64 etc types. * A lot more information is needed to produce good code (e.g. “this f() called here really means this function there, and will never be monkey-patched” – same with len() or list(), btw). * Most of this information cannot easily be expressed as a type * If the interpreter gathers all that information, it'll probably have gathered a superset of what PEP484 can provide anyway. All this is a red herring. If type hints are useful to PyPy, that's a bonus. Cython uses its own system of type hints, a future version may be able to use PEP 484 hints instead. But any performance benefit is a bonus. PEP 484 is for increasing correctness, not speed. -- Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 22 April 2015 at 03:03, Carol Willing willi...@willingconsulting.com wrote: 2. Clearly, great thought has been put into this PEP. If anyone has a good analysis of the potential impact on Python 3 adoption, please do pass along. I would be interested in reading the information. I don't have hard data, but I do get to see quite a few of the challenges that dynamically typed languages (including Python) have scaling to large development teams. When you look at the way folks actually *use* languages like Java and C# in practice, you find that they're almost universally tool mavens (in Oliver Steele's sense: http://blog.osteele.com/posts/2004/11/ides/) where automated tools are taking care of most of the boilerplate for them. Find me a C# developer, and I'll bet you they're a Visual Studio user, find me a Java developer, and I'll bet you they're an Eclipse or IntelliJ user. This approach actually offers a lot of benefits in putting a skill floor under a development team - while you can't make people think, you can at least automatically rule out broad categories of mundane errors, and focus on the business logic of the problem you're aiming to solve. As a result, my main reaction to PEP 484 in a Python 3 adoption context is that Python 3 offers all the agility and flexibility of Python 2, with all the structural assurances of Java or C# is actually a huge selling point for anyone in an institutional context attempting to persuade their management chain to back a migration effort from Python 2 to Python 3. Another factor to consider here is that my understanding is that one of the *reasons* folks want better structural analysis (by annotating the Python 2 stdlib and key third part libraries in typeshed) is to help automate Python 2 - Python 3 conversions in the absence of comprehensive test coverage. While to some degree this works against the previous point, there's a difference between having this as an addon vs having it as a standard feature (and unlike the network security changes and bundling pip, this is one where I'm entirely happy leaving it as the kind of carrot that can take pride of place in a corporate business case). The legitimate concerns that arise are around what happens to *community* code bases, including the standard library itself, as well as what folks are likely to see if they run inspect.getsource() on standard library components. For that, I think there's a lot of value in continuing to have explicit type hints be the exception rather than the rule in the upstream community, so the idea of the typeshed project is enormously appealing to me. If anyone doesn't want to deal with type hints themselves, but has a contributor that really wants to annotate their library, then take it to typeshed will hopefully become a recurring refrain :) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Le 23/04/15 14:55, Paul Sokolovsky a écrit : Hello, On Thu, 23 Apr 2015 09:15:44 -0400 Daniel Holth dho...@gmail.com wrote: [] Also ask why no one used type specifier, they are possible since Python 3.0 ? Because it is the wrong way for Python. That's an example of how perceptions differ. In my list, everyone(*) uses them - MyPy, MicroPython, etc. Even more should use them (any JIT module, which are many), but sit in the bushes, waiting for a kick, like PEP484 provides. It's OK that type hints are only to assist the programmer. Yes, it's OK to have a situation where type hints assist only a programmer. It's not OK to think that type hints may be useful only for programmer, instead of bunch more purposes, several of which were already shown in the long previous discussion. PyPy's FAQ has an explanation of why type hints are not for performance. http://pypy.readthedocs.org/en/latest/faq.html#would-type-annotations-help-pypy-s-performance You probably intended to write why type hints are not for *PyPy's* performance. There're many other language implementations and modules for which it may be useful, please don't limit your imagination by a single case. Those points apply to basically any compliant implementation of Python relying on speculative optimisation. Python is simply too dynamic for PEP484-style hints to provide any useful performance improvement targets. And speaking of PyPy, it really should think how to improve its performance - not of generated programs, but of generation itself. If compilation of a trivial program on a pumpy hardware takes 5 minutes and gigabytes of RAM and diskspace, few people will use it for other purposes beyond curiosity. There's something very un-Pythonic in waiting 5 mins just to run 10-line script. Type hints can help here too ;-) (by not wasting resources propagating types thru the same old standard library for example). Sorry, but that's nonsense. PyPy would be a seriously useless interpreter if running a 10-line script required such a lengthy compilation, so, obviously, that's not what happens. You seem to misunderstand what PyPy is: it's an interpreter with a just-in-time compiler, not a static compiler. It doesn't generate programs in any meaningful sense. Instead, it interprets the program, and when it detects a hot code path, it compiles it to machine code based on the precise types it sees. No resources are wasted on code that isn't actually executed. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Hello, On Fri, 24 Apr 2015 18:27:29 +0100 Ronan Lamy ronan.l...@gmail.com wrote: Also ask why no one used type specifier, they are possible since Python 3.0 ? Because it is the wrong way for Python. That's an example of how perceptions differ. In my list, everyone(*) uses them - MyPy, MicroPython, etc. Even more should use them (any JIT module, which are many), but sit in the bushes, waiting for a kick, like PEP484 provides. It's OK that type hints are only to assist the programmer. Yes, it's OK to have a situation where type hints assist only a programmer. It's not OK to think that type hints may be useful only for programmer, instead of bunch more purposes, several of which were already shown in the long previous discussion. PyPy's FAQ has an explanation of why type hints are not for performance. http://pypy.readthedocs.org/en/latest/faq.html#would-type-annotations-help-pypy-s-performance You probably intended to write why type hints are not for *PyPy's* performance. There're many other language implementations and modules for which it may be useful, please don't limit your imagination by a single case. Those points apply to basically any compliant implementation of Python relying on speculative optimisation. Python is simply too dynamic for PEP484-style hints to provide any useful performance improvement targets. What's your point - saying that type annotations alone not enough to achieve the best (C-like) performance, which is true, or saying that if they are alone not enough, then they are not needed at all, which is ... strange ? And speaking of PyPy, it really should think how to improve its performance - not of generated programs, but of generation itself. If compilation of a trivial program on a pumpy hardware takes 5 minutes and gigabytes of RAM and diskspace, few people will use it for other purposes beyond curiosity. There's something very un-Pythonic in waiting 5 mins just to run 10-line script. Type hints can help here too ;-) (by not wasting resources propagating types thru the same old standard library for example). Sorry, but that's nonsense. PyPy would be a seriously useless interpreter if running a 10-line script required such a lengthy compilation, so, obviously, that's not what happens. You seem to misunderstand what PyPy is: it's an interpreter with a just-in-time compiler, not a static compiler. It doesn't generate programs in any meaningful sense. Instead, it interprets the program, and when it detects a hot code path, it compiles it to machine code based on the precise types it sees. No resources are wasted on code that isn't actually executed. Regardless of whether I understood that meta-meta stuff, I just followed couple of tutorials, each of them warning of memory and disk space issues, and both running long to get results. Everyone else following tutorials will get the same message I did - PyPy is a slow-to-work-with bloat. As for uber-meta stuff PyPy offers - I'm glad that's all done in my favorite language, leaving all other languages behind. I'm saddened there's no mundane JIT or static compiler usable and accepted by all community - many other languages have that. This all goes pretty offtopic wrt to the original discussion, so again, what's your point - you say that all these things can't be done in Python, or there's no need for it to be done? That people should look somewhere else? I submitted a bug to jinja2 project and posted message on its mailing list - I didn't get reply for 3 months. Why? Because its maintainer went hacking another language, how was it called, alGOl, or something. You want me and other folks to go too? Sorry, I'm staying so far, and keep dreaming of better Python's future (where for example if I need to get more performance from existing app, I can gradually optimize it based on need, not rewrite it in another language or be hitting not implemented in uber-meta stuff). -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
lol @ the fact that the type hints are breaking github's syntax highlighter :) On 23 April 2015 at 14:44, Paul Sokolovsky pmis...@gmail.com wrote: Hello, On Thu, 23 Apr 2015 14:48:58 +0200 Wolfgang Langner tds333+py...@gmail.com wrote: Hello, On Thu, Apr 23, 2015 at 11:59 AM, Paul Sokolovsky pmis...@gmail.com wrote: Hello, On Thu, 23 Apr 2015 10:43:52 +0200 Wolfgang Langner tds333+py...@gmail.com wrote: [] Also ask why no one used type specifier, they are possible since Python 3.0 ? Because it is the wrong way for Python. That's an example of how perceptions differ. In my list, everyone(*) uses them - MyPy, MicroPython, etc. Even more should use them (any JIT module, which are many), but sit in the bushes, waiting for a kick, like PEP484 provides. (*) Everyone of those who needs them. Otherwise, let's throw out metaclasses - noone uses them. They are there to be used and won't go away. But for most Libraries out there no one used it. JIT (Numba), Cython and other compilers/tools doing optimization have their own syntax and needs. That's exactly what needs to change, and where this PEP helps, as was already pointed out: http://code.activestate.com/lists/python-dev/135659/ They need even more information like i32, i64, different floats and so on. That's poor excuse for not trying to be a good member of Python community (and work on standard type annotation syntax, instead of digging own hole). MyPy is really new and in development. And the docstring type spec way is still possible. Anything is possible. The talk is about what makes most of sense. Docstrings were always arbitrary string designated at human's consumption, how (ab)using them for type annotations is better than having clean language-grammar syntax? MicroPython uses one annotation for their inline assembler stuff not type hints. Here's how MicroPython uses type annotations: https://github.com/micropython/micropython/blob/master/tests/micropython/viper_ptr8_load.py For the library there are no type hints for function definitions. Remark: I use Metaclasses, seldom but if needed very useful. :-) -- bye by Wolfgang -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/hjwp2%40cantab.net -- -- Harry J.W. Percival -- Twitter: @hjwp Mobile: +44 (0) 78877 02511 Skype: harry.percival ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Hello, On Thu, 23 Apr 2015 15:25:30 +0100 Harry Percival harry.perci...@gmail.com wrote: lol @ the fact that the type hints are breaking github's syntax highlighter :) What one can expect from software written in Ruby? ;-) -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Thu, Apr 23, 2015 at 03:25:30PM +0100, Harry Percival wrote: lol @ the fact that the type hints are breaking github's syntax highlighter :) That just tells us that Github's syntax highlighter has been broken for over five years. Function annotations go back to Python 3.0, more than five years ago. The only thing which is new about type hinting is that we're adding a standard *use* for those annotations. I just tested a version of kwrite from 2005, ten years old, and it highlights the following annotated function perfectly: def func(a:str='hello', b:int=int(x+1)) - None: print(a + b) Of course, I'm hoping that any decent type checker won't need the type hints. It should be able to infer from the default values that a is a string and b an int, and only require a type hint if you want to accept other types as well. (It should also highlight that a+b cannot succeed.) -- Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Wed, Apr 22, 2015 at 5:45 PM, Guido van Rossum gu...@python.org wrote: Given that even if Difference existed, and even if we had a predefined type alias for Difference[Iterable[str], str], you' still have to remember to mark up all those functions with that annotation. It almost sounds simpler to just predefine this function: def make_string_list(a: Union[str, Iterable[str]]) - Iterable[str]: if isinstance(a, str): return [a] else: return a fair enough -- and I do indeed have that code in various places already. Somehow, I've always been uncomfortable with checking specifically for the str type -- guess I want everything to be fully duck-typable. But then I wouldn't be doing type hints, either, would I? -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Hello, On Thu, 23 Apr 2015 09:15:44 -0400 Daniel Holth dho...@gmail.com wrote: [] Also ask why no one used type specifier, they are possible since Python 3.0 ? Because it is the wrong way for Python. That's an example of how perceptions differ. In my list, everyone(*) uses them - MyPy, MicroPython, etc. Even more should use them (any JIT module, which are many), but sit in the bushes, waiting for a kick, like PEP484 provides. It's OK that type hints are only to assist the programmer. Yes, it's OK to have a situation where type hints assist only a programmer. It's not OK to think that type hints may be useful only for programmer, instead of bunch more purposes, several of which were already shown in the long previous discussion. PyPy's FAQ has an explanation of why type hints are not for performance. http://pypy.readthedocs.org/en/latest/faq.html#would-type-annotations-help-pypy-s-performance You probably intended to write why type hints are not for *PyPy's* performance. There're many other language implementations and modules for which it may be useful, please don't limit your imagination by a single case. And speaking of PyPy, it really should think how to improve its performance - not of generated programs, but of generation itself. If compilation of a trivial program on a pumpy hardware takes 5 minutes and gigabytes of RAM and diskspace, few people will use it for other purposes beyond curiosity. There's something very un-Pythonic in waiting 5 mins just to run 10-line script. Type hints can help here too ;-) (by not wasting resources propagating types thru the same old standard library for example). -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Thu, Apr 23, 2015 at 9:55 AM, Paul Sokolovsky pmis...@gmail.com wrote: Hello, On Thu, 23 Apr 2015 09:15:44 -0400 Daniel Holth dho...@gmail.com wrote: [] Also ask why no one used type specifier, they are possible since Python 3.0 ? Because it is the wrong way for Python. That's an example of how perceptions differ. In my list, everyone(*) uses them - MyPy, MicroPython, etc. Even more should use them (any JIT module, which are many), but sit in the bushes, waiting for a kick, like PEP484 provides. It's OK that type hints are only to assist the programmer. Yes, it's OK to have a situation where type hints assist only a programmer. It's not OK to think that type hints may be useful only for programmer, instead of bunch more purposes, several of which were already shown in the long previous discussion. PyPy's FAQ has an explanation of why type hints are not for performance. http://pypy.readthedocs.org/en/latest/faq.html#would-type-annotations-help-pypy-s-performance You probably intended to write why type hints are not for *PyPy's* performance. There're many other language implementations and modules for which it may be useful, please don't limit your imagination by a single case. And speaking of PyPy, it really should think how to improve its performance - not of generated programs, but of generation itself. If compilation of a trivial program on a pumpy hardware takes 5 minutes and gigabytes of RAM and diskspace, few people will use it for other purposes beyond curiosity. There's something very un-Pythonic in waiting 5 mins just to run 10-line script. Type hints can help here too ;-) (by not wasting resources propagating types thru the same old standard library for example). Naturally, PyPy is very controversial. Type annotations can help to compile Python into a subset of Python. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 4/22/2015 8:45 PM, Guido van Rossum wrote: On Wed, Apr 22, 2015 at 2:38 PM, Chris Barker chris.bar...@noaa.gov mailto:chris.bar...@noaa.gov wrote: On Tue, Apr 21, 2015 at 11:32 PM, Terry Reedy tjre...@udel.edu mailto:tjre...@udel.edu wrote: I was just thinking today that for this, typing needs a subtraction (difference) operation in addition to an addition (union) operation: Difference(Iterable(str), str) Anyway, the point is that being able to say all these types, except this one would solve this particular problem -- but would it solve any others? Do we want this to work around a quirk in Pythons string type? More seriously, I doubt there are other important use cases for Difference. I thought about Difference(numbers.Number, complex) to get ordered numbers, but numbers.Real should probably work. I agree more real uses are needed before adding Difference. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
* Wolfgang Langner tds333+py...@gmail.com [2015-04-23 10:43:52 +0200]: 2. Using it in the language as part of the function signature, my first thought was oh good, then I changed my mind to: oh it can be very ugly and unreadable, it is the wrong place. Now I am against it, best is, if I have to specify type signatures, do it in one place, keep them up to date. Most of the time this is the documentation. Why not use the docstring with a standard type specifier for this. Suggested here: http://pydev.blogspot.de/2015/04/type-hinting-on-python.html While I happen to agree with you (but I'm happy with both variants really), I think that's a thing which has definitely been decided already. The idea is also listed in the PEP: https://www.python.org/dev/peps/pep-0484/#other-backwards-compatible-conventions Docstrings. There is an existing convention for docstrings, based on the Sphinx notation ( :type arg1: description ). This is pretty verbose (an extra line per parameter), and not very elegant. We could also make up something new, but the annotation syntax is hard to beat (because it was designed for this very purpose). For nearly every function I have written, there is a docstring and most of the time also a type specified. But if I must provide all this in a second place it is not the right way to go. Over time normally one place misses some changes and is wrong. It seems there's an extension for Sphinx already to use type annotations: https://pypi.python.org/pypi/sphinx-autodoc-annotation It seems to be older than PEP 484 (December 2013), so I hope it'll be updated or already work well with the ideas in the PEP. Also ask why no one used type specifier, they are possible since Python 3.0 ? Because it is the wrong way for Python. Well, except that Sphinx extension, and MyPy, and MicroPython, and a thing which already exists for run-time type checking[1] and probably a whole lot more. The whole *reason* for PEP 484 (at least from my perspective) is to have a common base for the existing usages of type annotations. Florian [1] https://github.com/ceronman/typeannotations -- http://www.the-compiler.org | m...@the-compiler.org (Mail/XMPP) GPG: 916E B0C8 FD55 A072 | http://the-compiler.org/pubkey.asc I love long mails! | http://email.is-not-s.ms/ pgp0dRiwqbI7j.pgp Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Hello, On Thu, 23 Apr 2015 10:43:52 +0200 Wolfgang Langner tds333+py...@gmail.com wrote: [] Also ask why no one used type specifier, they are possible since Python 3.0 ? Because it is the wrong way for Python. That's an example of how perceptions differ. In my list, everyone(*) uses them - MyPy, MicroPython, etc. Even more should use them (any JIT module, which are many), but sit in the bushes, waiting for a kick, like PEP484 provides. (*) Everyone of those who needs them. Otherwise, let's throw out metaclasses - noone uses them. Regards, Wolfgang -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Hi, having a lot experience with Python beginners and people programming Java/Python I have also an opinion about this. ;-) First reaction was, oh good. Then I read every thread and comment about it, looked at a lot internal code give all some time and the result is: I found a lot of code written like Java with assert and isinstance for type checks. Not very Pythonic but someone from Java thinks this is good. Because Python has no type checks. A lot of people think types help in the IDE and for tools to do automatic checks, find some errors earlier. My conclusion: 1. typing.py is good to make a standard how to specify type hints. Can also be used by tools and IDE's 2. Using it in the language as part of the function signature, my first thought was oh good, then I changed my mind to: oh it can be very ugly and unreadable, it is the wrong place. Now I am against it, best is, if I have to specify type signatures, do it in one place, keep them up to date. Most of the time this is the documentation. Why not use the docstring with a standard type specifier for this. Suggested here: http://pydev.blogspot.de/2015/04/type-hinting-on-python.html 3. Stub files are good if someone needs this feature, it is complete separate and if someone don't want to write them it is no problem. But it is good to have a way to name them and a place to find them and know this. IDE's parse the docstring already, it is optional to have one and every Editor has folding support to disable it if it is to much of information. Checkers like mypy can use the type info from there to do checks. If there is a standard way to specify types in docstrings. It is also possible to specify a type hint for a variable, this is good for documentations and can also be used for type checks. For stub files the same can be used. So no different syntax. Also If someone wants it and a module has no documentation it can be added from there. For nearly every function I have written, there is a docstring and most of the time also a type specified. But if I must provide all this in a second place it is not the right way to go. Over time normally one place misses some changes and is wrong. I am convinced something like: def myfunction(a, b): My special function. :param a: ... :type a: int :param b: second value :type b: int :rtype: None or shorter form also possible in Sphinx if only one type for parameter is specified: def myfunction(a, b): My special function. :param int a: ... :param int b: second value :rtype: None Is easier to maintain good to read, changes not the world how to write Python functions and is useful for IDE's and checkers. And the best, most editors and IDE's support folding of docstrings and if someone don't want to see them it can be done now. The docstring is a good place for type specification, because as stated Python never will do type checks on execution. So no need to make it part of the language syntax. All this can also be applied over time to the standard library. If it makes sense so specify a type it is good to have it in the documentation. And sphinx is the standard documentation tool for Python now. Also ask why no one used type specifier, they are possible since Python 3.0 ? Because it is the wrong way for Python. Regards, Wolfgang ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Hello, On Thu, Apr 23, 2015 at 11:59 AM, Paul Sokolovsky pmis...@gmail.com wrote: Hello, On Thu, 23 Apr 2015 10:43:52 +0200 Wolfgang Langner tds333+py...@gmail.com wrote: [] Also ask why no one used type specifier, they are possible since Python 3.0 ? Because it is the wrong way for Python. That's an example of how perceptions differ. In my list, everyone(*) uses them - MyPy, MicroPython, etc. Even more should use them (any JIT module, which are many), but sit in the bushes, waiting for a kick, like PEP484 provides. (*) Everyone of those who needs them. Otherwise, let's throw out metaclasses - noone uses them. They are there to be used and won't go away. But for most Libraries out there no one used it. JIT (Numba), Cython and other compilers/tools doing optimization have their own syntax and needs. They need even more information like i32, i64, different floats and so on. MyPy is really new and in development. And the docstring type spec way is still possible. MicroPython uses one annotation for their inline assembler stuff not type hints. For the library there are no type hints for function definitions. Remark: I use Metaclasses, seldom but if needed very useful. :-) -- bye by Wolfgang ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Thu, Apr 23, 2015 at 5:59 AM, Paul Sokolovsky pmis...@gmail.com wrote: Hello, On Thu, 23 Apr 2015 10:43:52 +0200 Wolfgang Langner tds333+py...@gmail.com wrote: [] Also ask why no one used type specifier, they are possible since Python 3.0 ? Because it is the wrong way for Python. That's an example of how perceptions differ. In my list, everyone(*) uses them - MyPy, MicroPython, etc. Even more should use them (any JIT module, which are many), but sit in the bushes, waiting for a kick, like PEP484 provides. It's OK that type hints are only to assist the programmer. PyPy's FAQ has an explanation of why type hints are not for performance. http://pypy.readthedocs.org/en/latest/faq.html#would-type-annotations-help-pypy-s-performance ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Hello, On Thu, 23 Apr 2015 14:48:58 +0200 Wolfgang Langner tds333+py...@gmail.com wrote: Hello, On Thu, Apr 23, 2015 at 11:59 AM, Paul Sokolovsky pmis...@gmail.com wrote: Hello, On Thu, 23 Apr 2015 10:43:52 +0200 Wolfgang Langner tds333+py...@gmail.com wrote: [] Also ask why no one used type specifier, they are possible since Python 3.0 ? Because it is the wrong way for Python. That's an example of how perceptions differ. In my list, everyone(*) uses them - MyPy, MicroPython, etc. Even more should use them (any JIT module, which are many), but sit in the bushes, waiting for a kick, like PEP484 provides. (*) Everyone of those who needs them. Otherwise, let's throw out metaclasses - noone uses them. They are there to be used and won't go away. But for most Libraries out there no one used it. JIT (Numba), Cython and other compilers/tools doing optimization have their own syntax and needs. That's exactly what needs to change, and where this PEP helps, as was already pointed out: http://code.activestate.com/lists/python-dev/135659/ They need even more information like i32, i64, different floats and so on. That's poor excuse for not trying to be a good member of Python community (and work on standard type annotation syntax, instead of digging own hole). MyPy is really new and in development. And the docstring type spec way is still possible. Anything is possible. The talk is about what makes most of sense. Docstrings were always arbitrary string designated at human's consumption, how (ab)using them for type annotations is better than having clean language-grammar syntax? MicroPython uses one annotation for their inline assembler stuff not type hints. Here's how MicroPython uses type annotations: https://github.com/micropython/micropython/blob/master/tests/micropython/viper_ptr8_load.py For the library there are no type hints for function definitions. Remark: I use Metaclasses, seldom but if needed very useful. :-) -- bye by Wolfgang -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 4/21/2015 6:41 PM, Chris Barker wrote: Well, it'll catch passing in a string instead of a sequence of strings -- one of teh common and semi-insidious type errors I see a lot (at least with newbies). Oh wait, maybe it won't -- a string IS a sequence of strings. That's why this is an insidioous bug in teh first place: List[string] will work, yes -- but now I'm locked down the user to an actual, list, yes? So I try: Iterable[string] ah yes -- now users can use tuples of strings, or a custom class, or -- but wait, darn it, a string IS an iterable of strings.. SIGH. I was just thinking today that for this, typing needs a subtraction (difference) operation in addition to an addition (union) operation: Difference(Iterable(str), str) -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 4/21/2015 6:50 PM, Chris Barker wrote: On Tue, Apr 21, 2015 at 11:58 AM, Paul Sokolovsky pmis...@gmail.com mailto:pmis...@gmail.com wrote: It does, and hope people won't be caught in static typechecking loop and consider other usages too. I an interested is using type hints for automatic or at least semi-automatic generation of tests, in particular for parameters hinted as generic and abstract. A parameter ints: Iterable(int) should be work with empty, length 1, and longer tuples, lists, sets, dicts (with int keys) and a custom class. Unless there is supplementary doc otherwise, it should handle negative, 0, and positive ints, small and large, in regular and random patterns. (Thinking about this more, some of this could be done without the PEP being approved, but there has to be *some* syntax for test generator input information, and using the standard vocabulary of typing instead of inventing something would be easier.) Im confused -- from the bit I've been skimming the discussion, over on python-ideas, and now here, is that this is all about static typechecking. The PEP is about a) defining type hints as the primary use of annotations* and b) adding a new module to standardize type hints instead of having multiple groups define multiple variations on the theme. The urgency of developing a standard now is that there *are* at least two or three independent efforts. * http://www.artima.com/weblogs/viewpost.jsp?thread=85551 (2004) makes it clear that Guido thought of static typing first and annotations second, in the form added to 3.0 years later, as an implementation thereof. The primary motivation of the people working on the new module is static typechecking by 3rd party programs. This seems to be Guido's primary interest and what he believes will be the primary use. It is also the primary promotional argument since type checking already exists. However, this does not stop others from having other motivations and making other uses of the information once available. It's not about run-time type checking. The PEP does not prevent that, and will make it more feasible by standardizing on one system. It's not about type-base performance optimization. Nothing is planned for CPython, but type hints are already being used by MicroPython for this purpose. (To me, this is one of the pluses of it having targeted 3.3.) It's not about any use of annotations other than types. Right, but there are multiple possible uses for type information. What is it about other than static typechecking? There is also documentation. Type hints are partially an alternative to stylized entries in doc strings. A program might combine the two or check consistency. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Wed, Apr 22, 2015 at 2:38 PM, Chris Barker chris.bar...@noaa.gov wrote: Oh wait, maybe it won't -- a string IS a sequence of strings. That's why this is an insidious bug in the first place. On Tue, Apr 21, 2015 at 11:32 PM, Terry Reedy tjre...@udel.edu wrote: I was just thinking today that for this, typing needs a subtraction (difference) operation in addition to an addition (union) operation: Difference(Iterable(str), str) Yup -- that might solve, it, but it feels a bit odd -- I can take any Iterable of string, except a string. -- but what if there are others that won't work??? But I guess that's the core of putting type hints on a dynamic language. Still, I tend to think that this particular issue is really a limitation with Python's type system -- nothing to do with type hinting. I can see that a character type seems useless in Python, but there are lessons from other places: a numpy array is a collection of (usually) numbers that can be treated as a single entity -- much like a string is a collection of characters that is treated as a single entity -- in both cases, it's core to convenience and performance to do that. But with numpy, when you index an array, you get something back with one less dimension: index into a 3-d array, you get a 2-d array index into a 2-d array, you get a 1-d array index into a 1-d array, you get a scalar -- NOT a length-one 1-d array Sometimes this is a pain for generic code, but more often than not it's critical to writing dynamic code -- not because you couldn't do the operations you want, but because it's important to distinguish between a scalar and an array that happens to have only one value. Anyway, the point is that being able to say all these types, except this one would solve this particular problem -- but would it solve any others? Do we want this to work around a quirk in Pythons string type? NOTE: I know full well that adding a character type to Python is not worth it. If you switch to bytes the problem goes away. :-P More seriously, I doubt there are other important use cases for Difference. Given that even if Difference existed, and even if we had a predefined type alias for Difference[Iterable[str], str], you' still have to remember to mark up all those functions with that annotation. It almost sounds simpler to just predefine this function: def make_string_list(a: Union[str, Iterable[str]]) - Iterable[str]: if isinstance(a, str): return [a] else: return a and call this in those functions that have an Interable[str] argument. Now instead of getting errors for all the places where a caller mistakenly passes a single str, you've *fixed* all those call sites. Isn't that more Pythonic? :-) -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Oh wait, maybe it won't -- a string IS a sequence of strings. That's why this is an insidious bug in the first place. On Tue, Apr 21, 2015 at 11:32 PM, Terry Reedy tjre...@udel.edu wrote: I was just thinking today that for this, typing needs a subtraction (difference) operation in addition to an addition (union) operation: Difference(Iterable(str), str) Yup -- that might solve, it, but it feels a bit odd -- I can take any Iterable of string, except a string. -- but what if there are others that won't work??? But I guess that's the core of putting type hints on a dynamic language. Still, I tend to think that this particular issue is really a limitation with Python's type system -- nothing to do with type hinting. I can see that a character type seems useless in Python, but there are lessons from other places: a numpy array is a collection of (usually) numbers that can be treated as a single entity -- much like a string is a collection of characters that is treated as a single entity -- in both cases, it's core to convenience and performance to do that. But with numpy, when you index an array, you get something back with one less dimension: index into a 3-d array, you get a 2-d array index into a 2-d array, you get a 1-d array index into a 1-d array, you get a scalar -- NOT a length-one 1-d array Sometimes this is a pain for generic code, but more often than not it's critical to writing dynamic code -- not because you couldn't do the operations you want, but because it's important to distinguish between a scalar and an array that happens to have only one value. Anyway, the point is that being able to say all these types, except this one would solve this particular problem -- but would it solve any others? Do we want this to work around a quirk in Pythons string type? NOTE: I know full well that adding a character type to Python is not worth it. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 21 April 2015 at 18:12, Steven D'Aprano st...@pearwood.info wrote: I expect that dealing with duck typing will be very high on the list of priorities for the future. In the meantime, for this specific use-case, you're probably not going to be able to statically check this type hint. Your choices would be: - don't type check anything; To be clear, for the moment this is what requests will do, unless the other maintainers strongly disagree with me (they don't). I am quite convinced that PEP 484 is insufficiently powerful to make it worthwhile for requests to provide 'official' type hints. I suspect someone will provide hints to typeshed, and I certainly hope they're good, because if they're bad we'll definitely field bug reports about them (more on this in a different thread I think). ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 21 April 2015 at 17:59, Guido van Rossum gu...@python.org wrote: For me, PEP 484 is a stepping stone. Among the authors of PEP 484 there was much discussion about duck typing, and mypy even has some limited support for duck typing (I think you can still find it by searching the mypy code for protocol). But we ran out of time getting all the details written up and agreed upon, so we decided to punt -- for now. But duck typing still needs to have a way to talk about things like seek method with this type signature (something like `def seek(self, offset: int, whence: int=SEEK_SET) - int`) so the current proposal gets us part of the way there. The hope is that once 3.5 is out (with PEP 484's typing.py included *provisional* mode) we can start working on the duck typing specification. The alternative would have been to wait until 3.6, but we didn't think that there would be much of an advantage to postponing the more basic type hinting syntax (it would be like refusing to include import until you've sorted out packages). During the run of 3.5 we'll hopefully get feedback on where duck typing is most needed and how to specify it -- valuable input that would be much harder to obtain of *no* part of the type hints notation were standardized. This makes a lot of sense. If PEP 484 is intended to be a stepping stone (or compromise, or beta, or whatever word one wants to use), then it is easy to forgive it its limitations, and I'm looking forward to seeing it improve. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 4/20/2015 9:07 PM, Chris Angelico wrote: On Tue, Apr 21, 2015 at 10:52 AM, Ben Finney ben+pyt...@benfinney.id.au wrote: Jack is not complaining only about *writing* code. He's complaining about the effect this will have on code that we all are expected to *read*. For reading, good function and parameter names and a good summary as the first line of the docstring are probably more useful *to humans* than formal type annotations. Ahh. Yes, that's a concern. When you go digging into that library to find out how it works, yes, you'd be face-to-face with their type annotations. Maybe it'd be of value to have a quick code stripper that takes away all the annotations, I was already thinking about that for Idle. The file should then be either semi read-only or renamed. Also options to merge annotations in from stub files for editing and to split back out when writing. plus any other junk/framing that you're not interested in, and gives you something you can browse in a text editor? It could take away all the Sphinx adornments from docstrings, any source control versioning markers, all that kind of thing. Interesting idea. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, 21 Apr 2015 10:10:06 -0700, Guido van Rossum gu...@python.org wrote: On Tue, Apr 21, 2015 at 9:17 AM, R. David Murray rdmur...@bitdance.com wrote: Please be respectful rather than inflammatory. If you read what I wrote, I did not say that I was going to stop contributing, I specifically talked about that gut reaction being both emotional and illogical. That doesn't make the reaction any less real, and the fact that such reactions exist is a data point you should consider in conducting your PR campaign for this issue. (I don't mean that last as a negative: this issue *requires* an honest PR campaign.) Well, my own reactions at this point in the flame war are also quite emotional. :-( I have done my best in being honest in my PR campaign. But I feel like the I believe you have. My inclusion of the word 'honest' was meant to contrast the kind of PR we need with the kind of PR people typically think about, which is often not particularly honest. opposition (not you, but definitely some others -- have you seen Twitter?) are spreading FUD based on an irrational conviction that this will destroy No, I tend only to peek in there occasionally. Python. It will not. It may not prove the solution to all Python's problems -- there's always 3.6. (Oh wait, Python 2.7 is perfect. I've heard that before -- Paul Everitt famously said the same of Python 1.5.2. Aren't you glad I didn't take him literally? :-P ) Yes. But somewhere there or not long before was my introduction to Python, so I remember it fondly :) --David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Hello, On Tue, 21 Apr 2015 09:50:59 -0700 Ethan Furman et...@stoneleaf.us wrote: On 04/21, Paul Sokolovsky wrote: And for example yesterday's big theme was people blackmailing that they stop contributing to stdlib if annotations are in [...] A volunteer's honest reaction is not blackmail, and categorizing it as such is not helpful to the discussion. Sure, that was rather humoresque note. Still, one may wonder why honest reaction is like that, if from reading PEP484 it's clear that it doesn't change status quo: https://www.python.org/dev/peps/pep-3107 added annotations long ago, and PEP484 just provides default semantics for them. Note that *default*, not the only. PEP484 is full of conditions and long transition windows. Did PEP3107 shutter everyone's world? No. And PEP484 is nothing but a logical continuation of PEP3107, coming forward with real use for annotations, but not intended to shutter everyone's world. Well, hopefully Guido now clarified what was already written PEP484 (or not written, like nowhere it says we add requirement of mandatory typehints in version 3.x [of stdlib or otherwise]). But now people try to come up with *anti*-patterns on how to use type annotation and argue that these anti-patterns are not useful. Surely, annotations are useful in some places and not useful in other. requests doesn't need them? Good. But it's quite useful to annotate FFT routines and subroutines as taking arrays of floats, we can't get faster than C(tm) without that, like some other languages did (or claim to have done, and now look attractive). (You don't do FFT in Python? OMG, that's old.) -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 10:03 AM, Carol Willing willi...@willingconsulting.com wrote: Two areas of clarification would be helpful for me: 1. Optional: What does this really mean in practice? Am I opting in to static type checking and type hints? Or must I opt out of type hints? Having to opt out would probably put more burden on the educational use cases than opting in would for a large corporate project. You're definitely opting in. You must do two things to opt in: (a) add type hints to some of your signatures; (b) run a type checker over your code. You will only get the benefits of type hints of you do both, and there is no pressure to opt in. If someone else adds type hints to their code, which you import, but you don't run the type checker, nothing changes for you when you run your code (you can still get away with things that happen to work even the type checker would reject them). When you are reading their source code, you will see their type hints -- like every other aspect of writing good code, some people's type hints will be readable, while others' less so. At least nobody will be writing type hints in Cyrillic. :-) 2. Clearly, great thought has been put into this PEP. If anyone has a good analysis of the potential impact on Python 3 adoption, please do pass along. I would be interested in reading the information. I wish I had a crystal ball, but this is hard to predict. Anecdotally, some people believe this will be catnip, while others believe it to be poison. The truth will surely be somewhere in the middle. At this point we don't know what drives Python 3 adoption except time -- it's definitely going up. :-) -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 21 April 2015 at 17:55, Gregory P. Smith g...@krypto.org wrote: I view most of this thread as FUD. The fear is understandable, I'm trying to tell people to stop panicing. I think (hope!) everyone is clear that what's being expressed in this thread is honest (emotional) reactions. There's a negative connotation to the term FUD that's uncomfortable in this context, although it's understandable that the authors and supporters of the PEP feel frustrated about the feedback, so merely using terms with negative connotations is a pretty measured response, actually :-) We've got people expressing fear and others telling them not to panic. Realistically, just being told not to panic won't help. What will help is feeling that the fears are understood and being considered, even if ultimately the response is we think it'll be alright - wait and see. Most people contributing to this thread have probably not been involved in the earlier discussions (I know I tuned out of the threads on python-ideas) so are now reacting at short notice to what looks like a big change. Things should calm down in due course, but what the heck is all this? reactions are to be expected. And honestly, the PEP is pretty difficult to read. That's probably the nature of the subject (my eyes glaze over when reading about type theory) but it doesn't help. When I read the PEP, after a few sections I found myself losing focus because I kept thinking but what about X? Result - I didn't manage to read the whole PEP and came away with a feeling that all those things I thought of couldn't be handled. Not a positive reaction, unfortunately. That's the fault of the reader rather than the PEP, but maybe the PEP could include a common concerns/FAQ section, with discussions of the issues people are worried about? One final point. The overwhelming feeling I'm getting about the debate is that the main response to people with concerns is you don't have to use it and it's optional. But that's not the point - we're talking past each other to an extent. I (and I presume most others) understand the fact that type hints are optional. We may or may not use them ourselves (I'm not actually ruling out that I might end up liking them!) But that's not the concern - what we're trying to understand is how we, as a community of programmers, and an ecosystem of interdependent codebases, deal with the possibility that there could be two differing philosophies (to hint or not to hint) trying to work together. Maybe it'll be a non-event, and we carry on as usual. Maybe nobody will use types in public code for years, because we all end up having to support 2.x whether we want to or not. Maybe we shouldn't try to solve big social issues like that and we should just trust in the fact that Python has a great community who *will* find a way to work together. I don't know, but that's my real question, and it's optional isn't really the response I'm looking for. (The response I probably deserve is likely don't take it all so seriously, and it's probably fair. I apologise - it's been that sort of day :-)) Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 9:47 AM, Antoine Pitrou solip...@pitrou.net wrote: On Tue, 21 Apr 2015 09:28:45 -0700 Guido van Rossum gu...@python.org wrote: On Tue, Apr 21, 2015 at 12:49 AM, Antoine Pitrou solip...@pitrou.net wrote: On Mon, 20 Apr 2015 20:43:38 -0400 R. David Murray rdmur...@bitdance.com wrote: +1 to this from me too. I'm afraid that means I'm -1 on the PEP. I didn't write this in my earlier email because I wasn't sure about it, but my gut reaction after reading Harry's email was if type annotations are used in the stdlib, I'll probably stop contributing. That doesn't mean that's *true*, but that's the first time I've ever had that thought, so it is probably worth sharing. I think it would be nice to know what the PEP means for daily stdlib development. If patches have to carry typing information each time they add/enhance an API that's an addition burden. If typing is done separately by interested people then it sounds like it wouldn't have much of an impact on everyone else's workflow. This point will be moot until new code appears in the stdlib whose author likes type hints. As I said, we won't be converting existing code to add type hints (I'm actively against that for the stdlib, for reasons I've explained already). I was thinking of potential stub files. Or are you also putting a ban on those for existing stdlib code? Sorry if that has already been answered... No ban, but for now the plan is to collect and manage those separately as part of typeshed. I imagine the typical failure case is where the stubs are more restrictive than the implementation -- after all if the implementation becomes *more* restrictive it's breaking backward compatibility. I think it's okay for the stubs to lag behind like that, so I don't expect pressure on stdlib contributors. If anything the pressure will be on typeshed. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, 21 Apr 2015 18:27:50 +0300 Paul Sokolovsky pmis...@gmail.com wrote: Let me try: MicroPython already uses type annotations for statically typed functions. E.g. def add(x:int, y:int): return x + y will translate the function to just 2 machine instructions. That's quite nice. Oh really, you care to support single-precisions in Numba? Numba is quite specialized. In our area, single-precision arithmetic can sometimes give double the speed on modern CPUs (especially when LLVM is able to vectorize the code). The speedup can be even larger on a GPU (where double-precision execution resources are scarce). I agree it doesn't make sense for a general-purpose Python compiler. Anyway, back to your example, it would be done like: SFloat = float DFloat = float For a random tool out there, SFloat and DFloat would be just aliases to floats, but Numba will know they have additional semantics behind them. (That assumes that typedefs like SFloat can be accessed in symbolic form - that's certainly possible if you have your own parser/VM, but might worth to think how to do it on CPython level). Fortunately, we don't have our own parser :-) We work from the CPython bytecode. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, 21 Apr 2015 16:55:49 -, Gregory P. Smith g...@krypto.org wrote: We will not be putting type annotations anywhere in the stdlib or expecting anyone else to maintain them there. That would never happen until tools that are convincing enough in their utility for developers to _want_ to use are available and accepted. That'll be a developer workflow thing we could address with a later PEP. IF it happens at all. This is the most reassuring statement I've heard so far ;) --David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Apr 21, 2015, at 11:23 AM, Guido van Rossum gu...@python.org wrote: 2. Clearly, great thought has been put into this PEP. If anyone has a good analysis of the potential impact on Python 3 adoption, please do pass along. I would be interested in reading the information. I wish I had a crystal ball, but this is hard to predict. Anecdotally, some people believe this will be catnip, while others believe it to be poison. The truth will surely be somewhere in the middle. At this point we don't know what drives Python 3 adoption except time -- it's definitely going up. :-) Anecdotal evidence shows that some organizations perceive this feature as one that justifies migration. Some of those organizations are pretty serious about open-sourcing things. That makes me believe that by sheer volume of the code they’re producing, Python 3 adoption will continue to increase. As Gregory Smith rightfully pointed out, nobody wants ugly code. I understand why people are afraid of that and it warms my heart that they are. The community cares so much about aesthetics and readability, it’s great! We will evolve this over time. This will be a learning process for everybody but we can’t learn to swim by only theorizing about it. We thought of the evolving nature of the solution from Day 1, hence the *provisional* nature of it. The wordy syntax is another example of that. Not requiring changes to the interpreter and the standard library was very high on the list of priorities. Once the *concept* proves itself, then we can improve on the syntax. Acknowledging PEP 484 being just the second step^ in a long journey is why some “obvious” parts are left out for now (hello, duck typing; hello, multiple dispatch; hello, runtime type checks in unit tests; etc. etc.). Those are often big enough to warrant their own PEPs. ^ PEP 3107 being the first. -- Lukasz Langa | Facebook Production Engineer | Global Consistency (+1) 650-681-7811___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 4/21/15 9:17 AM, R. David Murray wrote: Please be respectful rather than inflammatory. Thank you David. If you read what I wrote, I did not say that I was going to stop contributing, I specifically talked about that gut reaction being both emotional and illogical. That doesn't make the reaction any less real, and the fact that such reactions exist is a data point you should consider in conducting your PR campaign for this issue. (I don't mean that last as a negative: this issue *requires* an honest PR campaign.) As David stated, gut reactions are real. These reactions have the potential, if listened to and respected, to lead toward an optimal (not ideal) solution. Likely, the implementation of optional static type checking will evolve from reasoned, respectful debate of the issues not inflammatory quotes. Quite frankly, belittling someone's understanding or knowledge does not serve the PEP or technical issues at hand. I like the option of static type checking for critical high availability and high reliability applications (i.e. air traffic control, financial transactions). I'm less interested in static type checking of inputs when prototyping or developing less critical applications. There have been good technical points made by many on this thread especially given the different use cases. These use cases, and likely a few more, are important to an honest, continued technical refinement of the PEP. Two areas of clarification would be helpful for me: 1. Optional: What does this really mean in practice? Am I opting in to static type checking and type hints? Or must I opt out of type hints? Having to opt out would probably put more burden on the educational use cases than opting in would for a large corporate project. 2. Clearly, great thought has been put into this PEP. If anyone has a good analysis of the potential impact on Python 3 adoption, please do pass along. I would be interested in reading the information. Warmly, Carol P.S. I do member a time when tools to easily check for memory leaks in C were new and magical. Looking at the Coverity scans, I'm glad that the old magic is reaping real benefits. -- *Carol Willing* Developer | Willing Consulting https://willingconsulting.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Hello, On Tue, 21 Apr 2015 12:17:01 -0400 R. David Murray rdmur...@bitdance.com wrote: On Tue, 21 Apr 2015 18:27:50 +0300, Paul Sokolovsky pmis...@gmail.com wrote: I was replying to Steven's message. Did you read it? Yes. And I try to follow general course of discussion, as its hard to follow individual sub-threads. And for example yesterday's big theme was people blackmailing that they stop contributing to stdlib if annotations are in, and today's theme appear to be people telling that static type checking won't be useful. And just your reply to Steven was a final straw which prompted me to point out that static type checking is not a crux of it, but just one (not the biggest IMHO) usage. Please be respectful rather than inflammatory. If you read what I wrote, I did not say that I was going to stop contributing, I specifically talked about that gut reaction being both emotional and illogical. Sure, and I didn't mean you personally - your original message conveyed that it was gut feeling very well. More like following to your message, where people gave +1. To a gut feeling? To the thought of stopping contribution? I dunno, and that blackmail note wasn't intended to be more serious than burn the witch and other funny stylistic devices which already appeared and make the discussion a bit more lively ;-). That doesn't make the reaction any less real, and the fact that such reactions exist is a data point you should consider in conducting your PR campaign for this issue. (I don't mean that last as a negative: this issue *requires* an honest PR campaign.) It does, and hope people won't be caught in static typechecking loop and consider other usages too. --David -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 1:10 PM, Guido van Rossum gu...@python.org wrote: On Tue, Apr 21, 2015 at 9:17 AM, R. David Murray rdmur...@bitdance.com wrote: Please be respectful rather than inflammatory. If you read what I wrote, I did not say that I was going to stop contributing, I specifically talked about that gut reaction being both emotional and illogical. That doesn't make the reaction any less real, and the fact that such reactions exist is a data point you should consider in conducting your PR campaign for this issue. (I don't mean that last as a negative: this issue *requires* an honest PR campaign.) Well, my own reactions at this point in the flame war are also quite emotional. :-( I have done my best in being honest in my PR campaign. But I feel like the opposition (not you, but definitely some others -- have you seen Twitter?) are spreading FUD based on an irrational conviction that this will destroy Python. It will not. It may not prove the solution to all Python's problems -- there's always 3.6. (Oh wait, Python 2.7 is perfect. I've heard that before -- Paul Everitt famously said the same of Python 1.5.2. Aren't you glad I didn't take him literally? :-P ) -- --Guido van Rossum (python.org/~guido) I am looking forward to using type annotations. I am not very good at remembering the operations that are available on the objects passed around in my code, but I am very good at typing CTRL-space. To get that I am happy to modify my code with weird docstrings or any other notation. Good support for completion, aided by standard annotations, eliminates a huge amount of cross-referencing while coding. I'm also hopeful that static type checking, aided with annotations, will help with unicode porting. Duck typing does not work very well when you are trying to differentiate between bytes and str. Also, Python 1.5.2 was pretty good :-) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 2:23 PM, Guido van Rossum gu...@python.org wrote: At least nobody will be writing type hints in Cyrillic. :-) Why not? It works just fine: Список = list def sum(x: Список): ... pass ... (See https://en.wikipedia.org/wiki/Rapira for some prior art.) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, 21 Apr 2015 21:31:49 +0300, Paul Sokolovsky pmis...@gmail.com wrote: On Tue, 21 Apr 2015 09:50:59 -0700 Ethan Furman et...@stoneleaf.us wrote: On 04/21, Paul Sokolovsky wrote: And for example yesterday's big theme was people blackmailing that they stop contributing to stdlib if annotations are in [...] A volunteer's honest reaction is not blackmail, and categorizing it as such is not helpful to the discussion. Sure, that was rather humoresque note. Still, one may wonder why honest reaction is like that, if from reading PEP484 it's clear that it doesn't change status quo: https://www.python.org/dev/peps/pep-3107 added annotations long ago, and PEP484 just provides default But what concerned me as a Python core developer was the perception that as a core developer I would have to deal with type hints and type checking in the stdlib, because there was talk of including type hints for the stdlib (as stub files) in 3.6. *That* was the source of my concern, which is reduced by the statement that we'll only need to think about type checking in the stdlib once the tools are *clearly* adding value, in *our* (the stdlib module maintainers') opinion, so that we *want* to use the type hints. The discussion of stub files being maintained by interested parties in a separate repository is also helpful. Again, that means that wider adoption should mostly only happen if the developers see real benefit. I still dislike the idea of having to read type hints (I agree with whoever it was said a well written docstring is more helpful than abstract type hints when reading Python code), but perhaps I will get used to it if/when it shows up in libraries I use. I think it would serve your goal better if instead of dismissing concerns you were to strive to understand them and then figure out how to allay them. Overall, I suspect that those who are doubtful are put off by the rah rah this is great you are fools for not wanting to use it tone of (some) of the proponents (which makes us think this is going to get pushed on us willy-nilly), whereas hearing a consistent message that this will *enable* interested parties to collaborate on type checking without impacting the way the rest of you code *unless it proves its value to you* would be better received. Guido has done the latter, IMO, as have some others. --David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Mon, 20 Apr 2015 20:43:38 -0400 R. David Murray rdmur...@bitdance.com wrote: +1 to this from me too. I'm afraid that means I'm -1 on the PEP. I didn't write this in my earlier email because I wasn't sure about it, but my gut reaction after reading Harry's email was if type annotations are used in the stdlib, I'll probably stop contributing. That doesn't mean that's *true*, but that's the first time I've ever had that thought, so it is probably worth sharing. I think it would be nice to know what the PEP means for daily stdlib development. If patches have to carry typing information each time they add/enhance an API that's an addition burden. If typing is done separately by interested people then it sounds like it wouldn't have much of an impact on everyone else's workflow. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 21 April 2015 at 01:45, Chris Angelico ros...@gmail.com wrote: When you're writing a library, it can be a great help to provide type annotations, because every application that uses your library can benefit. It can be a great help to whom? Not to me (the library author), because I can't use them in my library code, because I have to support 2.7. That's by no means a bad thing (after all, most libraries are written to help others), but I found it really unclear who was being advantaged here. To show the downside, I decided to annotate requests' API a little bit. Then I stopped, because I really couldn't work out how to do it. For example, requests.get() takes a URL parameter. This can be an instance of basestring, or anything that provides a __str__ or __unicode__ method that provides us a string that looks like a URL (such as a URL abstraction class). We don't know ahead of time what those objects may be, so there's no type signature I can provide here that is of any use beyond 'Any'. We also provide headers/params options, whose type signature would be (as far as I can tell) Union[Iterable[Tuple[basestring, basestring]], Mapping[basestring, basestring]], which is a fairly unpleasant type (and also limiting, as we totally accept two-element lists here as well as two-tuples. That's got *nothing* on the type of the `files` argument, which is the most incredibly polymorphic argument I've ever seen: the best I can work out it would be: Optional[ Union[ Mapping[ basestring, Union[ Tuple[basestring, Optional[Union[basestring, file]]], Tuple[basestring, Optional[Union[basestring, file]], Optional[basestring]], Tuple[basestring, Optional[Union[basestring, file]], Optional[basestring], Optional[Headers]] ] ], Iterable[ Tuple[ basestring, Union[ Tuple[basestring, Optional[Union[basestring, file]]], Tuple[basestring, Optional[Union[basestring, file]], Optional[basestring]], Tuple[basestring, Optional[Union[basestring, file]], Optional[basestring], Optional[Headers]] ] ] ] ] Headers = Union[ Mapping[basestring, basestring], Iterable[Tuple[basestring, basestring]], ] This includes me factoring out the Headers type for my own sanity, and does not include the fact that in practice almost all instances of 'basestring' above actually mean 'anything that can be safely cast to basestring', and 'file' means 'any file-like object including BytesIO and StringIO' (speaking of which, the PEP doesn't even begin to go into how to handle that kind of requirement: do we have some kind of Readable interface that can be implemented? Or do I have to manually union together all types someone might want to use?). Now, this can be somewhat improved: the three types of tuple (again, not just tuples, often lists!) can be factored out, as can the Union[basestring, file]. However, there's still really no way around the fact that this is a seriously complex type signature! Python has had years of people writing APIs that feel natural, even if that means weird contortions around 'types'. Writing an argument as 'clever' as the 'files' argument in requests in a statically typed language is an absolute nightmare, but Python makes it an easy and natural thing to do. Further, Python's type system is not sufficiently flexible to allow library authors to adequately specify the types their code actually works on. I need to be able to talk about interfaces, because interfaces are the contract around which APIs are build in Python, but I cannot do that with this system in a way that makes any sense at all. To even begin to be useful for library authors this PEP would need to allow some kind of type hierarchy that is *not* defined by inheritance, but instead by interfaces. We've been discouraging use of 'type' and 'isinstance' for years because they break duck typing, but that has *nothing* on what this PEP appears to do to duck typing. I suppose the TLDR of this email is that I think that libraries with 'pythonic' APIs are the least likely to take up this typing system because it will provide the least value to them. Maintaining signatures will be a pain (stub files are necessary), writing clear signatures will be a pain (see above), and writing signatures that are both sufficiently open to be back-compatible and sufficiently restrictive to be able to catch bugs seems like it might be very nearly impossible. I'd love for someone to tell me I'm wrong, though. I want to like this PEP, I really do. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, 21 Apr 2015 22:47:23 +1000 Steven D'Aprano st...@pearwood.info wrote: Ironically, type hinting will *reduce* the need for intrusive, anti-duck-testing explicit calls to isinstance() at runtime: It won't, since as you pointed out yourself, type checks are purely optional and entirely separate from compilation and runtime evaluation. Why bother making that expensive isinstance call every single time the function is called, if the type checker can prove that x is always a float? Because the user might not run the type checker, obviously. To quote you: When we say that type checking is optional, we mean it. You can't at the same time point out that type checking has no power or control over runtime behaviour, and then claim that type checking makes runtime behaviour (for example, ability to accept or reject certain types) saner. It is a trivial contradiction. (and because of the former property, type hints can be entirely wrong - for example incomplete, or too strict, or too lax - without anyone noticing as long as they are self-consistent, since runtime behaviour is unaffected by them) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 11:56:15AM +0100, Rob Cliffe wrote: (Adding a type hint that restricted the argument to say a sequence of numbers turns out to be a mistake. Let's find out how big a mistake it is with an test run. py def sorter(alist: List[int]) - List[int]: ... return sorted(alist) ... py data = (chr(i) + 'ay' for i in range(97, 107)) py type(data) class 'generator' py sorter(data) ['aay', 'bay', 'cay', 'day', 'eay', 'fay', 'gay', 'hay', 'iay', 'jay'] When we say that type checking is optional, we mean it. [Disclaimer: I had to fake the List object, since I don't have the typing module, but everything else is exactly as you see it.] Annotations will be available for type checking. If you don't want to type check, don't type check. If you want to go against the type hints, you can go against the type hints, and get exactly the same runtime errors as you have now: py sorter(None) Traceback (most recent call last): File stdin, line 1, in module File stdin, line 2, in sorter TypeError: 'NoneType' object is not iterable There is no compile time checking unless you choose to run a type checker or linter -- and even if the type checker flags errors, you can ignore it and run the program regardless. Just like today with linters like PyFlakes, PyLint and similar. For those who choose not to run a type checker, the annotations will be nothing more than introspectable documentation. And what is a number? Is Fraction? What about complex numbers, which can't be sorted? What if the function were written before the Decimal class?) I know that you are intending these as rhetorical questions, but Python has had a proper numeric tower since version 2.5 or 2.6. So: py from numbers import Number py from decimal import Decimal py isinstance(Decimal(1.25), Number) True py isinstance(2+3j, Number) True Errors are often not caught until run time that would be caught at compile time in other languages (though static code checkers help). Yes they do help, which is exactly the point. (Not much of a disadvantage because of Python's superb error diagnostics.) That's certainly very optimistic of you. If I had to pick just one out of compile time type checking versus run time unit tests, I'd pick run time tests. But it is naive to deny the benefits of compile time checks in catching errors that you otherwise might not have found even with extensive unit tests (and lets face it, we never have enough unit tests). Ironically, type hinting will *reduce* the need for intrusive, anti-duck-testing explicit calls to isinstance() at runtime: def func(x:float): if isinstance(x, float): ... else: raise TypeError Why bother making that expensive isinstance call every single time the function is called, if the type checker can prove that x is always a float? Python code typically says what it is doing, with the minimum of syntactic guff. (Well, apart from colons after if/while/try etc. :-) ) Which makes it easy to read. Now it seems as if this proposal wants to start turning Python in the C++ direction, encouraging adding ugly boilerplate code. (This may only be tangentially relevant, but I want to scream when I see some combination of public/private/protected/static/extern etc., most of which I don't understand.) Perhaps if you understood it you would be less inclined to scream. Chris A makes the valid point (if I understand correctly) that Authors of libraries should make it as easy as possible to (i) know what object types can be passed to functions (ii) diagnose when the wrong type of object is passed Authors of apps are not under such obligation, they can basically do what they want. Well, (i) can be done with good documentation (docstrings etc.). (ii) can be done with appropriate runtime checks and good error messages. How ironic. After singing the praises of duck-typing, now you are recommending runtime type checks. As far as good error messages go, they don't help you one bit when the application suddenly falls over in a totally unexpected place due to a bug in your code. I can't go into too many details due to commercial confidentiality, but we experienced something similar recently. A situation nobody foresaw, that wasn't guarded against, and wasn't tested for, came up after deployment. There was a traceback, of course, but a failure in the field 200km away with a stressed customer and hundreds of angry users is not as useful as a compile-time failure during development. You see where I'm going with this - adding type hints to Python feels a bit like painting feet on the snake. Pythons are one of the few snakes which have vestigal legs: http://en.wikipedia.org/wiki/Pelvic_spur -- Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 21 April 2015 at 12:23, Gustavo Carneiro gjcarne...@gmail.com wrote: Documentation is not checked. It often loses sync with the actual code. Docs say one thing, code does another. Agreed. I don't think anyone would disagree here. I'm talking from the position of being a library author, where supporting versions of Python lower than 3.5 will be a reality for at least 5 more years. I will not be able to inline my type hints, so they'll have to go in stub files, and now we've got the same problem: type hints can go out of step just as easily as documentation can. Documenting types in docstrings forces you to repeat yourself. You have to write the parameter names twice, once in the function definition line, another in the docstring. And you need to understand the syntax of whatever docstring format will be used to check the code. Yes, but I'm repeating myself O(small integer) lines away from where I first wrote it. The maintenance burden of keeping those things in step is not very high. As for your last point, my code is not checked against my docstrings, primarily for the reasons I mentioned in my original email. My docstrings are merely rendered into documentation. I'm sure I'm not the only one that thinks that type hints actually *improve readability*. In some cases, sure. No contest. but I am certain that most of the time the type hints make the code easier to read because you don't have to spend so much mental effort trying to figure out gee, I wonder if this parameter is supposed to be bool, string, or int? Yes. If you spend a lot of your time doing this, then type hints will help you, *assuming* the programmer who originally wrote the ambiguous code diligently maintains the type hints. I have no reason to believe that a programmer whose code is that ambiguous is going to do that, or even write type hints at all. The bigger problem is that, previously, Python functions never took bool, string, or int. They took 'object that can be evaluated in a boolean context', 'object that behaves like strings in some undefined way', and 'objects that behave like ints in some undefined way'. The type hint you provide is *not helpful*. What I *need* is a type hint that says 'object that has the .endswith() method' or 'object that can be safely cast to a string', so I can tell whether I can pass you a custom URL class that happens to render into a textual URL or whether it definitely has to be a string object. But you can't express that relationship in this model, so you will just say 'string', and now I will turn off type checking because the type hinter keeps shouting at me for using my URLBuilder object. If you only work with primitive types and only want your users to use primitive types, this proposal almost certainly works brilliantly. If you *don't*, this proposal begins to become extremely uncomfortable extremely fast. Even if it only helps in the simplest cases, it saves the reader of the code a lot of effort if you add up all those little cases together. My suggestion, wherever an annotated type is too complex, don't annotate it. If you do /only/ the simple cases, the end result is easier to read. If that's genuinely the bar for success for this PEP, then fine, I wish it the best. I'd *like* a PEP that can deal with the complex cases too, but I freely admit to having no idea how such a thing may be implemented. All I can say is that I think this PEP will have limited uptake amongst many third-party libraries, particularly the complex ones. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 01:25:34PM +0100, Chris Withers wrote: Anyway, I've not posted much to python-dev in quite a while, but this is a topic that I would be kicking myself in 5-10 years time when I've had to move to Javascript or insert new language here because everyone else has drifted away from Python as it had become ugly... Facebook released Flow, a static typechecker for Javascript, to a very positive reaction. From their announcement: Flow’s type checking is opt-in — you do not need to type check all your code at once. However, underlying the design of Flow is the assumption that most JavaScript code is implicitly statically typed; even though types may not appear anywhere in the code, they are in the developer’s mind as a way to reason about the correctness of the code. Flow infers those types automatically wherever possible, which means that it can find type errors without needing any changes to the code at all. On the other hand, some JavaScript code, especially frameworks, make heavy use of reflection that is often hard to reason about statically. For such inherently dynamic code, type checking would be too imprecise, so Flow provides a simple way to explicitly trust such code and move on. This design is validated by our huge JavaScript codebase at Facebook: Most of our code falls in the implicitly statically typed category, where developers can check their code for type errors without having to explicitly annotate that code with types. Quoted here: http://blog.jooq.org/2014/12/11/the-inconvenient-truth-about-dynamic-vs-static-typing/ More about flow: http://flowtype.org/ Matz is interested in the same sort of gradual type checking for Ruby as Guido wants to add to Python: https://www.omniref.com/blog/blog/2014/11/17/matz-at-rubyconf-2014-will-ruby-3-dot-0-be-statically-typed/ Julia already includes this sort of hybrid dynamic+static type checking: http://julia.readthedocs.org/en/latest/manual/types/ I could keep going, but I hope I've made my point. Whatever language you are using in 5-10 years time, it will almost certainly be either mostly static with some dynamic features like Java, or dynamic with optional and gradual typing. -- Steven ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 21 April 2015 at 13:47, Steven D'Aprano st...@pearwood.info wrote: On Tue, Apr 21, 2015 at 11:56:15AM +0100, Rob Cliffe wrote: (Adding a type hint that restricted the argument to say a sequence of numbers turns out to be a mistake. Let's find out how big a mistake it is with an test run. py def sorter(alist: List[int]) - List[int]: ... return sorted(alist) ... py data = (chr(i) + 'ay' for i in range(97, 107)) py type(data) class 'generator' py sorter(data) ['aay', 'bay', 'cay', 'day', 'eay', 'fay', 'gay', 'hay', 'iay', 'jay'] When we say that type checking is optional, we mean it. That's a very good point - although I have to say that I sort of feel like I'm cheating doing that. It doesn't matter how many times you say optional, it does still feel like it's wrong, in basically the same way as using undocumented methods on a class, etc. Conceded, type checking is optional. But that message seems to be failing to get through. It's not a technical thing, it's a PR issue. Which is *very* close to meaning that the problem is FUD, but less in the sense of people deliberately spreading FUD as people being worried, not understanding the proposal, and not being sure how type hints will work in practice. I'm reasonably willing to just accept that the PEP will be getting implemented, and to wait and see how things turn out. But my fear, uncertainty and doubt remain. I'll try not to spread them, though. You see where I'm going with this - adding type hints to Python feels a bit like painting feet on the snake. Pythons are one of the few snakes which have vestigal legs: http://en.wikipedia.org/wiki/Pelvic_spur LOL, instant win :-) Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 21 April 2015 at 10:10, Chris Angelico ros...@gmail.com wrote: At this point, you may want to just stop caring about the exact type. Part of the point of gradual typing is that you can short-cut a lot of this. And quite frankly, this isn't really helping anything. Just skip it and say that it's Union[Mapping, Iterable, None]. I think this is my problem with the proposal. This change doesn't catch any of the bugs anyone was going to hit (no-one is passing a callable to this parameter), and it doesn't catch any of the bugs someone might actually hit (getting the tuple elements in the wrong order, for example). At this point the type signature is worse than useless. It seems like the only place the type annotations will get used is in relatively trivial cases where the types are obvious anyway. I don't deny that *some* bugs will be caught, but I suspect they'll overwhelmingly be crass ones that would have been caught quickly anyway. The minute a type signature might actually help prevent a subtle bug we can't use them anymore because the toolchain isn't powerful enough to do it so we just put 'Any' and call it a day. And even then we haven't helped the trivial case much because anyone who wants to provide a duck-typed object that should be perfectly suitable in this case no longer can. What this proposal *needs* to be generally useful, IMO, is the ability to specify types that actually match the way we have encouraged people to write code: duck typed, favouring composition over inheritance, etc. At the very least that means 'type constraints' (see Haskell, Rust, really any language with a powerful and robust type system: hell, even see Go's interfaces) need to be something we can specify. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 6:58 PM, Cory Benfield c...@lukasa.co.uk wrote: On 21 April 2015 at 01:45, Chris Angelico ros...@gmail.com wrote: When you're writing a library, it can be a great help to provide type annotations, because every application that uses your library can benefit. It can be a great help to whom? Not to me (the library author), because I can't use them in my library code, because I have to support 2.7. That's by no means a bad thing (after all, most libraries are written to help others), but I found it really unclear who was being advantaged here. Mainly application users get the benefit, I expect. But I may be wrong. ... That's got *nothing* on the type of the `files` argument, which is the most incredibly polymorphic argument I've ever seen: the best I can work out it would be: Optional[ Union[ Mapping[ basestring, Union[ Tuple[basestring, Optional[Union[basestring, file]]], Tuple[basestring, Optional[Union[basestring, file]], Optional[basestring]], Tuple[basestring, Optional[Union[basestring, file]], Optional[basestring], Optional[Headers]] ] ], Iterable[ Tuple[ basestring, Union[ Tuple[basestring, Optional[Union[basestring, file]]], Tuple[basestring, Optional[Union[basestring, file]], Optional[basestring]], Tuple[basestring, Optional[Union[basestring, file]], Optional[basestring], Optional[Headers]] ] ] ] ] At this point, you may want to just stop caring about the exact type. Part of the point of gradual typing is that you can short-cut a lot of this. And quite frankly, this isn't really helping anything. Just skip it and say that it's Union[Mapping, Iterable, None]. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 21 April 2015 at 15:31, Chris Angelico ros...@gmail.com wrote: Granted, there are some vague areas - how many functions take a file-like object, and are they all the same? - but between MyPy types and the abstract base types that already exist, there are plenty of ways to formalize duck typing. Are there? Can I have a link or an example, please? I feel like I don't know how I'm supposed to do this, and I'd like to see how that works. I'll even give a concrete use-case: I want to be able to take a file-like object that has a .read() method and a .seek() method. And frankly, even with the uncertainties, I'd still rather have a function declared as taking a file-like object than an object with a .read() method that takes an integer and returns up to that many bytes of data, and a .seek() method that blah blah blah blah. Sometimes, the precision is completely useless. It is completely useless. Happily, this is a strawman, and no-one was asking for it, so we can all live happily ever after! The correct specification is read method with this type signature and seek method with this type signature. I would even be prepared to waive the type signatures on read and seek, given that enforcing the type hinting on others is not a good idea. I suspect I have a mismatch with several others in this discussion. My position is that if I'm going to have a type system, I'd like to have a powerful one: Haskell, not Java. Otherwise I'll get by with the duck typing that has worked just fine for us over the last 20 years. I suspect, however, that many others in this conversation want any type system at all, so long as they can have one. Is that an accurate characterisation of your position, Chris? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Apr 20 2015, Chris Angelico ros...@gmail.com wrote: Maybe it'd be of value to have a quick code stripper that takes away all the annotations, plus any other junk/framing that you're not interested in, and gives you something you can browse in a text editor? If you need to preprocess your source code to make it suitable for human consumption something is very wrong with your language design. I can't believe you're seriously suggesting this. Best, -Nikolaus -- GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F »Time flies like an arrow, fruit flies like a Banana.« ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 20/04/2015 20:09, Paul Moore wrote: On 20 April 2015 at 19:41, Barry Warsaw ba...@python.org wrote: tldr; type hints in python source are scary. Would reserving them for stub files be better? I think so. I think PEP 8 should require stub files for stdlib modules and strongly encourage them for 3rd party code. Agreed. I have many of the same concerns as Harry, but I wouldn't have expressed them quite as well. I'm not too worried about actually removing annotations from the core language, but I agree that we should create a strong culture of type hints go in stub files to keep source files readable and clean. On that note, I'm not sure stub files is a particularly good name. Maybe type files would be better? Something that emphasises that they are the correct place to put type hints, not a workaround. Or we could just be honest and admit that we're choosing to add header files to Python. It's a shame, as it's more complexity, and it's being inflicted on those who might be writing a library for the first time, or those becoming core committers for the first time, or those just trying to do the right thing. Currently, the burden is a heavier one (type inference, rather than reading it from a file) but borne by people best placed to handle it (authors of IDEs, type checking software, etc). Chris ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Le 21/04/2015 15:50, Paul Sokolovsky a écrit : Hello, On Tue, 21 Apr 2015 15:08:27 +0200 Antoine Pitrou solip...@pitrou.net wrote: [] Because the user might not run the type checker, obviously. To quote you: When we say that type checking is optional, we mean it. You can't at the same time point out that type checking has no power or control over runtime behaviour, and then claim that type checking makes runtime behaviour (for example, ability to accept or reject certain types) saner. It is a trivial contradiction. I suspected there's either short-sightedness, or just a word play for for a purpose of word play. Type annotation are NOT introduced for the purpose of static type checking. I was replying to Steven's message. Did you read it? Runtime type checking (run as via make test, not as production) is much more interesting to catch errors. Obviously you're giving the word runtime a different meaning than I do. The type checker isn't supposed to actually execute the user's functions (it's not that it's forbidden, simply that it's not how it will work in all likelihood): therefore, it doesn't have any handle on what *actually* happens at runtime, vs. what is declared in the typing declarations. But if by runtime you mean any action that happens *out-of-band* during development (such as running pip or a syntax-checking linter), then sure, fine :-) Even more interesting usage is to allow ahead-of-time, and thus unbloated, optimization. There're bunch of JITters and AOTters for Python language, each of which uses own syntax (via decorators, etc.) to annotate functions. As a developer of one of those tools, I've already said that I find it unlikely for the PEP to be useful for that purpose. The issue is that the vocabulary created in the PEP is not extensible enough. Note I'm not saying it's impossible. I'm just skeptical that in its current form it will help us. And apparently none of our competitors seems very enthusiastic either (feel free to prove me wrong: I might have missed something :-)). Having language-specified type annotations allows for portable syntax for such optimized code. Only if the annotations allow expressing the subtleties required by the specific optimizer. For example, Float is too vague for Numba: we would like to know if that is meant to be a single- or double-precision float. Don't block the language if you're stuck with an unimaginative implementation, there's much more to Python than that. The Python language doesn't really have anything to do with that. It's just an additional library with a set of conventions. Which is also why a PEP wouldn't be required to make it alive, it's just there to make it an official standard. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Wed, Apr 22, 2015 at 12:51 AM, Cory Benfield c...@lukasa.co.uk wrote: On 21 April 2015 at 15:31, Chris Angelico ros...@gmail.com wrote: Granted, there are some vague areas - how many functions take a file-like object, and are they all the same? - but between MyPy types and the abstract base types that already exist, there are plenty of ways to formalize duck typing. Are there? Can I have a link or an example, please? I feel like I don't know how I'm supposed to do this, and I'd like to see how that works. I'll even give a concrete use-case: I want to be able to take a file-like object that has a .read() method and a .seek() method. Someone who's been more involved in the details of MyPy can probably say more specifically what ought to be done about file-like objects. And frankly, even with the uncertainties, I'd still rather have a function declared as taking a file-like object than an object with a .read() method that takes an integer and returns up to that many bytes of data, and a .seek() method that blah blah blah blah. Sometimes, the precision is completely useless. It is completely useless. Happily, this is a strawman, and no-one was asking for it, so we can all live happily ever after! The correct specification is read method with this type signature and seek method with this type signature. I would even be prepared to waive the type signatures on read and seek, given that enforcing the type hinting on others is not a good idea. I suspect I have a mismatch with several others in this discussion. My position is that if I'm going to have a type system, I'd like to have a powerful one: Haskell, not Java. Otherwise I'll get by with the duck typing that has worked just fine for us over the last 20 years. I suspect, however, that many others in this conversation want any type system at all, so long as they can have one. Is that an accurate characterisation of your position, Chris? Pretty accurate, yeah. Here's how I see it: def incremental_parser(input: FileLike) - List[Token]: tokens = [] data = while True: if not data: data = input.read(64) token = Token(data[0]); data = data[1:] while token.wants_more(): token.give_more(data[0]); data = data[1:] tokens.append(token) if token.is_end_of_stream(): break input.seek(-len(data), 1) return tokens If you were to exhaustively stipulate the requirements on the file-like object, you'd have to say: * Provides a read() method which takes an integer and returns up to that many bytes * Provides a seek() method which takes two integers * Is capable of relative seeking by at least 63 bytes backward * Is open for reading * Etcetera That's not the type system's job. Not in Python. Maybe in Haskell, but not in Python. So how much _should_ go into the type hint? I'm happy with FileLike or however it's to be spelled; maybe separate readable files from writable ones, as those are two fairly clear variants, but that's about all you really need. If you provide incremental_parser() with an input file that's non-seekable, it's going to have problems - and your editor may or may not even be able to detect that (some files are seekable but only in the forward direction, but they'll have the exact same seek() method). I might be wrong about where MyPy is trying to go with this, but doubtless someone will correct me on any inaccuracies above. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Hey, I just wanted to say to everyone, thanks for being so patient and willing to engage with this discussion, despite my not having done my research and read the (substantial) prior discussion on the topic. Here it is (or at least, some of it!) for any other newcomers: https://mail.python.org/pipermail/python-list/2015-January/697202.html https://mail.python.org/pipermail/python-list/2015-January/697315.html https://github.com/ambv/typehinting/issues/55 I'm not sure any of it will radically change anyone's mind. Everyone (I think) agrees that type hints carry some cost in terms of legibility and simplicity of the language. Everyone agrees they have some benefits. The disagreement remains over whether it will be worth it, and whether stub files vs inline vs something-else would be the best place to hide them. Is let's wait and see an acceptable position? Probably the only real way to resolve the disagreement is to start seeing the type hints in use, and form opinions based on real-world practice. Some people will like them inline. Lots of people will use stub files, since they're the only way to maintain 2+3 compatibility, so it's my home that the community will standardise around that. Still other people will want to have another crack at using docstrings, but the standardisation on notation of types will still help them. And still other others will want no part in them, and will argue against them in their communities. Hopefully a best practice will evolve. And hopefully it'll be the stub file one, and then we really can deprecate function annotations in 3.6 ;-) On 21 April 2015 at 11:56, Rob Cliffe rob.cli...@btinternet.com wrote: On 21/04/2015 10:33, Cory Benfield wrote: On 21 April 2015 at 10:10, Chris Angelico ros...@gmail.com ros...@gmail.com wrote: At this point, you may want to just stop caring about the exact type. Part of the point of gradual typing is that you can short-cut a lot of this. And quite frankly, this isn't really helping anything. Just skip it and say that it's Union[Mapping, Iterable, None]. I think this is my problem with the proposal. This change doesn't catch any of the bugs anyone was going to hit (no-one is passing a callable to this parameter), and it doesn't catch any of the bugs someone might actually hit (getting the tuple elements in the wrong order, for example). At this point the type signature is worse than useless. Exactly. At this point putting the type signatures in seems to be a pointless exercise in masochism (for the author) and sadism (for the reader). The refreshing thing about Python is that it is a fantastic, *concise*, dynamically typed language, at the opposite end of the spectrum from C++ (ugh!). We have got used to the consequences (good and bad) of this: Duck typing, e.g. a function to sort numbers (sorry Tim Peters bad example) turns out to support any kind of object (e.g. strings) that supports comparison. Not to mention objects of some class that will be written in 5 years time. (Adding a type hint that restricted the argument to say a sequence of numbers turns out to be a mistake. And what is a number? Is Fraction? What about complex numbers, which can't be sorted? What if the function were written before the Decimal class?) Errors are often not caught until run time that would be caught at compile time in other languages (though static code checkers help). (Not much of a disadvantage because of Python's superb error diagnostics.) Python code typically says what it is doing, with the minimum of syntactic guff. (Well, apart from colons after if/while/try etc. :-) ) Which makes it easy to read. Now it seems as if this proposal wants to start turning Python in the C++ direction, encouraging adding ugly boilerplate code. (This may only be tangentially relevant, but I want to scream when I see some combination of public/private/protected/static/extern etc., most of which I don't understand.) Chris A makes the valid point (if I understand correctly) that Authors of libraries should make it as easy as possible to (i) know what object types can be passed to functions (ii) diagnose when the wrong type of object is passed Authors of apps are not under such obligation, they can basically do what they want. Well, (i) can be done with good documentation (docstrings etc.). (ii) can be done with appropriate runtime checks and good error messages. E.g. (Cory's example) I'm sure it is possible to test somehow if an object is file-like (if only by trying to access it like a file). Is thorough argument checking and provision of good diagnostics going to be easy for the library author? No. Is it going to be a lot of work to do thoroughly? Almost certainly yes. But what the hell, writing a production-quality library is not an exercise for newbies. It seems to me that type hints are
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 20/04/2015 19:30, Harry Percival wrote: Hi all, tldr; type hints in python source are scary. Would reserving them for stub files be better? I was trying to find Jack's original post as I think his summary is excellent and aligns well with where I think I'm coming from on this: https://mail.python.org/pipermail/python-dev/2015-April/139253.html However, Harry makes just as many good points so I'll reply here, with a note that while switching to (type/header/stub - my ordered preference for how to describe .pyi files) is preferable to type hints inside files, it's still a massive change to the language, not one I can say I've missed over the past 15 years, and one if asked to vote on (I know that's not the case ;-)) that I would choose to vote against. Anyway, I've not posted much to python-dev in quite a while, but this is a topic that I would be kicking myself in 5-10 years time when I've had to move to Javascript or insert new language here because everyone else has drifted away from Python as it had become ugly... Chris ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 03:08:27PM +0200, Antoine Pitrou wrote: On Tue, 21 Apr 2015 22:47:23 +1000 Steven D'Aprano st...@pearwood.info wrote: Ironically, type hinting will *reduce* the need for intrusive, anti-duck-testing explicit calls to isinstance() at runtime: It won't, since as you pointed out yourself, type checks are purely optional and entirely separate from compilation and runtime evaluation. Perhaps you are thinking of libraries, where the library function has to deal with whatever junk people throw at it. To such libraries, I believe that the major benefit of type hints is not so much in proving the library's correctness in the face of random arguments, but as documentation. In any case, of course you are correct that public library functions and methods will continue to need to check their arguments. (Private functions, perhaps not.) But for applications, the situation is different. If my application talks to a database and extracts a string which it passes on to its own function spam(), then it will be a string. Not a string-like object. Not something that quacks like a string. A string. Once the type checker is satisfied that spam() always receives a string, then further isinstance checks inside spam() is a waste of time. If spam()'s caller changes and might return something which is not a string, then the type checker will flag that. Obviously to get this benefit you need to actually use a type checker. I didn't think I needed to mention that. -- Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
So. This is how you try and get me to care about Python 3. Can't speak for others, but this does the opposite for me. This makes me ecstatic that Python 2 has a nearly-frozen api. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Hello, On Tue, 21 Apr 2015 11:56:15 +0100 Rob Cliffe rob.cli...@btinternet.com wrote: On 21/04/2015 10:33, Cory Benfield wrote: On 21 April 2015 at 10:10, Chris Angelico ros...@gmail.com wrote: At this point, you may want to just stop caring about the exact type. Part of the point of gradual typing is that you can short-cut a lot of this. And quite frankly, this isn't really helping anything. Just skip it and say that it's Union[Mapping, Iterable, None]. I think this is my problem with the proposal. This change doesn't catch any of the bugs anyone was going to hit (no-one is passing a callable to this parameter), and it doesn't catch any of the bugs someone might actually hit (getting the tuple elements in the wrong order, for example). At this point the type signature is worse than useless. Exactly. At this point putting the type signatures in seems to be a pointless exercise in masochism (for the author) and sadism (for the reader). Or interesting and productive exercise, which will lead to further improvements and benefits - for other kinds of authors and readers. Those kinds left your kind alone by letting to keep not writing annotations. Why you feel so embarrassed to let the other kind progress? Python becomes multi-paradigm language, what's so bad with that? You're afraid that you will have discomfort reading other's code? My god, I love Python, and hate 50% of code written in it, most of it needs to be rewritten to be satisfactory! Why it bothers you that there will be direction on how other people may (re)write it? The refreshing thing about Python is that it is a fantastic, *concise*, dynamically typed language, at the opposite end of the spectrum from C++ (ugh!). C++ is fantastic, concise, statically typed language. You've just used to see 90% of code in it which needs to be rewritten. [] Apologies, Guido, but: [] You see where I'm going with this - adding type hints to Python feels a bit like painting feet on the snake. Or at least turning it into a different language. I'm sure type annotations never could be discussed for inclusion if Guido didn't feel that they fit with Python's way. Sometimes it makes sense to trust BDFL. It's much less obtrusive and qustionable change than forcing Unicode by default, making print a function, or keeping 3-headed sys.exc_info() ugliness instead throwing it out of Python3. Because well, type annotation *are optional*. And it's by now clear that there will be --strip-type-annotations flag to pip and stickers This system runs type annotations free. Best wishes Rob Cliffe -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Hello, On Tue, 21 Apr 2015 15:08:27 +0200 Antoine Pitrou solip...@pitrou.net wrote: [] Because the user might not run the type checker, obviously. To quote you: When we say that type checking is optional, we mean it. You can't at the same time point out that type checking has no power or control over runtime behaviour, and then claim that type checking makes runtime behaviour (for example, ability to accept or reject certain types) saner. It is a trivial contradiction. I suspected there's either short-sightedness, or just a word play for for a purpose of word play. Type annotation are NOT introduced for the purpose of static type checking. The current PEP just gives an example of their usage for static type checking as an example backed by the immediate availability of a tool which does that, MyPy. But granted, static type checking is the most boring of possible usages for type annotation. Runtime type checking (run as via make test, not as production) is much more interesting to catch errors. The tooling for that is yet to be written though. Even more interesting usage is to allow ahead-of-time, and thus unbloated, optimization. There're bunch of JITters and AOTters for Python language, each of which uses own syntax (via decorators, etc.) to annotate functions. Having language-specified type annotations allows for portable syntax for such optimized code. Granted, the most juicy usages described above won't be available in CPython out of the box. But there should be that motto: CPython is the most boring of all Pythons. Don't block the language if you're stuck with an unimaginative implementation, there's much more to Python than that. -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 7:56 PM, Arnaud Delobelle arno...@gmail.com wrote: If people constantly get told by their editor / IDE that they are calling function with the wrong argument types, what are they going to do? They may start adopting the same approach as in Java / C++ etc... where interfaces must be explicitly defined and the practice of duck typing may become forgotten because discouraged by the tools programmers use. Style guides should generally be recommending the use of concrete types for return values, but abstract types for parameters - you might well have a function that returns a list, but most functions that accept lists are really accepting sequences. Granted, there are some vague areas - how many functions take a file-like object, and are they all the same? - but between MyPy types and the abstract base types that already exist, there are plenty of ways to formalize duck typing. And frankly, even with the uncertainties, I'd still rather have a function declared as taking a file-like object than an object with a .read() method that takes an integer and returns up to that many bytes of data, and a .seek() method that blah blah blah blah. Sometimes, the precision is completely useless. What an editor would do with the hint that a function takes a file-like object I'm not sure, but if it issues a complaint because I'm passing it something that doesn't have a writelines() method, it's the fault of the editor, not the type hint. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, 21 Apr 2015 23:16:19 +1000 Steven D'Aprano st...@pearwood.info wrote: I could keep going, but I hope I've made my point. I don't think so. Just because other languages are looking at it doesn't mean it will end up successful. It means it's an interesting idea, that's all. A litmus test for this PEP would be to apply it to a sizable code base and evaluate the results. Looking at isolated function examples doesn't tell us how it will scale (and I'm obviously not talking about performance, but the ability to still give useful diagnostics in the presence of typing holes - i.e. untyped functions - or imprecisions, when factored in a complex graph of call dependencies). Whatever language you are using in 5-10 years time, it will almost certainly be either mostly static with some dynamic features like Java, or dynamic with optional and gradual typing. Anybody can make predictions. As a data point, 6 years ago people were predicting that the average desktop CPU would have 16 cores nowadays... (we don't hear much from them anymore :-)) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, 21 Apr 2015 at 09:59 Cory Benfield c...@lukasa.co.uk wrote: [...] Further, Python's type system is not sufficiently flexible to allow library authors to adequately specify the types their code actually works on. I need to be able to talk about interfaces, because interfaces are the contract around which APIs are build in Python, but I cannot do that with this system in a way that makes any sense at all. To even begin to be useful for library authors this PEP would need to allow some kind of type hierarchy that is *not* defined by inheritance, but instead by interfaces. We've been discouraging use of 'type' and 'isinstance' for years because they break duck typing, but that has *nothing* on what this PEP appears to do to duck typing. This is why I feel like this PEP may be a real threat to duck typing. If people constantly get told by their editor / IDE that they are calling function with the wrong argument types, what are they going to do? They may start adopting the same approach as in Java / C++ etc... where interfaces must be explicitly defined and the practice of duck typing may become forgotten because discouraged by the tools programmers use. I guess what I'm saying is that this could encourage a very significant cultural change in the way Python code is written towards a less flexible mindset. -- Arnaud ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Apr 21, 2015, at 01:34 PM, Steven D'Aprano wrote: Putting the type information in a stub file is an exponentially more distant fourth best, or to put it another way, *the worst* solution for where to put type hints. Not only do you Repeat Yourself with the name of the parameter, but also the name of the function (or method and class) AND module. The type information *isn't even in the same file*, which increases the chance of it being lost, forgotten, deleted, out of date, unmaintained, etc. All true, but the trade-off is the agility and ease of working on, reading, and understanding the stdlib, all of which IMHO will suffer if type hints are inlined there. What I don't want to have happen is for type hints to slowly infiltrate the stdlib to the point where no patch will be accepted unless it also has hints. I have the same gut reaction to this as RDM expressed a few posts back. One of the thing I love most about Python is its dynamic typing. I'm all for giving linter developers a hook for experimenting with their tools, I just don't care and I don't want to *have* to care. Maybe some day they will make it so compelling that I will care, but I want to be convinced first. So I think stub files in the stdlib are the right compromise today. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 20/04/2015 19:30, Harry Percival wrote: Hi all, tldr; type hints in python source are scary. Would reserving them for stub files be better? I think Jack's summary of this is excellent and aligns well with where I think I'm coming from on this: https://mail.python.org/pipermail/python-dev/2015-April/139253.html Harry Also makes just as many good points so I'll reply here, with a note that while switching to (type/header/stub - my ordered preference for how to describe .pyi files) is preferable to type hints inside files, it's still a massive change to the language, not one I can say I've missed over the past 15 years, and one if asked to vote on (I know that's not the case ;-)) that I would choose to vote against. Anyway, I've not posted much to python-dev in quite a while, but this is a topic that I would be kicking myself in 5-10 years time when I've had to move to Javascript or insert new language here because everyone else has drifted away from Python as it had become ugly... Chris ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 21/04/2015 10:33, Cory Benfield wrote: On 21 April 2015 at 10:10, Chris Angelico ros...@gmail.com wrote: At this point, you may want to just stop caring about the exact type. Part of the point of gradual typing is that you can short-cut a lot of this. And quite frankly, this isn't really helping anything. Just skip it and say that it's Union[Mapping, Iterable, None]. I think this is my problem with the proposal. This change doesn't catch any of the bugs anyone was going to hit (no-one is passing a callable to this parameter), and it doesn't catch any of the bugs someone might actually hit (getting the tuple elements in the wrong order, for example). At this point the type signature is worse than useless. Exactly. At this point putting the type signatures in seems to be a pointless exercise in masochism (for the author) and sadism (for the reader). The refreshing thing about Python is that it is a fantastic, *concise*, dynamically typed language, at the opposite end of the spectrum from C++ (ugh!). We have got used to the consequences (good and bad) of this: Duck typing, e.g. a function to sort numbers (sorry Tim Peters bad example) turns out to support any kind of object (e.g. strings) that supports comparison. Not to mention objects of some class that will be written in 5 years time. (Adding a type hint that restricted the argument to say a sequence of numbers turns out to be a mistake. And what is a number? Is Fraction? What about complex numbers, which can't be sorted? What if the function were written before the Decimal class?) Errors are often not caught until run time that would be caught at compile time in other languages (though static code checkers help). (Not much of a disadvantage because of Python's superb error diagnostics.) Python code typically says what it is doing, with the minimum of syntactic guff. (Well, apart from colons after if/while/try etc. :-) ) Which makes it easy to read. Now it seems as if this proposal wants to start turning Python in the C++ direction, encouraging adding ugly boilerplate code. (This may only be tangentially relevant, but I want to scream when I see some combination of public/private/protected/static/extern etc., most of which I don't understand.) Chris A makes the valid point (if I understand correctly) that Authors of libraries should make it as easy as possible to (i) know what object types can be passed to functions (ii) diagnose when the wrong type of object is passed Authors of apps are not under such obligation, they can basically do what they want. Well, (i) can be done with good documentation (docstrings etc.). (ii) can be done with appropriate runtime checks and good error messages. E.g. (Cory's example) I'm sure it is possible to test somehow if an object is file-like (if only by trying to access it like a file). Is thorough argument checking and provision of good diagnostics going to be easy for the library author? No. Is it going to be a lot of work to do thoroughly? Almost certainly yes. But what the hell, writing a production-quality library is not an exercise for newbies. It seems to me that type hints are attempting to be a silver bullet and to capture in a simple formula what is often, in practice, *not simple at all*, viz. Is this passed object suitable?. Attempting - and failing, except in the simplest cases. Apologies, Guido, but: There was once a Chinese student who was a marvellous painter. He painted a perfect life-like picture of a snake. But he was unable to stop and leave it alone. In his zeal he went on to paint feet on the snake. Which of course completely spoiled the picture, as snakes don't have feet. Hence to paint feet on the snake: to be unable to resist tinkering with something that is already good. (I suppose If it ain't broke, don't fix it is an approximate Western equivalent.) You see where I'm going with this - adding type hints to Python feels a bit like painting feet on the snake. Or at least turning it into a different language. Best wishes Rob Cliffe ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 21 April 2015 at 11:56, Rob Cliffe rob.cli...@btinternet.com wrote: On 21/04/2015 10:33, Cory Benfield wrote: On 21 April 2015 at 10:10, Chris Angelico ros...@gmail.com ros...@gmail.com wrote: At this point, you may want to just stop caring about the exact type. Part of the point of gradual typing is that you can short-cut a lot of this. And quite frankly, this isn't really helping anything. Just skip it and say that it's Union[Mapping, Iterable, None]. I think this is my problem with the proposal. This change doesn't catch any of the bugs anyone was going to hit (no-one is passing a callable to this parameter), and it doesn't catch any of the bugs someone might actually hit (getting the tuple elements in the wrong order, for example). At this point the type signature is worse than useless. Exactly. At this point putting the type signatures in seems to be a pointless exercise in masochism (for the author) and sadism (for the reader). The refreshing thing about Python is that it is a fantastic, *concise*, dynamically typed language, at the opposite end of the spectrum from C++ (ugh!). We have got used to the consequences (good and bad) of this: Duck typing, e.g. a function to sort numbers (sorry Tim Peters bad example) turns out to support any kind of object (e.g. strings) that supports comparison. Not to mention objects of some class that will be written in 5 years time. (Adding a type hint that restricted the argument to say a sequence of numbers turns out to be a mistake. And what is a number? Is Fraction? What about complex numbers, which can't be sorted? What if the function were written before the Decimal class?) Errors are often not caught until run time that would be caught at compile time in other languages (though static code checkers help). (Not much of a disadvantage because of Python's superb error diagnostics.) Python code typically says what it is doing, with the minimum of syntactic guff. (Well, apart from colons after if/while/try etc. :-) ) Which makes it easy to read. Now it seems as if this proposal wants to start turning Python in the C++ direction, encouraging adding ugly boilerplate code. (This may only be tangentially relevant, but I want to scream when I see some combination of public/private/protected/static/extern etc., most of which I don't understand.) Chris A makes the valid point (if I understand correctly) that Authors of libraries should make it as easy as possible to (i) know what object types can be passed to functions (ii) diagnose when the wrong type of object is passed Authors of apps are not under such obligation, they can basically do what they want. Well, (i) can be done with good documentation (docstrings etc.). Documentation is not checked. It often loses sync with the actual code. Docs say one thing, code does another. Documenting types in docstrings forces you to repeat yourself. You have to write the parameter names twice, once in the function definition line, another in the docstring. And you need to understand the syntax of whatever docstring format will be used to check the code. I'm sure I'm not the only one that thinks that type hints actually *improve readability*. I will not argue whether it makes the code uglier or prettier, but I am certain that most of the time the type hints make the code easier to read because you don't have to spend so much mental effort trying to figure out gee, I wonder if this parameter is supposed to be bool, string, or int? The type (or types) that a parameter is expected to have becomes explicit, otherwise you have to read the entire function body to understand. (ii) can be done with appropriate runtime checks and good error messages. E.g. (Cory's example) I'm sure it is possible to test somehow if an object is file-like (if only by trying to access it like a file). Is thorough argument checking and provision of good diagnostics going to be easy for the library author? No. Is it going to be a lot of work to do thoroughly? Almost certainly yes. But what the hell, writing a production-quality library is not an exercise for newbies. It seems to me that type hints are attempting to be a silver bullet and to capture in a simple formula what is often, in practice, *not simple at all*, viz. Is this passed object suitable?. Attempting - and failing, except in the simplest cases. Even if it only helps in the simplest cases, it saves the reader of the code a lot of effort if you add up all those little cases together. My suggestion, wherever an annotated type is too complex, don't annotate it. If you do /only/ the simple cases, the end result is easier to read. Apologies, Guido, but: There was once a Chinese student who was a marvellous painter. He painted a perfect life-like picture of a snake. But he was unable to stop and leave it alone.
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 20/04/2015 20:09, Paul Moore wrote: On 20 April 2015 at 19:41, Barry Warsaw ba...@python.org wrote: tldr; type hints in python source are scary. Would reserving them for stub files be better? I think so. I think PEP 8 should require stub files for stdlib modules and strongly encourage them for 3rd party code. Agreed. I have many of the same concerns as Harry, but I wouldn't have expressed them quite as well. I'm not too worried about actually removing annotations from the core language, but I agree that we should create a strong culture of type hints go in stub files to keep source files readable and clean. On that note, I'm not sure stub files is a particularly good name. Maybe type files would be better? Something that emphasises that they are the correct place to put type hints, not a workaround. Or we could just be honest and admit that we're choosing to add header files to Python. It's a shame, as it's more complexity, and it's being inflicted on those who might be writing a library for the first time, or those becoming core committers for the first time, or those just trying to do the right thing. Currently, the burden is a heavier one (type inference, rather than reading it from a file) but borne by people best placed to handle it (authors of IDEs, type checking software, etc). Chris ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 21/04/2015 12:23, Gustavo Carneiro wrote: Well, (i) can be done with good documentation (docstrings etc.). Documentation is not checked. It often loses sync with the actual code. Docs say one thing, code does another. That certainly something that could be fixed by formalising the documentation specification and having a checker check that rather than changing the syntax of the language... Chris ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Wed, 22 Apr 2015 01:09:52 +1000, Chris Angelico ros...@gmail.com wrote: def incremental_parser(input: FileLike) - List[Token]: tokens = [] data = while True: if not data: data = input.read(64) token = Token(data[0]); data = data[1:] while token.wants_more(): token.give_more(data[0]); data = data[1:] tokens.append(token) if token.is_end_of_stream(): break input.seek(-len(data), 1) return tokens If you were to exhaustively stipulate the requirements on the file-like object, you'd have to say: * Provides a read() method which takes an integer and returns up to that many bytes * Provides a seek() method which takes two integers * Is capable of relative seeking by at least 63 bytes backward * Is open for reading * Etcetera That's not the type system's job. Not in Python. Maybe in Haskell, but Just a note that if I'm reading the high level description right, this kind of analysis is exactly the kind of thing that Flow does for javascript. --David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Hello, On Tue, 21 Apr 2015 16:11:51 +0200 Antoine Pitrou anto...@python.org wrote: [] You can't at the same time point out that type checking has no power or control over runtime behaviour, and then claim that type checking makes runtime behaviour (for example, ability to accept or reject certain types) saner. It is a trivial contradiction. I suspected there's either short-sightedness, or just a word play for for a purpose of word play. Type annotation are NOT introduced for the purpose of static type checking. I was replying to Steven's message. Did you read it? Yes. And I try to follow general course of discussion, as its hard to follow individual sub-threads. And for example yesterday's big theme was people blackmailing that they stop contributing to stdlib if annotations are in, and today's theme appear to be people telling that static type checking won't be useful. And just your reply to Steven was a final straw which prompted me to point out that static type checking is not a crux of it, but just one (not the biggest IMHO) usage. Runtime type checking (run as via make test, not as production) is much more interesting to catch errors. Obviously you're giving the word runtime a different meaning than I do. The type checker isn't supposed to actually execute the user's functions (it's not that it's forbidden, simply that it's not how it will work in all likelihood): therefore, it doesn't have any handle on what *actually* happens at runtime, vs. what is declared in the typing declarations. Well, maybe typechecker is the wrong word then. I'm talking about instrumented VM which actually interprets type annotation while running bytecode - for example, on entry to a function, it takes argument annotations, and executes sequence of equivalent isinstance() checks, etc., etc. One won't use such instrumented VM at production runtime, but it will be useful to enable while running testsuite (or integration tests), to catch more issues. Even more interesting usage is to allow ahead-of-time, and thus unbloated, optimization. There're bunch of JITters and AOTters for Python language, each of which uses own syntax (via decorators, etc.) to annotate functions. As a developer of one of those tools, I've already said that I find it unlikely for the PEP to be useful for that purpose. The issue is that the vocabulary created in the PEP is not extensible enough. How so, if it essentially allows you make typedefs? Syntax of such typedef follow basic conventions (so *any* tool should understand its *basic* semantics), but you're free to assign (implicit, particular for your tool, but of course documented for it) semantics. Note I'm not saying it's impossible. I'm just skeptical that in its current form it will help us. And apparently none of our competitors That's my biggest fear - that JIT Python community is not yet ready to receive and value this feature. seems very enthusiastic either (feel free to prove me wrong: I might have missed something :-)). Let me try: MicroPython already uses type annotations for statically typed functions. E.g. def add(x:int, y:int): return x + y will translate the function to just 2 machine instructions. And we'd prefer to use standard language syntax, instead of having our own conventions (e.g. above you see that return type is inferred). Having language-specified type annotations allows for portable syntax for such optimized code. Only if the annotations allow expressing the subtleties required by the specific optimizer. For example, Float is too vague for Numba: we would like to know if that is meant to be a single- or double-precision float. Oh really, you care to support single-precisions in Numba? We have a lot of problems with sfloats in MicroPython, because whole Python stdlib API was written with implicit assumption that float is (at least) double. E.g., we cannot have time.time() which both returns fractional seconds and is based of Jan 1, 1970 - there's simply not enough bits in single-precision floats! It's on my backlog to bring up this and related issues before wider Python development community. Anyway, back to your example, it would be done like: SFloat = float DFloat = float For a random tool out there, SFloat and DFloat would be just aliases to floats, but Numba will know they have additional semantics behind them. (That assumes that typedefs like SFloat can be accessed in symbolic form - that's certainly possible if you have your own parser/VM, but might worth to think how to do it on CPython level). Don't block the language if you're stuck with an unimaginative implementation, there's much more to Python than that. The Python language doesn't really have anything to do with that. It's just an additional library with a set of conventions. Which is also why a PEP wouldn't be required to make it alive, it's just there to make it an official standard. Yes, and IMHO
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Hello, On Tue, 21 Apr 2015 08:05:59 -0700 Nikolaus Rath nikol...@rath.org wrote: On Apr 20 2015, Chris Angelico ros...@gmail.com wrote: Maybe it'd be of value to have a quick code stripper that takes away all the annotations, plus any other junk/framing that you're not interested in, and gives you something you can browse in a text editor? If you need to preprocess your source code to make it suitable for human consumption something is very wrong with your language design. I can't believe you're seriously suggesting this. I'm sure that was irony, d'oh. Just as my suggestion to have stickers This system runs free of type annotations for the most zealous anti-annotations folks. The proposed type annotations are very readable. Not as readable as C's type syntax, but ok. Sorry, irony again. I remember myself as a 10-years old kid, trying to wrap my head about C's types syntax, and not getting anything of it. So, please contrast it to C - PEP484 gives very readable syntax, and even both human- and machine-readable. If you feel like writing 3-story type annotation, just resist the temptation, Any is your best friend. -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 21 April 2015 at 16:09, Chris Angelico ros...@gmail.com wrote: Pretty accurate, yeah. Here's how I see it: def incremental_parser(input: FileLike) - List[Token]: tokens = [] data = while True: if not data: data = input.read(64) token = Token(data[0]); data = data[1:] while token.wants_more(): token.give_more(data[0]); data = data[1:] tokens.append(token) if token.is_end_of_stream(): break input.seek(-len(data), 1) return tokens If you were to exhaustively stipulate the requirements on the file-like object, you'd have to say: * Provides a read() method which takes an integer and returns up to that many bytes * Provides a seek() method which takes two integers * Is capable of relative seeking by at least 63 bytes backward * Is open for reading * Etcetera That's not the type system's job. Not in Python. Maybe in Haskell, but not in Python. So how much _should_ go into the type hint? I'm happy with FileLike or however it's to be spelled; maybe separate readable files from writable ones, as those are two fairly clear variants, but that's about all you really need. If you provide incremental_parser() with an input file that's non-seekable, it's going to have problems - and your editor may or may not even be able to detect that (some files are seekable but only in the forward direction, but they'll have the exact same seek() method). Ok, that makes sense to me. =) Looking at mypy's source code, I see shutil.copyfileobj has the following signature: def copyfileobj(fsrc: IO[AnyStr], fdst: IO[AnyStr], length: int = 16*1024) - None: so it seems that mypy defines a general IO datatype here. I continue to be worried about the lack of a *general* solution to this problem, but I'm glad that some specific solutions exist. I still think library authors will not do much to take up this proposal. =) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 9:09 AM, Chris Angelico ros...@gmail.com wrote: ... Pretty accurate, yeah. Here's how I see it: def incremental_parser(input: FileLike) - List[Token]: tokens = [] data = while True: if not data: data = input.read(64) token = Token(data[0]); data = data[1:] while token.wants_more(): token.give_more(data[0]); data = data[1:] tokens.append(token) if token.is_end_of_stream(): break input.seek(-len(data), 1) return tokens If you were to exhaustively stipulate the requirements on the file-like object, you'd have to say: * Provides a read() method which takes an integer and returns up to that many bytes * Provides a seek() method which takes two integers * Is capable of relative seeking by at least 63 bytes backward * Is open for reading * Etcetera Potentially you could use io.RawIOBase as the ABC for the type you need for FileLike, including read and seek. See the mixins in https://docs.python.org/3/library/io.html#class-hierarchy That's not the type system's job. Not in Python. Maybe in Haskell, Not in Haskell either FWIW in terms of what its type system can prove but not in Python. So how much _should_ go into the type hint? I'm happy with FileLike or however it's to be spelled; maybe separate readable files from writable ones, as those are two fairly clear variants, but that's about all you really need. RawIOBase is also potential overkill... maybe you just wanted something that duck typed for a few methods you implemented. Any and dynamic typing is a good choice then. If you provide incremental_parser() with an input file that's non-seekable, it's going to have problems - and your editor may or may not even be able to detect that (some files are seekable but only in the forward direction, but they'll have the exact same seek() method). With what has been proposed, there are no static guarantees about how many bytes can be read, or that input is even seekable (does seekable() return True?) or it is open for reading. Instead we can only *prove* a limited amount in static type systems about the runtime dynamic behavior of code, and PEP 484 is weaker than other approaches (for very good reasons IMHO). Still useful however :) - Jim ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
(Gmail messed up the attributions - apologies if I didn't fix them up correctly). 21 April 2015 at 19:55, Łukasz Langa luk...@langa.pl wrote: On Apr 21, 2015, at 11:23 AM, Guido van Rossum gu...@python.org wrote: 2. Clearly, great thought has been put into this PEP. If anyone has a good analysis of the potential impact on Python 3 adoption, please do pass along. I would be interested in reading the information. I wish I had a crystal ball, but this is hard to predict. Anecdotally, some people believe this will be catnip, while others believe it to be poison. The truth will surely be somewhere in the middle. At this point we don't know what drives Python 3 adoption except time -- it's definitely going up. :-) Anecdotal evidence shows that some organizations perceive this feature as one that justifies migration. Some of those organizations are pretty serious about open-sourcing things. That makes me believe that by sheer volume of the code they’re producing, Python 3 adoption will continue to increase. As Gregory Smith rightfully pointed out, nobody wants ugly code. I understand why people are afraid of that and it warms my heart that they are. The community cares so much about aesthetics and readability, it’s great! We will evolve this over time. This will be a learning process for everybody but we can’t learn to swim by only theorizing about it. We thought of the evolving nature of the solution from Day 1, hence the *provisional* nature of it. The wordy syntax is another example of that. Not requiring changes to the interpreter and the standard library was very high on the list of priorities. Once the *concept* proves itself, then we can improve on the syntax. Acknowledging PEP 484 being just the second step^ in a long journey is why some “obvious” parts are left out for now (hello, duck typing; hello, multiple dispatch; hello, runtime type checks in unit tests; etc. etc.). Those are often big enough to warrant their own PEPs. ^ PEP 3107 being the first. Thank you for this response. For some reason, it's reassured me a lot (I've no idea really why it struck a chord with me more than any of the other responses in the thread, just one of those things, I guess). Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 22 April 2015 at 04:28, Guido van Rossum gu...@python.org wrote: On Tue, Apr 21, 2015 at 12:49 AM, Antoine Pitrou solip...@pitrou.net wrote: On Mon, 20 Apr 2015 20:43:38 -0400 R. David Murray rdmur...@bitdance.com wrote: +1 to this from me too. I'm afraid that means I'm -1 on the PEP. I didn't write this in my earlier email because I wasn't sure about it, but my gut reaction after reading Harry's email was if type annotations are used in the stdlib, I'll probably stop contributing. That doesn't mean that's *true*, but that's the first time I've ever had that thought, so it is probably worth sharing. I think it would be nice to know what the PEP means for daily stdlib development. If patches have to carry typing information each time they add/enhance an API that's an addition burden. If typing is done separately by interested people then it sounds like it wouldn't have much of an impact on everyone else's workflow. This point will be moot until new code appears in the stdlib whose author likes type hints. As I said, we won't be converting existing code to add type hints (I'm actively against that for the stdlib, for reasons I've explained already). *If* type hints prove useful, I expect that adding type hints **to code that deserves them** is treated no different in the workflow than adding tests or docs. I.e. something that is the right thing to do because it has obvious benefits for users and/or future maintainers. If at some point running a type checker over the stdlib as part of continuous integration become routine, type hints can also replace certain silly tests. Until some point in a possible but distant future when we're all thinking back fondly about the argument we're currently having, it will be the choice of the author of new (and *only* new) stdlib modules whether and how to use type hints. Such a hypothetical author would also be reviewing updates to their module and point out lack of type hints just like you might point out an incomplete docstring, an outdated comment, or a missing test. (The type checker would be responsible for pointing out bugs. :-P ) What about major changes to existing modules? I have a backlog of intended feature uplifts from testtools into unittest - if the type hints thing works out I am likely to put them into testtools. Whats your view on type hints to such *new code* in existing modules? -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 22 April 2015 at 08:26, Guido van Rossum gu...@python.org wrote: In the end this should be up to you and the reviewers, but for such a venerable module like unittest I'd be hesitant to be an early adopter. I'd also expect that much of unittest is too dynamic in nature to benefit from type hints. But maybe you should just try to use them for testtools and see for yourself how beneficial or cumbersome they are in that particular case? Exactly yes. I've been experimenting recently with mypy to see. So far I've regressed backthrough 4 repos (unittest2, testtools, traceback2, linecache2) to get something small enough to work and experiment with. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 2:33 AM, Cory Benfield c...@lukasa.co.uk wrote: It seems like the only place the type annotations will get used is in relatively trivial cases where the types are obvious anyway. I don't deny that *some* bugs will be caught, but I suspect they'll overwhelmingly be crass ones that would have been caught quickly anyway. Well, it'll catch passing in a string instead of a sequence of strings -- one of teh common and semi-insidious type errors I see a lot (at least with newbies). Oh wait, maybe it won't -- a string IS a sequence of strings. That's why this is an insidioous bug in teh first place: List[string] will work, yes -- but now I'm locked down the user to an actual, list, yes? So I try: Iterable[string] ah yes -- now users can use tuples of strings, or a custom class, or -- but wait, darn it, a string IS an iterable of strings.. SIGH. NOTE: I've thought for years that Python should have a character type for this reason, but that's a side note. Oh, and I'm a heavy numpy user. And, in fact, I write a lot of functions that are essentially statically typed -- i.e. they will only work with a Nx3 array of float64 for instance. However, in this case, I want run-time type checking, and I DON'T want the API to be statically typed, so I use something like: the_arr = np.asarray( input_object, dypte-np.float64).reshape(-1, 3) and in my docstring: :param input_object: the input data to compute something from :type input_object: (Nx3) numpy array or floats or somethign that can be turned into one. Is there any way for type hinting to help here??? - Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 11:58 AM, Paul Sokolovsky pmis...@gmail.com wrote: It does, and hope people won't be caught in static typechecking loop and consider other usages too. Im confused -- from the bit I've been skimming the discussion, over on python-ideas, and now here, is that this is all about static typechecking. It's not about run-time type checking. It's not about type-base performance optimization. It's not about any use of annotations other than types. What is it about other than static typechecking? -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Thank you Jack. Jack: I hate code and I want as little of it as possible in our product I love that quote -- and I ALWAYS use it when I teach newbies Python. It's kind of the point of Python -- you can get a lot done by writing very little code. I'm still confused about what all this type annotation is about -- yes, I certainly see lots of advantages to static typing -- but also many disadvantages. And one of them is a bunch more stuff to read and write. But key is: If you like typing, use a statically typed language. If you want the advantages of dynamic typing -- then why do you want all the extra clutter and safety? And it seems to me that optional/partial typing buys you much of the hassle and restrictions, and essentially NONE of the safety (type casting pointers in C, anyone?). And I've had this debate with proponents of JAVA and the like a bunch -- sure, type checking will catch bugs in your code -- but they are almost always shallow bugs -- and bugs that if your tests didn't catch them, you have really lame tests! On the other had, you just can't get good performance doing low-level things with a dynamic language. So I use Cython, which, in a way, is Python with optional static typing. It gives you down to the metal performance, where, and only where, you need it, but not really much in the way of type checking. However, this is exactly the opposite of what everyone seem to be talking about using optional typing in Python for -- which is type checking ,but not any run-time optimizations -- in fact, AFAICT, not even run-time type checking. So a whole lot of clutter for very little gain. :-( NOTE: MyPy is out in the wild -- I'd be really interested to see how it all is really working out -- even on those enterprise code bases -- other than managers feeling better, are developers finding out that: WOW -- this baby caught some nasty, bugs that I would not have found with testing -- and would have been hard to debug after the fact! But in the end, I agree with the OP here -- stub files let pre-run-time static type checking happen without cluttering up Python for the rest of us -- so a nice compromise. So I guess we'll see. - Chris On Mon, Apr 20, 2015 at 4:41 PM, Jack Diederich jackd...@gmail.com wrote: Twelve years ago a wise man said to me I suggest that you also propose a new name for the resulting language I talked with many of you at PyCon about the costs of PEP 484. There are plenty of people who have done a fine job promoting the benefits. * It is not optional. Please stop saying that. The people promoting it would prefer that everyone use it. If it is approved it will be optional in the way that PEP8 is optional. If I'm reading your annotated code it is certainly /not/ optional that I understand the annotations. * Uploading stubs for other people's code is a terrible idea. Who do I contact when I update the interface to my library? The random Joe who helped by uploading annotations three months ago and then quit the internet? I don't even want to think about people maliciously adding stubs to PyPI. * The cognitive load is very high. The average function signature will double in length. This is not a small cost and telling me it is optional to pretend that every other word on the line doesn't exist is a farce. * Every company's style guide is about to get much longer. That in itself is an indicator that this is a MAJOR language change and not just some optional add-on. * People will screw it up. The same people who can't be trusted to program without type annotations are also going to be *writing* those type annotations. * Teaching python is about to get much less attractive. It will not be optional for teachers to say just pretend all this stuff over here doesn't exist * No new syntax is a lie. Or rather a red herring. There are lots of new things it will be required to know and just because the compiler doesn't have to change doesn't mean the language isn't undergoing a major change. If this wasn't in a PEP and it wasn't going to ship in the stdlib very few people would use it. If you told everyone they had to install a different python implementation they wouldn't. This is much worse than that - it is Python4 hidden away inside a PEP. There are many fine languages that have sophisticated type systems. And many bondage discipline languages that make you type things three times to make really really sure you meant to type that. If you find those other languages appealing I invite you to go use them instead. -Jack https://mail.python.org/pipermail/python-dev/2003-February/033291.html ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/chris.barker%40noaa.gov -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206)
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Cory Benfield c...@lukasa.co.uk python-dev@python.org python-dev@python.org On Tue, Apr 21, 2015 at 8:47 AM, Cory Benfield c...@lukasa.co.uk wrote: I'm talking from the position of being a library author, where supporting versions of Python lower than 3.5 will be a reality for at least 5 more years. I will not be able to inline my type hints, so they'll have to go in stub files, and now we've got the same problem: type hints can go out of step just as easily as documentation can. I imagine authors interested in both type hinting and backwards compatibility could write their code using the 3.0+ annotation syntax, while using the various backwards compatibility packages and schemes, then split the source file using some inevitably-available tool to generate both annotation-free source and associated stub files. The author maintains (directly, at least) files on a 1-to-1 ratio -- the fact that they are split prior to distribution is transparent (much like generating documentation from source / comments). Running such a preprocessing step may not be ideal, but it is certainly an option that the motivated package author would seem to have, if they were so inclined. -- FWIW, my gut reaction was the same as many on the list (the syntax can get verbose and negatively impact readability), but I think we're only seeing one side of the eventual coin, looking at individual, sometimes specifically chosen examples. In theory, if this takes off, IDEs could do a lot to help reduce the burden. Maybe they provide a read-only view of functions/methods where annotations are hidden until you put the caret inside the block, at which point the hints are revealed. Maybe even inside the block they're hidden, until you attempt to edit the arguments. Maybe they're just always gathered up by the IDE and accessible through some panel widget. Some of these approaches may sound helpful to some, others not so much, but the point is that there are a lot of creative ways that IDEs *could* help ease the transition, if type hinting becomes of significant interest to the community. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Mon, Apr 20, 2015 at 08:37:28PM -0700, Guido van Rossum wrote: On Mon, Apr 20, 2015 at 4:41 PM, Jack Diederich jackd...@gmail.com wrote: Twelve years ago a wise man said to me I suggest that you also propose a new name for the resulting language The barrage of FUD makes me feel like the woman who asked her doctor for a second opinion and was told you're ugly too. Don't worry Guido, some of us are very excited to see this coming to fruition :-) It's been over ten years since your first blog post on optional typing for Python. At least nobody can accuse you of rushing into this. http://www.artima.com/weblogs/viewpost.jsp?thread=85551 -- Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, 21 Apr 2015 09:28:45 -0700 Guido van Rossum gu...@python.org wrote: On Tue, Apr 21, 2015 at 12:49 AM, Antoine Pitrou solip...@pitrou.net wrote: On Mon, 20 Apr 2015 20:43:38 -0400 R. David Murray rdmur...@bitdance.com wrote: +1 to this from me too. I'm afraid that means I'm -1 on the PEP. I didn't write this in my earlier email because I wasn't sure about it, but my gut reaction after reading Harry's email was if type annotations are used in the stdlib, I'll probably stop contributing. That doesn't mean that's *true*, but that's the first time I've ever had that thought, so it is probably worth sharing. I think it would be nice to know what the PEP means for daily stdlib development. If patches have to carry typing information each time they add/enhance an API that's an addition burden. If typing is done separately by interested people then it sounds like it wouldn't have much of an impact on everyone else's workflow. This point will be moot until new code appears in the stdlib whose author likes type hints. As I said, we won't be converting existing code to add type hints (I'm actively against that for the stdlib, for reasons I've explained already). I was thinking of potential stub files. Or are you also putting a ban on those for existing stdlib code? Sorry if that has already been answered... Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 04/21, Paul Sokolovsky wrote: And for example yesterday's big theme was people blackmailing that they stop contributing to stdlib if annotations are in [...] A volunteer's honest reaction is not blackmail, and categorizing it as such is not helpful to the discussion. -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 12:50 AM Antoine Pitrou solip...@pitrou.net wrote: On Mon, 20 Apr 2015 20:43:38 -0400 R. David Murray rdmur...@bitdance.com wrote: +1 to this from me too. I'm afraid that means I'm -1 on the PEP. I didn't write this in my earlier email because I wasn't sure about it, but my gut reaction after reading Harry's email was if type annotations are used in the stdlib, I'll probably stop contributing. That doesn't mean that's *true*, but that's the first time I've ever had that thought, so it is probably worth sharing. I think it would be nice to know what the PEP means for daily stdlib development. If patches have to carry typing information each time they add/enhance an API that's an addition burden. If typing is done separately by interested people then it sounds like it wouldn't have much of an impact on everyone else's workflow. Separately by interested people. That won't change until tools appear and mature that help maintain the types for us. (if ever) Nobody wants unreadable code. Nobody is proposing to make unreadable code happen or encourage its creation. One thing I feel is often overlooked in the discussion on this PEP: It is about creating a unified type expression syntax for everyone working on Python typing to centralize around. Regardless of if the PEPs version falls short for some purposes. It allows for sharing work. There are multiple ongoing projects that are trying to make use of type information with Python, this allows them to all speak the same language. (MyPy, MicroPython, Cython, the static analyzer we are trying to create at Google, several others not on the top of my head I'm sure, etc.) We will not be putting type annotations anywhere in the stdlib or expecting anyone else to maintain them there. That would never happen until tools that are convincing enough in their utility for developers to _want_ to use are available and accepted. That'll be a developer workflow thing we could address with a later PEP. IF it happens at all. I view most of this thread as FUD. The fear is understandable, I'm trying to tell people to stop panicing. This PEP does not mean that Python is suddenly going to become unreadable. Just that a set of people working on a common goal have a way to communicate with one another. If that work bears fruit, great, it'll be shared and provided as tools that people want to use. If not, it won't matter in the slightest and the typing module and this PEP will be relegated to history. This is a 100% non-invasive PEP. No new keywords! Motivation behind static analysis and type checkers comes directly from the success of the type annotations and checking done to Javascript in Google's javascript Closure compiler that has been available for years. Steven mentioned Facebook's Flow which does a similar thing. These are both opt-in and by and large we've found that developers love using them in any decent sized code base. That model is the goal for any of our Python typing related projects to get to. If developers don't _want_ to use it in the end, we have failed and they can happily continue not using it because it was never required. The reason this PEP exists is for tool developers to be able to do their thing and prove to everyone that it is (a) possible and (b) genuinely useful. IF that proves successful, we can consider if we need a saner syntax for anyone to want to use it. For now we've got this PEP which is a bit of a hack using the Python 3 annotations and a typing module but at the same time doesn't involve any language changes we might regret. I call that a win! -gps ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 7:51 AM, Cory Benfield c...@lukasa.co.uk wrote: The correct specification is read method with this type signature and seek method with this type signature. I would even be prepared to waive the type signatures on read and seek, given that enforcing the type hinting on others is not a good idea. I suspect I have a mismatch with several others in this discussion. My position is that if I'm going to have a type system, I'd like to have a powerful one: Haskell, not Java. Otherwise I'll get by with the duck typing that has worked just fine for us over the last 20 years. I suspect, however, that many others in this conversation want any type system at all, so long as they can have one. For me, PEP 484 is a stepping stone. Among the authors of PEP 484 there was much discussion about duck typing, and mypy even has some limited support for duck typing (I think you can still find it by searching the mypy code for protocol). But we ran out of time getting all the details written up and agreed upon, so we decided to punt -- for now. But duck typing still needs to have a way to talk about things like seek method with this type signature (something like `def seek(self, offset: int, whence: int=SEEK_SET) - int`) so the current proposal gets us part of the way there. The hope is that once 3.5 is out (with PEP 484's typing.py included *provisional* mode) we can start working on the duck typing specification. The alternative would have been to wait until 3.6, but we didn't think that there would be much of an advantage to postponing the more basic type hinting syntax (it would be like refusing to include import until you've sorted out packages). During the run of 3.5 we'll hopefully get feedback on where duck typing is most needed and how to specify it -- valuable input that would be much harder to obtain of *no* part of the type hints notation were standardized. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Apr 21 2015, Paul Sokolovsky pmis...@gmail.com wrote: Hello, On Tue, 21 Apr 2015 08:05:59 -0700 Nikolaus Rath nikol...@rath.org wrote: On Apr 20 2015, Chris Angelico ros...@gmail.com wrote: Maybe it'd be of value to have a quick code stripper that takes away all the annotations, plus any other junk/framing that you're not interested in, and gives you something you can browse in a text editor? If you need to preprocess your source code to make it suitable for human consumption something is very wrong with your language design. I can't believe you're seriously suggesting this. I'm sure that was irony, d'oh. That'd be a relief. It didn't sound ironic to me. The proposed type annotations are very readable. [..] I don't have an informed opinion about that yet. I was just commenting on the general idea of stripping them away if they're not readable. Best, -Nikolaus -- GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F »Time flies like an arrow, fruit flies like a Banana.« ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 9:17 AM, R. David Murray rdmur...@bitdance.com wrote: Please be respectful rather than inflammatory. If you read what I wrote, I did not say that I was going to stop contributing, I specifically talked about that gut reaction being both emotional and illogical. That doesn't make the reaction any less real, and the fact that such reactions exist is a data point you should consider in conducting your PR campaign for this issue. (I don't mean that last as a negative: this issue *requires* an honest PR campaign.) Well, my own reactions at this point in the flame war are also quite emotional. :-( I have done my best in being honest in my PR campaign. But I feel like the opposition (not you, but definitely some others -- have you seen Twitter?) are spreading FUD based on an irrational conviction that this will destroy Python. It will not. It may not prove the solution to all Python's problems -- there's always 3.6. (Oh wait, Python 2.7 is perfect. I've heard that before -- Paul Everitt famously said the same of Python 1.5.2. Aren't you glad I didn't take him literally? :-P ) -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 03:51:05PM +0100, Cory Benfield wrote:
On 21 April 2015 at 15:31, Chris Angelico ros...@gmail.com wrote:
Granted, there are some
vague areas - how many functions take a file-like object, and are
they all the same? - but between MyPy types and the abstract base
types that already exist, there are plenty of ways to formalize duck
typing.
Are there? Can I have a link or an example, please? I feel like I
don't know how I'm supposed to do this, and I'd like to see how that
works. I'll even give a concrete use-case: I want to be able to take a
file-like object that has a .read() method and a .seek() method.
I've never done this before, so I might not quite have done it
correctly, but this appears to work just fine:
py import abc
py class SeekableReadable(metaclass=abc.ABCMeta):
... @classmethod
... def __subclasshook__(cls, C):
... if hasattr(C, 'seek') and hasattr(C, 'read'):
... return True
... return NotImplemented
...
py f = open('/tmp/foo')
py isinstance(f, SeekableReadable)
True
py from io import StringIO
py issubclass(StringIO, SeekableReadable)
True
py issubclass(int, SeekableReadable)
False
That gives you your runtime check for an object with seek() and read()
methods. For compile-time checking, I expect you would define
SeekableReadable as above, then make the declaration:
def read_from_start(f:SeekableReadable, size:int):
f.seek(0)
return f.read(size)
So now you have runtime interface checking via an ABC, plus
documentation for the function parameter type via annotation.
But will the static checker understand that annotation? My guess is,
probably not as it stands. According to the docs, MyPy currently
doesn't support this sort of duck typing, but will:
[quote]
There are also plans to support more Python-style “duck typing”
in the type system. The details are still open.
[end quote]
http://mypy.readthedocs.org/en/latest/class_basics.html#abstract-base-classes-and-multiple-inheritance
I expect that dealing with duck typing will be very high on the list
of priorities for the future. In the meantime, for this specific use-case,
you're probably not going to be able to statically check this type hint.
Your choices would be:
- don't type check anything;
- don't type check the read_from_start() function, but type check
everything else;
- don't type check the f parameter (remove the SeekableReadable
annotation, or replace it with Any, but leave the size:int
annotation);
- possibly some type checkers will infer from the function body that f
must have seek() and read() methods, and you don't have to declare
anything (structural typing instead of nominal?);
- (a bad idea, but just for the sake of completeness) leave the
annotation in, and ignore false negatives.
Remember that there is no built-in Python type checker. If you have no
checker, the annotations are just documentation and nothing else will
have changed. If you don't like the checker you have, you'll be able to
replace it with another.
--
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 1:18 PM, Robert Collins robe...@robertcollins.net wrote: On 22 April 2015 at 04:28, Guido van Rossum gu...@python.org wrote: Until some point in a possible but distant future when we're all thinking back fondly about the argument we're currently having, it will be the choice of the author of new (and *only* new) stdlib modules whether and how to use type hints. Such a hypothetical author would also be reviewing updates to their module and point out lack of type hints just like you might point out an incomplete docstring, an outdated comment, or a missing test. (The type checker would be responsible for pointing out bugs. :-P ) What about major changes to existing modules? I have a backlog of intended feature uplifts from testtools into unittest - if the type hints thing works out I am likely to put them into testtools. Whats your view on type hints to such *new code* in existing modules? In the end this should be up to you and the reviewers, but for such a venerable module like unittest I'd be hesitant to be an early adopter. I'd also expect that much of unittest is too dynamic in nature to benefit from type hints. But maybe you should just try to use them for testtools and see for yourself how beneficial or cumbersome they are in that particular case? -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, 21 Apr 2015 18:27:50 +0300, Paul Sokolovsky pmis...@gmail.com wrote: I was replying to Steven's message. Did you read it? Yes. And I try to follow general course of discussion, as its hard to follow individual sub-threads. And for example yesterday's big theme was people blackmailing that they stop contributing to stdlib if annotations are in, and today's theme appear to be people telling that static type checking won't be useful. And just your reply to Steven was a final straw which prompted me to point out that static type checking is not a crux of it, but just one (not the biggest IMHO) usage. Please be respectful rather than inflammatory. If you read what I wrote, I did not say that I was going to stop contributing, I specifically talked about that gut reaction being both emotional and illogical. That doesn't make the reaction any less real, and the fact that such reactions exist is a data point you should consider in conducting your PR campaign for this issue. (I don't mean that last as a negative: this issue *requires* an honest PR campaign.) --David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 12:49 AM, Antoine Pitrou solip...@pitrou.net wrote: On Mon, 20 Apr 2015 20:43:38 -0400 R. David Murray rdmur...@bitdance.com wrote: +1 to this from me too. I'm afraid that means I'm -1 on the PEP. I didn't write this in my earlier email because I wasn't sure about it, but my gut reaction after reading Harry's email was if type annotations are used in the stdlib, I'll probably stop contributing. That doesn't mean that's *true*, but that's the first time I've ever had that thought, so it is probably worth sharing. I think it would be nice to know what the PEP means for daily stdlib development. If patches have to carry typing information each time they add/enhance an API that's an addition burden. If typing is done separately by interested people then it sounds like it wouldn't have much of an impact on everyone else's workflow. This point will be moot until new code appears in the stdlib whose author likes type hints. As I said, we won't be converting existing code to add type hints (I'm actively against that for the stdlib, for reasons I've explained already). *If* type hints prove useful, I expect that adding type hints **to code that deserves them** is treated no different in the workflow than adding tests or docs. I.e. something that is the right thing to do because it has obvious benefits for users and/or future maintainers. If at some point running a type checker over the stdlib as part of continuous integration become routine, type hints can also replace certain silly tests. Until some point in a possible but distant future when we're all thinking back fondly about the argument we're currently having, it will be the choice of the author of new (and *only* new) stdlib modules whether and how to use type hints. Such a hypothetical author would also be reviewing updates to their module and point out lack of type hints just like you might point out an incomplete docstring, an outdated comment, or a missing test. (The type checker would be responsible for pointing out bugs. :-P ) -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Mon, Apr 20, 2015 at 4:00 PM, Harry Percival harry.perci...@gmail.com wrote: @Lukasz: Of course you're right, ugly is a matter of perspective, and I'm sure I could grow to love them, and they might evolve into a more polished direction they start to read more transparently after a while. But I'm still worried about beginners, and I'm even worried about me. I like to be able to scan through some code and see the essence of it. I learned Java at school, and I got it figured out, but i'm glad I left it behind. Every so often I read a TDD book and the examples are all in java and it just feels like obfuscation -- public void static private String[] class blabla... so many keywords and types getting in the way of *what the code is actually doing*. That's what's so appealing about Python, it strips it down to just the basics. And, to me, type hints are always going to be some unnecessary chaff that gets in the way of my understanding -- not useless, not that they don't have a purpose or that we should remove them completely. But if there was a way of just hiding them, so that I don't have to think about them, as a beginner, or as a normal programmer, most of the time, in the 90% of cases where I don't need to see them, I shouldn't have to... that's why i'm so keen on this stub files idea. One thing I don't understand is this local variable inference thing -- can that not be made to work in stub files? On 20 April 2015 at 21:50, Harry Percival harry.perci...@gmail.com wrote: stub files are only used to type-check *users* of a module. If you want a module itself to be type-checked you have to use inline type hints is this a fundamental limitation, or just the current state of tooling? On 20 April 2015 at 21:48, Harry Percival harry.perci...@gmail.com wrote: I hate stub files. [...] in my opinion, [it] just about guarantees a maintenance burden that will fall by the side of the road. I'm not so pessimistic. It's not like documentation or docstrings or comments -- the whole point is that it should be very easy to have an automated check for whether your stubs are in sync with your source, because both are in code. Unlike docs or comments which can easily become out of date, because there's no automated process to tell you they need updating... I'm thinking of it as a thing your editor will warn you of. Like pyflakes warnings about unused variables co, I'm never happy until I've silenced them all in a file, and similarly, your editor will keep bugging you until you've got your stubs inline with your code... On 20 April 2015 at 20:37, Isaac Morland ijmor...@uwaterloo.ca wrote: On Mon, 20 Apr 2015, Paul Moore wrote: On 20 April 2015 at 19:41, Barry Warsaw ba...@python.org wrote: tldr; type hints in python source are scary. Would reserving them for stub files be better? I think so. I think PEP 8 should require stub files for stdlib modules and strongly encourage them for 3rd party code. Agreed. I have many of the same concerns as Harry, but I wouldn't have expressed them quite as well. I'm not too worried about actually removing annotations from the core language, but I agree that we should create a strong culture of type hints go in stub files to keep source files readable and clean. On that note, I'm not sure stub files is a particularly good name. Maybe type files would be better? Something that emphasises that they are the correct place to put type hints, not a workaround. How about header files? (ducks...) Isaac Morland CSCF Web Guru DC 2619, x36650 WWW Software Specialist ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/hjwp2%40cantab.net So I've generally stayed out of this but I feel there is some context that people are missing in general. First, allow me to provide some context: I maintain a /lot/ of Python code[1] and nearly all of it is designed to be compatible with Pythons 2.6, 2.7, 3.2, 3.3, 3.4 (and eventually 3.5) and sometimes 2.5 (depending on the project). If I want to improve a developer's experience with some of that code using Type Hints I will essentially have no way to do that unless I write the code with the annotations and ship versions with annotations stripped and other versions with annotations? That's a lot more overhead. If I could provide the annotations in stubs that means that only the people who care about using them will have to use them. Is it more overhead to manage twice the number of files? Yes. Do I feel it would be worth it to not overly complicate how these packages are released? Yes. Further, there are far more reasons to make stubs the baseline (in my opinion) the biggest reason of all is that people want to provide stubs for popular yet unmaintained libraries as third party
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
exactly. yay stub files! we all agree! everyone loves them! boo type annotations inline in python source. only some people love them. and even then, only after a while, and only tentatively... and some people fear them, mightily... On 20 April 2015 at 23:14, Łukasz Langa luk...@langa.pl wrote: On Apr 20, 2015, at 3:02 PM, Ian Cordasco graffatcolmin...@gmail.com wrote: I think while the authors are currently seeing stubs as a necessary *evil* they're missing points where they're a better backwards compatible solution for people who want to give users with capable IDEs the ability to use stub (or hint) files. We might have not chosen the wording that makes this obvious to you but we definitely did consider all of the points you’re referring to: https://www.python.org/dev/peps/pep-0484/#stub-files -- Best regards, Łukasz Langa WWW: http://lukasz.langa.pl/ Twitter: @llanga IRC: ambv on #python-dev -- -- Harry J.W. Percival -- Twitter: @hjwp Mobile: +44 (0) 78877 02511 Skype: harry.percival ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
So I guess the main difference is that type annotations in stub files wouldn't be available at runtime? Ie, they wouldn't magically appear in __annotations__ (unless the python interpreter itself started to evaluate stub files too) On 20 April 2015 at 22:02, Guido van Rossum gu...@python.org wrote: On Mon, Apr 20, 2015 at 1:50 PM, Harry Percival harry.perci...@gmail.com wrote: stub files are only used to type-check *users* of a module. If you want a module itself to be type-checked you have to use inline type hints is this a fundamental limitation, or just the current state of tooling? It's not fundamental, it's just more in line with the original purpose of stubs (to describe C extensions). -- --Guido van Rossum (python.org/~guido) -- -- Harry J.W. Percival -- Twitter: @hjwp Mobile: +44 (0) 78877 02511 Skype: harry.percival ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
I hate stub files. [...] in my opinion, [it] just about guarantees a maintenance burden that will fall by the side of the road. I'm not so pessimistic. It's not like documentation or docstrings or comments -- the whole point is that it should be very easy to have an automated check for whether your stubs are in sync with your source, because both are in code. Unlike docs or comments which can easily become out of date, because there's no automated process to tell you they need updating... I'm thinking of it as a thing your editor will warn you of. Like pyflakes warnings about unused variables co, I'm never happy until I've silenced them all in a file, and similarly, your editor will keep bugging you until you've got your stubs inline with your code... On 20 April 2015 at 20:37, Isaac Morland ijmor...@uwaterloo.ca wrote: On Mon, 20 Apr 2015, Paul Moore wrote: On 20 April 2015 at 19:41, Barry Warsaw ba...@python.org wrote: tldr; type hints in python source are scary. Would reserving them for stub files be better? I think so. I think PEP 8 should require stub files for stdlib modules and strongly encourage them for 3rd party code. Agreed. I have many of the same concerns as Harry, but I wouldn't have expressed them quite as well. I'm not too worried about actually removing annotations from the core language, but I agree that we should create a strong culture of type hints go in stub files to keep source files readable and clean. On that note, I'm not sure stub files is a particularly good name. Maybe type files would be better? Something that emphasises that they are the correct place to put type hints, not a workaround. How about header files? (ducks...) Isaac Morland CSCF Web Guru DC 2619, x36650 WWW Software Specialist ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/hjwp2%40cantab.net -- -- Harry J.W. Percival -- Twitter: @hjwp Mobile: +44 (0) 78877 02511 Skype: harry.percival ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
I wrote a longer response and then realized it didn't really add much to the discussion. So let me be short: type annotations do *not* appeal to me, and I am not looking forward to the cognitive overhead of dealing with them. Perhaps I will eventually grow to like them if the tools that use them really add value. You'll have to sell me on it, though. On Mon, 20 Apr 2015 12:35:33 -0700, luk...@langa.pl wrote: Stub files have many downsides, too, unfortunately: - we donât *want* to have them, but we *need* to have them (C extensions, third-party modules, Python 2, â¦) - they bring cognitive overhead of having to switch between two files - they require the author to repeat himself quite a lot - they might go out of date much easier than annotations in the function signature The whole point of type hints is for the linters/IDEs, so IMO it is perfectly reasonable to put the burden of making them useful onto the linters/IDEs. The UI for it can unify the two files into a single view...I know because way back in the dark ages I wrote a small editor-based IDE that did something very analogous on an IBM Mainframe, and it worked really well as a development environment. --David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Twelve years ago a wise man said to me I suggest that you also propose a new name for the resulting language I talked with many of you at PyCon about the costs of PEP 484. There are plenty of people who have done a fine job promoting the benefits. * It is not optional. Please stop saying that. The people promoting it would prefer that everyone use it. If it is approved it will be optional in the way that PEP8 is optional. If I'm reading your annotated code it is certainly /not/ optional that I understand the annotations. * Uploading stubs for other people's code is a terrible idea. Who do I contact when I update the interface to my library? The random Joe who helped by uploading annotations three months ago and then quit the internet? I don't even want to think about people maliciously adding stubs to PyPI. * The cognitive load is very high. The average function signature will double in length. This is not a small cost and telling me it is optional to pretend that every other word on the line doesn't exist is a farce. * Every company's style guide is about to get much longer. That in itself is an indicator that this is a MAJOR language change and not just some optional add-on. * People will screw it up. The same people who can't be trusted to program without type annotations are also going to be *writing* those type annotations. * Teaching python is about to get much less attractive. It will not be optional for teachers to say just pretend all this stuff over here doesn't exist * No new syntax is a lie. Or rather a red herring. There are lots of new things it will be required to know and just because the compiler doesn't have to change doesn't mean the language isn't undergoing a major change. If this wasn't in a PEP and it wasn't going to ship in the stdlib very few people would use it. If you told everyone they had to install a different python implementation they wouldn't. This is much worse than that - it is Python4 hidden away inside a PEP. There are many fine languages that have sophisticated type systems. And many bondage discipline languages that make you type things three times to make really really sure you meant to type that. If you find those other languages appealing I invite you to go use them instead. -Jack https://mail.python.org/pipermail/python-dev/2003-February/033291.html ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Mon, Apr 20, 2015 at 1:17 PM, Robert Collins robe...@robertcollins.net wrote: On 21 April 2015 at 08:10, Eric Snow ericsnowcurren...@gmail.com wrote: While it helps, this sort of best-practice is still unsettled (and apparently not obvious). In the short term it would make more sense to recommend using stub files for all the reason Harry enumerated. Once the best practices are nailed down through experience with stub files, then we can make recommendations regarding inline type hints. -eric Forgive my ignorance, but can stub files can't annotate variables within functions? E.g. AIUI if there is a stub file, it is used in the static analysis instead of the actual source. Likely I've got it modelled wrong in my head :) -Rob Correct, stub files are only used to type-check *users* of a module. If you want a module itself to be type-checked you have to use inline type hints. (Though it has been suggested to combine the hints from the stub with the implementation and use this to type-check the implementation, and some tool chains may actually implement this.) -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
