Re: Commonly-used names in the Python standard library
On Fri, Feb 21, 2014 at 6:21 PM, Marko Rauhamaa ma...@pacujo.net wrote: Steven D'Aprano steve+comp.lang.pyt...@pearwood.info: On Thu, 20 Feb 2014 12:22:29 +0200, Marko Rauhamaa wrote: I'm looking forward to the day when every application can add its own keywords as is customary in Lisp. And what a wonderful day that will be! Reading any piece of code you didn't write yourself -- or wrote a long time ago -- will be an adventure! Every script will have it's own exciting new set of keywords doing who knows what, which makes every script nearly it's own language! Oh joy, I cannot wait! That's sarcasm, by the way. I don't hear Lispers or C programmers complaining. Yes, you can shoot yourself in the foot with macro trickery, but macros can greatly enhance code readability and remove the need for code generators. That's why they are used extensively in Linux kernel code and GOOPS (Guile's object system), for example. How does C let you create new keywords? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Commonly-used names in the Python standard library
On Fri, 21 Feb 2014 09:21:56 +0200, Marko Rauhamaa wrote: I don't hear Lispers or C programmers complaining. Lisp is not a popular language. Despite being more powerful, more efficient, and a lot older, I expect that there are far fewer people who know Lisp (let alone use it regularly) than Python. I wouldn't go so far as to say that the Lisp/Scheme family of languages is moribund, but they are certainly niche. And by the way, that niche includes some of the best and brightest developers. (Some of whom are too clever by half, but that's another story.) Merely mediocre programmers don't learn Lisp. So if you take the smartest 0.1% of programmers, and give them a language that lets them chainsaw their leg off, they won't complain, they're just keep their leg out of the way. But if you take the average 50% programmers, and give them a language that lets them chainsaw their leg off, there will be a lot of one-legged programmers. I'm really glad that Lisp exists, but I don't want Python to aspire to be Lisp. In the same way, I am very fond of Forth. Forth too lets you re- define *everything* about the language, including creating new flow- control commands. I love that language. But there is a very good reason why there are a thousand Python coders for every one Forth coder, and it isn't the stack or the reverse Polish notation. As for C, there are a lot of mediocre C programmers writing mediocre C programs filled with buffer overflows, null-pointer bugs and all sorts of other problems. And they don't complain about those either. Because the smart ones know how not to chainsaw their leg off (but even they still make mistakes, which is why there are periodic security vulnerabilities even in code written by people of the calibre of Linus Torvalds and the Linux kernel devs), or have moved to another language and write the bare minimum of code in C only when they really need to. Yes, you can shoot yourself in the foot with macro trickery, but macros can greatly enhance code readability Or greatly destroy it, which is precisely the reason why Python doesn't have a macro system. When Guido van Rossum reads Python code, he wants it to look like Python code, not some arbitrary custom-built language. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: Commonly-used names in the Python standard library
Chris Angelico ros...@gmail.com:
How does C let you create new keywords?
With #define. Nowhere near as elegant (flexible, hygienic) as in Lisp,
but used to create new syntax:
include/linux/list.h:
#define list_for_each(pos, head) \
for (pos = (head)-next; pos != (head); pos = pos-next)
include/linux/wait.h:
#define __wait_event_interruptible(wq, condition, ret) \
do {\
DEFINE_WAIT(__wait);\
\
for (;;) { \
prepare_to_wait(wq, __wait, TASK_INTERRUPTIBLE); \
if (condition) \
break; \
if (!signal_pending(current)) { \
schedule(); \
continue; \
} \
ret = -ERESTARTSYS; \
break; \
} \
finish_wait(wq, __wait); \
} while (0)
In the latter example, note how condition is embedded in the middle of
the macro and evaluated repeatedly.
Marko
--
https://mail.python.org/mailman/listinfo/python-list
Re: Commonly-used names in the Python standard library
On Fri, Feb 21, 2014 at 9:26 PM, Marko Rauhamaa ma...@pacujo.net wrote: Chris Angelico ros...@gmail.com: How does C let you create new keywords? With #define. Nowhere near as elegant (flexible, hygienic) as in Lisp, but used to create new syntax: That can't create new syntax, though. All it can do is create a thing that looks like a symbol or a function call and plonks a bit of code in at that point. That's all. It's more akin to creating a function that's able to work with blocks of unexecuted code. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Commonly-used names in the Python standard library
Chris Angelico ros...@gmail.com: On Fri, Feb 21, 2014 at 9:26 PM, Marko Rauhamaa ma...@pacujo.net wrote: With #define. Nowhere near as elegant (flexible, hygienic) as in Lisp, but used to create new syntax: That can't create new syntax, though. All it can do is create a thing that looks like a symbol or a function call and plonks a bit of code in at that point. That's all. It's more akin to creating a function that's able to work with blocks of unexecuted code. Exactly! Lisp functions and macros (including special forms) are closely related. The whole scheme (no pun intended) is so attractive because of the S expression supersyntax. A while macro is syntactically no different from a while function. A macro is a function whose automatic argument evaluation is disabled; the macro function gets the ASTs of the arguments as input. A macro can always simulate a function but not the other way round. With the addition of macros, Python would become a (remote) Lisp dialect. Defining macros would become more complicated because of Python's more complex supersyntax. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Commonly-used names in the Python standard library
On 02/21/2014 04:03 AM, Marko Rauhamaa wrote: With the addition of macros, Python would become a (remote) Lisp dialect. Defining macros would become more complicated because of Python's more complex supersyntax. Have you tried MacroPy? If not, you should. If so, what were its failings? -- ~Ethan~ -- https://mail.python.org/mailman/listinfo/python-list
Re: Commonly-used names in the Python standard library
On Friday, February 21, 2014 12:26:00 AM UTC+8, Marko Rauhamaa wrote: Chris Angelico ros...@gmail.com: Also, what happens if two modules (one of which might be your script) written for different versions both import some third module? Should they get different versions, based on what version tags they use themselves? Compatibility can't be changed that easily. You either run on the new version, or run on the old. Not both. Shared C libraries face the exact same issue. Java seems pretty good on this front as well. When there is a will, there is a way. Marko Well, please check jython first under OS Andrioids( LINUX+JAVA-XWIN). The mobile phone SW applications are getting hoter and hoter these days. -- https://mail.python.org/mailman/listinfo/python-list
Commonly-used names in the Python standard library
In working on a proposal that might result in the creation of a new keyword, I needed to ascertain what names were used extensively in existing Python code. Out of random curiosity, I asked my script what names were the most commonly used. The script responded with 21854 names and a total of 297164 references, averaging 13-14 refs per name. A good number of names are referenced just once - set and never referenced. They're there for applications to use. That takes out 6217 names. But I'm more interested in the ones that see a lot of use. By far the most common name is 'self', for obvious reasons; after that, it's a fairly steady drop-off. The most popular names in the standard library are... *drumroll* 45298 - self 3750 - os 3744 - name 3166 - i 3140 - s 2685 - value 2648 - a 2451 - len 2348 - c 2331 - sys 2255 - b 2238 - line 2132 - print 2131 - x Few surprises there. The os and sys modules are used extensively, and short variable names are reused frequently. To the print-detractors: That's two thousand uses in the standard library, more than any other single function bar 'len'! (And by the way, this is ignoring any file with /test/ in the name.) I find the pairing of 'name' and 'value' interesting. There are 40% more names than values in Python, apparently. And on that bombshell, as they say on Top Gear, it's time to end! ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Commonly-used names in the Python standard library
Chris Angelico ros...@gmail.com: In working on a proposal that might result in the creation of a new keyword, I'm looking forward to the day when every application can add its own keywords as is customary in Lisp. I needed to ascertain what names were used extensively in existing Python code One advantage of Perl is that names and keywords are in separate namespaces so introducing new keywords is easy. Should Python have something like: from py35 import * That way, you could choose between: unless x 7: return and: py35.unless x 7: return in case you have already made use of the name unless in your program. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Commonly-used names in the Python standard library
On Thu, Feb 20, 2014 at 9:22 PM, Marko Rauhamaa ma...@pacujo.net wrote: from py35 import * That way, you could choose between: unless x 7: return and: py35.unless x 7: return in case you have already made use of the name unless in your program. What about return? Are you allowed to namespace that? And 'from' and 'import' and '*'? In languages with keywords, they're there to signal things to the parser. There are languages that have no keywords at all, like REXX, and their grammars are usually restricted to non-alphabetic tokens (for instance, REXX has and | instead of and and or). Python already has most of its important names in either builtins (which can be shadowed) or actual modules, so it's only actual language keywords that can't be reused; and there aren't all that many of those. But more can be created, and it's worth being careful. In this instance, various proposals included then, when, use, and raises. My script reported the following: 1 instances of the name 'use' 12 instances of the name 'when' and none of either of the others. Granted, the stdlib isn't everything, and isn't even reliably representative, but that supported my gut feeling that keywording 'when' would be likely to trip code up. If you're curious about the full proposal, it's PEP 463, an expression form of the 'except' statement. The latest draft PEP can be found here: https://raw2.github.com/Rosuav/ExceptExpr/master/pep-0463.txt and the official repo (currently out of date, but later on will be the official and maintained version) has it here: http://www.python.org/dev/peps/pep-0463/ ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Commonly-used names in the Python standard library
Chris Angelico ros...@gmail.com: On Thu, Feb 20, 2014 at 9:22 PM, Marko Rauhamaa ma...@pacujo.net wrote: py35.unless x 7: return What about return? Are you allowed to namespace that? And 'from' and 'import' and '*'? Old keywords are guaranteed not to clash with programs. Introducing new keywords runs that risk. Hence, C had to introduce the ugly _Bool keyword. If you're curious about the full proposal, it's PEP 463, an expression form of the 'except' statement. The latest draft PEP can be found here: https://raw2.github.com/Rosuav/ExceptExpr/master/pep-0463.txt A coworker pointed out that the gist of the PEP has already been implemented by URL: https://pypi.python.org/pypi/fuckit. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Commonly-used names in the Python standard library
On Thu, Feb 20, 2014 at 10:28 PM, Marko Rauhamaa ma...@pacujo.net wrote: A coworker pointed out that the gist of the PEP has already been implemented by URL: https://pypi.python.org/pypi/fuckit. I love how that's categorized Topic :: Software Development :: Quality Assurance. It certainly assures _something_ about quality... ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Commonly-used names in the Python standard library
On Thu, Feb 20, 2014 at 10:28 PM, Marko Rauhamaa ma...@pacujo.net wrote: Chris Angelico ros...@gmail.com: On Thu, Feb 20, 2014 at 9:22 PM, Marko Rauhamaa ma...@pacujo.net wrote: py35.unless x 7: return What about return? Are you allowed to namespace that? And 'from' and 'import' and '*'? Old keywords are guaranteed not to clash with programs. Introducing new keywords runs that risk. Hence, C had to introduce the ugly _Bool keyword. Okay, so what you're saying is that there are three states: Before Python X.Y, the unless keyword simply doesn't exist. (It can't be coded in as a module, so it can't exist until someone implements the code.) From X.Y, it can be called up by importing it from pyAB and used in its namespace. From A.B onward, it always exists. Python has a facility like this. It doesn't namespace the keywords, but it does let you choose whether to have them or not. In Python 2.5, you could type from __future__ import with_statement to turn 'with' into a keyword. After Python 2.6, it's always a keyword. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Commonly-used names in the Python standard library
Chris Angelico ros...@gmail.com: Python has a facility like this. It doesn't namespace the keywords, but it does let you choose whether to have them or not. In Python 2.5, you could type from __future__ import with_statement to turn 'with' into a keyword. After Python 2.6, it's always a keyword. That certainly softens the blow but might still cause unnecessary suffering when maintaining/resurrecting legacy Python code. How about blocking the introduction of new keywords for ever except if you specify: from __py35__ import syntax Eventually, every Python module would likely begin with a statement like that, and it would document the assumption more clearly than __future__. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Commonly-used names in the Python standard library
On Thu, Feb 20, 2014 at 11:09 PM, Marko Rauhamaa ma...@pacujo.net wrote: How about blocking the introduction of new keywords for ever except if you specify: from __py35__ import syntax Eventually, every Python module would likely begin with a statement like that, and it would document the assumption more clearly than __future__. It's more self-documenting with the __future__ directive, because it says *what* syntax you're importing from the future. And at some point, the new keywords must just become standard. There's no point polluting every Python script forever with these directives, and no point maintaining two branches of code in the interpreter. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Commonly-used names in the Python standard library
Chris Angelico ros...@gmail.com: On Thu, Feb 20, 2014 at 11:09 PM, Marko Rauhamaa ma...@pacujo.net wrote: from __py35__ import syntax It's more self-documenting with the __future__ directive, because it says *what* syntax you're importing from the future. As a developer, I will probably want to state the Python dialect that was used to write the module. Each dialect comes with hundreds of features. I don't want to list them individually (even if I could). And at some point, the new keywords must just become standard. That's an explicit program of destroying backwards-compatibility: a war on legacy code. That may be the Python way, but it's not a necessary strategy. There's no point polluting every Python script forever with these directives, and no point maintaining two branches of code in the interpreter. Two branches? I would imagine there would be dozens of branches in the interpreter if the latest interpreter were to support all past Python dialects (as it should, IMO). Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Commonly-used names in the Python standard library
On Thu, Feb 20, 2014 at 11:46 PM, Marko Rauhamaa ma...@pacujo.net wrote: And at some point, the new keywords must just become standard. That's an explicit program of destroying backwards-compatibility: a war on legacy code. That may be the Python way, but it's not a necessary strategy. There's no point polluting every Python script forever with these directives, and no point maintaining two branches of code in the interpreter. Two branches? I would imagine there would be dozens of branches in the interpreter if the latest interpreter were to support all past Python dialects (as it should, IMO). Indeed. If the interpreter were to include every dialect of old Python, then it would have a lot more than two branches. They would, in fact, increase exponentially with every Python version. Fortunately, there is an alternative. You can specify the version of Python like this: #!/usr/local/bin/python3.4 or any of several other ways. You then choose exactly which versions of Python to have installed, and continue to use them for as long as you wish. There's no reason for the 3.4 interpreter to be able to run code as if it were the 3.1 interpreter, when you can just have the 3.1 interpreter itself right there. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Commonly-used names in the Python standard library
Chris Angelico ros...@gmail.com: If the interpreter were to include every dialect of old Python, then it would have a lot more than two branches. They would, in fact, increase exponentially with every Python version. It shouldn't be *that bad*; the linux kernel is grappling with the glut of system calls, but they are managing it reasonably well. I don't see why Python, especially at this mature stage, couldn't adopt a similar stance *going forward*. In fact, not every syntax change requires special backwards-compatibility treatment in the compiler. Constructs that used to be illegal might become legal (say, try-except-finally). They don't require any attention. Even new keywords have a very small impact on the parser; it should be a simple matter of enabling dictionary entries. Fortunately, there is an alternative. You can specify the version of Python like this: #!/usr/local/bin/python3.4 Well, * you won't be finding old Python versions on newer operating system distributions, * even URL: http://www.python.org/downloads/ isn't all that extensive and * the program may import modules that were written in different Python dialects. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Commonly-used names in the Python standard library
On Fri, Feb 21, 2014 at 2:14 AM, Marko Rauhamaa ma...@pacujo.net wrote: * you won't be finding old Python versions on newer operating system distributions, * even URL: http://www.python.org/downloads/ isn't all that extensive and * the program may import modules that were written in different Python dialects. You can always build your own Python, if it really matters... but more likely, if you care about old versions, you actually care about *one specific old version* which your program uses. That's why Red Hat still supports Python 2.4 and, I think, 2.3. You can't randomly pick up 2.2 or 1.5, but if you want 2.4, you can keep on using that for as long as this RHEL is supported. As to importing modules written for other versions... that can be a major problem. Often the new keywords come with new functionality. Take string exceptions, for instance. Say you import a module that was written for a version that still supported them - if it raises a string, you can't catch it. There is a limit to how far the compatibility can be taken. Also, what happens if two modules (one of which might be your script) written for different versions both import some third module? Should they get different versions, based on what version tags they use themselves? Compatibility can't be changed that easily. You either run on the new version, or run on the old. Not both. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Commonly-used names in the Python standard library
Chris Angelico ros...@gmail.com: Also, what happens if two modules (one of which might be your script) written for different versions both import some third module? Should they get different versions, based on what version tags they use themselves? Compatibility can't be changed that easily. You either run on the new version, or run on the old. Not both. Shared C libraries face the exact same issue. Java seems pretty good on this front as well. When there is a will, there is a way. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Commonly-used names in the Python standard library
On Fri, Feb 21, 2014 at 3:26 AM, Marko Rauhamaa ma...@pacujo.net wrote: Chris Angelico ros...@gmail.com: Also, what happens if two modules (one of which might be your script) written for different versions both import some third module? Should they get different versions, based on what version tags they use themselves? Compatibility can't be changed that easily. You either run on the new version, or run on the old. Not both. Shared C libraries face the exact same issue. Java seems pretty good on this front as well. When there is a will, there is a way. Shared C libraries usually do it by linking against a particular version. That's why you often need to keep multiple versions around. Once it's all binary code, there's no more compatibility question - it all runs on the same CPU. With Python code, the module's written to run on a particular interpreter, and that can't just switch around - it's like the weird and wonderful life I enjoyed as 32-bit computing started coming along, and I wanted to call on code that used the other word length... ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Commonly-used names in the Python standard library
On Thu, 20 Feb 2014 14:09:19 +0200, Marko Rauhamaa wrote: Chris Angelico ros...@gmail.com: Python has a facility like this. It doesn't namespace the keywords, but it does let you choose whether to have them or not. In Python 2.5, you could type from __future__ import with_statement to turn 'with' into a keyword. After Python 2.6, it's always a keyword. That certainly softens the blow but might still cause unnecessary suffering when maintaining/resurrecting legacy Python code. How about blocking the introduction of new keywords for ever except if you specify: from __py35__ import syntax Eventually, every Python module would likely begin with a statement like that, and it would document the assumption more clearly than __future__. What *actual* problem is this supposed to solve? Do you often find that Python has introduced new keywords, breaking your code? -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: Commonly-used names in the Python standard library
On Thu, 20 Feb 2014 14:46:35 +0200, Marko Rauhamaa wrote: I would imagine there would be dozens of branches in the interpreter if the latest interpreter were to support all past Python dialects (as it should, IMO). Well thank goodness you're not in charge of Python's future development. That way leads to madness: madness for the core developers (if you think maintaining Python 2 and 3 branches is hard imagine maintaining *dozens* of them, *forever*), madness of the programmers using the language, and madness for anyone trying to learn the language. It's hard enough for newbies to deal with *two* dialects, 2 and 3. And you want to introduce dozens. Thanks, but no thanks. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: Commonly-used names in the Python standard library
On Thu, 20 Feb 2014 20:39:10 +1100, Chris Angelico wrote: In working on a proposal that might result in the creation of a new keyword, I needed to ascertain what names were used extensively in existing Python code. I would love to steal^W see your script for doing this :-) -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: Commonly-used names in the Python standard library
On Thu, 20 Feb 2014 12:22:29 +0200, Marko Rauhamaa wrote: Chris Angelico ros...@gmail.com: In working on a proposal that might result in the creation of a new keyword, I'm looking forward to the day when every application can add its own keywords as is customary in Lisp. And what a wonderful day that will be! Reading any piece of code you didn't write yourself -- or wrote a long time ago -- will be an adventure! Every script will have it's own exciting new set of keywords doing who knows what, which makes every script nearly it's own language! Oh joy, I cannot wait! That's sarcasm, by the way. I needed to ascertain what names were used extensively in existing Python code One advantage of Perl is that names and keywords are in separate namespaces so introducing new keywords is easy. Then I can write code like: for for in in: while while: if if: raise raise which will go a long way to ensuring that my code is an hostile and unreadable as possible. (Sometimes, less can be more. That's especially true of programming languages.) -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: Commonly-used names in the Python standard library
On Fri, Feb 21, 2014 at 5:51 PM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: On Thu, 20 Feb 2014 20:39:10 +1100, Chris Angelico wrote: In working on a proposal that might result in the creation of a new keyword, I needed to ascertain what names were used extensively in existing Python code. I would love to steal^W see your script for doing this :-) No probs! It's part of my ancillary stuff for the PEP 463 research: https://github.com/Rosuav/ExceptExpr/blob/master/find_except_expr.py It basically just runs over one file at a time, parses it into an AST, and walks the tree. Pretty simple. Actually, some of these sorts of things might make neat examples of what can be done with the ast module. Until this week, I had no idea how easy it was to analyze Python code this way. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Commonly-used names in the Python standard library
On Fri, Feb 21, 2014 at 5:59 PM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: Then I can write code like: for for in in: while while: if if: raise raise which will go a long way to ensuring that my code is an hostile and unreadable as possible. REXX allows that. Most people wouldn't use classic keywords like 'if', as that'll only cause confusion (although if if then then; else else is legal), but some of the other keywords are useful in other contexts. The main advantage is that, for instance, the PARSE command can freely use keywords: PARSE VAR x blah blah PARSE VALUE linein(blah) WITH blah blah All those words (parse, var, value, with) are keywords - in that context. But I can happily use var and value elsewhere, and will do so. Python, on the other hand, has to be more careful; so you see things like cls instead of class, or import_ and so on, with the trailing underscore. Trade-offs. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Commonly-used names in the Python standard library
Steven D'Aprano steve+comp.lang.pyt...@pearwood.info: On Thu, 20 Feb 2014 12:22:29 +0200, Marko Rauhamaa wrote: I'm looking forward to the day when every application can add its own keywords as is customary in Lisp. And what a wonderful day that will be! Reading any piece of code you didn't write yourself -- or wrote a long time ago -- will be an adventure! Every script will have it's own exciting new set of keywords doing who knows what, which makes every script nearly it's own language! Oh joy, I cannot wait! That's sarcasm, by the way. I don't hear Lispers or C programmers complaining. Yes, you can shoot yourself in the foot with macro trickery, but macros can greatly enhance code readability and remove the need for code generators. That's why they are used extensively in Linux kernel code and GOOPS (Guile's object system), for example. One advantage of Perl is that names and keywords are in separate namespaces so introducing new keywords is easy. Then I can write code like: for for in in: while while: if if: raise raise which will go a long way to ensuring that my code is an hostile and unreadable as possible. Perl does that by forcing you to prefix you variables with $ et al. The analogy would be: for $for in @in: while $while: if $if: raise $raise I'm not saying I like the look of that, but it does have a distinct advantage in introducing new keywords. Marko -- https://mail.python.org/mailman/listinfo/python-list
