Re: Why no warnings when re-assigning builtin names?
(I realize this thread is old. I have been away for a few weeks. I read through the whole thread, though, and did not see anyone bring up this one particular point: there is already a linting script that handles this.) On Mon, Aug 15, 2011 at 10:52 PM, Gerrat Rickert grick...@coldstorage.com wrote: With surprising regularity, I see program postings (eg. on StackOverflow) from inexperienced Python users accidentally re-assigning built-in names. For example, they'll innocently call some variable, `list', and assign a list of items to it. In article mailman.22.1313446504.27778.python-l...@python.org Chris Angelico ros...@gmail.com wrote: It's actually masking, not reassigning. That may make it easier or harder to resolve the issue. If you want a future directive that deals with it, I'd do it the other way - from __future__ import mask_builtin_warning or something - so the default remains as it currently is. But this may be a better job for a linting script. The pylint program already does this: $ cat shado.py module doc def func(list): func doc return list $ pylint shado.py * Module shado W0622: 2:func: Redefining built-in 'list' ... Your code has been rated at 6.67/10 If your shadowing is done on purpose, you can put in a pylint comment directive to suppress the warning. Pylint is the American Express Card of Python coding: don't leave $HOME without it! :-) -- In-Real-Life: Chris Torek, Wind River Systems Intel require I note that my opinions are not those of WRS or Intel Salt Lake City, UT, USA (40°39.22'N, 111°50.29'W) +1 801 277 2603 email: gmail (figure it out) http://web.torek.net/torek/index.html -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On 2011-08-31, Chris Torek nos...@torek.net wrote: (I realize this thread is old. I have been away for a few weeks. I read through the whole thread, though, and did not see anyone bring up this one particular point: there is already a linting script that handles this.) Yes. I've found pylint... A weird mix of very helpful, thanks and oh, come off it. A thread about pylint is where I got my example of the natural Python way to express a parabola: theValueRepresentingTheYAxisLocationOfThePoint = theValueRepresentingTheXAxisLocationOfThe Point * theValueRepresentingTheXAxisLocationOfThe Point I still say that there are times when short names are natural and idiomatic, and much clearer than long names. :P But I do think that, given the basic assumption that pylint is a core tool for vetting code, it is probably adequate for it to provide the warnings. -s -- Copyright 2011, all wrongs reversed. Peter Seebach / usenet-nos...@seebs.net http://www.seebs.net/log/ -- lawsuits, religion, and funny pictures http://en.wikipedia.org/wiki/Fair_Game_(Scientology) -- get educated! I am not speaking for my employer, although they do rent some of my opinions. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On Wed, Aug 17, 2011 at 2:35 AM, Seebs usenet-nos...@seebs.net wrote: On 2011-08-17, Chris Angelico ros...@gmail.com wrote: It mightn't be very significant, but there'd still be some cost. However, IMHO the greatest cost is the spamminess; forcing the user to deal with lines and lines of warnings is not a useful way to design a language. Lines and lines? I'd say if it's to be allowed to shadow built-ins (and I'm not sure that's a good thing at all), you'd still be looking at one warning per shadowing, which shouldn't be too many. One warning per shadowing. Okay. def foo(list): Foo's the list provided and returns True on success or False on failure. def bar(list): Counts the number of bars in the list, assuming it to be made of music. if not foo(list): return You call foo() once and bar() twice. How many shadowings have there been? How many warnings do you get? A simple implementation would give five warnings for this case - once for each invocation that shadows a builtin. Another simple implementation would give two warnings, at the time that the def statements are executed; this is preferred, but it's still two warnings, and if you have a huge set of functions that do this, that can easily be lines and lines of warnings. Or should it set a flag and say I've already warned this session about shadowing 'list', so suppress all others? That seems confusing. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On Wed, Aug 17, 2011 at 1:58 AM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: My thoughts would be: 1. It's hard to avoid shadowing anything unless you know the entire language and never forget things. Define the entire language. Does that include the names of all the plethora of exceptions? How about the standard library? The shadowing issue applies to the standard library as well as the builtins, so yes; to avoid shadowing *anything*, you would have to know the entire language. I posit that this is a practical impossibility, and that unexpected shadowing will always be possible (and won't always be prevented by syntax highlighting). Some day you'll discover that you can't use module X because you have a function called X, and you'll have to rename. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On Wed, 17 Aug 2011 01:17 pm Seebs wrote: [...] Another scope is normally a horizontal thing -- you're talking about a different scope such that you are *either* in this one *or* in that one. Built-ins are not in a scope you are never not in. That's technically incorrect. Built-ins are a scope you are never in, if by in you mean code is executed in this scope. You have three scopes in general: Local Global Built-ins (There's also zero or more nonlocal scopes, between local and global, that applies to closures and nested functions, but never mind that.) Code is almost(?) always executed in the local scope. (eval and exec let you mess around with that, somewhat.) E.g. an assignment x = 1 applies to the local namespace unless you explicitly declare it global. If you are in the top level of a module, the local namespace is also the global one, the global statement does nothing, and the assignment occurs in the global namespace. However, name lookup rules are such that while assignments are always local, unsuccessful lookups may fall through to global then built-in. See also: http://www.python.org/dev/peps/pep-0227/ The only way to put something into the built-in namespace is by using the fully-qualified name: import builtins # Use __builtin__ in Python 2 builtins.x = 1 # Kids! Don't try this at home! x 1 del x # prove it isn't a global or local Traceback (most recent call last): File stdin, line 1, in module NameError: name 'x' is not defined So a top-level (global) assignment, say: def sum(list): ... *shadows* the built-ins sum and list, it doesn't replace them. It defines a local variable list and a global variable sum, but it doesn't touch either built-in. They are still available via the fully qualified name builtins.sum and builtins.list. [...] (4) And then, after a while, you decide that perhaps shadowing is not always so bad. (It helps if the Effbot tells you off for objecting to shadowing in a two-line function.) At first, you limit yourself to using parameter names that shadow built-ins in small tool-like functions. Then when the world doesn't implode, you rethink the use of top-level function names like izip and realise that namespaces exist so that you don't need to care about shadowing functions you don't use, and if people call import * they have nobody to blame but themselves if things break. Hmm. See, I've never reached that, in Python or any other language. I figure it creates a new potential for confusion, and that I would rather avoid any ambiguity. I don't *like* ambiguity in code. Ah, well you see the thing is, this is Python. As soon as you call any function you don't control, you no longer know what your environment is with any certainty. For all you know, the harmless-looking function is monkey-patching builtins like there's no tomorrow. Speaking broadly, perhaps too broadly, we're incredibly proud of the ability to modify nearly everything at runtime. Fortunately, while we are proud of having that ability, actually *using* it is considered a mortal sin. We're not Ruby developers -- if you actually monkey-patch something, especially built-ins, you can expect to be taken outside and slapped around with a fish if you get caught. http://www.youtube.com/watch?v=IhJQp-q1Y1s So I don't shadow stuff that's part of the language, because doing that makes it possible for a line of code to have a clear and obvious meaning to someone who looks at that line out of context, and a *completely different* clear and obvious meaning to someone who looks at it with a bit more context. I don't like that! It means that someone reading my code can never, ever, assume that a standard language feature is actually itself and not a local shadow which does something different unless they go walking back up the scope chain checking. And that's a pretty big cost to attach to stuff that is, by design, basic and universally available. Sure. But they can't have that certainty regardless of whether you shadow something, because how do they know whether you've shadowed it or not? In theory, anything could be anything at any time, and you have no protection. In practice, I worry more about being eaten by genetically-engineered flying piranhas than about rogue monkey-patching code. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
RE: Why no warnings when re-assigning builtin names?
On 8/16/2011 7:29 PM, Terry Reedy wrote: On 8/16/2011 1:15 PM, Gerrat Rickert wrote: I think that best practices would suggest that one shouldn't use variable names that shadow builtins (except in specific, special circumstances), so I don't really think this would be an annoyance at all. The number of *unwanted* warnings they'd get would be pretty close to zero. OTOH, in response to a question I asked on StackOverflow, someone posted a large list of times where this isn't followed in the std lib, so there seems to be a precedent for just using the builtin names for anything one feels like at the time. If you run across that again and email me the link, I will take a look and see if I think the issue should be raised on pydev. Of course, some modules *intentionally* define an open function, intended to be accessed as 'mod.open' and not as 'from mod import *; open'. Also, class/instance attributes can also reuse builtin names. But 'open = True/False' would be bad. -- Terry Jan Reedy See the accepted answer to this question: http://stackoverflow.com/questions/7079519/is-there-python-code-in-the-python-standard-library-that-uses-variable-names-that -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On 2011-08-17, Chris Angelico ros...@gmail.com wrote: def foo(list): Foo's the list provided and returns True on success or False on failure. def bar(list): Counts the number of bars in the list, assuming it to be made of music. if not foo(list): return You call foo() once and bar() twice. How many shadowings have there been? How many warnings do you get? I'd say two, one when def foo... is parsed, one when def bar... is parsed. A simple implementation would give five warnings for this case - once for each invocation that shadows a builtin. Another simple implementation would give two warnings, at the time that the def statements are executed; this is preferred, but it's still two warnings, and if you have a huge set of functions that do this, that can easily be lines and lines of warnings. Or should it set a flag and say I've already warned this session about shadowing 'list', so suppress all others? That seems confusing. I guess I don't object to multiple warnings if I do the same thing multiple times. I was just thinking in terms of a single parse-time warning for the actual point at which something is shadowed, rather than, say, a warning every time a name is hit during execution of statements and refers to a shadow. -s -- Copyright 2011, all wrongs reversed. Peter Seebach / usenet-nos...@seebs.net http://www.seebs.net/log/ -- lawsuits, religion, and funny pictures http://en.wikipedia.org/wiki/Fair_Game_(Scientology) -- get educated! I am not speaking for my employer, although they do rent some of my opinions. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On 2011-08-17, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: On Wed, 17 Aug 2011 01:17 pm Seebs wrote: [...] Another scope is normally a horizontal thing -- you're talking about a different scope such that you are *either* in this one *or* in that one. Built-ins are not in a scope you are never not in. That's technically incorrect. Built-ins are a scope you are never in, if by in you mean code is executed in this scope. No, by in I mean lookups from your code will reach this scope if they don't find something sooner. Hmm. See, I've never reached that, in Python or any other language. I figure it creates a new potential for confusion, and that I would rather avoid any ambiguity. I don't *like* ambiguity in code. Ah, well you see the thing is, this is Python. As soon as you call any function you don't control, you no longer know what your environment is with any certainty. For all you know, the harmless-looking function is monkey-patching builtins like there's no tomorrow. Speaking broadly, perhaps too broadly, we're incredibly proud of the ability to modify nearly everything at runtime. Heh. Fortunately, while we are proud of having that ability, actually *using* it is considered a mortal sin. We're not Ruby developers -- if you actually monkey-patch something, especially built-ins, you can expect to be taken outside and slapped around with a fish if you get caught. Okay, so. Here's what I don't get. If it's such a bad thing, *why is it allowed*? Why are you proud of the ability to do something that you are never socially-allowed to do? I have mixed feelings about Ruby's general tolerance for monkey-patching, although I've seen it do some pretty awesome things, so I am somewhat inclined to accept that it may be worth it. But it's clearly a feature which is used intentionally. It sounds to me like Python is very proud of having a feature which no one would ever use. ... Why? Sure. But they can't have that certainty regardless of whether you shadow something, because how do they know whether you've shadowed it or not? Well, they could trust me. :) In theory, anything could be anything at any time, and you have no protection. In practice, I worry more about being eaten by genetically-engineered flying piranhas than about rogue monkey-patching code. I do too, if I know that people don't shadow built-ins. If I know that people are shadowing built-ins, then some of the time, when I'm editing other peoples' code, they HAVE effectively monkey-patched it. -s -- Copyright 2011, all wrongs reversed. Peter Seebach / usenet-nos...@seebs.net http://www.seebs.net/log/ -- lawsuits, religion, and funny pictures http://en.wikipedia.org/wiki/Fair_Game_(Scientology) -- get educated! I am not speaking for my employer, although they do rent some of my opinions. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
Seebs wrote: On 2011-08-17, Steven D'Aprano wrote: On Wed, 17 Aug 2011 01:17 pm Seebs wrote: Hmm. See, I've never reached that, in Python or any other language. I figure it creates a new potential for confusion, and that I would rather avoid any ambiguity. I don't *like* ambiguity in code. Ah, well you see the thing is, this is Python. As soon as you call any function you don't control, you no longer know what your environment is with any certainty. For all you know, the harmless-looking function is monkey-patching builtins like there's no tomorrow. Speaking broadly, perhaps too broadly, we're incredibly proud of the ability to modify nearly everything at runtime. Heh. Fortunately, while we are proud of having that ability, actually *using* it is considered a mortal sin. We're not Ruby developers -- if you actually monkey-patch something, especially built-ins, you can expect to be taken outside and slapped around with a fish if you get caught. Okay, so. Here's what I don't get. If it's such a bad thing, *why is it allowed*? Why are you proud of the ability to do something that you are never socially-allowed to do? Monkey-patching built-ins would be something along the lines of import sys sys.modules['__builtin__'].str = my_super_string and is what stands you in jeopardy of being fish-slapped. ;) Merely shadowing a built-in, or stdlib, or whatever, isn't monkey-patching. ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On 2011-08-17, Ethan Furman et...@stoneleaf.us wrote: Seebs wrote: On 2011-08-17, Steven D'Aprano wrote: Ah, well you see the thing is, this is Python. As soon as you call any function you don't control, you no longer know what your environment is with any certainty. For all you know, the harmless-looking function is monkey-patching builtins like there's no tomorrow. Speaking broadly, perhaps too broadly, we're incredibly proud of the ability to modify nearly everything at runtime. Fortunately, while we are proud of having that ability, actually *using* it is considered a mortal sin. We're not Ruby developers -- if you actually monkey-patch something, especially built-ins, you can expect to be taken outside and slapped around with a fish if you get caught. Here's what I don't get. If it's such a bad thing, *why is it allowed*? Why are you proud of the ability to do something that you are never socially-allowed to do? Monkey-patching built-ins would be something along the lines of import sys sys.modules['__builtin__'].str = my_super_string and is what stands you in jeopardy of being fish-slapped. ;) Merely shadowing a built-in, or stdlib, or whatever, isn't monkey-patching. Oh, I know. I was just noticing that Steven's post is specifically talking about how Python users are proud of having the ability to monkey-patch. If monkey-patching like that is a mortal sin, leads to fish-slapping, and so on.. Why is it possible? Why not just reject such things as invalid code? -s -- Copyright 2011, all wrongs reversed. Peter Seebach / usenet-nos...@seebs.net http://www.seebs.net/log/ -- lawsuits, religion, and funny pictures http://en.wikipedia.org/wiki/Fair_Game_(Scientology) -- get educated! I am not speaking for my employer, although they do rent some of my opinions. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On Wed, Aug 17, 2011 at 5:33 PM, Seebs usenet-nos...@seebs.net wrote: If it's such a bad thing, *why is it allowed*? Why are you proud of the ability to do something that you are never socially-allowed to do? Going back to my original three examples: 1) Deliberate shadowing because you want to change the behavior of the name. Extremely rare. 2) Shadowing simply by using the name of an unusual builtin (like 'file') in a context where you never use it. Very common. 3) Unintentional shadowing where you create a variable, but then intend to use the builtin. This is the only one that's a problem. All three are allowed, but it's the first one that's considered unusual. The second one is simply that Python doesn't have a million and one reserved words. Yes, you probably don't want to use 'print' as a variable name, but shadowing it with an exact equivalent would be fine (eg to automatically date-stamp or log your output, without changing your code). And as described above, using list/str/id etc is not uncommon. I greatly prefer this to the alternative, which is another 133 reserved words (based on Python 3.2 for Windows). ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On 2011-08-17, Chris Angelico ros...@gmail.com wrote: On Wed, Aug 17, 2011 at 5:33 PM, Seebs usenet-nos...@seebs.net wrote: If it's such a bad thing, *why is it allowed*? ?Why are you proud of the ability to do something that you are never socially-allowed to do? Going back to my original three examples: I may have been unclear about jumping topics; that comment was about monkey-patching, not about shadowing. I greatly prefer this to the alternative, which is another 133 reserved words (based on Python 3.2 for Windows). You have a point there. I guess I'm just used to the assumption that the confusion caused by shadowing names outweighs the benefits of using them -- the world is rich with plausible names. -s -- Copyright 2011, all wrongs reversed. Peter Seebach / usenet-nos...@seebs.net http://www.seebs.net/log/ -- lawsuits, religion, and funny pictures http://en.wikipedia.org/wiki/Fair_Game_(Scientology) -- get educated! I am not speaking for my employer, although they do rent some of my opinions. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
Seebs wrote: On 2011-08-17, Ethan Furman et...@stoneleaf.us wrote: Seebs wrote: On 2011-08-17, Steven D'Aprano wrote: Ah, well you see the thing is, this is Python. As soon as you call any function you don't control, you no longer know what your environment is with any certainty. For all you know, the harmless-looking function is monkey-patching builtins like there's no tomorrow. Speaking broadly, perhaps too broadly, we're incredibly proud of the ability to modify nearly everything at runtime. Fortunately, while we are proud of having that ability, actually *using* it is considered a mortal sin. We're not Ruby developers -- if you actually monkey-patch something, especially built-ins, you can expect to be taken outside and slapped around with a fish if you get caught. Here's what I don't get. If it's such a bad thing, *why is it allowed*? Why are you proud of the ability to do something that you are never socially-allowed to do? Monkey-patching built-ins would be something along the lines of import sys sys.modules['__builtin__'].str = my_super_string and is what stands you in jeopardy of being fish-slapped. ;) Merely shadowing a built-in, or stdlib, or whatever, isn't monkey-patching. Oh, I know. I was just noticing that Steven's post is specifically talking about how Python users are proud of having the ability to monkey-patch. If monkey-patching like that is a mortal sin, leads to fish-slapping, and so on.. Why is it possible? Why not just reject such things as invalid code? Well, the mortal sin part is a bit of an exaggeration -- it's more like you'd better have a really darn good reason to do it. And it is absolutely one of my favorite parts about Python. If I want to inject a custom Path class into builtins so it's readily available, and then change os.listdir to return it instead of normal strings, I can. If my application is truly case-insensitive, I can make my own istr class and monkey-patch builtins so it's what is used. Can this blow-up in my face? Certainly. But I would rather have the option open to me instead of being told No, I'm sorry, you can't do that because I (developers in question) didn't imagine a good use case for it. Part of the fun of Python is experimentation. And how much fun is it to be told over and over, No, you can't do that? As an example of something that could easily have been outlawed, but wasn't, check out http://stackoverflow.com/questions/7068340 ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On Wed, Aug 17, 2011 at 6:38 PM, Seebs usenet-nos...@seebs.net wrote: I may have been unclear about jumping topics; that comment was about monkey-patching, not about shadowing. Ah, apologies. Monkey-patching is a way of using the middle of a stack of code without using what's below it. If everything is statically linked, then every function does exactly what it was written to do and no more; if it accepts a file object as an argument, you could give it some other file-like object, but that's all. However, if the function was written to use print() and you don't want it to print to the screen, what do you do? Obviously it's your responsibility to ensure that your replacement is 100% equivalent in the required functionality, but it's a great flexibility to simply change the meaning of print; the alternative is to copy and paste the code, make one change (or even no change at all, if your replacement print() is a global), and run that. As Ethan posted while I was typing this, you're allowed to do it, but you just need to have a good reason for it. I like to explain/justify myself in comments when these things crop up in my code. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On 2011-08-17, Ethan Furman et...@stoneleaf.us wrote: Part of the fun of Python is experimentation. And how much fun is it to be told over and over, No, you can't do that? Okay, I buy that. Actually, this sort of fits with my experience of how (sane) people do it in Ruby. And I'm really the wrong person to criticize people for monkey-patching: http://www.yoctoproject.org/projects/pseudo (In which I monkeypatch *the entire Unix filesystem API*. On Linux and OS X. For C programs.) I guess maybe my question shouldn't have been why is it allowed, but and why is it bad to use it? It just seemed like you should never do this, it's horrible and we're proud of being able to do this were not entirely compatible. -s -- Copyright 2011, all wrongs reversed. Peter Seebach / usenet-nos...@seebs.net http://www.seebs.net/log/ -- lawsuits, religion, and funny pictures http://en.wikipedia.org/wiki/Fair_Game_(Scientology) -- get educated! I am not speaking for my employer, although they do rent some of my opinions. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
Seebs wrote: On 2011-08-17, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: Fortunately, while we are proud of having that ability, actually *using* it is considered a mortal sin. We're not Ruby developers -- if you actually monkey-patch something, especially built-ins, you can expect to be taken outside and slapped around with a fish if you get caught. Okay, so. Here's what I don't get. If it's such a bad thing, *why is it allowed*? Why are you proud of the ability to do something that you are never socially-allowed to do? Why does any language allow monkey-patching? Because: (1) it is a consequence of a clean language design that doesn't special case things unless absolutely necessary -- monkey-patching wasn't added to the language, it just emerged given the basic language design; (2) consequently it isn't some special technique, it is just a special name given to ordinary, humdrum, everyday things like name-binding within a namespace; (3) and yet it is a powerful and useful ability that lets you extend both the language and libraries (yours or third party) while still writing very clean code; (4) it's also pretty cool that you can do these things; and most importantly (5) all of the above. Even if we shouldn't (ab)use it in production, it is still greatly useful for quick and dirty scripts, testing, experimentation and debugging. And sometimes monkey-patches end up in production. For example, the standard library site.py file adds help() and quit() commands to builtins at startup. Or you might grab an instance of some third-party class, and dynamically adding a method or an attribute to it. Why bother sub-classing it? There's a scene in James Clavell's Shogun which is relevant. Toranaga, the Japanese warlord, discovers that the Englishman John Blackthorne has betrayed his rightful ruler. Blackthorne protests that there are mitigating circumstances. Toranaga says that there can never be any mitigating circumstances for disloyalty to one's liege-lord. Blackthorne replies that there is one: if you win. The same applies for monkey-patching and other dangerous techniques. There can be no excuse for doing something like that in production... unless it is better than the alternatives. [...] It sounds to me like Python is very proud of having a feature which no one would ever use. ... Why? Don't get me wrong, there are plenty of people who would give up Python's extreme dynamicism is a heartbeat, if it were up to them. It does play havoc with the ability to optimise Python code, and is one of the reasons why CPython isn't as fast as (say) Lua. But the PyPy people are well on the way to blunting that criticism, with a fast, effective JIT optimising compiler that will be fast in the common case and no worse off in the rare times that built-in functions have been shadowed or modified. (PyPy is already about twice as fast as CPython, and in a few carefully crafted examples faster than C code.) -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On Tue, Aug 16, 2011 at 2:32 AM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: On Tue, 16 Aug 2011 08:15 am Chris Angelico wrote: It's actually masking, not reassigning. That may make it easier or harder to resolve the issue. The usual term is shadowing builtins, and it's a feature, not a bug :) Sorry, shadowing. And yeah, that's why I said that fixing this issue was the domain of linting scripts. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On Aug 16, 2011, at 1:15 AM, Steven D'Aprano wrote: On Tue, 16 Aug 2011 01:23 pm Philip Semanchuk wrote: On Aug 15, 2011, at 9:32 PM, Steven D'Aprano wrote: On Tue, 16 Aug 2011 08:15 am Chris Angelico wrote: If you want a future directive that deals with it, I'd do it the other way - from __future__ import mask_builtin_warning or something - so the default remains as it currently is. But this may be a better job for a linting script. Agreed. It's a style issue, nothing else. There's nothing worse about: def spam(list): pass compared to class thingy: pass def spam(thingy): pass Why should built-ins be treated as more sacred than your own objects? Because built-ins are described in the official documentation as having a specific behavior, while my objects are not. *My* objects certainly are, because I write documentation for my code. My docs are no less official than Python's docs. I'm sure they are no less official to you. But you are you, and then there's...everyone else. =) I (and I think most people) give far more credibility to the Python docs than to the documentation of an individual. That's not a reflection on you, it reflects the limits of one person's ability versus organizationally produced docs which are heavily used, discussed, and have been iteratively developed over many years. Sometimes shadowing is safe, sometimes it isn't. Sometimes X is safe and sometimes it isn't can be said of many, many things, from taking a walk down the street to juggling with knives. But it has little to do with whether or not Python should issue a warning in the specific case we're talking about. A warning that is off by default won't help the people who need it, because they don't know enough to turn the warning on. I agree that it wouldn't help the people who need it most (absolute raw newcomers). But you're asserting that once one learned the incantation to enable the theoretical warning we're discussing, one would have graduated to a level where it's no longer useful. That's not the case. There's a lot of ground to cover between newcomer who has learned about a particular warning and coder who regularly shadows builtins on purpose. I am an example. I know enough to turn the theoretical warning on, and I would if I could. I have never shadowed a builtin deliberately. I've done it accidentally plenty of times. There are 84 builtins in my version of Python and I don't have them all memorized. The fact that my editor colors them differently is the only thing I have to back up my leaky memory. Not all editors are so gracious. Yes, it can be useful to replace some of the builtins with one's own implementation, and yes, doing so fits in with Python's we're all consenting adults philosophy. But replacing (shadowing, masking -- call it what you will) builtins is not everyday practice. On the contrary, as the OP Gerrat pointed out, it's most often done unwittingly by newcomers to the language who have no idea that they've done anything out of the ordinary or potentially confusing. Protecting n00bs from their own errors is an admirable aim, but have you considered that warnings for something which may be harmless could do more harm than good? Isn't the whole point of a warning to highlight behavior that's not strictly wrong but looks iffy? Sort of, I can't be sure, but this looks like trouble to me. I hope you know what you're doing. If we are to eschew warnings in cases where they might be highlighting something harmless, then we would have no warnings at all. Again, shadowing builtins is not everyday practice. I have been trying to remember if I've ever seen it done deliberately, and I can't remember a case. Now, a comment like that is an invitation for people come out of the woodwork with cases where they found it useful, and I would welcome some examples as I'm sure they'd be interesting. But I think it's safe to say that if you look at random samples of code, builtins are shadowed unintentionally hundreds of times for every time they're shadowed deliberately and usefully. If a language feature is most often invoked accidentally without knowledge of or regard for its potential negative consequences, then it might be worth making it easier to avoid those accidents. Perhaps. But I'm not so sure it is worth the cost of extra code to detect shadowing and raise a warning. After all, the average coder probably never shadows anything, One need look no further than the standard library to see a strong counterexample. grep through the Python source for file =. I see dozens of examples of this builtin being used as a common variable name. I would call contributors to the standard library above-average coders, and we can see them unintentionally shadowing builtins many times. and for those that do, once they get bitten *once* they either never do it again or learn how to shadow safely. I have done
Re: Why no warnings when re-assigning builtin names?
On Tue, Aug 16, 2011 at 3:13 PM, Philip Semanchuk phi...@semanchuk.com wrote: I am an example. I know enough to turn the theoretical warning on, and I would if I could. I have never shadowed a builtin deliberately. I've done it accidentally plenty of times. There are 84 builtins in my version of Python and I don't have them all memorized. The fact that my editor colors them differently is the only thing I have to back up my leaky memory. Not all editors are so gracious. Rather than turn a warning on you can run your code through a linting script. There are several excellent ones. Add it to your makefile or test suite; then you get the testing done over _all_ of your code, instead of waiting until the moment when you actually execute it. One need look no further than the standard library to see a strong counterexample. grep through the Python source for file =. I see dozens of examples of this builtin being used as a common variable name. I would call contributors to the standard library above-average coders, and we can see them unintentionally shadowing builtins many times. There are several types of shadowing: 1) Deliberate shadowing because you want to change the behavior of the name. Extremely rare. 2) Shadowing simply by using the name of an unusual builtin (like 'file') in a context where you never use it. Very common. 3) Unintentional shadowing where you create a variable, but then intend to use the builtin. This is the only one that's a problem. list = [...] is not a problem unless you later on use foo = list(map(...)) which is more common in Python 3 than Python 2, but fortunately, it'll throw a nice quick error - nobody's going to use list operations on the normal 'list' type, nor is anybody going to call an instance of list. Definitely a job for linting. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
Philip Semanchuk wrote: On Aug 16, 2011, at 1:15 AM, Steven D'Aprano wrote: Protecting n00bs from their own errors is an admirable aim, but have you considered that warnings for something which may be harmless could do more harm than good? Isn't the whole point of a warning to highlight behavior that's not strictly wrong but looks iffy? Sort of, I can't be sure, but this looks like trouble to me. I hope you know what you're doing. If we are to eschew warnings in cases where they might be highlighting something harmless, then we would have no warnings at all. Sounds good to me. ;) Keep such things in the IDE's, and then those who desire such behavior can have it there. Do not clutter Python with such. Perhaps. But I'm not so sure it is worth the cost of extra code to detect shadowing and raise a warning. After all, the average coder probably never shadows anything, One need look no further than the standard library to see a strong counterexample. grep through the Python source for file =. I see dozens of examples of this builtin being used as a common variable name. I would call contributors to the standard library above-average coders, and we can see them unintentionally shadowing builtins many times. What makes you think it's unintentional? file makes a good variable name, and if you don't need it to actually open a file there's nothing wrong with using it yourself. and for those that do, once they get bitten *once* they either never do it again or learn how to shadow safely. I have done it plenty of times, never been bitten (thankfully) and still do it by accident now and again. Seems to me the real issue is somebody using a builtin, such as str or int, and that they somehow manage to do this without realizing, wait a sec', that's one of my variables! I don't see that as a problem that Python needs to solve. ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On Aug 16, 2011, at 11:12 AM, Chris Angelico wrote: On Tue, Aug 16, 2011 at 3:13 PM, Philip Semanchuk phi...@semanchuk.com wrote: One need look no further than the standard library to see a strong counterexample. grep through the Python source for file =. I see dozens of examples of this builtin being used as a common variable name. I would call contributors to the standard library above-average coders, and we can see them unintentionally shadowing builtins many times. There are several types of shadowing: 1) Deliberate shadowing because you want to change the behavior of the name. Extremely rare. 2) Shadowing simply by using the name of an unusual builtin (like 'file') in a context where you never use it. Very common. 3) Unintentional shadowing where you create a variable, but then intend to use the builtin. This is the only one that's a problem. Yes, but before you get to #3 you have to go through #2. The way I see it, #2 is setting a trap, #3 is actually stepping in it. I don't want to do either. Neither do I like working with code that has set trap #2 for me. Cheers Philip -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On Aug 16, 2011, at 11:41 AM, Ethan Furman wrote: Philip Semanchuk wrote: On Aug 16, 2011, at 1:15 AM, Steven D'Aprano wrote: Protecting n00bs from their own errors is an admirable aim, but have you considered that warnings for something which may be harmless could do more harm than good? Isn't the whole point of a warning to highlight behavior that's not strictly wrong but looks iffy? Sort of, I can't be sure, but this looks like trouble to me. I hope you know what you're doing. If we are to eschew warnings in cases where they might be highlighting something harmless, then we would have no warnings at all. Sounds good to me. ;) Keep such things in the IDE's, and then those who desire such behavior can have it there. Do not clutter Python with such. You wink, yet you sound serious. What's with the mixed message? Do you honestly advocate removing all warnings from Python, or not? I sincerely would like to know what you think. Perhaps. But I'm not so sure it is worth the cost of extra code to detect shadowing and raise a warning. After all, the average coder probably never shadows anything, One need look no further than the standard library to see a strong counterexample. grep through the Python source for file =. I see dozens of examples of this builtin being used as a common variable name. I would call contributors to the standard library above-average coders, and we can see them unintentionally shadowing builtins many times. What makes you think it's unintentional? file makes a good variable name, and if you don't need it to actually open a file there's nothing wrong with using it yourself. Unintentional as in, I'm using file as a variable name because it's handy as opposed to intentional as in Yes, I am deliberately changing the meaning of this builtin. and for those that do, once they get bitten *once* they either never do it again or learn how to shadow safely. I have done it plenty of times, never been bitten (thankfully) and still do it by accident now and again. Seems to me the real issue is somebody using a builtin, such as str or int, and that they somehow manage to do this without realizing, wait a sec', that's one of my variables! Yes I don't see that as a problem that Python needs to solve. need is a strong word. Python will be fine regardless of whether this changes or not. I believe Python could be improved; that's all I'm arguing. Cheers Philip -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
Philip Semanchuk wrote: On Aug 16, 2011, at 11:41 AM, Ethan Furman wrote: Philip Semanchuk wrote: If we are to eschew warnings in cases where they might be highlighting something harmless, then we would have no warnings at all. Sounds good to me. ;) Keep such things in the IDE's, and then those who desire such behavior can have it there. Do not clutter Python with such. You wink, yet you sound serious. The smiley is an attempt to not sound harsh. What's with the mixed message? Do you honestly advocate removing all warnings from Python, or not? I sincerely would like to know what you think. I think warnings should be reserved for language changes and such (like DeprecationWarning, RuntimeWarning, and FutureWarning), not for possible programmer mistakes. What makes you think it's unintentional? file makes a good variable name... Unintentional as in, I'm using file as a variable name because it's handy as opposed to intentional as in Yes, I am deliberately changing the meaning of this builtin. That's not what 'unintentional' means. Further, there's no way to tell whether it was or was not from the code alone. Unless it caused a bug, in which case I'd be willing to call it unintentional. ;) I don't see that as a problem that Python needs to solve. need is a strong word. Python will be fine regardless of whether this changes or not. I believe Python could be improved; that's all I'm arguing. Python can be improved -- I don't see 'hand-holding' as an improvement. IDEs and lints can do this. ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On 08/16/2011 10:31 AM, Philip Semanchuk wrote: On Aug 16, 2011, at 11:12 AM, Chris Angelico wrote: There are several types of shadowing: 1) Deliberate shadowing because you want to change the behavior of the name. Extremely rare. 2) Shadowing simply by using the name of an unusual builtin (like 'file') in a context where you never use it. Very common. 3) Unintentional shadowing where you create a variable, but then intend to use the builtin. This is the only one that's a problem. Yes, but before you get to #3 you have to go through #2. The way I see it, #2 is setting a trap, #3 is actually stepping in it. I don't want to do either. Neither do I like working with code that has set trap #2 for me. Chris succinctly described the times I've done shadowing. Fortunately, the shadowing done in #3 (which you appropriately describe as being a superset of #2) is fairly remedied with most editors...since it usually occurs when you have oh, I accidentally shadowed builtin X, so you just do a global search-and-replace for all those places you shadowed X and rename it to something like my_X and proceed to use X as the builtin. The bigger issue I have is module shadowing which is trickier to catch and produces weird symptoms (i.e. cryptic errors). The most common one I see is creating a local module called email.py and then having issues when trying to use standard-library email calls which find your local email.py before they find the email.py file in the standard library. I actually wrote a tool to scan for duplicate modules in $PYTHONPATH (pretty dumb tool, could easily have broken on things like zipfile imports, DLLs, etc), but it made diagnosing the issue easier. -tkc -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On Aug 16, 2011, at 12:19 PM, Ethan Furman wrote: Philip Semanchuk wrote: On Aug 16, 2011, at 11:41 AM, Ethan Furman wrote: Philip Semanchuk wrote: If we are to eschew warnings in cases where they might be highlighting something harmless, then we would have no warnings at all. Sounds good to me. ;) Keep such things in the IDE's, and then those who desire such behavior can have it there. Do not clutter Python with such. You wink, yet you sound serious. The smiley is an attempt to not sound harsh. Thanks. It's hard to know on the Internet. I don't see that as a problem that Python needs to solve. need is a strong word. Python will be fine regardless of whether this changes or not. I believe Python could be improved; that's all I'm arguing. Python can be improved -- I don't see 'hand-holding' as an improvement. IDEs and lints can do this. When you say hand-holding, I hear a pejorative. That makes I don't see 'hand-holding' as an improvement a tautology. Have I misheard you? I think Python does lots of beneficial hand-holding. Garbage collection is a good example. $DIETY knows, people have been struggling with manual memory management in C and its ilk for a long time. Even though there are good tools to help, memory leaks still happen. Python increases our productivity by allowing us to forget about manual memory management altogether. I can do it with tools like valgrind, but Python's makes the point moot. Is that hand-holding? If so, I'm all for it. Cheers Philip -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On Aug 16, 9:13 am, Philip Semanchuk phi...@semanchuk.com wrote:
Sometimes X is safe and sometimes it isn't can be said
of many, many things, from taking a walk down the street
to juggling with knives. But it has little to do with
whether or not Python should issue a warning in the
specific case we're talking about.
I think any Python n00b should be writing code in n editor that is
safe for beginners AND is also teaching tool. IDLE is probably the
best thing a new Python programmer could use. It has syntax hilight,
smart indention, call tips, code completion, a class browser, greping
tools, and more! I do admit the tool is lacking in many areas
(including code base) that would be atrocious to an experienced
Pythonista, however, the tool is perfect for n00bs (and myself and
other pythoinistas use it all the time!)
There's a lot of ground to cover between newcomer who has
learned about a particular warning and coder who
regularly shadows builtins on purpose.
One word: SYNTAX HILIGHT
I have never shadowed a builtin deliberately. I've done it
accidentally plenty of times. There are 84 builtins in my
version of Python and I don't have them all memorized. The
fact that my editor colors them differently is the only
thing I have to back up my leaky memory. Not all editors
are so gracious.
So you have syntax hilight however you still shadow builtins? Oh
dear, this problem is worse than i initially suspected!
You can coerce any example to apply to an argument for or
against such a warning, but I think the general case is
that Python could reduce unintended consequences by
warning when vars erase builtins. (=== How many builtins
did I use in that sentence?)
Well let's see:
py import keyword
py import __builtin__
py PY_KWS = keyword.kwlist
py PY_BLT = dir(globals()['__builtins__'])
py s = \
You can coerce any example to apply to an argument for or against such
a warning, but I think the general case is that Python could reduce
unintended consequences by warning when vars erase builtins.
py fmtstr = '{0} - {1}'
py for word in s.split(' '):
... if word in PY_BLT:
... print fmtstr.format('builtin', word)
... elif word in PY_KWS:
... print fmtstr.format('KeyWord', word)
...
builtin - coerce
builtin - any
builtin - apply
KeyWord - for
KeyWord - or
KeyWord - is
builtin - reduce
builtin - vars
--
http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
Philip Semanchuk wrote: I think Python does lots of beneficial hand-holding. Garbage collection is a good example. $DIETY knows, people have been struggling with manual memory management in C and its ilk for a long time. Even though there are good tools to help, memory leaks still happen. Python increases our productivity by allowing us to forget about manual memory management altogether. I can do it with tools like valgrind, but Python's makes the point moot. Is that hand-holding? If so, I'm all for it. Good point. There is an important difference, however, between offering warnings to newbie programmers (which is how this thread started), and memory management. ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On 2011-08-16, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: On Tue, 16 Aug 2011 01:23 pm Philip Semanchuk wrote: Why should built-ins be treated as more sacred than your own objects? Because built-ins are described in the official documentation as having a specific behavior, while my objects are not. *My* objects certainly are, because I write documentation for my code. My docs are no less official than Python's docs. Sure they are. I can't get yours from python.org. You can shadow anything. Sometimes shadowing is safe, sometimes it isn't. I don't see why we should necessarily fear safe shadowing of built-ins more than we fear unsafe shadowing of non-built-ins. I think largely because anyone coming to your code will have expectations about the built-ins. For non-built-ins, they'll have to look around the code to see what they do, but for built-ins, they come to the table with an otherwise-reasonable expectation that they already know what that word means. A warning that is off by default won't help the people who need it, because they don't know enough to turn the warning on. A warning that is on by default will be helpful to the newbie programmer for the first week or so, and then will be nothing but an annoyance for the rest of their career. Will it? I am pretty sure that I'd keep it on and fix anything that triggered it, because shadowing built-ins strikes me as Asking For Trouble. Protecting n00bs from their own errors is an admirable aim, but have you considered that warnings for something which may be harmless could do more harm than good? I would distinguish between may not be causing bugs and may be harmless. I think code which shadows a built-in has a pretty real risk of being harmful at some unspecified future point when some maintainer who hasn't memorized every last line of the code makes the totally reasonable assumption that basic language features are still working and available. Perhaps. But I'm not so sure it is worth the cost of extra code to detect shadowing and raise a warning. After all, the average coder probably never shadows anything, and for those that do, once they get bitten *once* they either never do it again or learn how to shadow safely. I don't see it as a problem. I would guess that it happens moderately often entirely by accident. Not a Python example, but I recently had a heck of a time with some JavaScript I was trying to maintain. Couldn't get it to work on a slightly older Firefox, and the diagnostics from Firebug came across as basically illucid. Problem: I was declaring an array named 'history'. My thoughts would be: 1. It's hard to avoid shadowing anything unless you know the entire language and never forget things. 2. In particular, Python likes to use clear, obvious, names for things. Meaning that your choice of a clear, obvious, name for a similar thing could be the name of a thing in the language. 3. I am not sure at all that shadowing can be safe in code which will some day be maintained. The chances of someone coming along and trying to write clean, idiomatic, Python which happens to blow up interestingly because their code runs in an environment where part of the standard language has been shadowed strike me as Too High. MHO, but speaking only for myself, I'd want that warning on and I'd not consider it harmless. -s -- Copyright 2011, all wrongs reversed. Peter Seebach / usenet-nos...@seebs.net http://www.seebs.net/log/ -- lawsuits, religion, and funny pictures http://en.wikipedia.org/wiki/Fair_Game_(Scientology) -- get educated! I am not speaking for my employer, although they do rent some of my opinions. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On 2011-08-16, Ethan Furman et...@stoneleaf.us wrote: I think warnings should be reserved for language changes and such (like DeprecationWarning, RuntimeWarning, and FutureWarning), not for possible programmer mistakes. I disagree, on the basis of the following: The quality of C code I have to deal with has increased dramatically as gcc's aggressive use of warnings has spread. Python can be improved -- I don't see 'hand-holding' as an improvement. IDEs and lints can do this. -W ignore -s -- Copyright 2011, all wrongs reversed. Peter Seebach / usenet-nos...@seebs.net http://www.seebs.net/log/ -- lawsuits, religion, and funny pictures http://en.wikipedia.org/wiki/Fair_Game_(Scientology) -- get educated! I am not speaking for my employer, although they do rent some of my opinions. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On 2011-08-16, Ethan Furman et...@stoneleaf.us wrote: What makes you think it's unintentional? Programming experience. People *often* do things unintentionally. Seems to me the real issue is somebody using a builtin, such as str or int, and that they somehow manage to do this without realizing, wait a sec', that's one of my variables! I don't see that as a problem that Python needs to solve. I think the word my prejudices the case. Imagine stepping into a large project that uses multiple frameworks and class libraries and so on. You know Python but you're new to the project. Under which circumstance will you have more problems? 1. There is not a single shadowed built-in in the entire project. 2. There are dozens of shadowed built-ins based on when the original programmer felt there wasn't going to be a need for a given built-in feature, or possibly just didn't know about it. Also, how easy or hard do you think it will be to debug those problems? What's your first response going to be if, on a screen which doesn't contain the word file at all, you try to use the file built-in and you get some cryptic error? Are you going to know to go looking for the shadow right away? -s -- Copyright 2011, all wrongs reversed. Peter Seebach / usenet-nos...@seebs.net http://www.seebs.net/log/ -- lawsuits, religion, and funny pictures http://en.wikipedia.org/wiki/Fair_Game_(Scientology) -- get educated! I am not speaking for my employer, although they do rent some of my opinions. -- http://mail.python.org/mailman/listinfo/python-list
RE: Why no warnings when re-assigning builtin names?
On Aug 16, 2011, at 1:15 AM, Steven D'Aprano wrote: ... A warning that is off by default won't help the people who need it, because they don't know enough to turn the warning on. A warning that is on by default will be helpful to the newbie programmer for the first week or so, and then will be nothing but an annoyance for the rest of their career. I think that best practices would suggest that one shouldn't use variable names that shadow builtins (except in specific, special circumstances), so I don't really think this would be an annoyance at all. The number of *unwanted* warnings they'd get would be pretty close to zero. OTOH, in response to a question I asked on StackOverflow, someone posted a large list of times where this isn't followed in the std lib, so there seems to be a precedent for just using the builtin names for anything one feels like at the time. Thinking about it, what about if, by default, python was configured to emit warnings about this sort of thing, but a simple environment variable or config option would turn them off. That way, new users would get warnings when doing this sort of thing, and any experienced user that wanted the option of using these variables anywhere would just have a one-time thing to change when they installed python (or any time later). They'd turn these warnings off when they installed Python, and would never have to think about it again. New users (or experienced ones that prefer this) would be warned by default, and then could make the conscious decision to shadow the builtin names, instead of accidentally doing it. All the suggestions to just use a linter aren't helpful, since new users aren't likely to start with one and they are the thrust of this discussion. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On 2011.08.16 10:44 AM, rantingrick wrote: One word: SYNTAX HILIGHT And I had thought your troll skills had disappeared. Good one. -- CPython 3.2.1 | Windows NT 6.1.7601.17592 | Thunderbird 5.0 PGP/GPG Public Key ID: 0xF88E034060A78FCB -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
Seebs wrote: On 2011-08-16, Ethan Furman et...@stoneleaf.us wrote: I think warnings should be reserved for language changes and such (like DeprecationWarning, RuntimeWarning, and FutureWarning), not for possible programmer mistakes. I disagree, on the basis of the following: The quality of C code I have to deal with has increased dramatically as gcc's aggressive use of warnings has spread. With gcc you pay the cost once, with Python you would pay it with every run. A linter would be more along the lines of 'pay it once'. ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
Seebs wrote: On 2011-08-16, Ethan Furman et...@stoneleaf.us wrote: What makes you think it's unintentional? Programming experience. People *often* do things unintentionally. Seems to me the real issue is somebody using a builtin, such as str or int, and that they somehow manage to do this without realizing, wait a sec', that's one of my variables! I don't see that as a problem that Python needs to solve. I think the word my prejudices the case. But fits in well with the OP. Imagine stepping into a large project that uses multiple frameworks and class libraries and so on. You know Python but you're new to the project. Under which circumstance will you have more problems? 1. There is not a single shadowed built-in in the entire project. 2. There are dozens of shadowed built-ins based on when the original programmer felt there wasn't going to be a need for a given built-in feature, or possibly just didn't know about it. My first course of action would be to learn what I'm patching before I patch it. Scope helps in the case of shadowing; lint helps to give warnings about that and other things as well. Also, how easy or hard do you think it will be to debug those problems? What's your first response going to be if, on a screen which doesn't contain the word file at all, you try to use the file built-in and you get some cryptic error? Are you going to know to go looking for the shadow right away? I should have a pretty good clue from the traceback, and a couple prints or logs around the offending lines should give me the rest. ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On Tue, Aug 16, 2011 at 6:43 PM, Ethan Furman et...@stoneleaf.us wrote: Seebs wrote: On 2011-08-16, Ethan Furman et...@stoneleaf.us wrote: I think warnings should be reserved for language changes and such (like DeprecationWarning, RuntimeWarning, and FutureWarning), not for possible programmer mistakes. I disagree, on the basis of the following: The quality of C code I have to deal with has increased dramatically as gcc's aggressive use of warnings has spread. With gcc you pay the cost once, with Python you would pay it with every run. A linter would be more along the lines of 'pay it once'. Agreed; in addition, it's spam. Spam at compile time isn't much of an issue; you type ./configure and make, and a billion messages scroll past you. Spam at run time? Everyone who USES the program has to deal with it, and in amongst the program's own output. I don't know how to have Python emit warnings that wouldn't cause these issues. The only thing I can think of is to have the interactive interpreter default to showing more warnings, which is far from perfect itself. As to 'file' specifically, my 3.2 doesn't seem to have it as a builtin. The return value from open() is _io.TextIOWrapper, so the whole issue of file=open(...) may have been completely dodged. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On 08/16/2011 12:11 PM, Seebs wrote: Under which circumstance will you have more problems? 1. There is not a single shadowed built-in in the entire project. 2. There are dozens of shadowed built-ins based on when the original programmer felt there wasn't going to be a need for a given built-in feature, or possibly just didn't know about it. In practice, I've never hit such a snag. For #2, the only way it would impact me is if I did the ill-advised from shadowy import * which might tromp on things I care about. Otherwise, I'd just do something like from shadowy import list as shadowy_list, id as shadowy_id If I'm altering another person's ill-created module, I might consider doing a search for builtins and then just simply patching the module so that it renamed all the shadowing usages. Unless the author was being particularly obscure (using scope to impact where a builtin meant the builtin, and other scopes to shadow the builtins), most authors are pretty consistent in their shadowing (usually ignorant of the fact they're shadowing the builtin) which makes the search-n-replace an easy tweak. -tkc -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On 2011-08-16, Ethan Furman et...@stoneleaf.us wrote: Seebs wrote: The quality of C code I have to deal with has increased dramatically as gcc's aggressive use of warnings has spread. With gcc you pay the cost once, with Python you would pay it with every run. A linter would be more along the lines of 'pay it once'. Huh! That is a really good point, which I had not considered. I still prefer to get warnings, but... Hmm. I wonder whether there's a way to mitigate the cost of these things by messing with -W settings, such that runtime that wants to be fast can omit the checks, but the default could still be to, well, prevent likely errors. -s -- Copyright 2011, all wrongs reversed. Peter Seebach / usenet-nos...@seebs.net http://www.seebs.net/log/ -- lawsuits, religion, and funny pictures http://en.wikipedia.org/wiki/Fair_Game_(Scientology) -- get educated! I am not speaking for my employer, although they do rent some of my opinions. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On 8/16/2011 1:15 PM, Gerrat Rickert wrote: I think that best practices would suggest that one shouldn't use variable names that shadow builtins (except in specific, special circumstances), so I don't really think this would be an annoyance at all. The number of *unwanted* warnings they'd get would be pretty close to zero. OTOH, in response to a question I asked on StackOverflow, someone posted a large list of times where this isn't followed in the std lib, so there seems to be a precedent for just using the builtin names for anything one feels like at the time. If you run across that again and email me the link, I will take a look and see if I think the issue should be raised on pydev. Of course, some modules *intentionally* define an open function, intended to be accessed as 'mod.open' and not as 'from mod import *; open'. Also, class/instance attributes can also reuse builtin names. But 'open = True/False' would be bad. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On 8/16/2011 2:56 PM, Seebs wrote: I wonder whether there's a way to mitigate the cost of these things by messing with -W settings, such that runtime that wants to be fast can omit the checks, but the default could still be to, well, prevent likely errors. Warning messages have a cost even if suppressed. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On 2011-08-16, Terry Reedy tjre...@udel.edu wrote: On 8/16/2011 2:56 PM, Seebs wrote: I wonder whether there's a way to mitigate the cost of these things by messing with -W settings, such that runtime that wants to be fast can omit the checks, but the default could still be to, well, prevent likely errors. Warning messages have a cost even if suppressed. Yes, but is it a *significant* cost? My assumption is that the suppression would be of checking, not just of displaying messages. -s -- Copyright 2011, all wrongs reversed. Peter Seebach / usenet-nos...@seebs.net http://www.seebs.net/log/ -- lawsuits, religion, and funny pictures http://en.wikipedia.org/wiki/Fair_Game_(Scientology) -- get educated! I am not speaking for my employer, although they do rent some of my opinions. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On Wed, Aug 17, 2011 at 12:49 AM, Seebs usenet-nos...@seebs.net wrote: Yes, but is it a *significant* cost? My assumption is that the suppression would be of checking, not just of displaying messages. It mightn't be very significant, but there'd still be some cost. However, IMHO the greatest cost is the spamminess; forcing the user to deal with lines and lines of warnings is not a useful way to design a language. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On Aug 16, 2011, at 7:29 PM, Terry Reedy wrote: On 8/16/2011 1:15 PM, Gerrat Rickert wrote: I think that best practices would suggest that one shouldn't use variable names that shadow builtins (except in specific, special circumstances), so I don't really think this would be an annoyance at all. The number of *unwanted* warnings they'd get would be pretty close to zero. OTOH, in response to a question I asked on StackOverflow, someone posted a large list of times where this isn't followed in the std lib, so there seems to be a precedent for just using the builtin names for anything one feels like at the time. If you run across that again and email me the link, I will take a look and see if I think the issue should be raised on pydev. Of course, some modules *intentionally* define an open function, intended to be accessed as 'mod.open' and not as 'from mod import *; open'. Also, class/instance attributes can also reuse builtin names. But 'open = True/False' would be bad. Hi Terry, To generalize from your example, are you saying that there's a mild admonition against shadowing builtins with unrelated variable names in standard lib code? Here's an example from Python 3.2.1's argparse.py, lines 466-473. open is shadowed on the second line. # clean up separators for mutually exclusive groups open = r'[\[(]' close = r'[\])]' text = _re.sub(r'(%s) ' % open, r'\1', text) text = _re.sub(r' (%s)' % close, r'\1', text) text = _re.sub(r'%s *%s' % (open, close), r'', text) text = _re.sub(r'\(([^|]*)\)', r'\1', text) text = text.strip() Thanks Philip -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
Seebs wrote: On 2011-08-16, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: On Tue, 16 Aug 2011 01:23 pm Philip Semanchuk wrote: Why should built-ins be treated as more sacred than your own objects? Because built-ins are described in the official documentation as having a specific behavior, while my objects are not. *My* objects certainly are, because I write documentation for my code. My docs are no less official than Python's docs. Sure they are. I can't get yours from python.org. And what makes that unofficial? python.org is not the sole arbiter of official documentation in the world. You can't get docs for Django or Scipy or NLTK from python.org either, but just try telling the authors of those libraries that their docs are somehow unofficial and see how far that gets you. [...] I think code which shadows a built-in has a pretty real risk of being harmful at some unspecified future point when some maintainer who hasn't memorized every last line of the code makes the totally reasonable assumption that basic language features are still working and available. Am I the only person who writes functions and methods any more? *wink* Modern languages, and by modern I mean most languages more than, oh, about fifty years old, provide ways to limit the scope of variables. You don't need to memorise every last line of the code to safely edit a function. def process(list, arg): spam(list) ham(arg) cheese(list, arg) The scope of parameter list is limited to within the function process itself. Inside, it shadows the built-in list. Outside, it doesn't do squat. [...] My thoughts would be: 1. It's hard to avoid shadowing anything unless you know the entire language and never forget things. Define the entire language. Does that include the names of all the plethora of exceptions? How about the standard library? For what it's worth, I don't care about memorising all the built-ins. I delegate that job to my editor, which has a syntax highlighter for Python. It never forgets built-ins. (In fact, sometimes this is a nuisance. When I'm editing Python 3 code, it still insists that apply and reduce are built-ins.) 2. In particular, Python likes to use clear, obvious, names for things. Meaning that your choice of a clear, obvious, name for a similar thing could be the name of a thing in the language. Yes, and Python also encourages the use of scopes, so that the clear, obvious name for something in one scope does not clash with the clear, obvious, identical name for something completely different in another scope. 3. I am not sure at all that shadowing can be safe in code which will some day be maintained. Oh there's no doubt that shadowing *can* be unsafe. But then, very few things can't be abused. As I see it, there are (at least) four stages related to shadowing. (1) At first, you don't even know enough to be concerned by shadowing. You blithely call variables list and str without a care in the world... until something goes wrong. (2) Second stage, you know enough to realise that shadowing can be bad. You avoid shadowing everything. Your code is full of variables called my_list and astr and some_tuple. You call your functions things like izip even though it is designed as a replacement for zip, because the caller might use from itertools import * and accidentally replace the built-in zip with my zip. You even avoid using string as a variable name because it might shadow the string module -- even though you haven't actually imported or used the string module in the last four years. (3) Eventually, you get frustrated writing doc strings like this: def function(astr, myint=0): function(astr [, myint]) - list Arguments: astr - string myint - integer Do something tool-like to a string and optional integer... and begin writing them like this: def function(astr, myint): function(str [, int]) - list Do something tool-like to a string and optional integer... (4) And then, after a while, you decide that perhaps shadowing is not always so bad. (It helps if the Effbot tells you off for objecting to shadowing in a two-line function.) At first, you limit yourself to using parameter names that shadow built-ins in small tool-like functions. Then when the world doesn't implode, you rethink the use of top-level function names like izip and realise that namespaces exist so that you don't need to care about shadowing functions you don't use, and if people call import * they have nobody to blame but themselves if things break. More-or-less-auto-biographically-ly y'rs, -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
Philip Semanchuk wrote: I think Python does lots of beneficial hand-holding. Garbage collection is a good example. $DIETY knows, people have been struggling with manual memory management in C and its ilk for a long time. Even though there are good tools to help, memory leaks still happen. Python increases our productivity by allowing us to forget about manual memory management altogether. I can do it with tools like valgrind, but Python's makes the point moot. Is that hand-holding? If so, I'm all for it. Hand-holding is not a well-defined term. According to Mel, garbage collectors would be hand-holding designed for soft nellies. http://www.cs.utah.edu/~elb/folklore/mel.html But Mel would probably make his own nails rather than buy them from the hardware store too. To some secret recipe involving rare metals that most people have never even heard of. And use a hammer forged from a single piece of meteor iron, shaped to fit his hand precisely. And it would be too heavy for mortals to lift. Of course the term hand-holding is a pejorative. The implication is that the person needing the hand-holding is like a child who needs somebody to hold their hand while they cross the road. Adults are supposed to know better (if only it were true in real life!). On the other hand, a real adult knows their limitations, and if you need a bit of hand-holding in a certain area of your life, so be it. Some errors aren't exactly a matter of inexperience and poor judgement, and so everyone can benefit from guard rails, seat belts and garbage collection. In that case, it isn't hand-holding, because there's no expectation that you should grow up from it. Even Donald Knuth probably would get benefit from a good garbage collector. I have no objection to lint tools. But separation of concerns should apply: the Python compiler should just compile what I tell it to, the linter should warn me if I'm running with scissors. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On 2011-08-17, Chris Angelico ros...@gmail.com wrote: On Wed, Aug 17, 2011 at 12:49 AM, Seebs usenet-nos...@seebs.net wrote: Yes, but is it a *significant* cost? ?My assumption is that the suppression would be of checking, not just of displaying messages. It mightn't be very significant, but there'd still be some cost. However, IMHO the greatest cost is the spamminess; forcing the user to deal with lines and lines of warnings is not a useful way to design a language. Lines and lines? I'd say if it's to be allowed to shadow built-ins (and I'm not sure that's a good thing at all), you'd still be looking at one warning per shadowing, which shouldn't be too many. -s -- Copyright 2011, all wrongs reversed. Peter Seebach / usenet-nos...@seebs.net http://www.seebs.net/log/ -- lawsuits, religion, and funny pictures http://en.wikipedia.org/wiki/Fair_Game_(Scientology) -- get educated! I am not speaking for my employer, although they do rent some of my opinions. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On 8/16/2011 7:49 PM, Seebs wrote: On 2011-08-16, Terry Reedytjre...@udel.edu wrote: On 8/16/2011 2:56 PM, Seebs wrote: I wonder whether there's a way to mitigate the cost of these things by messing with -W settings, such that runtime that wants to be fast can omit the checks, but the default could still be to, well, prevent likely errors. Warning messages have a cost even if suppressed. Yes, but is it a *significant* cost? My assumption is that the suppression would be of checking, not just of displaying messages. The -W settings suppress display or printing of warnings, not their generation. import warnings if some_condition: warnings.warn(message, Warning) The test and warn call are made regardless of disposition. Filters determine what is done with each Warning subclass. The logging module can grab them too. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On Aug 16, 2011, at 9:29 PM, Steven D'Aprano wrote: I have no objection to lint tools. But separation of concerns should apply: the Python compiler should just compile what I tell it to, the linter should warn me if I'm running with scissors. This point (also made by Ethan) I can agree with. I haven't looked through all the warnings the Python compiler emits, but it seems like it currently doesn't dispense advice (unlike, say, gcc). It only warns about changes in the language standard library. In that context, asking it to warn about shadowing builtins would be an expansion of scope. bye, Philip -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On 8/16/2011 8:18 PM, Philip Semanchuk wrote: Hi Terry, To generalize from your example, are you saying that there's a mild admonition against shadowing builtins with unrelated variable names in standard lib code? I would expect that there might be. I would have to check PEP8. Here's an example from Python 3.2.1's argparse.py, lines 466-473. open is shadowed on the second line. # clean up separators for mutually exclusive groups open = r'[\[(]' close = r'[\])]' text = _re.sub(r'(%s) ' % open, r'\1', text) text = _re.sub(r' (%s)' % close, r'\1', text) text = _re.sub(r'%s *%s' % (open, close), r'', text) text = _re.sub(r'\(([^|]*)\)', r'\1', text) text = text.strip() -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On Aug 16, 2011, at 10:15 PM, Terry Reedy wrote: On 8/16/2011 8:18 PM, Philip Semanchuk wrote: Hi Terry, To generalize from your example, are you saying that there's a mild admonition against shadowing builtins with unrelated variable names in standard lib code? I would expect that there might be. I would have to check PEP8. I was curious, so I checked. I didn't see anything specifically referring to builtins. This is as close as it gets: If a function argument's name clashes with a reserved keyword, it is generally better to append a single trailing underscore rather than use an abbreviation or spelling corruption. Thus print_ is better than prnt. (Perhaps better is to avoid such clashes by using a synonym.) bye Philip -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On 2011-08-17, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: Seebs wrote: On 2011-08-16, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: *My* objects certainly are, because I write documentation for my code. My docs are no less official than Python's docs. Sure they are. I can't get yours from python.org. And what makes that unofficial? python.org is not the sole arbiter of official documentation in the world. But it is the sole arbiter of official *Python* documentation. If I am looking at code in Python, I reasonably expect the Python docs to be correct about that code. Incomplete, sure, but *correct*. I think code which shadows a built-in has a pretty real risk of being harmful at some unspecified future point when some maintainer who hasn't memorized every last line of the code makes the totally reasonable assumption that basic language features are still working and available. Am I the only person who writes functions and methods any more? *wink* Yes. Everyone else converted to single giant blocks of code because they are easier to develop. :P Modern languages, and by modern I mean most languages more than, oh, about fifty years old, provide ways to limit the scope of variables. You don't need to memorise every last line of the code to safely edit a function. Only the parts that are in scope, but... I seem to recall (and I'm pretty Python newbiesh, so I could be wrong) that Python classes create a scope in which bits of the class become visible, and some classes are sorta biggish. def process(list, arg): spam(list) ham(arg) cheese(list, arg) The scope of parameter list is limited to within the function process itself. Inside, it shadows the built-in list. Outside, it doesn't do squat. Yes. But what about the built-in spam? :) Define the entire language. Does that include the names of all the plethora of exceptions? How about the standard library? I'd think standard library or close to it. For what it's worth, I don't care about memorising all the built-ins. I delegate that job to my editor, which has a syntax highlighter for Python. It never forgets built-ins. (In fact, sometimes this is a nuisance. When I'm editing Python 3 code, it still insists that apply and reduce are built-ins.) Heh. I mostly don't use syntax highlighters; at best, they distract me, at worst, they distract me a lot. I also don't use one in English, although I am sure some people would love to have nouns in blue and punctuation in green. Yes, and Python also encourages the use of scopes, so that the clear, obvious name for something in one scope does not clash with the clear, obvious, identical name for something completely different in another scope. Another scope is normally a horizontal thing -- you're talking about a different scope such that you are *either* in this one *or* in that one. Built-ins are not in a scope you are never not in. Oh there's no doubt that shadowing *can* be unsafe. But then, very few things can't be abused. Yup. As I see it, there are (at least) four stages related to shadowing. (1) At first, you don't even know enough to be concerned by shadowing. You blithely call variables list and str without a care in the world... until something goes wrong. (2) Second stage, you know enough to realise that shadowing can be bad. You avoid shadowing everything. Your code is full of variables called my_list and astr and some_tuple. You call your functions things like izip even though it is designed as a replacement for zip, because the caller might use from itertools import * and accidentally replace the built-in zip with my zip. You even avoid using string as a variable name because it might shadow the string module -- even though you haven't actually imported or used the string module in the last four years. Heh. (I got advised by pylint not to grab something from it, but I no longer remember why; I seem to recall being totally unable to find a way to avoid that warning and still have the string processing I needed.) (3) Eventually, you get frustrated writing doc strings like this: def function(astr, myint=0): function(astr [, myint]) - list Arguments: astr - string myint - integer Do something tool-like to a string and optional integer... and begin writing them like this: def function(astr, myint): function(str [, int]) - list Do something tool-like to a string and optional integer... That seems safe enough to me. :) (4) And then, after a while, you decide that perhaps shadowing is not always so bad. (It helps if the Effbot tells you off for objecting to shadowing in a two-line function.) At first, you limit yourself to using parameter names that shadow built-ins in small tool-like functions. Then when the world doesn't implode, you rethink the use of top-level function names
Why no warnings when re-assigning builtin names?
With surprising regularity, I see program postings (eg. on StackOverflow) from inexperienced Python users accidentally re-assigning built-in names. For example, they'll innocently call some variable, list, and assign a list of items to it. ...and if they're _unlucky_ enough, their program may actually work (encouraging them to re-use this name in other programs). If they try to use an actual keyword, both the interpreter and compiler are helpful enough to give them a syntax error, but I think the builtins should be pseudo-reserved, and a user should explicitly have to do something *extra* to not receive a warning. I'd suggest: from __future__ import allow_reassigning_builtins, but I think this abuse of the __future__ module likely isn't welcome. I know that for testing purposes, this functionality is very convenient, and I'm not suggesting it be removed. In these cases, it would be trivial to just require something explicit, telling the interpreter that the programmer was aware they were assigning to a builtin name. The situation is slightly different for modules that come with Python. Most of us would cringe when seeing something like: `string = Some string`; but at least the user has to explicitly import the string module for this to actually cause issues (other than readability). What sayest the Python community about having an explicit warning against such un-pythonic behaviour (re-assigning builtin names)? Regards, Gerrat -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On Aug 15, 2011, at 5:52 PM, Gerrat Rickert wrote: With surprising regularity, I see program postings (eg. on StackOverflow) from inexperienced Python users accidentally re-assigning built-in names. For example, they'll innocently call some variable, list, and assign a list of items to it. ...and if they're _unlucky_ enough, their program may actually work (encouraging them to re-use this name in other programs). Or they'll assign a class instance to 'object', only to cause weird errors later when they use it as a base class. I agree that this is a problem. The folks on my project who are new-ish to Python overwrite builtins fairly often. Since there's never been any consequence other than my my vague warnings that something bad might happen as a result, it's difficult for them to develop good habits in this regard. It doesn't help that Eclipse (their editor of choice) doesn't seem to provide a way of coloring builtins differently. (That's what I'm told, anyway. I don't use it.) If they try to use an actual keyword, both the interpreter and compiler are helpful enough to give them a syntax error, but I think the builtins should be pseudo-reserved, and a user should explicitly have to do something *extra* to not receive a warning. Unfortunately you're suggesting a change to the language which could break existing code. I could see a use for from __future__ import squawk_if_i_reassign_a_builtin or something like that, but the current default behavior has to remain as it is. JMO, Philip -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On Mon, Aug 15, 2011 at 10:52 PM, Gerrat Rickert grick...@coldstorage.com wrote: With surprising regularity, I see program postings (eg. on StackOverflow) from inexperienced Python users accidentally re-assigning built-in names. For example, they’ll innocently call some variable, “list”, and assign a list of items to it. It's actually masking, not reassigning. That may make it easier or harder to resolve the issue. If you want a future directive that deals with it, I'd do it the other way - from __future__ import mask_builtin_warning or something - so the default remains as it currently is. But this may be a better job for a linting script. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
Gerrat Rickert wrote: What sayest the Python community about having an explicit warning against such un-pythonic behaviour (re-assigning builtin names)? What makes you think this behavior is unpythonic? Python is not about hand-holding. ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On 2011-08-15, Ethan Furman et...@stoneleaf.us wrote: Gerrat Rickert wrote: What sayest the Python community about having an explicit warning against such un-pythonic behaviour (re-assigning builtin names)? What makes you think this behavior is unpythonic? Python is not about hand-holding. It seems like something which is sufficiently likely to be a mistake might deserve a warning -- especially since, so far as I can tell, there's never going to be a program which can't easily be written to avoid the problematic behavior. -s -- Copyright 2011, all wrongs reversed. Peter Seebach / usenet-nos...@seebs.net http://www.seebs.net/log/ -- lawsuits, religion, and funny pictures http://en.wikipedia.org/wiki/Fair_Game_(Scientology) -- get educated! I am not speaking for my employer, although they do rent some of my opinions. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
Seebs wrote: On 2011-08-15, Ethan Furman et...@stoneleaf.us wrote: Gerrat Rickert wrote: What sayest the Python community about having an explicit warning against such un-pythonic behaviour (re-assigning builtin names)? What makes you think this behavior is unpythonic? Python is not about hand-holding. It seems like something which is sufficiently likely to be a mistake might deserve a warning -- especially since, so far as I can tell, there's never going to be a program which can't easily be written to avoid the problematic behavior. sufficiently likely depends entirely on who is doing the coding. I use `open()` for opening my files, and so regularly use `file` as a name. It can also be very handy to mask a built-in when doing something even more fun and entertaining and I, for one, have zero desire to have Python start warning me about perfectly legitimate code. Programmers need to learn whichever language they are choosing to code in, and if extra help is needed beyond whatever is basic for that language, find (or write! ;) the third-party tool to help out. There are at least two linters for Python, and multiple IDEs that can help with these, and other, problems. (I don't much care for IDEs, but I am thinking of starting to use a linter, myself.) ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On Aug 15, 2011 5:56 PM, Gerrat Rickert grick...@coldstorage.com wrote: With surprising regularity, I see program postings (eg. on StackOverflow) from inexperienced Python users accidentally re-assigning built-in names. For example, they’ll innocently call some variable, “list”, and assign a list of items to it. ...and if they’re _unlucky_ enough, their program may actually work (encouraging them to re-use this name in other programs). If they try to use an actual keyword, both the interpreter and compiler are helpful enough to give them a syntax error, but I think the builtins should be “pseudo-reserved”, and a user should explicitly have to do something *extra* ... What sayest the Python community about having an explicit warning against such un-pythonic behaviour (re-assigning builtin names)? One of Python's greatest strength's in my opinion is that it strives for consistency. As much as possible, Python avoids differentiating between built-in objects (types or otherwise) and user-defined objects. I think it should stay that way. There are tools that can detect these errors and their use should be encouraged, but the Python interpreter shouldn't single out variables which are types that happen to be built-in from any other variable or any other type. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On Mon, Aug 15, 2011 at 2:52 PM, Gerrat Rickert grick...@coldstorage.comwrote: With surprising regularity, I see program postings (eg. on StackOverflow) from inexperienced Python users accidentally re-assigning built-in names. http://pypi.python.org/pypi/pylint checks for this and many other issues. I don't know if pyflakes or pychecker do. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On Aug 15, 5:13 pm, Philip Semanchuk phi...@semanchuk.com wrote:
On Aug 15, 2011, at 5:52 PM, Gerrat Rickert wrote:
With surprising regularity, I see program postings (eg. on
StackOverflow) from inexperienced Python users accidentally
re-assigning built-in names.
For example, they'll innocently call some variable, list, and assign a
list of items to it.
...and if they're _unlucky_ enough, their program may actually work
(encouraging them to re-use this name in other programs).
Or they'll assign a class instance to 'object', only to cause weird errors
later when they use it as a base class.
I agree that this is a problem. The folks on my project who are new-ish to
Python overwrite builtins fairly often.
Simple syntax hilighting can head off these issues with great ease.
Heck, python even has a keyword module, and you get a list of built-
ins from the dir() function!
import keyword
import __builtin__
PY_BUILTINS = [str(name) for name in dir(__builtin__) if not
name.startswith('_')]
PY_KEYWORDS = keyword.kwlist
Also Python ships with IDLE (which is a simplistic IDE) and although i
find it needs a bit of work to be what GvR initially dreamed, it works
good enough to get you by. I always say, you must use the correct tool
for the job, and syntax hilight is a must have to avoid these
accidents.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On Tue, 16 Aug 2011 08:15 am Chris Angelico wrote: On Mon, Aug 15, 2011 at 10:52 PM, Gerrat Rickert grick...@coldstorage.com wrote: With surprising regularity, I see program postings (eg. on StackOverflow) from inexperienced Python users accidentally re-assigning built-in names. For example, they’ll innocently call some variable, “list”, and assign a list of items to it. It's actually masking, not reassigning. That may make it easier or harder to resolve the issue. The usual term is shadowing builtins, and it's a feature, not a bug :) If you want a future directive that deals with it, I'd do it the other way - from __future__ import mask_builtin_warning or something - so the default remains as it currently is. But this may be a better job for a linting script. Agreed. It's a style issue, nothing else. There's nothing worse about: def spam(list): pass compared to class thingy: pass def spam(thingy): pass Why should built-ins be treated as more sacred than your own objects? -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On Aug 15, 2011, at 9:32 PM, Steven D'Aprano wrote: On Tue, 16 Aug 2011 08:15 am Chris Angelico wrote: If you want a future directive that deals with it, I'd do it the other way - from __future__ import mask_builtin_warning or something - so the default remains as it currently is. But this may be a better job for a linting script. Agreed. It's a style issue, nothing else. There's nothing worse about: def spam(list): pass compared to class thingy: pass def spam(thingy): pass Why should built-ins be treated as more sacred than your own objects? Because built-ins are described in the official documentation as having a specific behavior, while my objects are not. Yes, it can be useful to replace some of the builtins with one's own implementation, and yes, doing so fits in with Python's we're all consenting adults philosophy. But replacing (shadowing, masking -- call it what you will) builtins is not everyday practice. On the contrary, as the OP Gerrat pointed out, it's most often done unwittingly by newcomers to the language who have no idea that they've done anything out of the ordinary or potentially confusing. If a language feature is most often invoked accidentally without knowledge of or regard for its potential negative consequences, then it might be worth making it easier to avoid those accidents. bye, Philip -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no warnings when re-assigning builtin names?
On Tue, 16 Aug 2011 01:23 pm Philip Semanchuk wrote: On Aug 15, 2011, at 9:32 PM, Steven D'Aprano wrote: On Tue, 16 Aug 2011 08:15 am Chris Angelico wrote: If you want a future directive that deals with it, I'd do it the other way - from __future__ import mask_builtin_warning or something - so the default remains as it currently is. But this may be a better job for a linting script. Agreed. It's a style issue, nothing else. There's nothing worse about: def spam(list): pass compared to class thingy: pass def spam(thingy): pass Why should built-ins be treated as more sacred than your own objects? Because built-ins are described in the official documentation as having a specific behavior, while my objects are not. *My* objects certainly are, because I write documentation for my code. My docs are no less official than Python's docs. You can shadow anything. Sometimes shadowing is safe, sometimes it isn't. I don't see why we should necessarily fear safe shadowing of built-ins more than we fear unsafe shadowing of non-built-ins. (I'm not even convinced that making None a reserved word was the right decision.) A warning that is off by default won't help the people who need it, because they don't know enough to turn the warning on. A warning that is on by default will be helpful to the newbie programmer for the first week or so, and then will be nothing but an annoyance for the rest of their career. (For some definition of a week -- some people are slower learners than others.) Yes, it can be useful to replace some of the builtins with one's own implementation, and yes, doing so fits in with Python's we're all consenting adults philosophy. But replacing (shadowing, masking -- call it what you will) builtins is not everyday practice. On the contrary, as the OP Gerrat pointed out, it's most often done unwittingly by newcomers to the language who have no idea that they've done anything out of the ordinary or potentially confusing. Protecting n00bs from their own errors is an admirable aim, but have you considered that warnings for something which may be harmless could do more harm than good? Beginners often lack the skill to distinguish between harmless warnings that can safely be ignored, and fatal errors that need to be fixed. Even user friendly warning or error messages tend to unnerve some beginner coders. There's not much we can do about outright errors, except to make sure that the error string is as useful as possible, but we can avoid overloading beginners with warnings they don't need to care about: WARNING WARNING WARNING WILL ROBINSON, DANGER DANGER DANGER: YOUR SISTER'S NAME 'PENNY' SHADOWS THE BRITISH CURRENCY, POTENTIAL AMBIGUITY ALERT DANGER DANGER DANGER! *wink* Depending on their personality, you may end up teaching them to ignore warnings, or a superstitious dread of anything that leads to a warning. Neither is a good outcome. If a language feature is most often invoked accidentally without knowledge of or regard for its potential negative consequences, then it might be worth making it easier to avoid those accidents. Perhaps. But I'm not so sure it is worth the cost of extra code to detect shadowing and raise a warning. After all, the average coder probably never shadows anything, and for those that do, once they get bitten *once* they either never do it again or learn how to shadow safely. I don't see it as a problem. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
