Re: Questions about `locals` builtin
On Wed, Feb 28, 2018 at 10:59 PM, Steven D'Aprano wrote: > On Wed, 28 Feb 2018 18:01:42 +1100, Chris Angelico wrote: > >> If you really want a list of ALL the local names in a function, you can >> look at its __code__ object, which has a tuple of variable names: >> >> print(func1.__code__.co_varnames) >> >> That information is static to the function, as it is indeed determined >> when the function is compiled. > > Ho ho ho, not in Python 2 it isn't!!! > > py> def bizarre(): > ... x = 1 > ... from math import * # oww my aching head! > ... print 'sin' in locals() > ... print sin > ... The sin in question is that you're using a star import inside a function. The penance is to kneel before the altar and recite PEP 20 nineteen times. And then cut down the tallest tree in the forest... with... a debugger. (When I tried that, I got a SyntaxWarning. Yay!) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Questions about `locals` builtin
On Wed, Feb 28, 2018 at 10:58 PM, Steven D'Aprano wrote: > On Wed, 28 Feb 2018 18:04:11 +1100, Chris Angelico wrote: >> But if you know that >> there's only a handful of variables that you'd actually want to do that >> to, you can simply put those into an object of some form, and then >> mutate that object. > > o_O > > If I knew ahead of time which variables held the wrong value, I'd fix > them so they held the right value ahead of time, and not bother using a > debugger. > > :-) Ahh but that requires that you know the right value as well. It's plausible that you have a handful of "tweakables" but you have no idea what to tweak them *to*. That's where this would be plausible. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Questions about `locals` builtin
On Wed, 28 Feb 2018 18:01:42 +1100, Chris Angelico wrote: > If you really want a list of ALL the local names in a function, you can > look at its __code__ object, which has a tuple of variable names: > > print(func1.__code__.co_varnames) > > That information is static to the function, as it is indeed determined > when the function is compiled. Ho ho ho, not in Python 2 it isn't!!! py> def bizarre(): ... x = 1 ... from math import * # oww my aching head! ... print 'sin' in locals() ... print sin ... :1: SyntaxWarning: import * only allowed at module level py> bizarre() True py> bizarre.__code__.co_varnames ('x',) In Python 2, you can also use exec inside a function, for extra obfuscatory goodness. Python 3 locks down these loopholes, and ensures that locals inside functions can be statically determined. -- Steve -- https://mail.python.org/mailman/listinfo/python-list
Re: Questions about `locals` builtin
On Wed, 28 Feb 2018 18:04:11 +1100, Chris Angelico wrote: > On Wed, Feb 28, 2018 at 5:54 PM, dieter wrote: [...] >> I am still working with Python 2 (Python 3 may behave differently). >> There, during debugging, I would sometimes like to change the value of >> variables (I know that the variable has got a wrong value and would >> like to fix it to continue senseful debugging without a restart). This >> works for variables not yet known inside the function but does not work >> for true local variables. >> I assume that this is one effect of the "locals()" restriction. >> >> > That seems like a hairy thing to do, honestly. Well, yeah, but it is during debugging, not production code. > But if you know that > there's only a handful of variables that you'd actually want to do that > to, you can simply put those into an object of some form, and then > mutate that object. o_O If I knew ahead of time which variables held the wrong value, I'd fix them so they held the right value ahead of time, and not bother using a debugger. :-) -- Steve -- https://mail.python.org/mailman/listinfo/python-list
Re: Questions about `locals` builtin
On Wed, Feb 28, 2018 at 5:54 PM, dieter wrote: > Ned Batchelder writes: >> On 2/27/18 3:52 AM, Kirill Balunov wrote: >>> a. Is this restriction for locals desirable in the implementation of >>> CPython in Python 3? >>> b. Or is it the result of temporary fixes for Python 2? >> >> My understanding is that the behavior of locals() is determined mostly >> by what is convenient for the implementors, so that they can keep >> regular code running as quickly as possible. The answer to the >> question, "why can't we make locals() work more like I expect?" is, >> "because that would make things slower." >>> >>> Personally, I find the convenient functionality to update the local symbol >>> table inside a function, similar to `globals`. >> >> Can you show us an example of why you would want to update locals >> through locals()? There might be more natural ways to solve your >> problem. > > I am still working with Python 2 (Python 3 may behave differently). > There, during debugging, I would sometimes like to change the value > of variables (I know that the variable has got a wrong value > and would like to fix it to continue senseful debugging without a restart). > This works for variables not yet known inside the function but does > not work for true local variables. > I assume that this is one effect of the "locals()" restriction. > That seems like a hairy thing to do, honestly. But if you know that there's only a handful of variables that you'd actually want to do that to, you can simply put those into an object of some form, and then mutate that object. That's guaranteed to work in any Python. Personally, I'd be inclined to break things up into separate functions, and then if you want to change state and continue, it would be by interactively calling one of those functions with slightly different parameters. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Questions about `locals` builtin
On Tue, Feb 27, 2018 at 5:55 AM, Kirill Balunov wrote: > 2. The documentation has a note that "The contents of this dictionary > should not be modified". Which implies that it is a read only mapping. So > the question why it is `dict` instead of `types.MappingProxyType`? A dict is smaller and faster. In some Python implementations, locals() returns an actual dictionary of the actual locals; to construct a proxy would be a waste of effort, and to have the internal locals use a proxy is also a complete waste. Python tends not to enforce rules that won't actually cause a crash, so it's simpler and more efficient to give you a real dictionary and say "changing this might not do what you think it does". > 3. There is one more moment: local variables had been determined when > function was compiled. But `locals` returns _some_ current runtime copy. I > find this also confusing: > > def func1(): > > loc = locals() > > b = 12 > > return loc > > def func2(): > > b = 12 > > loc = locals() > > return loc > > func1() > { } func2() > {'b': 12} The values of those locals is calculated at run-time, but the set of names is not. If a name has not been bound, it's still flagged as "local", but it doesn't appear in locals(), because there's no corresponding value. So if "b" is not in locals(), attempting to access the variable named b will raise UnboundLocalError. Dictionaries don't have a way to say "this thing exists but has no value"; so this says "this thing doesn't exist", which is close enough. If you really want a list of ALL the local names in a function, you can look at its __code__ object, which has a tuple of variable names: print(func1.__code__.co_varnames) That information is static to the function, as it is indeed determined when the function is compiled. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Questions about `locals` builtin
Ned Batchelder writes: > On 2/27/18 3:52 AM, Kirill Balunov wrote: >> a. Is this restriction for locals desirable in the implementation of >> CPython in Python 3? >> b. Or is it the result of temporary fixes for Python 2? > > My understanding is that the behavior of locals() is determined mostly > by what is convenient for the implementors, so that they can keep > regular code running as quickly as possible. The answer to the > question, "why can't we make locals() work more like I expect?" is, > "because that would make things slower." >> >> Personally, I find the convenient functionality to update the local symbol >> table inside a function, similar to `globals`. > > Can you show us an example of why you would want to update locals > through locals()? There might be more natural ways to solve your > problem. I am still working with Python 2 (Python 3 may behave differently). There, during debugging, I would sometimes like to change the value of variables (I know that the variable has got a wrong value and would like to fix it to continue senseful debugging without a restart). This works for variables not yet known inside the function but does not work for true local variables. I assume that this is one effect of the "locals()" restriction. -- https://mail.python.org/mailman/listinfo/python-list
Re: Questions about `locals` builtin
Kirill Balunov writes: > 2018-02-27 2:57 GMT+03:00 Terry Reedy : > >> The point of point 3 is that terminology and details would likely be >> different if Python were freshly designed more or less as it is today, and >> some things only make more or less sense in historical context. Learn what >> you need to know to write code that works. >> > > Thank you, I'm fairly familiar with the scope and namespace concepts in > Python 3, and they are also very well described in the "Execution model" > https://docs.python.org/3/reference/executionmodel.html#execution-model, > for this special thanks to the person who wrote it ;-) > > I started using Python with СPython 3.5 and I'm not familiar with the > Python 2 features. But since Python 2 got into our discussion, I still have > a question: > > a. Is this restriction for locals desirable in the implementation of > CPython in Python 3? > b. Or is it the result of temporary fixes for Python 2? I think it is an implementation artefact: for efficiency reasons, local variables in a function are handled differently than "local" variables elsewhere. A side effect of the concrete implementation causes that truely local variables cannot be changed by updating the result of "locals()". The "locals" documentation tries to document the restrictions. -- https://mail.python.org/mailman/listinfo/python-list
Re: Questions about `locals` builtin
2018-02-27 14:59 GMT+03:00 Ned Batchelder : > On 2/27/18 3:52 AM, Kirill Balunov wrote: > >> a. Is this restriction for locals desirable in the implementation of >> CPython in Python 3? >> b. Or is it the result of temporary fixes for Python 2? >> > > My understanding is that the behavior of locals() is determined mostly by > what is convenient for the implementors, so that they can keep regular code > running as quickly as possible. The answer to the question, "why can't we > make locals() work more like I expect?" is, "because that would make things > slower." > Ok, but I in this case what is the benefit in Python 3.3+ in returning a copy of dict instead of MappingProxy? Personally, I find the convenient functionality to update the local symbol >> table inside a function, similar to `globals`. >> > > Can you show us an example of why you would want to update locals through > locals()? There might be more natural ways to solve your problem. > > The example from "Is there are good DRY fix for this painful design pattern?" https://mail.python.org/pipermail/python-list/2018- February/731218.html class Foo: def __init__(self, bashful, doc, dopey, grumpy, happy, sleepy, sneezy): self.bashful = bashful # etc def spam(self, bashful=None, doc=None, dopey=None, grumpy=None, happy=None, sleepy=None, sneezy=None): if bashful is None: bashful = self.bashful if doc is None: doc = self.doc if dopey is None: dopey = self.dopey if grumpy is None: grumpy = self.grumpy if happy is None: happy = self.happy if sleepy is None: sleepy = self.sleepy if sneezy is None: sneezy = self.sneezy # now do the real work... def eggs(self, bashful=None, # etc... ): if bashful is None: bashful = self.bashful # and so on and with the possibility to update `locals` the `spam` can be rewritten with: def spam(self, bashful=None, doc=None, dopey=None, grumpy=None, happy=None, sleepy=None, sneezy=None): loc = locals() for key, val in loc.items(): if val is None: loc[key] = getattr(self, key) In fact, I do not have a strict opinion on this matter. And I'd rather be glad that Pнthon was less dynamic in some moments in favor of some optimizations. With kind regards, -gdg -- https://mail.python.org/mailman/listinfo/python-list
Re: Questions about `locals` builtin
On 2/27/18 3:52 AM, Kirill Balunov wrote: a. Is this restriction for locals desirable in the implementation of CPython in Python 3? b. Or is it the result of temporary fixes for Python 2? My understanding is that the behavior of locals() is determined mostly by what is convenient for the implementors, so that they can keep regular code running as quickly as possible. The answer to the question, "why can't we make locals() work more like I expect?" is, "because that would make things slower." Personally, I find the convenient functionality to update the local symbol table inside a function, similar to `globals`. Can you show us an example of why you would want to update locals through locals()? There might be more natural ways to solve your problem. --Ned. -- https://mail.python.org/mailman/listinfo/python-list
Re: Questions about `locals` builtin
2018-02-27 2:57 GMT+03:00 Terry Reedy : > The point of point 3 is that terminology and details would likely be > different if Python were freshly designed more or less as it is today, and > some things only make more or less sense in historical context. Learn what > you need to know to write code that works. > Thank you, I'm fairly familiar with the scope and namespace concepts in Python 3, and they are also very well described in the "Execution model" https://docs.python.org/3/reference/executionmodel.html#execution-model, for this special thanks to the person who wrote it ;-) I started using Python with СPython 3.5 and I'm not familiar with the Python 2 features. But since Python 2 got into our discussion, I still have a question: a. Is this restriction for locals desirable in the implementation of CPython in Python 3? b. Or is it the result of temporary fixes for Python 2? Personally, I find the convenient functionality to update the local symbol table inside a function, similar to `globals`. Of course, I do not have the full picture and the difficulties with the flaws in implementing such functionality in CPython3. On the other hand, I understand that this additional dynamism as a whole may not be beneficial, and that local and global variables are determined at function compilation time. At this point, I only know one way to achieve this: `exec ('% s =% s'% (var_name, var_val))`, which I find clumsy enough. Funny, contradictory thoughts in my head :) Calling surrounding function local names collectively 'nonlocals' is the > result of months of bikeshedding. > Yes `nonlocals` is a cool fix in Python3. 2018-02-27 3:25 GMT+03:00 Steven D'Aprano : > Mostly because locals() predates MappingProxyType by many years, and also > because that restriction doesn't apply to other implementations of Python > such as Jython and IronPython. > > In CPython, the dict returned by locals() is a copy of the local > namespace, so modifying the dict doesn't modify the real local variables. > > (Well, sometimes it does, but not always. The story in Python 2 is really > complex.) Yes, I understand all the more the documentation specifies *New in version 3.3: class.MappingProxyType.* I'm not saying that this should be a standard for the Python language, especially in the context of what I wrote above. But the Python documentation already contains links to the features of the CPython implementation, (`id` for example). If the answer to the previous question is "Yes, such a restriction is desirable in CPython because, because, because ... and thereafter it is not planned to be changed." Then I do not see why it can not be made more explicit by changing `dict` to` types.MappingProxyType`. With the following change in the documentation: "Note The contents of this dictionary should be perceived as read-only mapping and should not be modified; The returned mapping type is implementation-dependent: changes may not affect the values of local variables used by the interpreter or the returned object may not support item assignment. CPython implementation detail: The returned object is `types.MappingProxyType` a read-only proxy of the current local symbol table." p.s.: Steven, this question was somewhat inspired by yours "Is there are good DRY fix for this painful design pattern?" With kind regards, -gdg -- https://mail.python.org/mailman/listinfo/python-list
Re: Questions about `locals` builtin
On Mon, 26 Feb 2018 17:05:46 -0800, Dan Stromberg wrote: [...] > I don't have IronPython handy, but according my (quite possibly flawed) > test program, locals() is a copy on CPython 3, CPython 2, Pypy3, Pypy, > Jython and MicroPython. > > I didn't see any interpreters that returned the namespace itself. > > What am I missing? Interesting... it seems that Jython is less consistent than I remembered. Here is my test function: def test(): y = 2 ns = locals() ns['x'] = 1 ns['y'] = 1 print(x, y) return # Fool the compiler into treating x as a local. if False: x = 999 In Jython 2.5, it prints (1, 2). So assigning to locals creates a new local variable (x) but doesn't update an existing one (y). In CPython, I get: UnboundLocalError: local variable 'x' referenced before assignment in versions 3.5, 2.7, 2,5, 2,4 and even 1.5 (it's just a regular NameError in 1.5). I broke my IronPython installation, but my recollection is that it would print (1, 1). (That's what I thought Jython did too, but I was mistaken.) So correction is noted: in Jython, at least, locals() is not the namespace itself, but some sort of weird proxy to it. -- Steve -- https://mail.python.org/mailman/listinfo/python-list
Re: Questions about `locals` builtin
Kirill Balunov writes: > I am a little bit confused with `locals` builtin in these moments: > > 1. The documentation says that _free varaibles_ are returned, which seems > incorrect description. In my mind the term free variable refers to > variables used in a function that are not local variables nor parameters of > that function. > > In docs: "Update and return a dictionary representing the current local > symbol table. Free variables are returned by `locals()` when it is called > in function blocks, but not in class blocks." I agree with your definition of "free variable" and I think that the documentation partially speaks of it. In a function body, you have two kinds of "free variables": implicitly used free variables and explicitly declared (via "global", "nonlocal") free variables. I think the documentation wants to make clear, that "locals" does not treat "free variables" in a completely consistent way (there is a difference between class and function blocks). I assume that "locals" may return some (but not necessary all) free variables in a function block -- depending on the actual implementation. -- https://mail.python.org/mailman/listinfo/python-list
Re: Questions about `locals` builtin
On Mon, Feb 26, 2018 at 4:25 PM, Steven D'Aprano wrote: > On Mon, 26 Feb 2018 21:55:35 +0300, Kirill Balunov wrote: >> 2. The documentation has a note that "The contents of this dictionary >> should not be modified". Which implies that it is a read only mapping. >> So the question why it is `dict` instead of `types.MappingProxyType`? > > Mostly because locals() predates MappingProxyType by many years, and also > because that restriction doesn't apply to other implementations of Python > such as Jython and IronPython. > > In CPython, the dict returned by locals() is a copy of the local > namespace, so modifying the dict doesn't modify the real local variables. > > (Well, sometimes it does, but not always. The story in Python 2 is really > complex.) > > But in Jython and IronPython, it actually is the namespace, so writing to > it works like writing to globals. > > So writing to locals *sometimes* works, and cannot be prohibited > outright, but if you want portable code, you have to avoid it. I find this interesting. So I decided to write a little test program, to see it first hand. I don't have IronPython handy, but according my (quite possibly flawed) test program, locals() is a copy on CPython 3, CPython 2, Pypy3, Pypy, Jython and MicroPython. I didn't see any interpreters that returned the namespace itself. What am I missing? Here's the test program. Perhaps there's something wrong in it? #!/usr/local/cpython-3.6/bin/python3 """Test whether locals() is a copy of the local namespace, or a reference.""" import sys def main(): """Run the test.""" var = 1 locs = locals() locs['var'] = 2 if var == 2: sys.stdout.write('locals not just a copy\n') else: sys.stdout.write('locals is just a copy\n') main() -- https://mail.python.org/mailman/listinfo/python-list
Re: Questions about `locals` builtin
On Mon, 26 Feb 2018 21:55:35 +0300, Kirill Balunov wrote: > Hi, > > I am a little bit confused with `locals` builtin in these moments: > > 1. The documentation says that _free varaibles_ are returned, which > seems incorrect description. I can't answer that, sorry. > 2. The documentation has a note that "The contents of this dictionary > should not be modified". Which implies that it is a read only mapping. > So the question why it is `dict` instead of `types.MappingProxyType`? Mostly because locals() predates MappingProxyType by many years, and also because that restriction doesn't apply to other implementations of Python such as Jython and IronPython. In CPython, the dict returned by locals() is a copy of the local namespace, so modifying the dict doesn't modify the real local variables. (Well, sometimes it does, but not always. The story in Python 2 is really complex.) But in Jython and IronPython, it actually is the namespace, so writing to it works like writing to globals. So writing to locals *sometimes* works, and cannot be prohibited outright, but if you want portable code, you have to avoid it. > 3. There is one more moment: local variables had been determined when > function was compiled. But `locals` returns _some_ current runtime copy. Have you tried it in Python 2 or 3? The behaviour may be less confusing in 3. -- Steve -- https://mail.python.org/mailman/listinfo/python-list
Re: Questions about `locals` builtin
On 2/26/2018 1:55 PM, Kirill Balunov wrote: Hi, I am a little bit confused with `locals` builtin in these moments: 1. The documentation says that _free varaibles_ are returned, which seems incorrect description. In my mind the term free variable refers to variables used in a function that are not local variables nor parameters of that function. I disagree with the terminology used in this section of the docs, so I won't defend or explain. The important points are that 1. *Python* code is that executed in the context of multiple directly accessible namespaces; 2. there is more than one way to implement a namespace; 3. the Python namespace system has grown in complexity since the beginning; and 4. some details depend on the implementation (CPython versus others) and Python and therefore implementation version. The point of point 3 is that terminology and details would likely be different if Python were freshly designed more or less as it is today, and some things only make more or less sense in historical context. Learn what you need to know to write code that works. The program namespace is called 'builtins'. Treat it as readonly. The namespace, as opposed to its contents, cannot be accessed unless imported. Each module namespace is accessed as globals(). It must be a dict. The names of builtins are accessed as it they were in globals, even though, in CPython, they are not. (I believe 'globals' predates multi-module programs.) The local namespace is the default recipient of name bindings from assignment, class, and def statements. At module scope, the local namespace is the module or global namespace. A binding in module locals == globals can mask a builtin. 'def print(): pass'. Deletion of that binding unmasks the builtin. 'del print'. (If an implementation actually initialized each globals() by copying builtins, then each deletion would have to check for possible unmasking.) Class statements get a new local namespace, which defaults to some sort of mapping. When the class statement finishes, the local names become class attributes. Be default, function local namespaces in CPython are implemented as arrays, with local names compiles as indexes in the array. Calling surrounding function local names collectively 'nonlocals' is the result of months of bikeshedding. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Questions about `locals` builtin
Hi, I am a little bit confused with `locals` builtin in these moments: 1. The documentation says that _free varaibles_ are returned, which seems incorrect description. In my mind the term free variable refers to variables used in a function that are not local variables nor parameters of that function. In docs: "Update and return a dictionary representing the current local symbol table. Free variables are returned by `locals()` when it is called in function blocks, but not in class blocks." >>> def func(): b = 12 print(locals(), ' : ', id(locals)) c = 13 print(locals(), ' : ', id(locals)) >>> print("Free variables: ", func.__code__.co_names) >>> print("Local variables: ", func.__code__.co_varnames) Free variables: ('print', 'locals', 'id') Local variables: ('b', 'c') 2. The documentation has a note that "The contents of this dictionary should not be modified". Which implies that it is a read only mapping. So the question why it is `dict` instead of `types.MappingProxyType`? 3. There is one more moment: local variables had been determined when function was compiled. But `locals` returns _some_ current runtime copy. I find this also confusing: >>> def func1(): loc = locals() b = 12 return loc >>> def func2(): b = 12 loc = locals() return loc >>> func1() { } >>> func2() {'b': 12} With kind regards, -gdg -- https://mail.python.org/mailman/listinfo/python-list