Re: Questions about `locals` builtin

[Chris Angelico]

On Wed, Feb 28, 2018 at 10:59 PM, Steven D'Aprano
 wrote:
> On Wed, 28 Feb 2018 18:01:42 +1100, Chris Angelico wrote:
>
>> If you really want a list of ALL the local names in a function, you can
>> look at its __code__ object, which has a tuple of variable names:
>>
>> print(func1.__code__.co_varnames)
>>
>> That information is static to the function, as it is indeed determined
>> when the function is compiled.
>
> Ho ho ho, not in Python 2 it isn't!!!
>
> py> def bizarre():
> ... x = 1
> ... from math import *  # oww my aching head!
> ... print 'sin' in locals()
> ... print sin
> ...

The sin in question is that you're using a star import inside a
function. The penance is to kneel before the altar and recite PEP 20
nineteen times. And then cut down the tallest tree in the forest...
with... a debugger.

(When I tried that, I got a SyntaxWarning. Yay!)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Questions about `locals` builtin

[Chris Angelico]

On Wed, Feb 28, 2018 at 10:58 PM, Steven D'Aprano
 wrote:
> On Wed, 28 Feb 2018 18:04:11 +1100, Chris Angelico wrote:
>> But if you know that
>> there's only a handful of variables that you'd actually want to do that
>> to, you can simply put those into an object of some form, and then
>> mutate that object.
>
> o_O
>
> If I knew ahead of time which variables held the wrong value, I'd fix
> them so they held the right value ahead of time, and not bother using a
> debugger.
>
> :-)

Ahh but that requires that you know the right value as well. It's
plausible that you have a handful of "tweakables" but you have no idea
what to tweak them *to*. That's where this would be plausible.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Questions about `locals` builtin

[Steven D'Aprano]

On Wed, 28 Feb 2018 18:01:42 +1100, Chris Angelico wrote:

> If you really want a list of ALL the local names in a function, you can
> look at its __code__ object, which has a tuple of variable names:
> 
> print(func1.__code__.co_varnames)
> 
> That information is static to the function, as it is indeed determined
> when the function is compiled.

Ho ho ho, not in Python 2 it isn't!!!

py> def bizarre():
... x = 1
... from math import *  # oww my aching head!
... print 'sin' in locals()
... print sin
...
:1: SyntaxWarning: import * only allowed at module level
py> bizarre()
True

py> bizarre.__code__.co_varnames
('x',)


In Python 2, you can also use exec inside a function, for extra 
obfuscatory goodness.

Python 3 locks down these loopholes, and ensures that locals inside 
functions can be statically determined.



-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Questions about `locals` builtin

[Steven D'Aprano]

On Wed, 28 Feb 2018 18:04:11 +1100, Chris Angelico wrote:

> On Wed, Feb 28, 2018 at 5:54 PM, dieter  wrote:
[...]
>> I am still working with Python 2 (Python 3 may behave differently).
>> There, during debugging, I would sometimes like to change the value of
>> variables (I know that the variable has got a wrong value and would
>> like to fix it to continue senseful debugging without a restart). This
>> works for variables not yet known inside the function but does not work
>> for true local variables.
>> I assume that this is one effect of the "locals()" restriction.
>>
>>
> That seems like a hairy thing to do, honestly.

Well, yeah, but it is during debugging, not production code.


> But if you know that
> there's only a handful of variables that you'd actually want to do that
> to, you can simply put those into an object of some form, and then
> mutate that object.

o_O

If I knew ahead of time which variables held the wrong value, I'd fix 
them so they held the right value ahead of time, and not bother using a 
debugger.

:-)



-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Questions about `locals` builtin

[Chris Angelico]

On Wed, Feb 28, 2018 at 5:54 PM, dieter  wrote:
> Ned Batchelder  writes:
>> On 2/27/18 3:52 AM, Kirill Balunov wrote:
>>> a.  Is this restriction for locals desirable in the implementation of
>>> CPython in Python 3?
>>> b.  Or is it the result of temporary fixes for Python 2?
>>
>> My understanding is that the behavior of locals() is determined mostly
>> by what is convenient for the implementors, so that they can keep
>> regular code running as quickly as possible.  The answer to the
>> question, "why can't we make locals() work more like I expect?" is,
>> "because that would make things slower."
>>>
>>> Personally, I find the convenient functionality to update the local symbol
>>> table inside a function, similar to `globals`.
>>
>> Can you show us an example of why you would want to update locals
>> through locals()?  There might be more natural ways to solve your
>> problem.
>
> I am still working with Python 2 (Python 3 may behave differently).
> There, during debugging, I would sometimes like to change the value
> of variables (I know that the variable has got a wrong value
> and would like to fix it to continue senseful debugging without a restart).
> This works for variables not yet known inside the function but does
> not work for true local variables.
> I assume that this is one effect of the "locals()" restriction.
>

That seems like a hairy thing to do, honestly. But if you know that
there's only a handful of variables that you'd actually want to do
that to, you can simply put those into an object of some form, and
then mutate that object. That's guaranteed to work in any Python.

Personally, I'd be inclined to break things up into separate
functions, and then if you want to change state and continue, it would
be by interactively calling one of those functions with slightly
different parameters.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Questions about `locals` builtin

[Chris Angelico]

On Tue, Feb 27, 2018 at 5:55 AM, Kirill Balunov  wrote:
> 2. The documentation has a note that "The contents of this dictionary
> should not be modified". Which implies that it is a read only mapping. So
> the question why it is `dict` instead of `types.MappingProxyType`?

A dict is smaller and faster. In some Python implementations, locals()
returns an actual dictionary of the actual locals; to construct a
proxy would be a waste of effort, and to have the internal locals use
a proxy is also a complete waste. Python tends not to enforce rules
that won't actually cause a crash, so it's simpler and more efficient
to give you a real dictionary and say "changing this might not do what
you think it does".

> 3. There is one more moment: local variables had been determined when
> function was compiled. But `locals` returns _some_ current runtime copy. I
> find this also confusing:
>
>
 def func1():
>
> loc = locals()
>
> b = 12
>
> return loc
>
>
 def func2():
>
> b = 12
>
> loc = locals()
>
> return loc
>
>
 func1()
> { }
 func2()
> {'b': 12}

The values of those locals is calculated at run-time, but the set of
names is not. If a name has not been bound, it's still flagged as
"local", but it doesn't appear in locals(), because there's no
corresponding value. So if "b" is not in locals(), attempting to
access the variable named b will raise UnboundLocalError.

Dictionaries don't have a way to say "this thing exists but has no
value"; so this says "this thing doesn't exist", which is close
enough. If you really want a list of ALL the local names in a
function, you can look at its __code__ object, which has a tuple of
variable names:

print(func1.__code__.co_varnames)

That information is static to the function, as it is indeed determined
when the function is compiled.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Questions about `locals` builtin

[dieter]

Ned Batchelder  writes:
> On 2/27/18 3:52 AM, Kirill Balunov wrote:
>> a.  Is this restriction for locals desirable in the implementation of
>> CPython in Python 3?
>> b.  Or is it the result of temporary fixes for Python 2?
>
> My understanding is that the behavior of locals() is determined mostly
> by what is convenient for the implementors, so that they can keep
> regular code running as quickly as possible.  The answer to the
> question, "why can't we make locals() work more like I expect?" is,
> "because that would make things slower."
>>
>> Personally, I find the convenient functionality to update the local symbol
>> table inside a function, similar to `globals`.
>
> Can you show us an example of why you would want to update locals
> through locals()?  There might be more natural ways to solve your
> problem.

I am still working with Python 2 (Python 3 may behave differently).
There, during debugging, I would sometimes like to change the value
of variables (I know that the variable has got a wrong value
and would like to fix it to continue senseful debugging without a restart).
This works for variables not yet known inside the function but does
not work for true local variables.
I assume that this is one effect of the "locals()" restriction.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Questions about `locals` builtin

[dieter]

Kirill Balunov  writes:
>  2018-02-27 2:57 GMT+03:00 Terry Reedy :
>
>> The point of point 3 is that terminology and details would likely be
>> different if Python were freshly designed more or less as it is today, and
>> some things only make more or less sense in historical context. Learn what
>> you need to know to write code that works.
>>
>
> Thank you, I'm fairly familiar with the scope and namespace concepts in
> Python 3, and they are also very well described in the "Execution model"
> https://docs.python.org/3/reference/executionmodel.html#execution-model,
> for this special thanks to the person who wrote it ;-)
>
> I started using Python with СPython 3.5 and I'm not familiar with the
> Python 2 features. But since Python 2 got into our discussion, I still have
> a question:
>
> a.  Is this restriction for locals desirable in the implementation of
> CPython in Python 3?
> b.  Or is it the result of temporary fixes for Python 2?

I think it is an implementation artefact: for efficiency reasons,
local variables in a function are handled differently than "local"
variables elsewhere. A side effect of the concrete implementation
causes that truely local variables cannot be changed by updating
the result of "locals()". The "locals" documentation tries to
document the restrictions.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Questions about `locals` builtin

[Kirill Balunov]

 2018-02-27 14:59 GMT+03:00 Ned Batchelder :

> On 2/27/18 3:52 AM, Kirill Balunov wrote:
>
>> a.  Is this restriction for locals desirable in the implementation of
>> CPython in Python 3?
>> b.  Or is it the result of temporary fixes for Python 2?
>>
>
> My understanding is that the behavior of locals() is determined mostly by
> what is convenient for the implementors, so that they can keep regular code
> running as quickly as possible.  The answer to the question, "why can't we
> make locals() work more like I expect?" is, "because that would make things
> slower."
>

Ok, but I in this case what is the benefit in Python 3.3+ in returning a
copy of dict instead of MappingProxy?

Personally, I find the convenient functionality to update the local symbol
>> table inside a function, similar to `globals`.
>>
>
> Can you show us an example of why you would want to update locals through
> locals()?  There might be more natural ways to solve your problem.
>
>
The example from "Is there are good DRY fix for this painful design
pattern?" https://mail.python.org/pipermail/python-list/2018-
February/731218.html

class Foo:
def __init__(self, bashful, doc, dopey, grumpy,
   happy, sleepy, sneezy):
self.bashful = bashful  # etc

def spam(self, bashful=None, doc=None, dopey=None,
   grumpy=None, happy=None, sleepy=None,
   sneezy=None):
if bashful is None:
bashful = self.bashful
if doc is None:
doc = self.doc
if dopey is None:
dopey = self.dopey
if grumpy is None:
grumpy = self.grumpy
if happy is None:
happy = self.happy
if sleepy is None:
sleepy = self.sleepy
if sneezy is None:
sneezy = self.sneezy
# now do the real work...

def eggs(self, bashful=None, # etc...
   ):
if bashful is None:
bashful = self.bashful
# and so on

and with the possibility to update  `locals` the `spam` can be rewritten
with:

def spam(self, bashful=None, doc=None, dopey=None,
   grumpy=None, happy=None, sleepy=None,
   sneezy=None):
loc = locals()
for key, val in loc.items():
if val is None:
loc[key] = getattr(self, key)

In fact, I do not have a strict opinion on this matter. And I'd rather be
glad that Pнthon was less dynamic in some moments in favor of some
optimizations.


With kind regards,
-gdg
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Questions about `locals` builtin

[Ned Batchelder]

On 2/27/18 3:52 AM, Kirill Balunov wrote:

a.  Is this restriction for locals desirable in the implementation of
CPython in Python 3?
b.  Or is it the result of temporary fixes for Python 2?


My understanding is that the behavior of locals() is determined mostly 
by what is convenient for the implementors, so that they can keep 
regular code running as quickly as possible.  The answer to the 
question, "why can't we make locals() work more like I expect?" is, 
"because that would make things slower."


Personally, I find the convenient functionality to update the local symbol
table inside a function, similar to `globals`.


Can you show us an example of why you would want to update locals 
through locals()?  There might be more natural ways to solve your problem.


--Ned.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Questions about `locals` builtin

[Kirill Balunov]

 2018-02-27 2:57 GMT+03:00 Terry Reedy :

> The point of point 3 is that terminology and details would likely be
> different if Python were freshly designed more or less as it is today, and
> some things only make more or less sense in historical context. Learn what
> you need to know to write code that works.
>

Thank you, I'm fairly familiar with the scope and namespace concepts in
Python 3, and they are also very well described in the "Execution model"
https://docs.python.org/3/reference/executionmodel.html#execution-model,
for this special thanks to the person who wrote it ;-)

I started using Python with СPython 3.5 and I'm not familiar with the
Python 2 features. But since Python 2 got into our discussion, I still have
a question:

a.  Is this restriction for locals desirable in the implementation of
CPython in Python 3?
b.  Or is it the result of temporary fixes for Python 2?

Personally, I find the convenient functionality to update the local symbol
table inside a function, similar to `globals`. Of course, I do not have the
full picture and the difficulties with the flaws in implementing such
functionality in CPython3. On the other hand, I understand that this
additional dynamism as a whole may not be beneficial, and that local and
global variables are determined at function compilation time. At this
point, I only know one way to achieve this: `exec ('% s =% s'% (var_name,
var_val))`, which I find clumsy enough. Funny, contradictory thoughts in my
head :)

Calling surrounding function local names collectively 'nonlocals' is the
> result of months of bikeshedding.
>

Yes `nonlocals` is a cool fix in Python3.

  2018-02-27 3:25 GMT+03:00 Steven D'Aprano :

> Mostly because locals() predates MappingProxyType by many years, and also
> because that restriction doesn't apply to other implementations of Python
> such as Jython and IronPython.
>
> In CPython, the dict returned by locals() is a copy of the local
> namespace, so modifying the dict doesn't modify the real local variables.
>
> (Well, sometimes it does, but not always. The story in Python 2 is really
> complex.)

Yes, I understand all the more the documentation specifies *New in version
3.3: class.MappingProxyType.* I'm not saying that this should be a standard
for the Python language, especially in the context of what I wrote above.
But the Python documentation already contains links to the features of the
CPython implementation, (`id` for example). If the answer to the previous
question is "Yes, such a restriction is desirable in CPython because,
because, because ... and thereafter it is not planned to be changed." Then
I do not see why it can not be made more explicit by changing `dict` to`
types.MappingProxyType`. With the following change in the documentation:

"Note
The contents of this dictionary should be perceived as read-only mapping
and should not be modified; The returned mapping type is
implementation-dependent: changes may not affect the values of local
variables used by the interpreter or the returned object may not support
item assignment.

CPython implementation detail: The returned object is
`types.MappingProxyType` a read-only proxy of the current local symbol
table."

p.s.: Steven, this question was somewhat inspired by yours "Is there are
good DRY fix for this painful design pattern?"

With kind regards,
-gdg
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Questions about `locals` builtin

[Steven D'Aprano]

On Mon, 26 Feb 2018 17:05:46 -0800, Dan Stromberg wrote:

[...]
> I don't have IronPython handy, but according my (quite possibly flawed)
> test program, locals() is a copy on CPython 3, CPython 2, Pypy3, Pypy,
> Jython and MicroPython.
> 
> I didn't see any interpreters that returned the namespace itself.
> 
> What am I missing?

Interesting... it seems that Jython is less consistent than I remembered. 
Here is my test function:

def test():
y = 2
ns = locals()
ns['x'] = 1
ns['y'] = 1
print(x, y)
return
# Fool the compiler into treating x as a local.
if False: x = 999


In Jython 2.5, it prints (1, 2). So assigning to locals creates a new 
local variable (x) but doesn't update an existing one (y).

In CPython, I get:

UnboundLocalError: local variable 'x' referenced before assignment

in versions 3.5, 2.7, 2,5, 2,4 and even 1.5 (it's just a regular 
NameError in 1.5).

I broke my IronPython installation, but my recollection is that it would 
print (1, 1). (That's what I thought Jython did too, but I was mistaken.)

So correction is noted: in Jython, at least, locals() is not the 
namespace itself, but some sort of weird proxy to it.


-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Questions about `locals` builtin

[dieter]

Kirill Balunov  writes:
> I am a little bit confused with `locals` builtin in these moments:
>
> 1. The documentation says that _free varaibles_ are returned, which seems
> incorrect description. In my mind the term free variable refers to
> variables used in a function that are not local variables nor parameters of
> that function.
>
> In docs: "Update and return a dictionary representing the current local
> symbol table. Free variables are returned by `locals()` when it is called
> in function blocks, but not in class blocks."

I agree with your definition of "free variable" and I think that
the documentation partially speaks of it.

In a function body, you have two kinds of "free variables":
implicitly used free variables and explicitly declared (via "global",
"nonlocal") free variables.

I think the documentation wants to make clear, that "locals"
does not treat "free variables" in a completely consistent way
(there is a difference between class and function blocks).

I assume that "locals" may return some (but not necessary all)
free variables in a function block -- depending on the actual
implementation.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Questions about `locals` builtin

[Dan Stromberg]

On Mon, Feb 26, 2018 at 4:25 PM, Steven D'Aprano
 wrote:
> On Mon, 26 Feb 2018 21:55:35 +0300, Kirill Balunov wrote:
>> 2. The documentation has a note that "The contents of this dictionary
>> should not be modified". Which implies that it is a read only mapping.
>> So the question why it is `dict` instead of `types.MappingProxyType`?
>
> Mostly because locals() predates MappingProxyType by many years, and also
> because that restriction doesn't apply to other implementations of Python
> such as Jython and IronPython.
>
> In CPython, the dict returned by locals() is a copy of the local
> namespace, so modifying the dict doesn't modify the real local variables.
>
> (Well, sometimes it does, but not always. The story in Python 2 is really
> complex.)
>
> But in Jython and IronPython, it actually is the namespace, so writing to
> it works like writing to globals.
>
> So writing to locals *sometimes* works, and cannot be prohibited
> outright, but if you want portable code, you have to avoid it.

I find this interesting.

So I decided to write a little test program, to see it first hand.

I don't have IronPython handy, but according my (quite possibly
flawed) test program, locals() is a copy on CPython 3, CPython 2,
Pypy3, Pypy, Jython and MicroPython.

I didn't see any interpreters that returned the namespace itself.

What am I missing?

Here's the test program.  Perhaps there's something wrong in it?

#!/usr/local/cpython-3.6/bin/python3

"""Test whether locals() is a copy of the local namespace, or a reference."""

import sys


def main():
"""Run the test."""
var = 1

locs = locals()
locs['var'] = 2

if var == 2:
sys.stdout.write('locals not just a copy\n')
else:
sys.stdout.write('locals is just a copy\n')


main()
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Questions about `locals` builtin

[Steven D'Aprano]

On Mon, 26 Feb 2018 21:55:35 +0300, Kirill Balunov wrote:

> Hi,
> 
> I am a little bit confused with `locals` builtin in these moments:
> 
> 1. The documentation says that _free varaibles_ are returned, which
> seems incorrect description.

I can't answer that, sorry.


> 2. The documentation has a note that "The contents of this dictionary
> should not be modified". Which implies that it is a read only mapping.
> So the question why it is `dict` instead of `types.MappingProxyType`?

Mostly because locals() predates MappingProxyType by many years, and also 
because that restriction doesn't apply to other implementations of Python 
such as Jython and IronPython.

In CPython, the dict returned by locals() is a copy of the local 
namespace, so modifying the dict doesn't modify the real local variables.

(Well, sometimes it does, but not always. The story in Python 2 is really 
complex.)

But in Jython and IronPython, it actually is the namespace, so writing to 
it works like writing to globals.

So writing to locals *sometimes* works, and cannot be prohibited 
outright, but if you want portable code, you have to avoid it.


> 3. There is one more moment: local variables had been determined when
> function was compiled. But `locals` returns _some_ current runtime copy.

Have you tried it in Python 2 or 3? The behaviour may be less confusing 
in 3.



-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Questions about `locals` builtin

[Terry Reedy]

On 2/26/2018 1:55 PM, Kirill Balunov wrote:

Hi,

I am a little bit confused with `locals` builtin in these moments:

1. The documentation says that _free varaibles_ are returned, which seems
incorrect description. In my mind the term free variable refers to
variables used in a function that are not local variables nor parameters of
that function.


I disagree with the terminology used in this section of the docs, so I 
won't defend or explain.


The important points are that 1. *Python* code is that executed in the 
context of multiple directly accessible namespaces; 2. there is more 
than one way to implement a namespace; 3. the Python namespace system 
has grown in complexity since the beginning; and 4. some details depend 
on the implementation (CPython versus others) and Python and therefore 
implementation version.


The point of point 3 is that terminology and details would likely be 
different if Python were freshly designed more or less as it is today, 
and some things only make more or less sense in historical context. 
Learn what you need to know to write code that works.


The program namespace is called 'builtins'.  Treat it as readonly.  The 
namespace, as opposed to its contents, cannot be accessed unless imported.


Each module namespace is accessed as globals().  It must be a dict.  The 
names of builtins are accessed as it they were in globals, even though, 
in CPython, they are not.  (I believe 'globals' predates multi-module 
programs.)


The local namespace is the default recipient of name bindings from 
assignment, class, and def statements.


At module scope, the local namespace is the module or global namespace. 
A binding in module locals == globals can mask a builtin.  'def print(): 
pass'.  Deletion of that binding unmasks the builtin.  'del print'.  (If 
an implementation actually initialized each globals() by copying 
builtins, then each deletion would have to check for possible unmasking.)


Class statements get a new local namespace, which defaults to some sort 
of mapping.  When the class statement finishes, the local names become 
class attributes.


Be default, function local namespaces in CPython are implemented as 
arrays, with local names compiles as indexes in the array.


Calling surrounding function local names collectively 'nonlocals' is the 
result of months of bikeshedding.


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Questions about `locals` builtin

[Kirill Balunov]

Hi,

I am a little bit confused with `locals` builtin in these moments:

1. The documentation says that _free varaibles_ are returned, which seems
incorrect description. In my mind the term free variable refers to
variables used in a function that are not local variables nor parameters of
that function.

In docs: "Update and return a dictionary representing the current local
symbol table. Free variables are returned by `locals()` when it is called
in function blocks, but not in class blocks."

>>> def func():

b = 12

print(locals(), ' : ', id(locals))

c = 13

print(locals(), ' : ', id(locals))


>>> print("Free variables: ", func.__code__.co_names)
>>> print("Local variables: ", func.__code__.co_varnames)

Free variables: ('print', 'locals', 'id')
Local variables: ('b', 'c')


2. The documentation has a note that "The contents of this dictionary
should not be modified". Which implies that it is a read only mapping. So
the question why it is `dict` instead of `types.MappingProxyType`?

3. There is one more moment: local variables had been determined when
function was compiled. But `locals` returns _some_ current runtime copy. I
find this also confusing:


>>> def func1():

loc = locals()

b = 12

return loc


>>> def func2():

b = 12

loc = locals()

return loc


>>> func1()
{ }
>>> func2()
{'b': 12}


With kind regards,
-gdg
-- 
https://mail.python.org/mailman/listinfo/python-list