[issue16781] execfile/exec execution in other than global scope uses locals(), leading to undefined behavior

[Terry J. Reedy]

Terry J. Reedy  added the comment:


New changeset 863eb7170b3017399fb2b786a1e3feb6457e54c2 by Miss Islington (bot) 
in branch '3.9':
bpo-16781: In 'exec' doc, add 'nonlocal' to 'yield' and 'return' (GH-2446)
https://github.com/python/cpython/commit/863eb7170b3017399fb2b786a1e3feb6457e54c2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16781] execfile/exec execution in other than global scope uses locals(), leading to undefined behavior

[miss-islington]

miss-islington  added the comment:


New changeset 920bf6a3a656e329c2bcbb761eb8c13c46c8cd05 by Miss Islington (bot) 
in branch '3.8':
bpo-16781: In 'exec' doc, add 'nonlocal' to 'yield' and 'return' (GH-2446)
https://github.com/python/cpython/commit/920bf6a3a656e329c2bcbb761eb8c13c46c8cd05


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16781] execfile/exec execution in other than global scope uses locals(), leading to undefined behavior

[Terry J. Reedy]

Terry J. Reedy  added the comment:

PR is based on 'Note:...' directly above.  I am still thinking about a patch 
for the namespace paragraph.

--
nosy:  -miss-islington
stage: patch review -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16781] execfile/exec execution in other than global scope uses locals(), leading to undefined behavior

[miss-islington]

Change by miss-islington :


--
pull_requests: +23265
pull_request: https://github.com/python/cpython/pull/24471

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16781] execfile/exec execution in other than global scope uses locals(), leading to undefined behavior

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 6.0 -> 7.0
pull_requests: +23264
pull_request: https://github.com/python/cpython/pull/24470

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16781] execfile/exec execution in other than global scope uses locals(), leading to undefined behavior

[Terry J. Reedy]

Terry J. Reedy  added the comment:


New changeset 0ec57e25c918b859b9f8d464e34e0ac859c2f8b3 by Terry Jan Reedy in 
branch 'master':
bpo-16781: In 'exec' doc, add 'nonlocal' to 'yield' and 'return' (GH-2446)
https://github.com/python/cpython/commit/0ec57e25c918b859b9f8d464e34e0ac859c2f8b3


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16781] execfile/exec execution in other than global scope uses locals(), leading to undefined behavior

[Terry J. Reedy]

Change by Terry J. Reedy :


--
keywords: +patch
pull_requests: +23263
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/24469

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16781] execfile/exec execution in other than global scope uses locals(), leading to undefined behavior

[Irit Katriel]

Change by Irit Katriel :


--
versions: +Python 3.10, Python 3.8, Python 3.9 -Python 2.6, Python 2.7, Python 
3.2, Python 3.3, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16781] execfile/exec execution in other than global scope uses locals(), leading to undefined behavior

[Terry J. Reedy]

Terry J. Reedy added the comment:

This is at most a further doc clarification issue as the code is working as 
documented. In a previous issue, I added the following sentence to all current 
versions to try to clarify this type of behavior:  If exec gets two separate 
objects as globals and locals, the code will be executed as if it were embedded 
in a class definition. (This follows  Remember that at module level, globals 
and locals are the same dictionary) Let's try it:

class Dummy:
from ctypes import wintypes
print(wintypes.LONG)

class LOGFONT(object):
 field = wintypes.LONG

class 'ctypes.c_long'
Traceback (most recent call last):
...
  File F:\Python\mypy\tem.py, line 6, in LOGFONT
field = wintypes.LONG
NameError: name 'wintypes' is not defined

Bingo! Anatoly's result is just as documented.

The doc later says modifications to the default locals dictionary should not 
be attempted. Anatoly's result is an example of why not.

Lets distill the situation:

1. The globals dict and locals mapping passed to exec are either the same 
object or different objects. This relation determines the execution behavior.

2. If they are the same object, the code is executed as if at module scope. 
They are the same if exec is called with both defaults at module scope, where 
globals() is locals(), or if they are explicitly made the same (globals = 
locals(), locals = globals(), or globals=dic, locals=dic).

3. If they are different objects, the code is executed as if embedded in a 
(dummy) class definition. They are different if exec is called with both 
defaults within a class or function scope*, where globals() is not locals(), or 
if explicit settings leave them different (globals = dic where dic is not 
locals(), locals=map, where map is not globals, or globals=dic, locals=map, 
where dic is not map).

I believe this nails the situation#.

* In 2.x, comprehensions do not create a function scope, but in 3.x, they do. 
Lambda expressions always do. This is why I did not write 'within a class or 
function definition', as some might not see that as including comprehensions.

# The new last sentence of the second paragraph, quoted above, contradicts the 
older first sentence: In all cases, if the optional parts are omitted, the 
code is executed in the current scope. Before 2.2, when the 'current scope' of 
a function was limited to global and local namespaces, that sentence was true. 
Indeed, it summarized points 1,2,3 above. I believe that it is not true now, 
and should be revised, as nonlocal namespaces cannot be seen by exec'ed code. I 
believe I checked that before adding the new sentence, but I would recheck 
before revising. I am thinking about how to perhaps rewrite the paragraph.

--
assignee:  - docs@python
components: +Documentation -Interpreter Core
nosy: +docs@python, terry.reedy
stage:  - needs patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16781
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16781] execfile/exec execution in other than global scope uses locals(), leading to undefined behavior

[R. David Murray]

R. David Murray added the comment:

It looks like you are correct Terry.  The problem, I think, is that the 
behavior of name spaces inside a class definition is probably the least 
intuitive aspect of python scoping, so that sentence, while technically 
complete, doesn't provide enough guidance.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16781
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16781] execfile/exec execution in other than global scope uses locals(), leading to undefined behavior

[Terry J. Reedy]

Terry J. Reedy added the comment:

I suppose you could say that I kicked that particular can over to the class doc 
;-).

The fundamental problem with exec is that it is at least as complicated as 
Python, since it executes any legal python code, and in fact is even more 
complicated* because there are various possible relationships with the calling 
context. Moreover, it always returns None, so that *any* effect is a 
side-effect, which tends to be 'magical'.

* For one thing, people can write and run normal python code without knowing 
that a = b (and other binding statements, like import) at module scope means 
locals()['a'] rather than globals()['a']. At module scope, there are the same 
because globals() is locals(). Within exec'ed code, they may not be the same 
thing even for 'top level' code. This is exactly what tripped up Anatoly in his 
example with the import.

I am thinking that a short How To Exec() might be a good idea, since a real 
explanation is too much for even a half-page entry in the built-ins chapter.

Note: the following doc statement Be aware that the return and yield 
statements may not be used outside of function definitions needs to have 
nonlocal added.

 nonlocal a
SyntaxError: nonlocal declaration not allowed at module level
 exec('nonlocal a')
Traceback (most recent call last):
  File pyshell#19, line 1, in module
exec('nonlocal a')
  File string, line None
SyntaxError: nonlocal declaration not allowed at module level

 def f(): exec('nonlocal a') 

f()
...
SyntaxError: nonlocal declaration not allowed at module level

This again points to why exec can be confusing. compile() considers the string 
it compiles to be top-level code without any surrounding context. However, 
exec() enables one to run 'top level' code with different globals and locals. 
There is no exact precedent for this in normal operation. The closest is 
execution of code within a class statement (before the type(name, dic, bases) 
part). But even that is not absolutely the same for nonlocal (though this could 
be the only exception ;-).

  class C: nonlocal a
SyntaxError: no binding for nonlocal 'a' found

A different error here (arguably not the best) -- the same as

 def f(): nonlocal a
SyntaxError: no binding for nonlocal 'a' found

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16781
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16781] execfile/exec execution in other than global scope uses locals(), leading to undefined behavior

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com:


--
nosy: +Arfrever

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16781
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16781] execfile/exec execution in other than global scope uses locals(), leading to undefined behavior

[R. David Murray]

R. David Murray added the comment:

Do you mean that *modifying* locals() in the function scope is undefined 
behavior?  That makes sense, and is a documented limitation.

So we need to clarify the execfile/exec documentation to note that if called in 
anything other than the global scope, locals() will get used as the locals 
dictionary, and this will lead to undefined behavior if any operation is 
performed that updates the local namespace...and thus you are best recommend to 
always pass one or two dictionaries to execfile/exec, so that you *know* what 
is getting updated.

Although I have to say that the exec/execfile doing something that is specified 
to lead to undefined behavior smells like a bug.  (Not that we could fix it 
even if we agreed that it was, for backward compatibility reasons.)

--
title: execfile/exec execution of class statement does not access locals() - 
execfile/exec execution in other than global scope uses locals(), leading to 
undefined behavior

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16781
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16781] execfile/exec execution in other than global scope uses locals(), leading to undefined behavior

[Benjamin Peterson]

Benjamin Peterson added the comment:

The best thing would be if we could kill the default use of locals() and 
globals() in execfile, but that's probably Py4 material.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16781
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com