Re: [Tutor] set locals
On 12/18/2013 09:45 PM, Alan Gauld wrote: On 18/12/13 17:45, Mark Lawrence wrote: Can I be so bold as to ask how discussing metaclasses and __setattr__ on a tutor mailing list is going to help the newbie who's having problems with their hello world program? It won't, but the tutor list is also for experienced programmers new to Python, so it might help there. Although I do grant it's a tad more exotic (I nearly said erotic!) than our usual fare. And I'd probably expect to see it on the main Python list. But IMO it is within the group's remit - just! But there is the equally valid counterpoint that overly exotic discussion here can turn away the real newbies who are our bread and butter audience. You are right. Actually, I preferred to susbscribe to tutor (1) because there's far too much traffic on the main python list (2) because other people's questions help and exploring python further (or remember forgotten points) (3) I can at times help myself, which is also enjoyable. Thus, in fact, in practice I consider tutor as if it were a mutual help list rather than dedicated to novice (python) programmers (but I do know this is what it is intended to be). So, sorry for the occasionnal e(x|r)otic stuff; and if you ask to stop it, I won't protest. (You may even ask for it off-list, Alan, if you prefer not to pollute the list more with meta-threads.) Denis ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] set locals
spir wrote:
Hello,
is it at all possible to set new vars (or any symbol) into an existing
scope (typically locals())?
locals() normally contains a copy of the current namespace as a dict.
Setting items is possible but only alters the dict and has no effect on the
original namespace:
def f():
... x = 1
... namespace = locals()
... print(namespace)
... namespace[x] = 2
... print(namespace)
... print(x)
...
f()
{'x': 1}
{'x': 2}
1
On the module level the local is identical with the global namespace, and
you can define variables with
x
Traceback (most recent call last):
File stdin, line 1, in module
NameError: name 'x' is not defined
locals()[x] = 42
x
42
In Python 2 you can introduce local variables with exec:
def f():
... exec 'foo = bar'
... print foo
...
f()
bar
In Python 3 that is no longer possible:
def f():
... exec('foo = bar')
... print(locals())
... print(eval(foo))
... print(foo)
...
f()
{'foo': 'bar'}
bar
Traceback (most recent call last):
File stdin, line 1, in module
File stdin, line 5, in f
NameError: name 'foo' is not defined
scope[name] = value
raises by me an error like:
TypeError: 'mappingproxy' object does not support item assignment
I guess 'mappingproxy' is the implementation name of a scope (here,
local), and I thought scopes were just dicts; so what is the issue? Do you
see an alternative?
Like Steven I have no idea how you produced the mappingproxy. Are you trying
to use a class as a namespace (in Python 3.3)?
class A: pass
...
A.__dict__[foo] = bar
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: 'mappingproxy' object does not support item assignment
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] set locals
On 12/18/2013 10:02 AM, Peter Otten wrote: spir wrote: [...] Like Steven I have no idea how you produced the mappingproxy. Are you trying to use a class as a namespace (in Python 3.3)? class A: pass ... A.__dict__[foo] = bar Traceback (most recent call last): File stdin, line 1, in module TypeError: 'mappingproxy' object does not support item assignment I have no idea neither, for me it's just a term in an error message. Here is a minimal reduction of the context, that produces similar outcome (py 3.3): EDIT: you are right about class dicts, see below. def set_var(scope, id, value): scope[id] = value set_var(locals(), foo, 'FOO') assert(foo == 'FOO')# OK bar = 'bar' set_var(globals(), bar, 'BAR') assert(bar == 'BAR')# OK class C: pass set_var(C.__dict__, baz, 'BAZ') # == TypeError: 'mappingproxy' object does not support item assignment assert(C.baz == 'BAZ') # statement never reached C.baz = trial to define symbol first set_var(C.__dict__, baz, 'BAZ') # == TypeError: 'mappingproxy' object does not support item assignment assert(C.baz == 'BAZ') # statement never reached It is effectively the case that I like to use classes as namespaces (to define grammars / parsers): they are highly practicle, one can even have computation in their def! They're kind of sub-modules, in fact; free scopes. So, it was actually so that the error happened in such a case; and thus mappingproxy seems to be an implementation term for a class's dict, or so. This does explain why we cannot modify them (as we can do it in plain python code, but symbol names must be constant). But I have no solution for my issue: there is no way to write: c.id = value with id beeing a variable. Or is there? As fat as I know I really need to do c.__dict__[id_var] = value EDIT: works!!! I tried C.__setattr__(C, baz, BAZ) which fails, for any reason, with TypeError: can't apply this __setattr__ to type object But the direct way works fine, for any reason: setattr(C, baz, BAZ) Thus we can modify set_var like eg: def set_var(scope, id, value): if isinstance(scope, type): setattr(scope, id, value) else: scope[id] = value Which then enables: class C: pass set_var(C, baz, 'BAZ') assert(C.baz == 'BAZ') # OK === For the record, here in (the code of) my final tool func: def name (scope): # name all defined pattern in given scope # Note: if scope is a class, there is a distinction between the scope # and the actual dict we traverse (c.__dict__): for any reason, we # cannot set entries in a class's dict directly, but must use setattr. dic = scope.__dict__ if isinstance(scope, type) else scope # all patterns in scope (in dic, in fact): names_pats = ((name, pat) for (name, pat) in dic.items() \ if isinstance(pat, Pattern)) for (name, pat) in names_pats: # If the original pat is already named, leave it as is and # instead create a copy with the new name: if pat.name: new_pat = copy(pat) new_pat.name = name # give it its name if isinstance(scope, type): # case scope is a class setattr(scope, name, new_pat) else: # case scope is module or local scope[name] = new_pat # Else, this is a new pattern, which just demands a name: else: pat.name = name # Special case of recursive pattern, # also name its actual matching pattern: if isinstance(pat, Recurs): pat.pattern.name = name Issue solved ;-). Thank you vey much, Denis ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] set locals
On Tue, Dec 17, 2013 at 10:52 AM, spir denis.s...@gmail.com wrote:
is it at all possible to set new vars (or any symbol) into an existing scope
(typically locals())?
scope[name] = value
raises by me an error like:
TypeError: 'mappingproxy' object does not support item assignment
I guess 'mappingproxy' is the implementation name of a scope (here, local),
and I thought scopes were just dicts; so what is the issue? Do you see an
alternative?
For a CPython function, built-in locals() creates/updates the frame's
f_locals mapping against the fast locals by calling the C function
PyFrame_FastToLocals. The reverse update can be done (but why?) with
the help of an extension module or ctypes to call
PyFrame_LocalsToFast. For example:
import sys
from ctypes import *
pythonapi.PyFrame_LocalsToFast.argtypes = [py_object, c_int]
def f(x, clear=0):
if not clear:
locals()['x'] += 1
else:
del locals()['x']
frame = sys._getframe()
pythonapi.PyFrame_LocalsToFast(frame, clear)
return x
f(1)
2
If cleared this way (akin to `del x`), the fast local for x is set to
NULL (unbound), so trying to return it raises an UnboundLocalError:
f(1, clear=1)
Traceback (most recent call last):
File stdin, line 1, in module
File stdin, line 9, in f
UnboundLocalError: local variable 'x' referenced before assignment
There's no facilitated way to add new fast locals. The memory used for
the frame's stack, fast locals, and closure cells is allocated when
the frame is instantiated, based on attributes of the compiled code
object.
On the other hand, a class body is unoptimized (intentionally), so it
uses the frame's f_locals mapping (that's a dict, unless you're using
the PEP 3115 __prepare__ hook). The populated mapping gets passed to
the metaclass __new__ (e.g. type.__new__), which copies it to a new
dict for the class.
You can use locals() to your heart's content in a class body:
class Cls:
locals()['x'] = 'spam'
Cls.x
'spam'
A class's __dict__ attribute is a descriptor defined by the metaclass.
In CPython, for example, this descriptor needs to know the offset into
the PyTypeObject where the dict is stored. The code it calls is simple
enough to include here:
static PyObject *
type_dict(PyTypeObject *type, void *context)
{
if (type-tp_dict == NULL) {
Py_INCREF(Py_None);
return Py_None;
}
return PyDictProxy_New(type-tp_dict);
}
If you aren't familiar with descriptors, read the following:
http://docs.python.org/3/howto/descriptor.html
And try the following:
class Cls(object, metaclass=type):
pass
type_dict_descr = vars(type)['__dict__']
type_dict_descr.__get__(Cls, type)
Calling the __get__ method eventually makes its way to the above C
function type_dict, which creates the mappingproxy by calling
PyDictProxy_New.
The proxy wraps the type's dict to keep you from hacking it directly.
The functions to do so aren't even defined (i.e. mp_ass_subscript and
sq_ass_item). So the abstract C API function PyObject_SetItem just
raises a TypeError. If you need to set attributes dynamically, use
built-in setattr().
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] set locals
On Wed, Dec 18, 2013 at 5:40 AM, spir denis.s...@gmail.com wrote:
C.__setattr__(C, baz, BAZ)
which fails, for any reason, with
TypeError: can't apply this __setattr__ to type object
You need __setattr__ from the metaclass:
class C: pass
...
type(C).__setattr__(C, baz, BAZ)
C.baz
'BAZ'
You can bind the method like this:
C_setattr = type(C).__setattr__.__get__(C)
C_setattr('foo', 'bar')
C.foo
'bar'
But just use built-in setattr(), really.
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] set locals
On 12/18/2013 12:07 PM, eryksun wrote:
On Wed, Dec 18, 2013 at 5:40 AM, spir denis.s...@gmail.com wrote:
C.__setattr__(C, baz, BAZ)
which fails, for any reason, with
TypeError: can't apply this __setattr__ to type object
You need __setattr__ from the metaclass:
class C: pass
...
type(C).__setattr__(C, baz, BAZ)
C.baz
'BAZ'
Oh, that makes sense: so, __setattr__ on a class is for its instances, right?
(thus, to set attrs on a class itself, we need ___setattr__ from its own class,
if I understand rightly?)
You can bind the method like this:
C_setattr = type(C).__setattr__.__get__(C)
C_setattr('foo', 'bar')
C.foo
'bar'
But just use built-in setattr(), really.
Really, yes!
Denis
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] set locals
On 18/12/2013 11:16, spir wrote:
On 12/18/2013 12:07 PM, eryksun wrote:
On Wed, Dec 18, 2013 at 5:40 AM, spir denis.s...@gmail.com wrote:
C.__setattr__(C, baz, BAZ)
which fails, for any reason, with
TypeError: can't apply this __setattr__ to type object
You need __setattr__ from the metaclass:
class C: pass
...
type(C).__setattr__(C, baz, BAZ)
C.baz
'BAZ'
Oh, that makes sense: so, __setattr__ on a class is for its instances,
right? (thus, to set attrs on a class itself, we need ___setattr__ from
its own class, if I understand rightly?)
You can bind the method like this:
C_setattr = type(C).__setattr__.__get__(C)
C_setattr('foo', 'bar')
C.foo
'bar'
But just use built-in setattr(), really.
Really, yes!
Denis
Can I be so bold as to ask how discussing metaclasses and __setattr__ on
a tutor mailing list is going to help the newbie who's having problems
with their hello world program?
--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.
Mark Lawrence
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] set locals
On 18/12/13 17:45, Mark Lawrence wrote: Can I be so bold as to ask how discussing metaclasses and __setattr__ on a tutor mailing list is going to help the newbie who's having problems with their hello world program? It won't, but the tutor list is also for experienced programmers new to Python, so it might help there. Although I do grant it's a tad more exotic (I nearly said erotic!) than our usual fare. And I'd probably expect to see it on the main Python list. But IMO it is within the group's remit - just! But there is the equally valid counterpoint that overly exotic discussion here can turn away the real newbies who are our bread and butter audience. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] set locals
On Wed, Dec 18, 2013 at 05:45:02PM +, Mark Lawrence wrote: Can I be so bold as to ask how discussing metaclasses and __setattr__ on a tutor mailing list is going to help the newbie who's having problems with their hello world program? It's not just newbies who need tutoring. Sometimes I've learned things here too. -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] set locals
On Wed, Dec 18, 2013 at 6:16 AM, spir denis.s...@gmail.com wrote:
On 12/18/2013 12:07 PM, eryksun wrote:
You need __setattr__ from the metaclass:
class C: pass
...
type(C).__setattr__(C, baz, BAZ)
C.baz
'BAZ'
Oh, that makes sense: so, __setattr__ on a class is for its instances,
right? (thus, to set attrs on a class itself, we need ___setattr__ from its
own class, if I understand rightly?)
Note that vars(type)['__dict__'] is a data descriptor, so the lookup
gives it precedence. There actually is a vars(C)['__dict__'], the
descriptor meant for instances of C, but the lookup stops before it
gets there. On the other hand, vars(type)['__setattr__'] is a non-data
descriptor (i.e. no __set__ method), so the lookup for C.__setattr__
skips it to bind and return vars(object)['__setattr__'] instead.
That's akin to shadowing a method with instance data. For example:
c = C()
c.__setattr__ = 'spam'
vars(c)
{'__setattr__': 'spam'}
c.__setattr__
'spam'
Here are 2 reasons why object.__setattr__ is wrong for a CPython type object:
type.__setattr__ includes a check to prevent setting
and deleting attributes of built-in types.
type.__setattr__ updates the affected slot (C struct
field) in the type and its subclasses. This also
invalidates the method cache for the affected types.
For example, if you just modify a type's dict entry for '__len__',
this doesn't update the sq_length and mp_length slots in the
PyHeapTypeObject, so built-in len() called on an instance will raise a
TypeError. Hence also the reason that modifying the dict directly is
disallowed by the __dict__ proxy.
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
[Tutor] set locals
Hello, is it at all possible to set new vars (or any symbol) into an existing scope (typically locals())? scope[name] = value raises by me an error like: TypeError: 'mappingproxy' object does not support item assignment I guess 'mappingproxy' is the implementation name of a scope (here, local), and I thought scopes were just dicts; so what is the issue? Do you see an alternative? Context: This is used in a tool func that names defined objects of a given (matching patterns) in a given scope (which forms a grammar/parser). The base issue is that objects don't know their names, which in my case (and many more) are useful, in fact necessary, for programmer feedback, testing, debugging and more. However, at times a developper needs variants of a given pattern (matching same format, eg name=symbol), but they should be distinct objects: meaning copy, meaning setting the new variant into the scope. The tool func (a static method of Pattern, in fact), is just: def name (scope): ''' Name all defined patterns of a given grammar's scope. [... more doc ...] ''' for name, pat in scope.items(): if isinstance(pat, Pattern): # Just name pat: pat.name = name # ... plus detail ... But the body should be: for name, pat in scope.items(): if isinstance(pat, Pattern): # Possibly make copy in case pat is already named: if pat.name: new_pat = copy(pat) new_pat.name = name scope[name] = new_pat # error *** # Else, just name pat: else: pat.name = name # ... plus detail ... I'm blocked. Users can do it by hand, but it's pretty annoying and a sure source of bugs (there is more than just naming: if users define a variant, they probably want to define a different match action: if no copy, then the action is set on the original pattern...). This should be automagic. Denis ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] set locals
On Tue, Dec 17, 2013 at 04:52:25PM +0100, spir wrote: Hello, is it at all possible to set new vars (or any symbol) into an existing scope (typically locals())? In general, no. The only time that writing to locals() is guaranteed to work is when you are in the top-level scope and locals returns the same dict as globals(). scope[name] = value raises by me an error like: TypeError: 'mappingproxy' object does not support item assignment I guess 'mappingproxy' is the implementation name of a scope (here, local), I cannot reproduce locals() returning a mappingproxy. What version of Python are you using, and how did you generate scope? -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
