Re: [Python-ideas] Enhancing vars()

[Steven D'Aprano]

On Wed, Dec 14, 2016 at 12:12:39AM +, Emanuel Barry wrote:
> > From Steven D'Aprano
> > Sent: Tuesday, December 13, 2016 6:49 PM
> > To: python-ideas@python.org
> > Subject: Re: [Python-ideas] Enhancing vars()
> > 
> > But if the object has __slots__, with or without a __dict__, then vars
> > should return a proxy which direct reads and writes to the correct slot
> > or dict.
> > 
> > It might be helpful to have a slotsproxy object which provides a
> > dict-like interface to an object with __slots__ but no __dict__, and
> > build support for both __slots__ and a __dict__ on top of that.
> 
> That might be a bit tricky, for example, it's possible that a class has a
> `foo` slot *and* a `foo` instance attribute (by virtue of subclasses). What
> would you do in that case?

vars() shouldn't need to care about inheritance: it only cares about the 
object's own individual namespace, not attributes inherited from the 
class or superclasses. That's how vars() works now:

py> class C:
... cheese = 1
...
py> obj = C()
py> ns = vars(obj)
py> 'cheese' in ns
False

The only difference here is that if the direct parent class has 
__slots__, the instance will use them instead of (or in addition to) a 
__dict__. We don't need to care about superclass __slots__, because they 
aren't inherited.



> Or what if there's a slot that doesn't have any
> value (i.e. raises AttributeError on access, but exists on the class
> nonetheless), but an instance attribute with the same name exists? And so
> on.

Only the *list of slot names* exists on the class. The slots themselves 
are part of the instance. Nevertheless, you are right: a slot can be 
defined, but not assigned to. That has to be treated as if the slot 
didn't exist:

py> class D:
... __slots__ = ['spam']
...
py> d = D()
py> hasattr(d, 'spam')
False

So I would expect that 

'spam' in vars(d)

should likewise return False, until such time that d.spam is assigned 
too.

The same applies even if the object has a __dict__. The slot always 
takes precedence, even if the slot isn't filled in.

py> class E:
... __slots__ = ['spam', '__dict__']
...
py> e = E()
py> e.__dict__['spam'] = 1
py> hasattr(e, 'spam')
False



> > If the objects has *neither* __slots__ nor __dict__, vars can probably
> > raise a TypeError.
> 
> Is that even possible in pure Python? The only object I know that can do
> this is `object`, but some other C objects might do that too.

I don't think pure Python classes can do this, at least not without some 
metaclass trickery, but certainly `object` itself lacks both __slots__ 
and instance __dict__, and C objects can do the same (so I'm told).



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Enhancing vars()

[Steven D'Aprano]

On Mon, Dec 12, 2016 at 10:45:39PM -0500, Alexander Belopolsky wrote:
> On Mon, Dec 12, 2016 at 6:45 PM, Steven D'Aprano 
> wrote:
> 
> > Proposal: enhance vars() to return a proxy to the object namespace,
> > regardless of whether said namespace is __dict__ itself, or a number of
> > __slots__, or both.
> >
> 
> How do you propose dealing with classes defined in C?  Their objects don't
> have __slots__.

I don't see any clean way to do so. Maybe we should have a convention 
that such objects provide a __slots__ attribute listing public 
attributes, but I'm not too concerned. Let vars(weird_c_object) raise 
TypeError, just as it does now.


> One possibility is to use __dir__ or dir(), but those can return anything
> and in the past developers
> were encouraged to put only "useful" attributes in __dir__.

Indeed.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Enhancing vars()

[Steven D'Aprano]

On Wed, Dec 14, 2016 at 12:58:16AM +0200, Serhiy Storchaka wrote:
> On 13.12.16 01:45, Steven D'Aprano wrote:
> >One of the lesser-known ones is vars(obj), which should be used in place
> >of obj.__dict__.
> >
> >Unfortunately, vars() is less useful than it might be, since not all
> >objects have a __dict__. Some objects have __slots__ instead, or even
> >both. That is considered an implementation detail of the object.
> 
> http://bugs.python.org/issue13290
> http://mail.python.org/pipermail/python-dev/2012-October/122011.html

Thanks Serhiy!

Glad to see I'm not the only one with this idea.

I think:

- the behaviour of locals() (and vars() when given no argument, where it 
returns locals()) is anomalous and should not be copied unless we really 
need to.

- Other Python implementations don't always emulate the weird behaviour 
of locals(), for example I think IronPython locals() is writeable, and 
the local variables do change.

steve@orac:~$ ipy
IronPython 2.6 Beta 2 DEBUG (2.6.0.20) on .NET 2.0.50727.1433
Type "help", "copyright", "credits" or "license" for more information.
>>> def test():
... a = 1
... locals()['a'] = 99
... print a
...
>>> test()
99

CPython will print 1 instead.

So CPython locals() is an implementation detail and we shouldn't feel 
the need to copy it's weird behaviour.

When given an object, vars(obj) should return a dict-like object which 
is a read/write proxy to the object's namespace. If the object has a 
__dict__ but no __slots__, then there's no need to change anything: it 
can keep the current behaviour and just return the dict itself:

assert vars(obj) is obj.__dict__


But if the object has __slots__, with or without a __dict__, then vars 
should return a proxy which direct reads and writes to the correct slot 
or dict.

It might be helpful to have a slotsproxy object which provides a 
dict-like interface to an object with __slots__ but no __dict__, and 
build support for both __slots__ and a __dict__ on top of that.

If the objects has *neither* __slots__ nor __dict__, vars can probably 
raise a TypeError.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Enhancing vars()

[Serhiy Storchaka]

On 13.12.16 01:45, Steven D'Aprano wrote:

One of the lesser-known ones is vars(obj), which should be used in place
of obj.__dict__.

Unfortunately, vars() is less useful than it might be, since not all
objects have a __dict__. Some objects have __slots__ instead, or even
both. That is considered an implementation detail of the object.


http://bugs.python.org/issue13290
http://mail.python.org/pipermail/python-dev/2012-October/122011.html


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Enhancing vars()

[Stephen J. Turnbull]

Nick Coghlan writes:

 > (With the lack of an underscore being due to the precedent set by
 > dict.fromkeys())
 > 
 > Armed with those, the "give me all the attributes from __dir__"
 > command would be:
 > 
 > attrs = dict.from_attrs(obj, dir(obj))
A
Urk +

You sure you want to follow precedent?  My fingers really like that
typo, too!

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Enhancing vars()

[Nick Coghlan]

On 13 December 2016 at 20:02, Steven D'Aprano  wrote:
> On Tue, Dec 13, 2016 at 10:29:38AM +0100, Marco Buttu wrote:
>
>> +1. Would it be possible in the future (Py4?) to change the name `vars`
>> to a more meaningful name? Maybe `namespace`, or something more appropriate.
>
> I'm not keen on the name vars() either, but it does make a certain
> sense: short for "variables", where "variable" here refers to attributes
> of an instance rather than local or global variables.

It also refers to local and global variables, as vars() is effectively
an alias for locals() if you don't pass an argument, and locals() is
effectively an alias for globals() at module level:

>>> locals() is globals()
True
>>> vars() is globals()
True
>>> def f(): return vars() is locals()
...
>>> f()
True

To be honest, rather than an enhanced vars(), I'd prefer to see a
couple more alternate dict constructors:

dict.fromattrs(obj, attributes)
dict.fromitems(obj, keys)

(With the lack of an underscore being due to the precedent set by
dict.fromkeys())

Armed with those, the "give me all the attributes from __dir__"
command would be:

attrs = dict.from_attrs(obj, dir(obj))

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Enhancing vars()

[Steven D'Aprano]

On Tue, Dec 13, 2016 at 10:29:38AM +0100, Marco Buttu wrote:

> +1. Would it be possible in the future (Py4?) to change the name `vars` 
> to a more meaningful name? Maybe `namespace`, or something more appropriate.

I'm not keen on the name vars() either, but it does make a certain 
sense: short for "variables", where "variable" here refers to attributes 
of an instance rather than local or global variables.

I'm not sure that namespace is a better name: namespace, it seems to me, 
it likely to be used as the name of the target:

namespace = vars(obj)

But if there is a lot of popular demand for a name change, then I 
suppose it could happen.

Ask again around Python 3.9 :-)



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Enhancing vars()

[Marco Buttu]

On 13/12/2016 00:45, Steven D'Aprano wrote:


In general, directly accessing dunders is a bit of a code smell. (I
exclude writing dunder methods in your classes, of course.) There's
usually a built-in or similar to do the job for you, e.g. instead of
iterator.__next__() we should use next(iterator).

One of the lesser-known ones is vars(obj), which should be used in place
of obj.__dict__.

[...]

Proposal: enhance vars() to return a proxy to the object namespace,
regardless of whether said namespace is __dict__ itself, or a number of
__slots__, or both.


+1. Would it be possible in the future (Py4?) to change the name `vars` 
to a more meaningful name? Maybe `namespace`, or something more appropriate.


--
Marco Buttu

INAF-Osservatorio Astronomico di Cagliari
Via della Scienza n. 5, 09047 Selargius (CA)
Phone: 070 711 80 217
Email: mbu...@oa-cagliari.inaf.it

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Enhancing vars()

[Steve Dower]

I'm +1. This bites me far too often.

> in the past developers
were encouraged to put only "useful" attributes in __dir__.

Good. If I'm getting vars() I really only want the useful ones. If I need 
interesting/secret ones then I'll getattr for them.

Cheers,
Steve

Top-posted from my Windows Phone

-Original Message-
From: "Alexander Belopolsky" <alexander.belopol...@gmail.com>
Sent: ‎12/‎12/‎2016 19:47
To: "Steven D'Aprano" <st...@pearwood.info>
Cc: "python-ideas" <python-ideas@python.org>
Subject: Re: [Python-ideas] Enhancing vars()



On Mon, Dec 12, 2016 at 6:45 PM, Steven D'Aprano <st...@pearwood.info> wrote:

Proposal: enhance vars() to return a proxy to the object namespace,
regardless of whether said namespace is __dict__ itself, or a number of
__slots__, or both.

How do you propose dealing with classes defined in C?  Their objects don't have 
__slots__.


One possibility is to use __dir__ or dir(), but those can return anything and 
in the past developers
were encouraged to put only "useful" attributes in __dir__.___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Enhancing vars()

[Ethan Furman]

On 12/12/2016 03:45 PM, Steven D'Aprano wrote:


Proposal: enhance vars() to return a proxy to the object namespace,
regardless of whether said namespace is __dict__ itself, or a number of
__slots__, or both.


+1

--
~Ethan~
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Enhancing vars()

[Terry Reedy]

On 12/12/2016 6:45 PM, Steven D'Aprano wrote:

In general, directly accessing dunders is a bit of a code smell. (I
exclude writing dunder methods in your classes, of course.) There's
usually a built-in or similar to do the job for you, e.g. instead of
iterator.__next__() we should use next(iterator).

One of the lesser-known ones is vars(obj), which should be used in place
of obj.__dict__.

Unfortunately, vars() is less useful than it might be, since not all
objects have a __dict__. Some objects have __slots__ instead, or even
both. That is considered an implementation detail of the object.

Proposal: enhance vars() to return a proxy to the object namespace,
regardless of whether said namespace is __dict__ itself, or a number of
__slots__, or both. Here is a woefully incompete and untested prototype:


+1  I believe this was mentioned as a possibility on some issue , but I 
cannot find it.  Does vars currently work for things with dict proxies 
instead of dicts?



class VarsProxy(object):
def __init__(self, obj):
if not (hasattr(obj, '__dict__') or hasattr(obj, '__slots__')):
raise TypeError('object has no namespace')
self._obj = obj

def __getitem__(self, key):
slots = getattr(type(self), '__slots__', None)
# see inspect.getattr__static for a more correct implementation
if slots is not None and key in slots:
# return the content of the slot, without any inheritance.
return getattr(self._obj, key)
else:
return self._obj.__dict__[key]

def __setitem__(self, key, value): ...
def __delitem__(self, key): ...



One complication: it is possible for the slot and the __dict__ to
both contain the key. In 3.5 that ambiguity is resolved in favour of the
slot:

py> class X:
... __slots__ = ['spam', '__dict__']
... def __init__(self):
... self.spam = 'slot'
... self.__dict__['spam'] = 'dict'
...
py> x = X()
py> x.spam
'slot'


Although __slots__ are uncommon, this would clearly distinguish
vars(obj) from obj.__dict__ and strongly encourage the use of vars()
over direct access to the dunder attribute.


Thoughts?






--
Terry Jan Reedy

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/