[Python-Dev] __dir__, part 2
so, if you remember, i suggested adding __dir__ to objects, so as to make dir() customizable, remove the deprecated __methods__ and __members__, and make it symmetrical to other built-in functions. you can see the original post here: http://mail.python.org/pipermail/python-dev/2006-July/067095.html which was generally accepted by the forum: http://mail.python.org/pipermail/python-dev/2006-July/067139.html so i went on, now that i have some spare time, to research the issue. the current dir() works as follows: (*) builtin_dir calls PyObject_Dir to do the trick (*) if the object is NULL (dir with no argument), return the frame's locals (*) if the object is a *module*, we're just using it's __dict__ (*) if the object is a *type*, we're using it's __dict__ and __bases__, but not __class__ (so as not to show the metaclass) (*) otherwise, it's a "normal object", so we take it's __dict__, along with __methods__, __members__, and dir(__class__) (*) create a list of keys from the dict, sort, return we'll have to change that if we were to introduce __dir__. my design is: (*) builtin_dir, if called without an argument, returns the frame's locals (*) otherwise, it calls PyObject_Dir(self), which would dispatch self.__dir__() (*) if `self` doesn't have __dir__, default to object.__dir__(self) (*) the default object.__dir__ implementation would do the same as today: collect __dict__, __members__, __methods__, and dir(__class__). by py3k, we'll remove looking into __methods__ and __members__. (*) type objects and module objects would implement __dir__ to their liking (as PyObject_Dir does today) (*) builtin_dir would take care of sorting the list returned by PyObject_Dir so first i'd want you people to react on my design, maybe you'd find flaws whatever. also, should this become a PEP? and last, how do i add a new method slot? does it mean i need to change all type-object definitions throughout the codebase? do i add it to some protocol? or directly to the "object protocol"? -tomer ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __dir__, part 2
Sounds like a good plan, though I'm not sure if it's worth doing in 2.6 -- I'd be happy with doing this just in 3k. I'm not sure what you mean by "adding a method slot" -- certainly it's possible to define a method __foo__ and call it directly without having a special tp_foo in the type object, and I recommend doing it that way since the tp_foo slots are just there to make things fast; in this case I don't see a need for dir() to be fast. --Guido On 11/6/06, tomer filiba <[EMAIL PROTECTED]> wrote: > so, if you remember, i suggested adding __dir__ to objects, so as to make > dir() customizable, remove the deprecated __methods__ and __members__, > and make it symmetrical to other built-in functions. > > you can see the original post here: > http://mail.python.org/pipermail/python-dev/2006-July/067095.html > which was generally accepted by the forum: > http://mail.python.org/pipermail/python-dev/2006-July/067139.html > > so i went on, now that i have some spare time, to research the issue. > the current dir() works as follows: > (*) builtin_dir calls PyObject_Dir to do the trick > (*) if the object is NULL (dir with no argument), return the frame's locals > (*) if the object is a *module*, we're just using it's __dict__ > (*) if the object is a *type*, we're using it's __dict__ and __bases__, > but not __class__ (so as not to show the metaclass) > (*) otherwise, it's a "normal object", so we take it's __dict__, along with > __methods__, __members__, and dir(__class__) > (*) create a list of keys from the dict, sort, return > > we'll have to change that if we were to introduce __dir__. my design is: > (*) builtin_dir, if called without an argument, returns the frame's locals > (*) otherwise, it calls PyObject_Dir(self), which would dispatch > self.__dir__() > (*) if `self` doesn't have __dir__, default to object.__dir__(self) > (*) the default object.__dir__ implementation would do the same as > today: collect __dict__, __members__, __methods__, and dir(__class__). > by py3k, we'll remove looking into __methods__ and __members__. > (*) type objects and module objects would implement __dir__ to their > liking (as PyObject_Dir does today) > (*) builtin_dir would take care of sorting the list returned by PyObject_Dir > > so first i'd want you people to react on my design, maybe you'd find > flaws whatever. also, should this become a PEP? > > and last, how do i add a new method slot? does it mean i need to > change all type-object definitions throughout the codebase? > do i add it to some protocol? or directly to the "object protocol"? > > > -tomer > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __dir__, part 2
cool. first time i build the entire interpreter, 'twas fun :) currently i "retained" support for __members__ and __methods__, so it doesn't break anything and is compatible with 2.6. i really hope it will be included in 2.6 as today i'm using ugly hacks in RPyC to make remote objects appear like local ones. having __dir__ solves all of my problems. besides, it makes a lot of sense of define __dir__ for classes that define __getattr__. i don't think it should be pushed back to py3k. here's the patch: http://sourceforge.net/tracker/index.php?func=detail&aid=1591665&group_id=5470&atid=305470 here's a demo: >>> class foo(object): ... def __dir__(self): ... return ["kan", "ga", "roo"] ... >>> f = foo() >>> f <__main__.foo object at 0x00A90C78> >>> dir() ['__builtins__', '__doc__', '__name__', 'f', 'foo'] >>> dir(f) ['ga', 'kan', 'roo'] >>> dir(foo) ['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__getattribute__ ', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__ ', '__repr__', '__setattr__', '__str__', '__weakref__'] >>> class bar(object): ... __members__ = ["bow", "wow"] ... >>> b=bar() >>> dir(b) ['__class__', '__delattr__', '__dict__', '__doc__', '__getattribute__', '__hash_ _', '__init__', '__members__', '__module__', '__new__', '__reduce__', '__reduce_ ex__', '__repr__', '__setattr__', '__str__', '__weakref__', 'bow', 'wow'] -tomer On 11/6/06, Guido van Rossum <[EMAIL PROTECTED]> wrote: > Sounds like a good plan, though I'm not sure if it's worth doing in > 2.6 -- I'd be happy with doing this just in 3k. > > I'm not sure what you mean by "adding a method slot" -- certainly it's > possible to define a method __foo__ and call it directly without > having a special tp_foo in the type object, and I recommend doing it > that way since the tp_foo slots are just there to make things fast; in > this case I don't see a need for dir() to be fast. > > --Guido > > On 11/6/06, tomer filiba <[EMAIL PROTECTED]> wrote: > > so, if you remember, i suggested adding __dir__ to objects, so as to make > > dir() customizable, remove the deprecated __methods__ and __members__, > > and make it symmetrical to other built-in functions. > > > > you can see the original post here: > > http://mail.python.org/pipermail/python-dev/2006-July/067095.html > > which was generally accepted by the forum: > > http://mail.python.org/pipermail/python-dev/2006-July/067139.html > > > > so i went on, now that i have some spare time, to research the issue. > > the current dir() works as follows: > > (*) builtin_dir calls PyObject_Dir to do the trick > > (*) if the object is NULL (dir with no argument), return the frame's locals > > (*) if the object is a *module*, we're just using it's __dict__ > > (*) if the object is a *type*, we're using it's __dict__ and __bases__, > > but not __class__ (so as not to show the metaclass) > > (*) otherwise, it's a "normal object", so we take it's __dict__, along with > > __methods__, __members__, and dir(__class__) > > (*) create a list of keys from the dict, sort, return > > > > we'll have to change that if we were to introduce __dir__. my design is: > > (*) builtin_dir, if called without an argument, returns the frame's locals > > (*) otherwise, it calls PyObject_Dir(self), which would dispatch > > self.__dir__() > > (*) if `self` doesn't have __dir__, default to object.__dir__(self) > > (*) the default object.__dir__ implementation would do the same as > > today: collect __dict__, __members__, __methods__, and dir(__class__). > > by py3k, we'll remove looking into __methods__ and __members__. > > (*) type objects and module objects would implement __dir__ to their > > liking (as PyObject_Dir does today) > > (*) builtin_dir would take care of sorting the list returned by PyObject_Dir > > > > so first i'd want you people to react on my design, maybe you'd find > > flaws whatever. also, should this become a PEP? > > > > and last, how do i add a new method slot? does it mean i need to > > change all type-object definitions throughout the codebase? > > do i add it to some protocol? or directly to the "object protocol"? > > > > > > -tomer > > ___ > > Python-Dev mailing list > > Python-Dev@python.org > > http://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: > > http://mail.python.org/mailman/options/python-dev/guido%40python.org > > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __dir__, part 2
tomer filiba wrote: > cool. first time i build the entire interpreter, 'twas fun :) > currently i "retained" support for __members__ and __methods__, > so it doesn't break anything and is compatible with 2.6. > > i really hope it will be included in 2.6 as today i'm using ugly hacks > in RPyC to make remote objects appear like local ones. > having __dir__ solves all of my problems. > > besides, it makes a lot of sense of define __dir__ for classes that > define __getattr__. i don't think it should be pushed back to py3k. > > here's the patch: > http://sourceforge.net/tracker/index.php?func=detail&aid=1591665&group_id=5470&atid=305470 As I noted on the tracker, PyObject_Dir is a public C API function, so it's behaviour needs to be preserved as well as the behaviour of calling dir() from Python code. So the final form of the patch will likely need to include stronger tests for that section of the API, as well as updating the documentation in various places (the dir and PyObject_Dir documentation, obviously, but also the list of magic methods in the language reference). +1 on targeting 2.6, too. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __dir__, part 2
No objection on targetting 2.6 if other developers agree. Seems this is well under way. good work! On 11/6/06, Nick Coghlan <[EMAIL PROTECTED]> wrote: > tomer filiba wrote: > > cool. first time i build the entire interpreter, 'twas fun :) > > currently i "retained" support for __members__ and __methods__, > > so it doesn't break anything and is compatible with 2.6. > > > > i really hope it will be included in 2.6 as today i'm using ugly hacks > > in RPyC to make remote objects appear like local ones. > > having __dir__ solves all of my problems. > > > > besides, it makes a lot of sense of define __dir__ for classes that > > define __getattr__. i don't think it should be pushed back to py3k. > > > > here's the patch: > > http://sourceforge.net/tracker/index.php?func=detail&aid=1591665&group_id=5470&atid=305470 > > As I noted on the tracker, PyObject_Dir is a public C API function, so it's > behaviour needs to be preserved as well as the behaviour of calling dir() from > Python code. > > So the final form of the patch will likely need to include stronger tests for > that section of the API, as well as updating the documentation in various > places (the dir and PyObject_Dir documentation, obviously, but also the list > of magic methods in the language reference). > > +1 on targeting 2.6, too. > > Cheers, > Nick. > > -- > Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia > --- > http://www.boredomandlaziness.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __dir__, part 2
On Tuesday 07 November 2006 19:00, Guido van Rossum wrote: > No objection on targetting 2.6 if other developers agree. Seems this > is well under way. good work! Sounds fine to me! Less magic under the hood is less magic, and that's always a good thing. The use case for it seems completely appropriate, too. Anthony -- Anthony Baxter <[EMAIL PROTECTED]> It's never too late to have a happy childhood. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __dir__, part 2
okay, everything's fixed. i updated the patch and added a small test to: Lib/test/test_builtins.py::test_dir -tomer On 11/7/06, Nick Coghlan <[EMAIL PROTECTED]> wrote: > tomer filiba wrote: > > cool. first time i build the entire interpreter, 'twas fun :) > > currently i "retained" support for __members__ and __methods__, > > so it doesn't break anything and is compatible with 2.6. > > > > i really hope it will be included in 2.6 as today i'm using ugly hacks > > in RPyC to make remote objects appear like local ones. > > having __dir__ solves all of my problems. > > > > besides, it makes a lot of sense of define __dir__ for classes that > > define __getattr__. i don't think it should be pushed back to py3k. > > > > here's the patch: > > http://sourceforge.net/tracker/index.php?func=detail&aid=1591665&group_id=5470&atid=305470 > > As I noted on the tracker, PyObject_Dir is a public C API function, so it's > behaviour needs to be preserved as well as the behaviour of calling dir() from > Python code. > > So the final form of the patch will likely need to include stronger tests for > that section of the API, as well as updating the documentation in various > places (the dir and PyObject_Dir documentation, obviously, but also the list > of magic methods in the language reference). > > +1 on targeting 2.6, too. > > Cheers, > Nick. > > -- > Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia > --- > http://www.boredomandlaziness.org > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __dir__, part 2
> as well as updating the documentation in various > places (the dir and PyObject_Dir documentation, obviously, but also the list > of magic methods in the language reference). oops, i meant everything except that -tomer On 11/7/06, tomer filiba <[EMAIL PROTECTED]> wrote: > okay, everything's fixed. > i updated the patch and added a small test to: > Lib/test/test_builtins.py::test_dir > > > -tomer > > On 11/7/06, Nick Coghlan <[EMAIL PROTECTED]> wrote: > > tomer filiba wrote: > > > cool. first time i build the entire interpreter, 'twas fun :) > > > currently i "retained" support for __members__ and __methods__, > > > so it doesn't break anything and is compatible with 2.6. > > > > > > i really hope it will be included in 2.6 as today i'm using ugly hacks > > > in RPyC to make remote objects appear like local ones. > > > having __dir__ solves all of my problems. > > > > > > besides, it makes a lot of sense of define __dir__ for classes that > > > define __getattr__. i don't think it should be pushed back to py3k. > > > > > > here's the patch: > > > http://sourceforge.net/tracker/index.php?func=detail&aid=1591665&group_id=5470&atid=305470 > > > > As I noted on the tracker, PyObject_Dir is a public C API function, so it's > > behaviour needs to be preserved as well as the behaviour of calling dir() > > from > > Python code. > > > > So the final form of the patch will likely need to include stronger tests > > for > > that section of the API, as well as updating the documentation in various > > places (the dir and PyObject_Dir documentation, obviously, but also the list > > of magic methods in the language reference). > > > > +1 on targeting 2.6, too. > > > > Cheers, > > Nick. > > > > -- > > Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia > > --- > > http://www.boredomandlaziness.org > > > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __dir__, part 2
Guido van Rossum wrote: > No objection on targetting 2.6 if other developers agree. Seems this > is well under way. good work! given that dir() is used extensively by introspection tools, I'm not sure I'm positive to a __dir__ that *overrides* the standard dir() behaviour. *adding* to the default dir() list is okay, re- placing it is a lot more questionable. (what about vars(), btw?) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __dir__, part 2
On 11/10/06, Fredrik Lundh <[EMAIL PROTECTED]> wrote:
> Guido van Rossum wrote:
>
> > No objection on targetting 2.6 if other developers agree. Seems this
> > is well under way. good work!
>
> given that dir() is used extensively by introspection tools, I'm
> not sure I'm positive to a __dir__ that *overrides* the standard
> dir() behaviour. *adding* to the default dir() list is okay, re-
> placing it is a lot more questionable.
I think that ought to go into the guidlines for what's an acceptable
__dir__ implementation. We don't try to stop people from overriding
__add__ as subtraction either.
> (what about vars(), btw?)
Interesting question! Right now vars() and dir() don't seem to use the
same set of keys; e.g.:
>>> class C: pass
...
>>> c = C()
>>> c.foo = 42
>>> vars(c)
{'foo': 42}
>>> dir(c)
['__doc__', '__module__', 'foo']
>>>
It makes some sense for vars(x) to return something like
dict((name, getattr(x, name)) for name in dir(x) if hasattr(x, name))
and for the following equivalence to hold between vars() and dir() without args:
dir() == sorted(vars().keys())
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __dir__, part 2
Fredrik Lundh schrieb: > Guido van Rossum wrote: > >> No objection on targetting 2.6 if other developers agree. Seems this >> is well under way. good work! > > given that dir() is used extensively by introspection tools, I'm > not sure I'm positive to a __dir__ that *overrides* the standard > dir() behaviour. *adding* to the default dir() list is okay, re- > placing it is a lot more questionable. One part that *I* would like about a complete overridable __dir__ implementation is that it would be nice to customize what help(something) prints. Thomas ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __dir__, part 2
Thomas Heller wrote: >>> No objection on targetting 2.6 if other developers agree. Seems this >>> is well under way. good work! >> >> given that dir() is used extensively by introspection tools, I'm >> not sure I'm positive to a __dir__ that *overrides* the standard >> dir() behaviour. *adding* to the default dir() list is okay, re- >> placing it is a lot more questionable. > > One part that *I* would like about a complete overridable __dir__ > implementation > is that it would be nice to customize what help(something) prints. I don't think you should confuse reliable introspection with the help system, though. introspection is used for a lot more than implementing help(). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __dir__, part 2
Guido van Rossum wrote: > I think that ought to go into the guidlines for what's an acceptable > __dir__ implementation. We don't try to stop people from overriding > __add__ as subtraction either. to me, overriding dir() is a lot more like overriding id() than over- riding "+". I don't think an object should be allowed to lie to the introspection mechanisms. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __dir__, part 2
On 11/10/06, Fredrik Lundh <[EMAIL PROTECTED]> wrote: > Guido van Rossum wrote: > > > I think that ought to go into the guidlines for what's an acceptable > > __dir__ implementation. We don't try to stop people from overriding > > __add__ as subtraction either. > > to me, overriding dir() is a lot more like overriding id() than over- > riding "+". I don't think an object should be allowed to lie to the > introspection mechanisms. Why not? You can override __class__ already. With a metaclass you can probably override inspection of the class, too. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __dir__, part 2
Guido van Rossum wrote:
>> (what about vars(), btw?)
>
> Interesting question! Right now vars() and dir() don't seem to use the
> same set of keys; e.g.:
>
class C: pass
> ...
c = C()
c.foo = 42
vars(c)
> {'foo': 42}
dir(c)
> ['__doc__', '__module__', 'foo']
>
> It makes some sense for vars(x) to return something like
>
> dict((name, getattr(x, name)) for name in dir(x) if hasattr(x, name))
>
> and for the following equivalence to hold between vars() and dir() without
> args:
>
> dir() == sorted(vars().keys())
+1. This is easy and straightforward to explain, better than
"With a module, class or class instance object as argument (or anything else
that has a __dict__ attribute), returns a dictionary corresponding to the
object's symbol table."
Georg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __dir__, part 2
Fredrik Lundh wrote: > Guido van Rossum wrote: > >> No objection on targetting 2.6 if other developers agree. Seems this >> is well under way. good work! > > given that dir() is used extensively by introspection tools, I'm > not sure I'm positive to a __dir__ that *overrides* the standard > dir() behaviour. *adding* to the default dir() list is okay, re- > placing it is a lot more questionable. If the new default __dir__ implementation only yields the same set of attributes (or more), there should be no problem. If somebody overrides __dir__, he knows what he's doing. He will most likely do something like "return super.__dir__() + [my, custom, attributes]". regards, Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __dir__, part 2
Fredrik Lundh wrote: > Guido van Rossum wrote: > >> No objection on targetting 2.6 if other developers agree. Seems this >> is well under way. good work! > > given that dir() is used extensively by introspection tools, I'm > not sure I'm positive to a __dir__ that *overrides* the standard > dir() behaviour. *adding* to the default dir() list is okay, re- > placing it is a lot more questionable. If a class only overrides __getattr__, then I agree it should only add to __dir__ (most likely by using a super call as Georg suggests). If it overrides __getattribute__, however, then it can actually deliberately block access to attributes that would otherwise be accessible, so it may make sense for it to alter the basic result of dir() instead of just adding more attributes to the end. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
