[issue5062] Rlcompleter.Completer does not use __dir__ magic method

[Carl Johnson]

New submission from Carl Johnson :

The documentation at http://docs.python.org/library/rlcompleter.html
claims that

Completer.complete(text, state)¶

   Return the state*th completion for *text.

   If called for text that doesn’t include a period character ('.'), it
will complete from names currently defined in __main__, __builtin__ and
keywords (as defined by the keyword module).

   If called for a dotted name, it will try to evaluate anything without
obvious side-effects (functions will not be evaluated, but it can
generate calls to __getattr__()) up to the last part, and find matches
for the rest via the dir() function. Any exception raised during the
evaluation of the expression is caught, silenced and None is returned.

In other words, it claims to use dir(obj) as part of the tab completion
process. This is not true (using Python 2.6.1 on OS X):

>>> class B(object):
...  def __dir__(self): return dir(u"") #Makes B objects look like strings
...
>>> b = B()
>>> dir(b)
['__add__', '__class__', '__contains__', '__delattr__', '__doc__',
'__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__',
'__getnewargs__', '__getslice__', '__gt__', '__hash__', '__init__',
'__le__', '__len__', '__lt__', '__mod__', '__mul__', '__ne__',
'__new__', '__reduce__', '__reduce_ex__', '__repr__', '__rmod__',
'__rmul__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__',
'_formatter_field_name_split', '_formatter_parser', 'capitalize',
'center', 'count', 'decode', 'encode', 'endswith', 'expandtabs', 'find',
'format', 'index', 'isalnum', 'isalpha', 'isdecimal', 'isdigit',
'islower', 'isnumeric', 'isspace', 'istitle', 'isupper', 'join',
'ljust', 'lower', 'lstrip', 'partition', 'replace', 'rfind', 'rindex',
'rjust', 'rpartition', 'rsplit', 'rstrip', 'split', 'splitlines',
'startswith', 'strip', 'swapcase', 'title', 'translate', 'upper', 'zfill']
>>> c = rlcompleter.Completer()
>>> c.complete("b.", 0) #Notice that it does NOT return __add__
u'b.__class__('
>>> c.matches #Notice that this list is completely different from the
list given by dir(b)
[u'b.__class__(', u'b.__delattr__(', u'b.__doc__', u'b.__format__(',
u'b.__getattribute__(', u'b.__hash__(', u'b.__init__(', u'b.__new__(',
u'b.__reduce__(', u'b.__reduce_ex__(', u'b.__repr__(',
u'b.__setattr__(', u'b.__sizeof__(', u'b.__str__(',
u'b.__subclasshook__(', u'b.__class__(', u'b.__class__(',
u'b.__delattr__(', u'b.__dict__', u'b.__dir__(', u'b.__doc__',
u'b.__format__(', u'b.__getattribute__(', u'b.__hash__(',
u'b.__init__(', u'b.__module__', u'b.__new__(', u'b.__reduce__(',
u'b.__reduce_ex__(', u'b.__repr__(', u'b.__setattr__(',
u'b.__sizeof__(', u'b.__str__(', u'b.__subclasshook__(',
u'b.__weakref__', u'b.__class__(', u'b.__delattr__(', u'b.__doc__',
u'b.__format__(', u'b.__getattribute__(', u'b.__hash__(',
u'b.__init__(', u'b.__new__(', u'b.__reduce__(', u'b.__reduce_ex__(',
u'b.__repr__(', u'b.__setattr__(', u'b.__sizeof__(', u'b.__str__(',
u'b.__subclasshook__(']

Suggested course of action: 

* Change the documentation for Python 2.6/3.0.
* Update Completer to use __dir__ in Pythons 2.7/3.1 and revert the
documentation.

See
http://mail.python.org/pipermail/python-dev/2009-January/thread.html#85471

--
assignee: georg.brandl
components: Documentation, Extension Modules
messages: 80556
nosy: carlj, georg.brandl
severity: normal
status: open
title: Rlcompleter.Completer does not use __dir__ magic method
type: behavior
versions: Python 2.6, Python 3.0

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5062] Rlcompleter.Completer does not use __dir__ magic method

[Gabriel Genellina]

Gabriel Genellina  added the comment:

This is not a bug in rlcompleter; __dir__ is returning bogus items, and 
rlcompleter checks whether there exist actually an attribute with such 
name.

Defining __getattr__ (or __getattribute__) and a matching __dir__ works 
fine:

>>> class B(object):
...   def __dir__(self):
... return dir(object) + ["xa","xb","xc"]
...   def __getattr__(self, name):
... if name in ["xa","xb","xc"]:
...   return None
... raise AttributeError, name
...
>>> b = B()
>>> import rlcompleter
>>> c = rlcompleter.Completer()
>>> c.complete("b.", 0)
'b.__class__('
>>> c.matches
['b.__class__(', 'b.__delattr__(', 'b.__doc__', 'b.__format__(', 
'b.__getattribute__(', 
...
'b.xa', 'b.xb', 'b.xc', 'b.__class__(', 'b.__class__(', 
...]
>>> c.complete("b.x", 0)
'b.xa'
>>> c.matches
['b.xa', 'b.xb', 'b.xc']

Now, looking at this I saw there *is* a bug in rlcompleter, as it may 
return many duplicate items:

>>> c.complete("b.__c", 0)
'b.__class__('
>>> c.matches
['b.__class__(', 'b.__class__(', 'b.__class__(', 'b.__class__(']

The attached patch fixes that.

--
keywords: +patch
nosy: +gagenellina
Added file: http://bugs.python.org/file12872/rlcompleter.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5062] Rlcompleter.Completer does not use __dir__ magic method

[Carl Johnson]

Carl Johnson  added the comment:

It seems to me that it isn't tab completion's place to out think the
__dir__ method. A) Because the documentation doesn't tell you that it
does (although you are warned that it may call some stuff) and B)
because if someone set up a __dir__ method, they probably are listing
the things that they want listed for a particular reason. I think that
it would be less confusing for rlcompleter to follow the __dir__ method
when it exists and only do its own poking and prodding when it does not.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5062] Rlcompleter.Completer does not use __dir__ magic method

[Gabriel Genellina]

Gabriel Genellina  added the comment:

This is what rlcompleter does; it uses dir() to find out what names to 
return.
Or do you mean that it should not iterate along __bases__ because this 
has already been done by dir()?

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5062] Rlcompleter.Completer does not use __dir__ magic method

[Carl Johnson]

Carl Johnson  added the comment:

I think that checking to see which things really exist with
getattr/hasattr made sense back in the days before the __dir__, since in
those days the real API for an object could diverge wildly from what was
reported by dir(object), but nowadays, if someone goes to the trouble of
defining the __dir__ method, then we should just trust that as being
"the API" and not do any other checking.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5062] Rlcompleter.Completer does not use __dir__ magic method

[Gabriel Genellina]

Gabriel Genellina  added the comment:

The check is made to decide whether the attribute is a method or not 
(because methods get a "(" appended) -- for names that fail to exist, 
one could just omit the "(" and include the name anyway.

rlcompleter does nothing special with __dir__, it always uses dir() 
only.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5062] Rlcompleter.Completer does not use __dir__ magic method

[Carl Johnson]

Carl Johnson  added the comment:

Ah, I see. It does a dir(obj) then tests things to see which are
callable and while it is at that, it removes the names that don't really
exist according to getattr.

Actually, can we go back to the Python 2.5 behavior? I really hate those
auto-added parentheses. For one thing, it screws it up when you do
"help(name". Am I missing some really obvious switch that would
turn the behavior back to the old style of ignoring the
callable/non-callable thing?

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5062] Rlcompleter.Completer does not use __dir__ magic method

[Gabriel Genellina]

Gabriel Genellina  added the comment:

The current behaviour is actually a requested feature: see #449227

I see your point, it may be annoying sometimes -- but calling a method 
is far more common than just getting a reference to it, so I think the 
current behaviour is fine (I'm talking about the added "(", not the 
repeated entries, nor the unneeded __bases__ recursion, nor the deleted 
entries)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com