[issue37176] super() docs don't say what super() does

[Raymond Hettinger]

Change by Raymond Hettinger :


--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37176] super() docs don't say what super() does

[Raymond Hettinger]

Raymond Hettinger  added the comment:

I've add PR 15564 to clarify some known ambiguities for the super() docs 
including a specific example "what it actually does".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37176] super() docs don't say what super() does

[Carlos André Dantas de Lima]

Carlos André Dantas de Lima  added the comment:

The method says who you will use some recursion.

--
nosy: +Carlos André Dantas de Lima

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37176] super() docs don't say what super() does

[Steven D'Aprano]

Steven D'Aprano  added the comment:

On Sat, Jun 08, 2019 at 09:14:18PM +, Raymond Hettinger wrote:

> Ideally, the text can also be made more compact.  Having eight 
> paragraphs sends an implicit message that this is too complex to 
> understand and that it should be avoided.

But it is complex -- as you point out, there are *at least* seven 
potential misapprehensions here.

super does a lot: there are at least four ways to call it (zero-argument 
form, unbound super, class+instance and class+subclass). And as the old 
"super considered harmful" versus "super considered super" debate shows, 
there are pitfalls in multiple inheritance that people run into.

The challenge here is that for 95% of cases (plucking a number from thin 
air) using super is simple: just follow the example in the docs. But the 
remaining troublesome cases can be very troublesome, and people running 
into those cases do need to understand the gory details.

> One essential goal is that we need to overcome strong preconceptions 
> about about what super() might do versus what it actually does.
[...]

Here's two more that you missed:

- that you should call super with super(type(self), self);

- that unbound super objects dispatch to unbound methods.

At some point or another over the last decade I've misunderstood super 
to do almost all of those things. As this thread shows, I thought I had 
understood it but I still misunderstood the role of the class calling 
super.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37176] super() docs don't say what super() does

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Yes, issue 23674 seems to be at least a partial duplicate.  We should just get 
this fixed and close both issues.

Ideally, the text can also be made more compact.  Having eight paragraphs sends 
an implicit message that this is too complex to understand and that it should 
be avoided.

One essential goal is that we need to overcome strong preconceptions about 
about what super() might do versus what it actually does.  Those with 
experience of super() in other languages tend to presume that it would be a 
keyword rather than a builtin type that needs to be instantiated, that it can 
only work within a class, that it has lower computational overhead than it 
does, that it only calls parents rather than siblings, that search order is 
controlled by the method using super() rather than the child instance, that 
operators will work, or that the search order is something is something other 
than the C3 algorithm.

The core challenge of documenting super() is that it is so different from what 
a person might imagine.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37176] super() docs don't say what super() does

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

> Some of the problems brought up here (which sibling or subclass, and which 
> parameter’s MRO) also came up a few years ago in Issue 23674.

Indeed. I would actually say that these two issues are duplicates of each other.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37176] super() docs don't say what super() does

[Géry]

Géry  added the comment:

@Steven D'Aprano

> But neither can it *only* look at the MRO of the second class, because that 
> would restart the search at the top of the hierarchy; also if type(second 
> argument) was the only thing that mattered, that would make the first 
> argument redundant and pointless.

Yes of course, but nobody states the opposite. Both parameters are necessary 
super(type1, obj-or-type2):

— the first one for getting the start class in the MRO, which is the class 
following type1 in the MRO;
— the second one for getting the MRO, which is type(obj).__mro__ if it is an 
instance of type1 or type2.__mro__ if it is a subclass of type1, and for 
binding the function into a method in a super(type1, obj).function expression 
(as you correctly said in your last message).

Guido's Python equivalent implementation of super() given in his 2002 paper 
Unifying types and classes in Python 2.2 is very informative:
https://www.python.org/download/releases/2.2.3/descrintro/#cooperation

All this confusion around super() shows that there is room for improvement in 
the documentation.

@Raymond Hettinger

> Please make a concrete proposal (a PR or somesuch).

I will try to make a PR this weekend if I find some time.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37176] super() docs don't say what super() does

[Martin Panter]

Martin Panter  added the comment:

Some of the problems brought up here (which sibling or subclass, and which 
parameter’s MRO) also came up a few years ago in Issue 23674.

--
nosy: +martin.panter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37176] super() docs don't say what super() does

[Steven D'Aprano]

Steven D'Aprano  added the comment:

> > What matters is the __mro__ attribute of the first argument. It matters 
> because that is how the MRO actually is searched.
> 
> By the way, if it was true (it is not), 

Yes, I see that now.

> then what did you think was 
> the purpose of the second parameter of super(type, obj-or-type)?

Given super(T, obj).method the method object is bound to instance obj; 
how else could it return a bound method instead of an unbound one?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37176] super() docs don't say what super() does

[Raymond Hettinger]

Change by Raymond Hettinger :


--
assignee: docs@python -> rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37176] super() docs don't say what super() does

[Steven D'Aprano]

Steven D'Aprano  added the comment:

> I'm sorry to say that you're wrong here. 

I'm happy to be corrected.

It is fair to say I failed to take the multiple inheritance case into 
account. Clearly super can't *only* look at the MRO of the first object 
since that will miss the multiple inheritance case, as you point out. 
Thank you.

But neither can it *only* look at the MRO of the second class, because 
that would restart the search at the top of the hierarchy; also if 
type(second argument) was the only thing that mattered, that would make 
the first argument redundant and pointless.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37176] super() docs don't say what super() does

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Please make a concrete proposal (a PR or somesuch). That will make it much 
easier to determine whether a particular bit of word-smithing is a improvement.

Of the issues discussed so far, these are the most promising:
* Document how the zero argument form of super() infers its arguments.
* Be clear that it is the mro of type(self) that determines the search path and 
that the role of super() is to the next in the mro after the current class.

--
versions:  -Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37176] super() docs don't say what super() does

[Géry]

Géry  added the comment:

@Steven D'Aprano

> What matters is the __mro__ attribute of the first argument. It matters 
because that is how the MRO actually is searched.

By the way, if it was true (it is not), then what did you think was the purpose 
of the second parameter of super(type, obj-or-type)?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37176] super() docs don't say what super() does

[Géry]

Géry  added the comment:

@Jeroen Demeyer

> I'm sorry to say that you're wrong here. super() looks at the MRO of the type 
> of the object (the second argument) (*).

Exactly! It is funny because I was about to open the same issue this weekend. 
The documentation of super() is wrong here.

--
nosy: +maggyero

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37176] super() docs don't say what super() does

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

And this last comment is precisely the kind of information which should be 
explained in the super() docs.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37176] super() docs don't say what super() does

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

> What matters is the __mro__ attribute of the first argument. It matters 
because that is how the MRO actually is searched.

I'm sorry to say that you're wrong here. super() looks at the MRO of the type 
of the object (the second argument) (*). It has to do that in order to support 
diamonds. Consider a diamond like

  D
 / \
B   C
 \ /
  A

(with A as common base class). Now super(B, D()).attr will look in the MRO of D 
(which is D, B, C, A) and therefore delegate to C.attr. In this case, C does 
not even appear in the MRO of B.

(*) To be pedantic: in the special case that the second argument is a type 
itself, it looks at the MRO of the second argument.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37176] super() docs don't say what super() does

[Steven D'Aprano]

Steven D'Aprano  added the comment:

> If you have to explain in a bpo issue how the doc should be read, that 
> proves exactly my point that it's confusing. The fact that it's 
> technically correct if you read it the right way is irrelevant.

Do you expect the docs to be technically correct when read wrongly? How 
else should people read the docs, except the right way?

If you have *concrete* suggestions for improvements, of course we will 
consider them. I'll start with two concrete improvements:

1. Explicitly document that the zero-argument version is equivalent
to super(__class__, ), and note that 
__class__ here is a special variable populated by the compiler.

The first part of this is already documented in super.__doc__ and the 
second part used to be documented:

https://docs.python.org/3.1/library/functions.html?#super

and I don't think there's any harm in adding it back in.

2. Link to the essay on how the MRO is calculated.

https://www.python.org/download/releases/2.3/mro/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37176] super() docs don't say what super() does

[Steven D'Aprano]

Steven D'Aprano  added the comment:

On Fri, Jun 07, 2019 at 09:57:11AM +, Jeroen Demeyer wrote:

> I'm having problems with the first word of "a parent or sibling class of 
> type". 

The first word is "a". Did you mean something else or did you mean it 
literally?

If the second case, we have to talk about *a* parent class rather than 
*the* parent class, since Python supports multiple inheritance and a 
class can have more than one parent class.

> The most important part of super() is to *which* class that attribute 
> lookups are delegated to and this is not explained.

Lookups are delegated to a parent or sibling class, exactly as the 
documentation says. Which class that is (whether a parent or sibling) 
can only be determined at runtime.

> And the sentence "The __mro__ attribute of the type lists the method 
> resolution search order used by both getattr() and super()" is even 
> wrong or at least confusing: what matters is not the MRO of the type 
> (the first argument to super()) but the MRO of ***the type of*** the 
> object (the second argument to super()).

I believe that is still wrong.

What matters is the __mro__ attribute of the first argument. It matters 
because that is how the MRO actually is searched.

The MRO of the second argument is not relevant, except to the degree 
that it overlaps with the __mro__ attribute of the first. And it will 
always overlaps, because the second argument must be an instance or 
subclass of the first. But not necessarily *directly* of the first:

object > Parent > A > B > C > D > E > F > instance = F()

If the method is defined in class C, then the super call in C should be 
written:

class C(B):
def method(self, arg):
super().method(arg)  # like super(C, self).method(arg)

and the lookup will start at C.__mro__[1], namely B, precisely as 
expected. If it started back at the type of the instance (self), namely 
class F, the C.method would recursively call itself over and over and 
over again.

If you don't believe me, you can simulate super() working the way you 
suggest by doing this:

# -%<-

class A(object):
def method(self):
print("from class A")

class B(A):
def method(self):
print("bad method")
# This is bad, don't do it!
super(type(self), self).method()

class C(B): pass

C().method()

# -%<-

For multiple inheritance:

class C(B, X, Y, Z): ...

we can't generally tell in advance which will be looked at next, since 
that depends on the precise details of the inheritance diagram. But 
whatever the class is, at this point, it is still the __mro__ of C that 
is used, not the MRO of the instance.

The bottom line: what matters is .__mro__, exactly as 
documented.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37176] super() docs don't say what super() does

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

> The sentence doesn't talk about the MRO of *type* (the first argument), 
it talks about the __mro__ attribute.

If you have to explain in a bpo issue how the doc should be read, that proves 
exactly my point that it's confusing. The fact that it's technically correct if 
you read it the right way is irrelevant.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37176] super() docs don't say what super() does

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

> Only one of the two arguments is called "type". The other is called 
"object-or-type".

I'm having problems with the first word of "a parent or sibling class of type". 
The most important part of super() is to *which* class that attribute lookups 
are delegated to and this is not explained.

> The __mro__ attribute is on the type, not the instance:

Sorry for that. I meant to say:

And the sentence "The __mro__ attribute of the type lists the method resolution 
search order used by both getattr() and super()" is even wrong or at least 
confusing: what matters is not the MRO of the type (the first argument to 
super()) but the MRO of ***the type of*** the object (the second argument to 
super()).

> Yes it is. Look at the example given:

An example is not an explanation. But it's true, this is the least of my 
problems with this doc.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37176] super() docs don't say what super() does

[Steven D'Aprano]

Steven D'Aprano  added the comment:

On Fri, Jun 07, 2019 at 08:00:34AM +, Jeroen Demeyer wrote:
> 
> Jeroen Demeyer  added the comment:
> 
> > What more do you want?
> 
> Mainly: it says "a parent or sibling class of *type*" but it doesn't 
> explain which class it actually uses.

Only one of the two arguments is called "type". The other is called 
"object-or-type".

> And the sentence "The __mro__ attribute of the type lists the method 
> resolution search order used by both getattr() and super()" is even 
> wrong or at least confusing: what matters is not the MRO of the type 
> (the first argument to super()) but the MRO of the object (the second 
> argument to super()).

The sentence doesn't talk about the MRO of *type* (the first argument), 
it talks about the __mro__ attribute. It is correct. The __mro__ 
attribute is on the type, not the instance:

py> int.__mro__
(, )

py> int().__mro__
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: 'int' object has no attribute '__mro__'

super() look-ups never start at the instance, they delegate to the 
superclass (or superclasses), not back to the current class, or the 
instance itself.

> The zero-argument form super() is not explained at all.

Yes it is. Look at the example given:

super().method(arg)# This does the same thing as:
   # super(C, self).method(arg)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37176] super() docs don't say what super() does

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

> What more do you want?

Mainly: it says "a parent or sibling class of *type*" but it doesn't explain 
which class it actually uses.

And the sentence "The __mro__ attribute of the type lists the method resolution 
search order used by both getattr() and super()" is even wrong or at least 
confusing: what matters is not the MRO of the type (the first argument to 
super()) but the MRO of the object (the second argument to super()).

The zero-argument form super() is not explained at all.

> Perhaps there ought to be a "gentle guide to super" somewhere, and the docs 
> could link to that?

There are plenty of guides like that and in fact that docs already link to 
https://rhettinger.wordpress.com/2011/05/26/super-considered-super/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37176] super() docs don't say what super() does

[Raymond Hettinger]

Raymond Hettinger  added the comment:

> It only says "Return a proxy object that delegates method 
> calls to a parent or sibling class of type" and then gives
> a bunch of use cases and examples."  

That wording seems very reasonable to me.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37176] super() docs don't say what super() does

[Steven D'Aprano]

Steven D'Aprano  added the comment:

The docs do say what super does: it returns "a proxy object that delegates 
method calls to a parent or sibling class of type", just as you quoted. That 
concise description is (almost) completely accurate and precise.

(I say *almost* because it's not just method calls that it works with, but any 
attribute lookup.)

What more do you want? That's not a rhetorical question.

I'm not saying that the docs are perfect or cannot be improved, but they look 
pretty good to me: they tell you what super does, they tell you why you might 
use it, and show how to use it. What's missing? Again, not a rhetorical 
question.

I understand that super is a very advanced corner of the language: there's a 
lot of necessary technical jargon in the docs, e.g.:

- One already needs to have a good understanding of what super *does* in order 
to make sense of the sentence describing what it *is*, so there's an element of 
circular reasoning needed.

- The reader needs to understand that super isn't magical, it just returns an 
object like any other function, and understand what "proxy object" means, as 
well as delegation.

- And have an understanding of how inheritance and the MRO work in Python, and 
the difference between bound and unbound methods/proxies.

I think that to the experienced reader who knows these concepts, the docs 
should be pretty clear and complete.

I'm having a hard time seeing what you believe is missing from the docs. Can 
you explain further?

Perhaps there ought to be a "gentle guide to super" somewhere, and the docs 
could link to that?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37176] super() docs don't say what super() does

[Steven D'Aprano]

Change by Steven D'Aprano :


--
Removed message: https://bugs.python.org/msg344892

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37176] super() docs don't say what super() does

[Steven D'Aprano]

Steven D'Aprano  added the comment:

The docs do say what super does: it returns "a proxy object that delegates 
method calls to a parent or sibling class of type", just as you quoted. That 
concise description is (almost) completely accurate and precise.

(I say *almost* because it's not just method calls that it works with, but any 
attribute lookup.)

What more do you want? That's not a rhetorical question.

I'm not saying that the docs are perfect or cannot be improved, but they look 
pretty good to me: they tell you what super does, they tell you why you might 
use it, and show how to use it. What's missing?

(Again, not a rhetorical question.)

I understand that super is a very advanced corner of the language: there's a 
lot of necessary technical jargon in the docs:

- One already needs to have a good understanding of what super *does* in order 
to make sense of the sentence describing what it *is*, so there's an element of 
circular reasoning needed.

- The reader needs to understand that super isn't magical, it just returns an 
object like any other function, and understand what "proxy object" means, as 
well as delegation.

- And have an understanding of how inheritance and the MRO work in Python, and 
the difference between bound and unbound methods/proxies.

But to the experienced reader who knows these concepts, I'm having a hard time 
seeing what you believe is missing from the docs.

Perhaps there ought to be a "gentle guide to super" somewhere, and the docs 
could link to that?

--
nosy: +steven.daprano

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37176] super() docs don't say what super() does

[Jeroen Demeyer]

New submission from Jeroen Demeyer :

The documentation for super() at 
https://docs.python.org/3.8/library/functions.html#super does not actually say 
what super() does. It only says "Return a proxy object that delegates method 
calls to a parent or sibling class of type" and then gives a bunch of use cases 
and examples.

If there is one place where we should define exactly what super() does (as 
opposed to give guidance on how to use it), the stdlib reference should be it.

--
assignee: docs@python
components: Documentation
messages: 344827
nosy: docs@python, jdemeyer
priority: normal
severity: normal
status: open
title: super() docs don't say what super() does
type: enhancement
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com