On 06/08/2021 20:29, Marco Sulla wrote:
I've done an answer on SO about why subclassing `dict` makes the
subclass so much slower than `dict`. The answer is interesting:
https://stackoverflow.com/questions/59912147/why-does-subclassing-in-python-slow-things-down-so-much
What do you think about?
I have spent a lot of time reading typeobject.c over the years I've been
looking at an alternative implementation. It's quite difficult to
follow, and full of tweaks for special circumstances. So I'm impressed
with the understanding that "user2357112 supports Monica" brings to the
subject. (Yes, I want to call them Monica too, but I don't think that's
their actual name. ) I don't think I understand it better than they but
here's my reading of that, informed by my reading of typeobject.c, in
case it helps.
When a built-in type like dict is defined in C, pointers to its C
implementation functions are hard-coded into slots in the type object.
In order to make each appear as a method to Python, a descriptor is
created when building the type that delegates to the slot (so
sq_contains generates a descriptor __contains__ in the dictionary of the
type.
Conversely, if in a sub-class you define __contains__, then the type
builder will insert a function pointer in the slot of the new type that
arranges a call to __contains__. This will overwrite whatever was in the
slot.
In a C implementation, you can also define methods (by creating a
PyMethodDef the tp_methods table) that become descriptors in the
dictionary of the type. You would not normally define both a C function
to place in the slot *and* the corresponding method via a PyMethodDef.
If you do, the version from the dictionary of the type will win the
slot, *unless* you mark the method definition (in its PyMethodDef) as
METH_COEXIST.
This exception is used in the special case of dict (and hardly anywhere
else but set I think). I assume this is because some important code
calls __contains__ via the descriptor, rather than via the slot (which
would be quicker), and because an explicit definition is faster than a
descriptor created automatically to wrap the slot.
Now, when you create a sub-class, the table of slots is copied first,
then the type is checked for definitions of special methods, and these
are allowed to overwrite the slot, unless they are slot wrappers on the
same function pointer the slot already contains. I think at this point
the slot is re-written to contain a wrapper on __contains__, which has
been inherited from dict.__contains__, because it isn't a *slot wrapper*
on the same function. For example:
>>> dict.__contains__
<method '__contains__' of 'dict' objects>
>>> str.__contains__
<slot wrapper '__contains__' of 'str' objects>
>>> class S(str): pass
>>> S.__contains__
<slot wrapper '__contains__' of 'str' objects>
>>> D.__contains__
<method '__contains__' of 'dict' objects>
I think that when filling the slots of a sub-class, one could check for
the METH_COEXIST flag at the point one checks to see whether the
definition from look-up on the type is a PyWrapperDescr on the same
pointer. One might have to know that the slot and descriptor come from
the same base. I'm not suggesting this would be worthwhile.
FYI, in the approach I am toying with, the slot wrapper descriptor is
always created from the function definition, then the slot is filled
from the available definitions by lookup. Defining __contains__ twice
would be impossible or an error. I think this has the semantics required
by Python, but we'll have to wait for proof.
-- Jeff Allen
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/ZKMVZ5M3V76SOZH7FOURQ66VFZQY2BTG/
Code of Conduct: http://python.org/psf/codeofconduct/
