Re: Plumbing behind super()
On 30/06/2019 01:15, adam.pre...@gmail.com wrote: Whoops. Now I need to figure out how the interpreter knows that change_a is a method and knows what self to feed it. I'm assuming that's in the cell variables similar to what super()'s doing as explained here. I haven't implemented cell variables so this is where I'm stuck in a sand pit. Look up descriptors. Actually, carefully (re)read all of https://docs.python.org/3/reference/datamodel.html -- https://mail.python.org/mailman/listinfo/python-list
Re: Plumbing behind super()
Thanks for the replies from everybody. It looks like I should double check super_init and see what truck is coming from that which will hit me with a gotcha later. I'm very naively right now plucking the class from my locals and I was able to proceed in the very, very short term. I think I would have run into something like this earlier but I was doing something else incorrectly with self references in general. I was having my byte code push the object reference on the stack for method calls instead of using a naive one. For example: m.change_a(2) Disregarding unrelated code, it disassembles to this in a 3.6 intepreter: 3 6 LOAD_FAST0 (m) 8 LOAD_ATTR1 (change_a) 10 LOAD_CONST 1 (2) 12 CALL_FUNCTION1 I have been doing an oopsies of trying to push the self reference on the stack for the method. So I'm doing something like: 3 6 LOAD_FAST0 (m) 8 LOAD_ATTR1 (change_a) X LOAD_FAST0 (m) 10 LOAD_CONST 1 (2) 12 CALL_FUNCTION2 Whoops. Now I need to figure out how the interpreter knows that change_a is a method and knows what self to feed it. I'm assuming that's in the cell variables similar to what super()'s doing as explained here. I haven't implemented cell variables so this is where I'm stuck in a sand pit. -- https://mail.python.org/mailman/listinfo/python-list
Re: Plumbing behind super()
On 28/06/2019 02:13, adam.pre...@gmail.com wrote: I'm trying to mimick Python 3.6 as a .NET science project and have started to get into subclassing. The super() not-a-keyword-honestly-guys has tripped me up. I have to admit that I've professionally been doing a ton Python 2.7, so I'm not good on my Python 3.6 trivia yet. I think I have the general gist of this, but want some affirmation. If you use super() in a method, all it does is load super as a global on to the interpreter stack and call it without any arguments. So I'm left to wonder how it's able to figure anything out when it's being literally given nothing... Python is open source, so let's open the source! The magic happens when super() is constructed, in super_init https://github.com/python/cpython/blob/a8b27e623d75377aabe50df27e97cab4e81a174a/Objects/typeobject.c#L7814 Once super_getattro is called, the super object has already been set up, and it already knows which object/class to look at. As you can see, super_init takes the calling frame and checks the first argument of the previous call, which, when called from a method, will generally be 'self' https://github.com/python/cpython/blob/a8b27e623d75377aabe50df27e97cab4e81a174a/Objects/typeobject.c#L7849 There's some additional code that, I think, handles being called from a closure (but I don't know the C api well enough to be sure) We can sort-of emulate this in pure Python with the inspect module ### def not_quite_super(): f = inspect.stack()[1].frame obj = list(f.f_locals.values())[0] print(type(obj), obj) class A: def __init__(self, a=0, b=1): c = a + b # this is just to add some more locals to the mix not_quite_super() A() ### Once you have the caller object, the rest is just like calling super() with arguments. I'm not actually sure that Python example above is guaranteed to work, but I *think* it should in versions where dicts are ordered (3.6+). Obviously it doesn't handle all the cases super() handles. except that it's not being given literally nothing: static PyObject * super_getattro(PyObject *self, PyObject *name) I was thinking maybe self has become more special in Python 3.6, but I don't think that's true since I've ported code to Python3 before that had inner classes where I'd use "inner_self" to disambiguate with the outer self. And though I thought it was so at first, it just turned out I screwed up my little code snippet to expose it. If self was special then I presume I could find it in my lookups and inject it. So how do I go from CALL_FUNCTION on a super() global without anything else on the stack to somehow having all the information I need? Is there something tracking that I'm in an object scope when calling stuff? -- https://mail.python.org/mailman/listinfo/python-list
Re: Plumbing behind super()
I was wrong in the last email because I accidentally in super_gettro instead of super_init. Just for some helper context: >>> class Foo: ... pass ... >>> class Bar(Foo): ... def __init__(self): ... super().__init__() ... self.a = 2 ... >>> dis(Bar) Disassembly of __init__: 3 0 LOAD_GLOBAL 0 (super) 2 CALL_FUNCTION0 4 LOAD_ATTR1 (__init__) 6 CALL_FUNCTION0 8 POP_TOP 4 10 LOAD_CONST 1 (2) 12 LOAD_FAST0 (self) 14 STORE_ATTR 2 (a) 16 LOAD_CONST 0 (None) 18 RETURN_VALUE I originally set a breakpoint at super_getattro so I was seeing it getting the self pointer from TOS, but I think I needed super_init--especially since that is getting called from a CALL_FUNCTION opcode, instead of super_getattro and it's LOAD_ATTR. I think that's from the self.a assignment. The super_init code melted my brain! It looks like it's grabbing the current frame and interrogating it to find __class__. I think I have the same amount of visibility and similar structure in what I'm writing but... woah. -- https://mail.python.org/mailman/listinfo/python-list
Re: Plumbing behind super()
On Thu, Jun 27, 2019, at 23:32, adam.pre...@gmail.com wrote:
> On Thursday, June 27, 2019 at 8:30:21 PM UTC-5, DL Neil wrote:
> > I'm mystified by "literally given nothing".
>
> I'm focusing there particularly on the syntax of writing "super()"
> without any arguments to it. However, internally it's getting fed stuff.
Any function which is defined inside a class and accesses 'super' is provided a
'cell variable' (i.e. nonlocal) called __class__, which super inspects on the
stack to identify the class (along with finding the self argument)
>>> class C:
... def foo(): super
...
>>> C.foo.__closure__
(,)
>>> C.foo.__code__.co_freevars
('__class__',)
You can also provide this cell manually by defining it as a nested function
inside another function which has a local variable __class__.
>>> def bar():
... __class__ = C
... return lambda self: super()
...
>>> bar()(C())
, >
You can see all this happening in
https://github.com/python/cpython/blob/3.8/Objects/typeobject.c#L7812
--
https://mail.python.org/mailman/listinfo/python-list
Re: Plumbing behind super()
On Thursday, June 27, 2019 at 8:30:21 PM UTC-5, DL Neil wrote: > I'm mystified by "literally given nothing". I'm focusing there particularly on the syntax of writing "super()" without any arguments to it. However, internally it's getting fed stuff. > If a class has not defined an attribute, eg self.my_attribute, but the > base class has defined an attribute of that name, then self.my_attribute > will as initialised by the base class. Welcome to the intricacies of > managing scope! I'm thinking my problem was more fundamental here. I finally got the debugger to bypass all the calls to the internal super() implementation that are done on startup and run instead just a bit of user-specified code. It's looking like super() actually gobbles the self pointer that goes on to the stack when __init__ is first created and replaces it when its done by returning it again. I think. Maybe. -- https://mail.python.org/mailman/listinfo/python-list
Re: Plumbing behind super()
On 28/06/19 12:13 PM, adam.pre...@gmail.com wrote: I'm trying to mimick Python 3.6 as a .NET science project and have started to get into subclassing. The super() not-a-keyword-honestly-guys has tripped me up. I have to admit that I've professionally been doing a ton Python 2.7, so I'm not good on my Python 3.6 trivia yet. I think I have the general gist of this, but want some affirmation. If you use super() in a method, all it does is load super as a global on to the interpreter stack and call it without any arguments. So I'm left to wonder how it's able to figure anything out when it's being literally given nothing... ... I was thinking maybe self has become more special in Python 3.6, but I don't think that's true since I've ported code to Python3 before that had inner classes where I'd use "inner_self" to disambiguate with the outer self. And though I thought it was so at first, it just turned out I screwed up my little code snippet to expose it. If self was special then I presume I could find it in my lookups and inject it. So how do I go from CALL_FUNCTION on a super() global without anything else on the stack to somehow having all the information I need? Is there something tracking that I'm in an object scope when calling stuff? You are missing quite a bit by not becoming familiar with Python2 -> 3 differences! I'm mystified by "literally given nothing". 1 All 'new' classes descend from object. 2 class Son( Father ):... literally gives 'Father' (At present) The base class must have been defined (frame on the stack) before it may be referenced. Self does not require super, it refers to the class itself. If a class has not defined an attribute, eg self.my_attribute, but the base class has defined an attribute of that name, then self.my_attribute will as initialised by the base class. Welcome to the intricacies of managing scope! -- Regards =dn -- https://mail.python.org/mailman/listinfo/python-list
Plumbing behind super()
I'm trying to mimick Python 3.6 as a .NET science project and have started to get into subclassing. The super() not-a-keyword-honestly-guys has tripped me up. I have to admit that I've professionally been doing a ton Python 2.7, so I'm not good on my Python 3.6 trivia yet. I think I have the general gist of this, but want some affirmation. If you use super() in a method, all it does is load super as a global on to the interpreter stack and call it without any arguments. So I'm left to wonder how it's able to figure anything out when it's being literally given nothing... except that it's not being given literally nothing: static PyObject * super_getattro(PyObject *self, PyObject *name) I was thinking maybe self has become more special in Python 3.6, but I don't think that's true since I've ported code to Python3 before that had inner classes where I'd use "inner_self" to disambiguate with the outer self. And though I thought it was so at first, it just turned out I screwed up my little code snippet to expose it. If self was special then I presume I could find it in my lookups and inject it. So how do I go from CALL_FUNCTION on a super() global without anything else on the stack to somehow having all the information I need? Is there something tracking that I'm in an object scope when calling stuff? -- https://mail.python.org/mailman/listinfo/python-list
