Re: bound methods
Leopold Toetsch wrote: Sam Ruby [EMAIL PROTECTED] wrote: The common cases I want to optimize for are common functions being called as common functions. And common methods being called as methods. Yep, that's very reasonable. The easiest way to optimize for the common methods being called as methods is if the current object is not passed as the first argument. Why so? Python methods don't have a notion of an object - it's just the first argument passed into a function. So I don't quite understand your conclusion. In the case of x.y(z) The caller is passing one argument. The recipient is expecting two. The recipient defines what is expected as the first argument (in class methods it is the class, in instance methods it is the instance). So, in the general case, the callee is responsible for inserting a parameter at the beginning of the PMC parameters. Insertion by setting a value into P2 is cheaper than insertion by shifting and then setting a value into P5. Yes, copying will be required in cases where functions are called as methods. How do you detect this case and when are arguments shifted then - note: I prefer the term shifting as copying is done anyway in all sub calls (see src/sub.c:copy_regs). I agree that shifting is a more appropriate term. At definition time, the PyFunc PMC has a number of properties (or flags) which are set defining names of the formal arguments, defaults, and the like. A similar approach can be used to define whether the first parameter is expected in P2 or P5. Note: this is not yet implemented for Python on Parrot. Note: different languages may chose different strategies as to when to shift. What is important is that we come to an agreement on what is expected of the caller. - Sam Ruby
Re: bound methods
Sam Ruby [EMAIL PROTECTED] wrote: Leopold Toetsch wrote: Why so? Python methods don't have a notion of an object - it's just the first argument passed into a function. So I don't quite understand your conclusion. In the case of x.y(z) The caller is passing one argument. The recipient is expecting two. The recipient defines what is expected as the first argument (in class methods it is the class, in instance methods it is the instance). Yes. Let's start at the called sub/method. Specifically a user function y would looks this: 1) .sub y .param pmc x .param pmc z as this is the general translation of the user code y. If you know that it's a method, you currently could translate it like: 2) .sub y method .param pmc z x = interpinfo .INTERPINFO_CURRENT_OBJECT But at the caller site you don't know anything about y - so with your proposed argument passing the code in y would need a prologue to handle both cases in both callee variants, right? So, in the general case, the callee is responsible for inserting a parameter at the beginning of the PMC parameters. That's not all. Case 2) above - the method - can in all our target languages be called with a sub call syntax - AFAIK: y(x, z) This is the normal call syntax for Perl6 multi-subs. With two different schemes for the callee, we got two variants that would work w/o further argument adaption and two variants that need either shifting arguments up or down. But this prologue would be needed in every function or method. ... Insertion by setting a value into P2 is cheaper than insertion by shifting and then setting a value into P5. Already optimizing? - SCNR. Anyway: src/sub.c:copy_regs() already copies registers from the caller's register frame to the callee. Implementing argument shifting there is cheap - if it's still needed. By defining that all subs or methods get the arguments (including the object) passed in P5, P6 ... we'd only have half of the mis-matching argument orderings. At definition time, the PyFunc PMC has a number of properties (or flags) which are set defining names of the formal arguments, defaults, and the like. Yes, more complications. ... A similar approach can be used to define whether the first parameter is expected in P2 or P5. Sure. But why should we go the more complicated way in the first place? Note: different languages may chose different strategies as to when to shift. What is important is that we come to an agreement on what is expected of the caller. From the caller's POV its quite clear if it's a method or a function call. But if the other end has two incarnations, things get messy. - Sam Ruby leo
Re: bound methods
Leopold Toetsch wrote:
Sam Ruby [EMAIL PROTECTED] wrote:
Leopold Toetsch wrote:
... But that doesn't
work fur user methods, especially if there is no indication that a user
function is used as a method in the first place.
def find(s, sub):
...
In Python, this is statically determinable. If that sequence is
directly nested inside a class, it is a method, otherwise it is a
function.
No, we had that several times. You did show examples, where this isn't
true. Here is another one:
def add(l, r):
print in add
return 42
class C(object):
__add__ = add
c = C()
print c + 1
add is just a plain function. There is no indication whatsoever that
it might be used as a method, when add is compiled. But as infix+ is
a method call l.__add__(r) this doesn't work, if the object is in P2.
I've proposed several times that arguments including the object should be
passed from P5 up. The invocant - if any - can still be reachable with
the Cinterpinfo opcode, but it'll be probably used just in calls to SUPER
or next_method or such.
Less shifting will be required if the object is passed in P2.
That's only true for NCI methods. And IIRC, your arguments were the same
some time ago. How do you compile the add above, when the object is in
P2?
Notes: at the moment, every Python method call creates a bound object,
and shifts PMC arguments.
This shouldn't be necessary for normal method calls like:
s.f(r)
But this translates into two VTABLE calls. find_method and invoke.
find_method needs to return a bound method.
Why don't you just use the callmethodcc opcode? That's exactly what is
happening here.
A find_method_and_invoke VTABLE entry (or more simply call_method) would
not need to return the intermediary bound method.
There is no need for an intermediate object, if it's a plain method
call.
If there were a call_method VTABLE entry and if P2 were passed into
methods, all of this would be unecessary in the majority of cases.
A separate vtable slot doesn't really help. Code that looks like a
function call can actually be a method call (and vv). Separating the
call into two vtables will just duplicate the call sequence. But let's
first convince Dan that all arguments are passed from P5 up, then we'll
see what we have.
Less shifting will be required if the object is passed in P2.
See above.
And why are you duplicating object arguments into P5, if the object
should go into P2?
,--[ dynclasses/pyint.pmc ]---
| METHOD PMC* __hex__(PMC *self) {
`-
You are inventing an additional self argument here just to work around
the problem that the object is passed in P2. The actual object pmc is
ignored:
,--[ dynclasses/pyint.c ]---
| PMC*
| Parrot_PyInt___hex__(Interp* interpreter, PMC* pmc, PMC *self)
`---
As said in another message, there is no problem with NCI methods. It's
just a matter of translating the O signature char.
The problem are user functions. There might be an indication that's a
method (like a definition inside a class block) - but not always. And if
the latter is the case *only once*, we just have to call all methods
with plain function call argument passing. Or prepend *all* functions
with an argument check wrapper that shift arguments around, if the call
was actually a method call.
The current CVS is in a state of transition.
I originally implemented to the current set of PDDs. Unhappy with the
amount of copying of PMC registers, I optimistically converted over to
the pass self as P5 approach that you suggested. I found that that
simply moved the problem to other places, and copying was unavoidable.
In fact, it generally made things worse.
Given that copying is unavoidable, what I would like to do is to
optimize for what I consider the most common cases, and make all the
remaining cases work properly.
The common cases I want to optimize for are common functions being
called as common functions. And common methods being called as methods.
The easiest way to optimize for the common methods being called as
methods is if the current object is not passed as the first argument.
Yes, copying will be required in cases where functions are called as
methods.
- Sam Ruby
Re: bound methods
Sam Ruby [EMAIL PROTECTED] wrote: The common cases I want to optimize for are common functions being called as common functions. And common methods being called as methods. Yep, that's very reasonable. The easiest way to optimize for the common methods being called as methods is if the current object is not passed as the first argument. Why so? Python methods don't have a notion of an object - it's just the first argument passed into a function. So I don't quite understand your conclusion. This isn't a python-only problem. Perl5 doesn't have an object either. Perl6 multi-methods provide both features: foo($a, $b);# probably a MMD call on both $a.foo($b); # a method call on 1st invocant The function or multi-sub foo can be basically anything in the first case including a lexical function foo and not a method at all. As long as we have different calling conventions for these 2 variants we have just a better chance for a mismatch IMHO. Yes, copying will be required in cases where functions are called as methods. How do you detect this case and when are arguments shifted then - note: I prefer the term shifting as copying is done anyway in all sub calls (see src/sub.c:copy_regs). - Sam Ruby leo
Re: bound methods
Sam Ruby [EMAIL PROTECTED] wrote: Leopold Toetsch wrote: WRT functionality: for a call it has to shift up PMC arguments and insert the object as P5, right? At the moment, it contains this logic. My plans are to remove the shifting and set the object into P2 / INTERP-ctx.current_object. Dan didn't answer issues WRT argument passing yet. In the current scheme your plan would be ok - for e.g. a NCI find method. But that doesn't work fur user methods, especially if there is no indication that a user function is used as a method in the first place. def find(s, sub): ... I've proposed several times that arguments including the object should be passed from P5 up. The invocant - if any - can still be reachable with the Cinterpinfo opcode, but it'll be probably used just in calls to SUPER or next_method or such. Notes: at the moment, every Python method call creates a bound object, and shifts PMC arguments. This shouldn't be necessary for normal method calls like: s.f(r) ... Removing the shifting of PMC arguments will require the insertion of an interpinfo opcode into each method. Or fixing argument passsing like above. If there were a call_method VTABLE entry and if P2 were passed into methods, all of this would be unecessary in the majority of cases. A separate vtable slot doesn't really help. Code that looks like a function call can actually be a method call (and vv). Separating the call into two vtables will just duplicate the call sequence. But let's first convince Dan that all arguments are passed from P5 up, then we'll see what we have. - Sam Ruby leo
Re: bound methods
Sam Ruby [EMAIL PROTECTED] wrote: Leopold Toetsch wrote: [ provide Bound_Meth core PMC classes ] Cool. I'd likely still subclass it to provide a get_string. Yep. WRT implementation: I'd like to swap struct_val/pmc_val for all Sub classes. It's just cleaner but should be almost fully transparent to users of these classes. No objection. Done that now. PMC_sub(SELF) aka PMC_struct_val(SELF) holds now basically all the subroutine information: segment, address, name of Subs or the C function pointer for NCI. This makes PMC_pmc_val() consistently available as the bound object. leo
Re: bound methods
Leopold Toetsch wrote: Sam Ruby [EMAIL PROTECTED] wrote: Leopold Toetsch wrote: WRT functionality: for a call it has to shift up PMC arguments and insert the object as P5, right? At the moment, it contains this logic. My plans are to remove the shifting and set the object into P2 / INTERP-ctx.current_object. Dan didn't answer issues WRT argument passing yet. In the current scheme your plan would be ok - for e.g. a NCI find method. But that doesn't work fur user methods, especially if there is no indication that a user function is used as a method in the first place. def find(s, sub): ... In Python, this is statically determinable. If that sequence is directly nested inside a class, it is a method, otherwise it is a function. I've proposed several times that arguments including the object should be passed from P5 up. The invocant - if any - can still be reachable with the Cinterpinfo opcode, but it'll be probably used just in calls to SUPER or next_method or such. Less shifting will be required if the object is passed in P2. Notes: at the moment, every Python method call creates a bound object, and shifts PMC arguments. This shouldn't be necessary for normal method calls like: s.f(r) But this translates into two VTABLE calls. find_method and invoke. find_method needs to return a bound method. A find_method_and_invoke VTABLE entry (or more simply call_method) would not need to return the intermediary bound method. ... Removing the shifting of PMC arguments will require the insertion of an interpinfo opcode into each method. Or fixing argument passsing like above. Which amounts to not removing the shifting. If there were a call_method VTABLE entry and if P2 were passed into methods, all of this would be unecessary in the majority of cases. A separate vtable slot doesn't really help. Code that looks like a function call can actually be a method call (and vv). Separating the call into two vtables will just duplicate the call sequence. But let's first convince Dan that all arguments are passed from P5 up, then we'll see what we have. Less shifting will be required if the object is passed in P2. - Sam Ruby
Re: bound methods
Sam Ruby [EMAIL PROTECTED] wrote:
Leopold Toetsch wrote:
... But that doesn't
work fur user methods, especially if there is no indication that a user
function is used as a method in the first place.
def find(s, sub):
...
In Python, this is statically determinable. If that sequence is
directly nested inside a class, it is a method, otherwise it is a
function.
No, we had that several times. You did show examples, where this isn't
true. Here is another one:
def add(l, r):
print in add
return 42
class C(object):
__add__ = add
c = C()
print c + 1
add is just a plain function. There is no indication whatsoever that
it might be used as a method, when add is compiled. But as infix+ is
a method call l.__add__(r) this doesn't work, if the object is in P2.
I've proposed several times that arguments including the object should be
passed from P5 up. The invocant - if any - can still be reachable with
the Cinterpinfo opcode, but it'll be probably used just in calls to SUPER
or next_method or such.
Less shifting will be required if the object is passed in P2.
That's only true for NCI methods. And IIRC, your arguments were the same
some time ago. How do you compile the add above, when the object is in
P2?
Notes: at the moment, every Python method call creates a bound object,
and shifts PMC arguments.
This shouldn't be necessary for normal method calls like:
s.f(r)
But this translates into two VTABLE calls. find_method and invoke.
find_method needs to return a bound method.
Why don't you just use the callmethodcc opcode? That's exactly what is
happening here.
A find_method_and_invoke VTABLE entry (or more simply call_method) would
not need to return the intermediary bound method.
There is no need for an intermediate object, if it's a plain method
call.
If there were a call_method VTABLE entry and if P2 were passed into
methods, all of this would be unecessary in the majority of cases.
A separate vtable slot doesn't really help. Code that looks like a
function call can actually be a method call (and vv). Separating the
call into two vtables will just duplicate the call sequence. But let's
first convince Dan that all arguments are passed from P5 up, then we'll
see what we have.
Less shifting will be required if the object is passed in P2.
See above.
And why are you duplicating object arguments into P5, if the object
should go into P2?
,--[ dynclasses/pyint.pmc ]---
| METHOD PMC* __hex__(PMC *self) {
`-
You are inventing an additional self argument here just to work around
the problem that the object is passed in P2. The actual object pmc is
ignored:
,--[ dynclasses/pyint.c ]---
| PMC*
| Parrot_PyInt___hex__(Interp* interpreter, PMC* pmc, PMC *self)
`---
As said in another message, there is no problem with NCI methods. It's
just a matter of translating the O signature char.
The problem are user functions. There might be an indication that's a
method (like a definition inside a class block) - but not always. And if
the latter is the case *only once*, we just have to call all methods
with plain function call argument passing. Or prepend *all* functions
with an argument check wrapper that shift arguments around, if the call
was actually a method call.
- Sam Ruby
leo
Re: bound methods
Sam Ruby [EMAIL PROTECTED] wrote: Leopold Toetsch wrote: Not quite. It's just: f = getattribute Parrot_string, find nothing more. The Cget_attr_str vtable has to do the right thing, i.e. if the attribute is a callable, it has to return a bound method object. Exactly. [ I've checked in the Bound_NCI PMC now ] The default implementation of get_attr_str already did a method lookup (it should probably use VTABLE_find_method, though). For the easy case - the method is NCI - now a Bound_NCI object is returned. Invoking that places the bound object as P2, which is fine for NCI calls, as the calling conventions are fully under our control. It's just a matter, how the signature O is translated: either take the next argument from P2 of from the next in P5, P6 ... So actually for NCI methods the current scheme is fine as shifting arguments isn't needed. Still remaining and unsolved is the problem with user methods argument passing, which does just not match. Cset_pointer sets the pointer to the actual subroutine. That's not quite right. Cset_pointer set's the function address of a callable. Anyway, a Bound_NCI isa NCI - it's not a container object like your implementation is. This simplifies inheritance vastly. ... Cset_pmc sets the pointer to the bound object. And we've additionally CPMC* get_pmc(), which returns the bound object. - Sam Ruby leo
bound methods (was: Calling conventions, invocations, and suchlike things)
Dan Sugalski [EMAIL PROTECTED] wrote: At 5:04 PM -0500 1/18/05, Sam Ruby wrote: f = Parrot.find print f(r) Note that I referenced the method as an attribute, and then called it as a function. Mmm, syntax! :) Luckily it makes no difference to us at the parrot level. What that should translate to is something like: $P0 = find_method Parrot_string, find # Elided check for failed lookup and fallback to attribute fetch $P1 = make_bound_method(Parrot_string, $P0) Not quite. It's just: f = getattribute Parrot_string, find nothing more. The Cget_attr_str vtable has to do the right thing, i.e. if the attribute is a callable, it has to return a bound method object. Furthermore, the function remembers what object it is bound to. This is accomplished by VTABLE_find_method creating a new PyBoundMeth PMC which contains two references, one to the object, and one to the method. While a good idea, I think it's not the right way to handle this. Binding objects to methods to create invokable subs is going to be something we're going to need for a lot of the languages, so I think we'd be better served providing a general facility to do it rather than leaving it to each individual language designer to do it. Should save some work all around too. Yeah. When this came up last, I've proposed two ways to handle it: 1) inside the Sub/NCI PMC 2) by a distinct Bound_Meth PMC class derived from 1) The latter is probably cleaner. Binding the object to the callable could be done e.g. by the Cset_pmc vtable. leo
Re: bound methods
Leopold Toetsch wrote: Dan Sugalski [EMAIL PROTECTED] wrote: At 5:04 PM -0500 1/18/05, Sam Ruby wrote: f = Parrot.find print f(r) Note that I referenced the method as an attribute, and then called it as a function. Mmm, syntax! :) Luckily it makes no difference to us at the parrot level. What that should translate to is something like: $P0 = find_method Parrot_string, find # Elided check for failed lookup and fallback to attribute fetch $P1 = make_bound_method(Parrot_string, $P0) Not quite. It's just: f = getattribute Parrot_string, find nothing more. The Cget_attr_str vtable has to do the right thing, i.e. if the attribute is a callable, it has to return a bound method object. Exactly. Furthermore, the function remembers what object it is bound to. This is accomplished by VTABLE_find_method creating a new PyBoundMeth PMC which contains two references, one to the object, and one to the method. While a good idea, I think it's not the right way to handle this. Binding objects to methods to create invokable subs is going to be something we're going to need for a lot of the languages, so I think we'd be better served providing a general facility to do it rather than leaving it to each individual language designer to do it. Should save some work all around too. Yeah. When this came up last, I've proposed two ways to handle it: 1) inside the Sub/NCI PMC 2) by a distinct Bound_Meth PMC class derived from 1) The latter is probably cleaner. Binding the object to the callable could be done e.g. by the Cset_pmc vtable. That's exactly how PyBoundMeth works today. Cset_pointer sets the pointer to the actual subroutine. Cset_pmc sets the pointer to the bound object. - Sam Ruby
Re: bound methods
Sam Ruby [EMAIL PROTECTED] wrote: Leopold Toetsch wrote: 2) by a distinct Bound_Meth PMC class derived from 1) The latter is probably cleaner. Binding the object to the callable could be done e.g. by the Cset_pmc vtable. That's exactly how PyBoundMeth works today. Cset_pointer sets the pointer to the actual subroutine. Cset_pmc sets the pointer to the bound object. Great. I saw the checkin but didn't have a close look at it. So if even the interface is the same, we really should put it into the core classes. WRT implementation: I'd like to swap struct_val/pmc_val for all Sub classes. It's just cleaner but should be almost fully transparent to users of these classes. WRT functionality: for a call it has to shift up PMC arguments and insert the object as P5, right? - Sam Ruby leo
Re: bound methods
Leopold Toetsch wrote: Sam Ruby [EMAIL PROTECTED] wrote: Leopold Toetsch wrote: 2) by a distinct Bound_Meth PMC class derived from 1) The latter is probably cleaner. Binding the object to the callable could be done e.g. by the Cset_pmc vtable. That's exactly how PyBoundMeth works today. Cset_pointer sets the pointer to the actual subroutine. Cset_pmc sets the pointer to the bound object. Great. I saw the checkin but didn't have a close look at it. So if even the interface is the same, we really should put it into the core classes. Cool. I'd likely still subclass it to provide a get_string. WRT implementation: I'd like to swap struct_val/pmc_val for all Sub classes. It's just cleaner but should be almost fully transparent to users of these classes. No objection. WRT functionality: for a call it has to shift up PMC arguments and insert the object as P5, right? At the moment, it contains this logic. My plans are to remove the shifting and set the object into P2 / INTERP-ctx.current_object. Notes: at the moment, every Python method call creates a bound object, and shifts PMC arguments. Removing the shifting of PMC arguments will require the insertion of an interpinfo opcode into each method. If there were a call_method VTABLE entry and if P2 were passed into methods, all of this would be unecessary in the majority of cases. - Sam Ruby
