bound methods (was: Calling conventions, invocations, and suchlike things)

2005-01-30 Thread Leopold Toetsch
Dan Sugalski [EMAIL PROTECTED] wrote:
 At 5:04 PM -0500 1/18/05, Sam Ruby wrote:

 f = Parrot.find
 print f(r)

Note that I referenced the method as an attribute, and then called
it as a function.

 Mmm, syntax! :) Luckily it makes no difference to us at the parrot
 level. What that should translate to is something like:

  $P0 = find_method Parrot_string, find
   # Elided check for failed lookup and fallback to attribute fetch
  $P1 = make_bound_method(Parrot_string, $P0)

Not quite. It's just:

  f = getattribute Parrot_string, find

nothing more. The Cget_attr_str vtable has to do the right thing, i.e.
if the attribute is a callable, it has to return a bound method object.

Furthermore, the function remembers what object it is bound to.
This is accomplished by VTABLE_find_method creating a new
PyBoundMeth PMC which contains two references, one to the object,
and one to the method.

 While a good idea, I think it's not the right way to handle this.
 Binding objects to methods to create invokable subs is going to be
 something we're going to need for a lot of the languages, so I think
 we'd be better served providing a general facility to do it rather
 than leaving it to each individual language designer to do it. Should
 save some work all around too.

Yeah. When this came up last, I've proposed two ways to handle it:

1) inside the Sub/NCI PMC
2) by a distinct Bound_Meth PMC class derived from 1)

The latter is probably cleaner. Binding the object to the callable could
be done e.g. by the Cset_pmc vtable.

leo


Re: Calling conventions, invocations, and suchlike things

2005-01-28 Thread Dan Sugalski
At 5:04 PM -0500 1/18/05, Sam Ruby wrote:
Dan Sugalski wrote:
Hi folks.
Welcome back!
Parrot's got the interesting, and somewhat unfortunate, requirement 
of having to allow all subroutines behave as methods and all 
methods behave as subroutines. (This is a perl 5 thing, but we have 
to make it work) That is, an invokable PMC may be invoked as a 
method call and passed in an object, or as a plain subroutine and 
not have an object passed in. As far as perl 5 is concerned the 
object is the first parameter in the argument list, but for 
everyone else the object is a very distinct and separate thing.
Python essentially has the same requirement, with a few twists. 
Specifically, methods come in be static, class, and regular flavors.

But first, a simple example.  Strings in python have a find 
method, so and can do the following:

f = Parrot.find
print f(r)
Note that I referenced the method as an attribute, and then called 
it as a function.
Mmm, syntax! :) Luckily it makes no difference to us at the parrot 
level. What that should translate to is something like:

$P0 = find_method Parrot_string, find
 # Elided check for failed lookup and fallback to attribute fetch
$P1 = make_bound_method(Parrot_string, $P0)
$P1(r)
Furthermore, the function remembers what object it is bound to. 
This is accomplished by VTABLE_find_method creating a new 
PyBoundMeth PMC which contains two references, one to the object, 
and one to the method.
While a good idea, I think it's not the right way to handle this. 
Binding objects to methods to create invokable subs is going to be 
something we're going to need for a lot of the languages, so I think 
we'd be better served providing a general facility to do it rather 
than leaving it to each individual language designer to do it. Should 
save some work all around too.

Static methods differ in that the object is not passed.
How is this different from a subroutine, then?
Class methods differ in that the object passed is actually the class 
of the object in question.
I'm assuming this is different from just a method on the class somehow?
Note: all this is determined by the callee.  It is all transparent 
to the caller.
This is the part I'm not so sure about. It looks like, rather than 
having two sides (caller and calle) we have three, caller, callee, 
and the code that fetches the invokable in the first place.

I fully agree that the caller shouldn't know about this stuff, since 
it may well have been handed the invokable thing as part of a 
function call or pulled it out of a variable or something.

I don't think the callee should have to know anything special here, 
though -- it doesn't seem at all unreasonable to have the callee 
*not* have to do anything special, nor play any magic games. (And I 
think I'd be a bit peeved if I was writing code which passed in 
object A as the object being invoked on, but the method decided it 
wanted to use object B instead) This is especially true in a 
mixed-language environment when you've got a class with methods 
written in different languages -- setting up any conventions that'll 
actually be followed seems like an exercise in futility. :)

That leaves the code that actually fetches the invokable thing in the 
first place, and that seems like the right place for this to happen. 
The language the code is written in knows what should happen based on 
what it gets back when querying the object, so as long as we provide 
a standard means to do all the binding stuff, we shoul dbe fine.

First, observe that I don't have any control over the exception that 
is raised when a method is not found (fix: raise the exception 
within find_method).
Right. There's going to be one generic method-not-found exception -- 
there really has to be only one, otherwise we're going to run into 
all sorts of cross-language problems. Exception unification (and, 
more likely, aliasing) is going to be one of the tricky issues.

My one minor request here is P2 be made available on entry to the 
invoked method.  This would remove some special case logic for me 
requiring the use of interpinfo.  I don't expect any guarantees that 
this is preserved or restored across sub calls.
The one thing that leaving it in the interpreter structure and not 
explicitly passing it in gets us is we get notice if its actually 
extracted and used. Which  is going to be fairly common, so I'm not 
sure what it buys us. I think we'll leave things as-is, but I'm not 
sure for how much longer.

Not having objects handle their own method dispatch is less 
clear-cut, but I do have some reasons, so here they are.
Just be aware that in order to preserve Python semantics, 
find_method will need to return a bound method.
That can't happen. find_method has to return an unbound method, since 
there are just too many cases where that's what we need. If the 
method then needs to be bound then the fetching code can do the 
binding.

 This involves creating an object on the heap, 

Re: Calling conventions, invocations, and suchlike things

2005-01-28 Thread Sam Ruby
Dan Sugalski wrote:
At 5:04 PM -0500 1/18/05, Sam Ruby wrote:
Dan Sugalski wrote:
Hi folks.
Welcome back!
Parrot's got the interesting, and somewhat unfortunate, requirement 
of having to allow all subroutines behave as methods and all methods 
behave as subroutines. (This is a perl 5 thing, but we have to make 
it work) That is, an invokable PMC may be invoked as a method call 
and passed in an object, or as a plain subroutine and not have an 
object passed in. As far as perl 5 is concerned the object is the 
first parameter in the argument list, but for everyone else the 
object is a very distinct and separate thing.
Python essentially has the same requirement, with a few twists. 
Specifically, methods come in be static, class, and regular flavors.

But first, a simple example.  Strings in python have a find method, 
so and can do the following:

f = Parrot.find
print f(r)
Note that I referenced the method as an attribute, and then called it 
as a function.
Mmm, syntax! :) Luckily it makes no difference to us at the parrot 
level. What that should translate to is something like:

$P0 = find_method Parrot_string, find
 # Elided check for failed lookup and fallback to attribute fetch
$P1 = make_bound_method(Parrot_string, $P0)
$P1(r)
This will be a recurring theme in my replies.  Any thing which presumes 
a bit of knowledge at compile time will ultimately not work.

Consider the following:
  class c:
find = 7
  def f(x):
return x.find
  print f(c())
  print f(Parrot)(r)
Now, what should the code for function f look like?  The only reasonable 
answer is something along the lines of:

  getattribute $P0, P5, 'find'
This has to work.  In both of the two calls to f().
Furthermore, the function remembers what object it is bound to. This 
is accomplished by VTABLE_find_method creating a new PyBoundMeth PMC 
which contains two references, one to the object, and one to the method.
While a good idea, I think it's not the right way to handle this. 
Binding objects to methods to create invokable subs is going to be 
something we're going to need for a lot of the languages, so I think 
we'd be better served providing a general facility to do it rather than 
leaving it to each individual language designer to do it. Should save 
some work all around too.
This would not be necessary, but the current implementation of the 
callmethodcc opcode unfortunately decouples the find_method (which is 
subject to the not at compile time restrictions alluded to above), and 
invoke.  If these weren't decoupled (i.e., there was a 
VTABLE_find_method or equivalent entry), then this would not be necessary.

Static methods differ in that the object is not passed.
How is this different from a subroutine, then?
From a callee-perspective: not at all.
What is important to note is that from a caller-perspective, they will 
invoke such subroutines with the callmethcc opcode.

Class methods differ in that the object passed is actually the class 
of the object in question.
I'm assuming this is different from just a method on the class somehow?
From a callee perspective, this appears to be a method on the instance.
Note: all this is determined by the callee.  It is all transparent to 
the caller.
This is the part I'm not so sure about. It looks like, rather than 
having two sides (caller and calle) we have three, caller, callee, and 
the code that fetches the invokable in the first place.

I fully agree that the caller shouldn't know about this stuff, since it 
may well have been handed the invokable thing as part of a function call 
or pulled it out of a variable or something.

I don't think the callee should have to know anything special here, 
though -- it doesn't seem at all unreasonable to have the callee *not* 
have to do anything special, nor play any magic games. (And I think I'd 
be a bit peeved if I was writing code which passed in object A as the 
object being invoked on, but the method decided it wanted to use object 
B instead) This is especially true in a mixed-language environment when 
you've got a class with methods written in different languages -- 
setting up any conventions that'll actually be followed seems like an 
exercise in futility. :)
Perhaps you might not find Python to your liking.  That's OK.
But the more general question is whether or not Parrot will implement 
the above as a policy, and thereby preclude langages like Python from 
being implemented on top of Parrot.

That leaves the code that actually fetches the invokable thing in the 
first place, and that seems like the right place for this to happen. The 
language the code is written in knows what should happen based on what 
it gets back when querying the object, so as long as we provide a 
standard means to do all the binding stuff, we shoul dbe fine.
I'm not sure what you mean by the code that actually fetches the 
invokable thing in this instance.  If you mean an at compile time 
translation approach like you alluded above, 

Re: Calling conventions, invocations, and suchlike things

2005-01-28 Thread Sam Ruby
Luke Palmer wrote:
Sam Ruby writes:
Mmm, syntax! :) Luckily it makes no difference to us at the parrot 
level. What that should translate to is something like:

  $P0 = find_method Parrot_string, find
   # Elided check for failed lookup and fallback to attribute fetch
  $P1 = make_bound_method(Parrot_string, $P0)
  $P1(r)
This will be a recurring theme in my replies.  Any thing which presumes 
a bit of knowledge at compile time will ultimately not work.

Consider the following:
 class c:
   find = 7
 def f(x):
   return x.find
 print f(c())
 print f(Parrot)(r)
Now, what should the code for function f look like?  The only reasonable 
answer is something along the lines of:

 getattribute $P0, P5, 'find'
I doubt that.  All languages have different semantics, and we can't
implement them all, because they are conflicting.  You, as a compiler
designer, have the opportunity to design things so that they work.  And
you definitely have to be clever if you're looking for language features
that Parrot doesn't natively support.
I disagree.  As Dan once said, as long as we have a proper protocol 
that everyone can conform to, we should be OK.

I don't care if Perl, Python, Ruby, TCL, and others each implement 
different semantics.  I do care that we adopt a common protocol.

The current set of VTABLE entries is a excellent first order 
approximation for the common protocol.

My first tendency here is to echo all attributes as methods (like Perl 6
does), and then always call the method when you see a dot.  ParrotString
has a find method, and it knows how to curry itself.  Instances of c
also have a find method, and that method always returns 7.
The distinction between attributes and methods is a bit subtle.  Any 
object may implement the VTABLE_invoke method (or in Python terms, a 
__call__ method), so all attributes may already *BE* a method.

But largely, I could change the current implementation of Python on 
Parrot to follow such a protocol in a matter of minutes.

But my original comment (Any approach 'which presumes a bit of knowledge 
at compile time will ultimately not work.') still stands.

Although I agree that we should come up with a general, bare-bones
object model that allows all of our current target languages to operate
smoothly.  Parrot's current model makes far too many assumptions.  But
that doesn't mean that anything you're trying to do is impossible; it
just means it's harder.
I don't believe that Parrot should impose an object model.  I return to 
Dan's as long as we have a proper protocol that everyone can conform 
to, we should be OK.

- Sam Ruby


Re: Calling conventions, invocations, and suchlike things

2005-01-28 Thread Luke Palmer
Sam Ruby writes:
 Mmm, syntax! :) Luckily it makes no difference to us at the parrot 
 level. What that should translate to is something like:
 
 $P0 = find_method Parrot_string, find
  # Elided check for failed lookup and fallback to attribute fetch
 $P1 = make_bound_method(Parrot_string, $P0)
 $P1(r)
 
 This will be a recurring theme in my replies.  Any thing which presumes 
 a bit of knowledge at compile time will ultimately not work.
 
 Consider the following:
 
   class c:
 find = 7
 
   def f(x):
 return x.find
 
   print f(c())
   print f(Parrot)(r)
 
 Now, what should the code for function f look like?  The only reasonable 
 answer is something along the lines of:
 
   getattribute $P0, P5, 'find'

I doubt that.  All languages have different semantics, and we can't
implement them all, because they are conflicting.  You, as a compiler
designer, have the opportunity to design things so that they work.  And
you definitely have to be clever if you're looking for language features
that Parrot doesn't natively support.

My first tendency here is to echo all attributes as methods (like Perl 6
does), and then always call the method when you see a dot.  ParrotString
has a find method, and it knows how to curry itself.  Instances of c
also have a find method, and that method always returns 7.

Although I agree that we should come up with a general, bare-bones
object model that allows all of our current target languages to operate
smoothly.  Parrot's current model makes far too many assumptions.  But
that doesn't mean that anything you're trying to do is impossible; it
just means it's harder.

Luke


Re: Calling conventions, invocations, and suchlike things

2005-01-28 Thread Sam Ruby
Sam Ruby wrote:
Now, what should the code for function f look like?  The only 
reasonable answer is something along the lines of:

 getattribute $P0, P5, 'find'
I doubt that.  All languages have different semantics, and we can't
implement them all, because they are conflicting.  You, as a compiler
designer, have the opportunity to design things so that they work.  And
you definitely have to be clever if you're looking for language features
that Parrot doesn't natively support.
I disagree.  As Dan once said, as long as we have a proper protocol 
that everyone can conform to, we should be OK.

I don't care if Perl, Python, Ruby, TCL, and others each implement 
different semantics.  I do care that we adopt a common protocol.

The current set of VTABLE entries is a excellent first order 
approximation for the common protocol.
In case that isn't perfectly clear, let me illustrate this with actual 
source code:

  inline op getattribute(out PMC, in PMC, in STR) :object_classes {
  $1 = VTABLE_get_attr_str(interpreter, $2, $3);
  goto NEXT();
  }
Note that the opcode doesn't actually implement any semantics.  It 
merely delegates the request to the $2 PMC.

Languages which share semantics for this operation can chose to inherit 
a common implementation.  Those with unique semantics can override this 
(or can find other languages with similar requirements and pool their 
implementation).

It seems rather likely to me that this was the original intent to 
providing a VTABLE_get_attr_str VTABLE entry in the first place.

This seems to me to be a rather good design pattern.
- Sam Ruby


Re: Calling conventions, invocations, and suchlike things

2005-01-19 Thread Leopold Toetsch
Dan Sugalski [EMAIL PROTECTED] wrote:
 Hi folks.

The lost son is back, welcome.

 The easy one first -- why the object is out-of-band, rather than one
 of the parameters.

 Parrot's got the interesting, and somewhat unfortunate, requirement
 of having to allow all subroutines behave as methods and all methods
 behave as subroutines. (This is a perl 5 thing,

*and* a Python thing *and* a Perl6 thing, when it comes to Perl6 multi
subs. And finally *all* Parrot infix operators are multi subs too.
For all these the current scheme does not match.

  multi sub infix:+(Int $left, Int $right) {...}

The only remaining methods, where your prerequisite is true is for Perl6
single methods, which the Perl6 code generator can translate either way,
because they are declared with the method keyword.

Please reconsider this decision.

 ... Regardless invokable
 things need to know whether they were called as a method or a sub.

Where currently the first argument (object) is passed in out-of-bounds,
the proposal is to pass the invocant. Nothing would change WRT all the
advantages of the current solution.

 Not having objects handle their own method dispatch is less
 clear-cut, but I do have some reasons, so here they are.

This is ok. See also Proposed vtable changes WRT method lookup, where
the dispatch and the find_method functionality is split.

leo


Calling conventions, invocations, and suchlike things

2005-01-18 Thread Dan Sugalski
Hi folks.
Sorry I've been gone so long. Non-p6i stuff's been well past 
monopolizing my time. Not much of an excuse, I know, but the Real 
World intrudes at the most inconvenient times. Things are, I hope, 
easing up a little, though I apologize in advance if I get a little 
cranky while I get back into things.

Having (very lightly) skimmed the past month of list mail, I'm 
thinking the best place to start is with the things that've come up 
about objects and method calls. I want to explain why things are 
designed the way they are so (hopefully) everyone's on the same page. 
(And, hopefully, to forestall grumbling when I say things aren't 
going to change :)

The setup, for those following along at home, is that when we make a 
method call the object is passed out-of-band (that is, not as part of 
the regular parameter list), and that objects don't actually handle 
method dispatch -- we split it into a two step affair where we 
request an invokable method PMC for a named method from an object, 
and then invoke it as a separate step.

The easy one first -- why the object is out-of-band, rather than one 
of the parameters. (Something that I doubt anyone's that worked up 
over, and I think everyone's OK with things as they stand, but here 
are the reasons anyway)

Parrot's got the interesting, and somewhat unfortunate, requirement 
of having to allow all subroutines behave as methods and all methods 
behave as subroutines. (This is a perl 5 thing, but we have to make 
it work) That is, an invokable PMC may be invoked as a method call 
and passed in an object, or as a plain subroutine and not have an 
object passed in. As far as perl 5 is concerned the object is the 
first parameter in the argument list, but for everyone else the 
object is a very distinct and separate thing. Regardless invokable 
things need to know whether they were called as a method or a sub. We 
*could* set a flag and have them check, then have some convention 
where the first parameter is an object if the I'm a method flag is 
set, but... yech. Having the object be separate and standalone seems 
cleaner, while still giving us a way to distinguish method/sub 
invocation. (You check to see if there's an object)

This does make things a little tricker for the perl 5 code generator, 
but not that much trickier and, let's face it, we're below the layer 
where things are easy. This *also* makes building signature checking 
into parrot a lot simpler (something we should do), since the 
signature checking stuff doesn't have to deal with possible parameter 
shifting based on whether we've a sub or method invocation.

Not having objects handle their own method dispatch is less 
clear-cut, but I do have some reasons, so here they are.

First off, one of the things I'm very much concerned about is C stack 
usage, both because we don't have all that much we can count on (joys 
of threads -- we'll be lucky to scrape together 10k some places) and 
because continuations can't cross C stack level boundaries. We're 
pretty careful about that one (it's the big reason for the limitation 
that continuations taken from within vtable functions can't escape).

I realize we can continue to be careful with it, mandating that the 
invoke_method vtable function behaves the same as the plain invoke 
does (that is, returning the address to jump to) but that brings up a 
separate problem -- transfer of control is a relatively heavyweight 
thing for us. Method and sub calls can potentially cross bytecode and 
security boundaries. Doing that right requires (potentially) a fair 
amount of screwing around inside the interpreter, as well as the 
invokable thing carrying around enough metadata to properly do the 
transfer. I'd really prefer to limit the number of PMCs that have 
that amount of intimate knowledge. Since all methods and subs have 
the appropriate bits attached to them, I'd as soon just use them.

There's also the potential issue of curried methods, where we need to 
create a new invokable thing and bind some parameters to it. We can 
certainly do that now with the current scheme so adding an 
invoke_method to the mix won't get in the way as such, but it does 
mean we have two near-identical ways of doing the same thing 
(find_method  invoke, and invoke_method) and since we can't toss the 
find_method way, it doesn't feel like adding invoke_method to the mix 
will get us anywhere.

Anyway, there we go. (I fully expect to find that both topics are 
dead about an hour after this goes out, but there you go :)
--
Dan

--it's like this---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk


Re: Calling conventions, invocations, and suchlike things

2005-01-18 Thread Sam Ruby
Dan Sugalski wrote:
Hi folks.
Welcome back!
Parrot's got the interesting, and somewhat unfortunate, requirement of 
having to allow all subroutines behave as methods and all methods behave 
as subroutines. (This is a perl 5 thing, but we have to make it work) 
That is, an invokable PMC may be invoked as a method call and passed in 
an object, or as a plain subroutine and not have an object passed in. As 
far as perl 5 is concerned the object is the first parameter in the 
argument list, but for everyone else the object is a very distinct and 
separate thing.
Python essentially has the same requirement, with a few twists. 
Specifically, methods come in be static, class, and regular flavors.

But first, a simple example.  Strings in python have a find method, so 
and can do the following:

f = Parrot.find
print f(r)
Note that I referenced the method as an attribute, and then called it as 
a function.  Furthermore, the function remembers what object it is 
bound to.  This is accomplished by VTABLE_find_method creating a new 
PyBoundMeth PMC which contains two references, one to the object, and 
one to the method.  The sole responsibility of a bound method is to 
insert the first parameter into the call, and then to invoke the desired 
method.

Static methods differ in that the object is not passed.
Class methods differ in that the object passed is actually the class of 
the object in question.

Note: all this is determined by the callee.  It is all transparent to 
the caller.

Now, compare that to callmethod or callmethodcc in ops:
   object = REG_PMC(2);
   method_pmc = VTABLE_find_method(interpreter, object, REG_STR(0));
   if (!method_pmc) {
 real_exception(interpreter, next, METH_NOT_FOUND,
 Method '%Ss' not found, REG_STR(0));
   }
   REG_PMC(0) = method_pmc;
   interpreter-ctx.current_object = object;
   dest = (opcode_t *)VTABLE_invoke(interpreter, method_pmc, next);
First, observe that I don't have any control over the exception that is 
raised when a method is not found (fix: raise the exception within 
find_method).  Second, observe that this assumes that current object is 
the object in all cases.  This doesn't work well for static or class 
methods (fix: create a PyBoundMeth object which overrides this, and then 
call the real method).

This does make things a little tricker for the perl 5 code generator, 
but not that much trickier and, let's face it, we're below the layer 
where things are easy. This *also* makes building signature checking 
into parrot a lot simpler (something we should do), since the signature 
checking stuff doesn't have to deal with possible parameter shifting 
based on whether we've a sub or method invocation.
Parameter shifting actually is made easier by placing the object in 
question in a separate register... no shifting is required.  (Note: in 
cvs, I actually do the shifting, partially because I'm in the process of 
reverting a change, and partially do to the issue raised in the next 
paragraph).

My one minor request here is P2 be made available on entry to the 
invoked method.  This would remove some special case logic for me 
requiring the use of interpinfo.  I don't expect any guarantees that 
this is preserved or restored across sub calls.

Not having objects handle their own method dispatch is less clear-cut, 
but I do have some reasons, so here they are.
Just be aware that in order to preserve Python semantics, find_method 
will need to return a bound method.  This involves creating an object on 
the heap, garbage collection, and a minor addition to the number of 
instructions executed on invoke (including a nested C stack).

This could all be avoided if there was a VTABLE_callmethod interface as 
the code would know that the intent was to only use this found method 
exactly once.

*shrug*
Do you plan to choose banana cream again at OSCON 2005?
- Sam Ruby