Re: Q: scope exit (was: Exceptions, sub cleanup, and scope exit)
At 8:07 AM +0100 12/10/04, Leopold Toetsch wrote: Dan Sugalski [EMAIL PROTECTED] wrote: ... A scope exit action is put in place on the control stack with: pushaction Psub * What is the intended usage of the action handler? * Specifically is this also ment for lazy DOD runs? * How is the relationship to the Cpop_pad opcode? The action handler is there to provide the languages as a way to do something when scopes are left. It's a generic 'out' for stuff that we've not thought about. Most scope exit stuff is cleanup which we'd rather be done via the DOD/GC system (otherwise things go Horribly Wrong in the face of continuations) but there may well be things that need doing. The one thing that I figure *will* be done is that languages will push a sweep or collect op in their scope cleanup in those cases where the language knows that there's potentially easy temp cleanup that needs doing, for example filehandles that should be closed when the scope's exited if there are no outstanding references to them. (And I know we've got the aggressive GC system for things like that, but in most cases languages can use something a bit less aggressive -- do a full sweep to clean up anything that's actually dead, and anything that escapes scope can be picked up later, since it's lifetime's probably gone nondeterministic) As far as pop_pad goes, I think maybe we need to revisit the control stack ops to see if some of them can go. (There are a fair number of rough draft things in the pad handling design that need editing) Possibly push_pad too, which could be handled with pushaction, or come up with a lighter-weight scheme to do something similar. Or it may be that there are few enough things that it's worth keeping push/pop_pad around. Not quite sure right now, but we should nail that one down. -- Dan --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Q: scope exit (was: Exceptions, sub cleanup, and scope exit)
Dan Sugalski [EMAIL PROTECTED] wrote: ... A scope exit action is put in place on the control stack with: pushaction Psub * What is the intended usage of the action handler? * Specifically is this also ment for lazy DOD runs? * How is the relationship to the Cpop_pad opcode? Thanks, leo
Re: Exceptions, sub cleanup, and scope exit
Dan Sugalski [EMAIL PROTECTED] wrote: pushmark 12 popmark 12 pushaction Psub I've now implemented these bits. I hope it's correct, specifically, if a return continuation in only captured, the action handler is not run. See t/pmc/exceptions.t Still missing is the throw opcode. Or better that exists, just exception creation and the extended attributes like language is missing. I'm still voting for a more object-ish exception constructor to better accomodate HLLs different exception usage. E.g. e = new PyKeyError # presumably a constant singleton throw e That ought to be enough for heavily used exception and for Perl6 control exceptions. OTOH e = new Exception setattribute e, message, Pmsg setattribute e, language, PLang ... throw e construct a full exception object. Currently it is: e[_message] = foo e[_error] e[_severity] ... And it could be even something like: cl = getclass Exception e = cl.instantiate(foo, Perl, .error, .severity, ...) leo
Re: Exceptions, sub cleanup, and scope exit
Dan Sugalski [EMAIL PROTECTED] wrote: At 9:59 AM +0100 11/19/04, Leopold Toetsch wrote: Its in and named Creturncc since yesterday return with current continuation. Hrm. The name's not right, I've proposed ret_cc and returncc, about two weeks ago the first time. I've asked for names of the opcode. As no answer arrived I just used that name. ... since there's no current continuation involved. Well, current in the sense of context, not P1. So the comment is better: return with continuation in context or such. push_eh handler, label Nope. The only thing that push_eh (or whatever we name it) needs is the address of the code to jump to if an exception's thrown. The exception pushing code should take care of building whatever structure's needed to call into it. Ah, ok. That makes sense. Allowing just one additional object doesn't properly support Python, which has two optional expressions for the Craise statement and Python attaches a traceback object to the exception. Hrm. Well, the traceback object could just be the interpreter structure at the time of the the exception. That'd be cheap enough to pass in. I'm fine with adding that as a second PMC parameter. I don't think that this works. The handler continuation restores the context, which changes the interpreter context. During unwinding the control stack we should probably fill a (preconstructed) array-ish object with pointers to contexts up to the handler conext. leo
Re: Exceptions, sub cleanup, and scope exit
At 10:28 AM +0100 11/22/04, Leopold Toetsch wrote: Dan Sugalski [EMAIL PROTECTED] wrote: At 9:59 AM +0100 11/19/04, Leopold Toetsch wrote: Its in and named Creturncc since yesterday return with current continuation. Hrm. The name's not right, I've proposed ret_cc and returncc, about two weeks ago the first time. I've asked for names of the opcode. As no answer arrived I just used that name. Fair enough. The cc's going to imply current continuation, which is going to confuse folks. Maybe we should name it invoke_return. Allowing just one additional object doesn't properly support Python, which has two optional expressions for the Craise statement and Python attaches a traceback object to the exception. Hrm. Well, the traceback object could just be the interpreter structure at the time of the the exception. That'd be cheap enough to pass in. I'm fine with adding that as a second PMC parameter. I don't think that this works. The handler continuation restores the context, which changes the interpreter context. During unwinding the control stack we should probably fill a (preconstructed) array-ish object with pointers to contexts up to the handler conext. We'd talked at one point about swapping interpreter structures as part of sub invocation, though there wasn't any resolution to what the right way to do that was. It got tabled as part of the flareup. Right now, I'm fine mandating that a non-invokable continuation is passed as the traceback object. If benchmarks show that we're going to require doing the swapping interpreter stuff to get non-sucky performance, or we have to do it to get reliable traceback info in the face of some exception scenarios, then we'll do it at that point. (I expect we're going to have to, but I'm fine with waiting on it for now, since it'll be transparent to bytecode and nearly all the C code) -- Dan --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: Exceptions, sub cleanup, and scope exit
At 9:59 AM +0100 11/19/04, Leopold Toetsch wrote: Dan Sugalski [EMAIL PROTECTED] wrote: The 'invoke the current return continuation' op apparently got lost in the blowup. That needs to go in. Its in and named Creturncc since yesterday return with current continuation. Hrm. The name's not right, since there's no current continuation involved. (At least we shouldn't be passing in the current continuation on return -- instead the return continuation that was in effect when the code we're returning to was active should be the return continuation) I'd like pushing exception handlers to remain simple -- the current system is almost OK. What I'd like it to change to is: push_eh label Shouldn't that be: push_eh handler, label Nope. The only thing that push_eh (or whatever we name it) needs is the address of the code to jump to if an exception's thrown. The exception pushing code should take care of building whatever structure's needed to call into it. (Though other than making a return continuation and changing the address to the address used in the push_eh instruction I'm not sure we need to do anything else) Throwing exceptions should be: throw language, subsystem, severity, code, object Allowing just one additional object doesn't properly support Python, which has two optional expressions for the Craise statement and Python attaches a traceback object to the exception. Hrm. Well, the traceback object could just be the interpreter structure at the time of the the exception. That'd be cheap enough to pass in. I'm fine with adding that as a second PMC parameter. OTOH (again from Python's view) raising just a CStopIteration or a CKeyErrror ought to be fast as its used heavily. Yeah. That's partly the reason for the non-PMC parameters. The other reason for them is that we may be throwing an exception at a point where it's not possible to actually create a PMC. We can always stuff in constant strings and integers, though. And Python has an hierarchy of exception *objects*, with inheritance. So I think that for maximum flexibility we should just have exception objects, with some mandatory attributes (the exception class might already provide enough information) and additional information the HLL might stuff in. Python code can always pass in an exception object in the PMC parameter slot -- that's fine. Perl 5 throws plain strings or arbitrary PMCs as exceptional things, and the internals may throw exceptions with non-pmc parameters if for some reason a PMC couldn't be created. And I'm definitely unsure what types language and subsystem should be. Clanguage - I presume - is defining the HLL that emitted the code, where the exception occured. Yep. I think this information has to be in the interpreter context as we should eventually be able to execute mixed languages code. So Clanguage should probably end up in the traceback object, passed along with the exception. Possibly, but C code may well be throwing exceptions too, but still want to attach a language to it. BTW - how do we set Clanguage? In general, I'd figured it was an attribute placed at compile-time on a sub PMC. pushmark 12 . . random code here . popmark 12 where everything put on the control stack after the pushmark identifier is popped off by the corresponding popmark identifier. Marks can nest, and it's the code's responsibility to not reuse labels. popmark will *not* cross routine boundaries, so the control stack will never be cleared out past the point where it was when the sub/method/whatever was entered. This is intended for cleanup of a bunch of pushed handlers inside one routine? Yep. Now, with this, there's one other thing we need to talk about, and that's scope exit actions. We promised this a number of years back (and now I feel very old) and need to deliver on them. A scope exit action is put in place on the control stack with: pushaction Psub and when the action is popped off the stack because of a popmark, walking the stack back looking for exception handlers, a popaction op, or returning from a subroutine the sub is invoked with a *single* integer parameter that indicates whether the action is being called as part of a normal or exception-based stack unwind. (a status of 0 means the action was caused by normal code, a 1 because the action was popped off as parrot walked backwards looking for an exception handler) If folks want to argue that the action should get the exception information if called as part of an exception unwind I'm OK with that. Go for it, I'm unsure of that one. Python has try ... finally. It preserves (possibly) existing exceptions around the finally block. But if in the finally clause another exception occurs (or a return of break is executed), the saved exception is lost. This could mean that the exception object is needed, e.g. to nullify it in that case. I think that the python compiler will be able to
Re: Exceptions, sub cleanup, and scope exit
Dan Sugalski [EMAIL PROTECTED] wrote: Exceptions are not, by default, resumable. Are there non-default resumable exceptions? leo
Re: Exceptions, sub cleanup, and scope exit
Dan Sugalski [EMAIL PROTECTED] wrote: The 'invoke the current return continuation' op apparently got lost in the blowup. That needs to go in. Its in and named Creturncc since yesterday return with current continuation. I'd like pushing exception handlers to remain simple -- the current system is almost OK. What I'd like it to change to is: push_eh label Shouldn't that be: push_eh handler, label This is more restrictive than passing in a generic invokable object as the exception handler, but given that all the exception handling schemes in all our languages of interest do lexically scoped exceptions I think we're better off with the restriction. Good. ...Allowing random invokables as exception handlers feels like it's likely to have some subtle security or consistency problems associated with it, though I can't give a good example at the moment. Yep, it smells like that. Throwing exceptions should be: throw language, subsystem, severity, code, object Allowing just one additional object doesn't properly support Python, which has two optional expressions for the Craise statement and Python attaches a traceback object to the exception. OTOH (again from Python's view) raising just a CStopIteration or a CKeyErrror ought to be fast as its used heavily. And Python has an hierarchy of exception *objects*, with inheritance. So I think that for maximum flexibility we should just have exception objects, with some mandatory attributes (the exception class might already provide enough information) and additional information the HLL might stuff in. How will Perl6 do it? And I'm definitely unsure what types language and subsystem should be. Clanguage - I presume - is defining the HLL that emitted the code, where the exception occured. I think this information has to be in the interpreter context as we should eventually be able to execute mixed languages code. So Clanguage should probably end up in the traceback object, passed along with the exception. BTW - how do we set Clanguage? pushmark 12 . . random code here . popmark 12 where everything put on the control stack after the pushmark identifier is popped off by the corresponding popmark identifier. Marks can nest, and it's the code's responsibility to not reuse labels. popmark will *not* cross routine boundaries, so the control stack will never be cleared out past the point where it was when the sub/method/whatever was entered. This is intended for cleanup of a bunch of pushed handlers inside one routine? Now, with this, there's one other thing we need to talk about, and that's scope exit actions. We promised this a number of years back (and now I feel very old) and need to deliver on them. A scope exit action is put in place on the control stack with: pushaction Psub and when the action is popped off the stack because of a popmark, walking the stack back looking for exception handlers, a popaction op, or returning from a subroutine the sub is invoked with a *single* integer parameter that indicates whether the action is being called as part of a normal or exception-based stack unwind. (a status of 0 means the action was caused by normal code, a 1 because the action was popped off as parrot walked backwards looking for an exception handler) If folks want to argue that the action should get the exception information if called as part of an exception unwind I'm OK with that. Go for it, I'm unsure of that one. Python has try ... finally. It preserves (possibly) existing exceptions around the finally block. But if in the finally clause another exception occurs (or a return of break is executed), the saved exception is lost. This could mean that the exception object is needed, e.g. to nullify it in that case. So, parrot will automatically clear out the control stack and invoke any actions on it if: *) the current return continuation is invoked with the return op *) a tail call is made with either the tailcall or tailmethodcall ops How does the latter look like for recursive tail calls? So. Simple, right? Make sense to everyone? Yep. A lot. A final question: classifying the exception in the handler is still up to the HLL, I presume? And one remark WRT internally thrown exceptions: in case of a MemoryError exception (Python speak), we are not allowed to use any more resources, so we probably need to preconstruct a few needed items for such a case. leo
Re: Exceptions, sub cleanup, and scope exit
Tim Bunce [EMAIL PROTECTED] wrote: I guess the HLL compiler needs to ensure that for every push the control flow will always pass through a matching pop. Not necessarily. The handler is pushed onto the control stack. During a context change (e.g. from a subroutine return), the previous context with the according control stack top is copied into the interpreter. So effectively the handler is discarded. OTOH if several handlers are involved the emitted code ought to take care of it, yes. Couldn't (doesn't) parrot have a stronger concept of a 'scope' (as in perl5's scope.c) so that scope-exit cleanup can be automated and reliable? AFAIK a scope is either a lexical closure or optimized (maybe) to a region of code surrounded by Cnew_pad / Cpop_pad opcodes. Doing the rught thing on scope exit is up to the emitted code. Tim [ignore me if I'm talking nonsense] leo
Re: Exceptions, sub cleanup, and scope exit
Dan Sugalski wrote: It's also important for people writing these things to take into account the possibility that their exit actions may potentially be triggered multiple times, courtesy of the joys of continuations. Hmm, the first thing to take into the account is that return continuations can be promoted to the fully blown continuations. This should affect the handlers in the same way - so exception handlers could have become arbitrary invokable objects at the point when the exception is thrown. Miro
Re: Exceptions, sub cleanup, and scope exit
Miroslav Silovic [EMAIL PROTECTED] wrote: Hmm, the first thing to take into the account is that return continuations can be promoted to the fully blown continuations. Yes. But an exception handler is not a RetContinuation object. It's an Exception_Handler object (also derived from Continuatio), behaving more like a RetContinuation then a full one. Creating a full continuation involves the Cclone vtable of that object, which can do just the right thing. With some few missing patches there isn't any visible P1 or such object around. So the described behavior will work. Miro leo
Re: Exceptions, sub cleanup, and scope exit
At 10:03 AM +0100 11/19/04, Miroslav Silovic wrote: Dan Sugalski wrote: It's also important for people writing these things to take into account the possibility that their exit actions may potentially be triggered multiple times, courtesy of the joys of continuations. Hmm, the first thing to take into the account is that return continuations can be promoted to the fully blown continuations. This should affect the handlers in the same way - so exception handlers could have become arbitrary invokable objects at the point when the exception is thrown. I'd rather not, if we can avoid it. Assumptions on control flow and such are likely to be very different, and I can't think of a good reason to treat an error handler as a normal sub. (If you have one then we can think on it some) -- Dan --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: Exceptions, sub cleanup, and scope exit
At 10:58 PM + 11/18/04, Tim Bunce wrote:
On Thu, Nov 18, 2004 at 11:37:54AM -0800, chromatic wrote:
On Thu, 2004-11-18 at 13:36 -0500, Dan Sugalski wrote:
I'd like pushing exception handlers to remain simple -- the current
system is almost OK. What I'd like it to change to is:
push_eh label
with popping the top exception handler being:
pop_eh
I'm up for better names, too.
The push_ is okay but eh is meh. push_handler seems better, though
handler is terribly generic. If the documentation and comments use it
consistently only for exceptions, though, it could work.
Throwcatch, so
push_catcher ?
I guess the HLL compiler needs to ensure that for every push the
control flow will always pass through a matching pop.
Otherwise that'll be another hard to debug situation.
Hopefully not. This is partially handled by the push/popmark stuff,
partially handled by stack unwinding on 'returns', and partially
handled by the sub-private nature of stacks.
Within a sub (or method, whatever) generated code can use the
pushmark/popmark ops to manage control stack elements. I expect
this'll be done mostly for entering and leaving scope -- code like:
foo: {
bar: {
}
}
outsidefoo:
would push a mark for foo when that block is entered, then push one
for bar when *that* block is entered. When bar's left the exit code
does a popmark bar, while exiting the foo block would exit a popmark
foo. If code in bar did a goto outsidefoo, the compiler would see
that it was exiting both the bar and foo blocks and just do a popmark
foo (since that'd pop all bar's entries too)
Each sub in parrot basically has its own private set of stacks, and
doesn't directly connect to its invoker's stacks. (Those are only
available from the return continuation tucked away in the interpreter
structure, so parrot can get to it easily enough) If code invokes a
continuation then the current stacks all go away (more or less --
unless there's a continuation holding on to them, but even then
they're out of view) and the stacks in the invoked continuation get
put into play, so any exception handlers that were in effect no
longer are. Or, rather, the ones that were in scope when the
continuation was taken are put back in scope as part of switching
over to the control stack that's held in the continuation.
Finally, when a 'return' op is invoked (basically one that puts the
return continuation back in play either through a real return or a
tail call) the op itself walks back the current control chain (which,
remember, is just for the current sub so it should be short) and
virtually pops all the elements off it, which means any element that
does something when popped will get a chance to do its thing.
Now it's distinctly possible there are more things that should go on
the control stack besides exception handlers, marks, and run my code
when I'm popped entries, so we can add those in as we need.
Tim [ignore me if I'm talking nonsense]
Nope, not nonsense. Things are just a bit fuzzy, so some
explanation's in order. :)
--
Dan
--it's like this---
Dan Sugalski even samurai
[EMAIL PROTECTED] have teddy bears and even
teddy bears get drunk
Re: Exceptions, sub cleanup, and scope exit
Dan Sugalski wrote: Hmm, the first thing to take into the account is that return continuations can be promoted to the fully blown continuations. This should affect the handlers in the same way - so exception handlers could have become arbitrary invokable objects at the point when the exception is thrown. I'd rather not, if we can avoid it. Assumptions on control flow and such are likely to be very different, and I can't think of a good reason to treat an error handler as a normal sub. (If you have one then we can think on it some) While that's not what I commented on, sure. Common LISP and TOM both invoke error handler -before- unwinding the stack (then the handler explicitely unwinds if it can't recover). I still don't think it's something Parrot should care about, as their (hypothetical) compilers can install their own error handling, and other languages don't expect their own throw to ever return - so I wouldn't call this a good reason. :) Miro
Exceptions, sub cleanup, and scope exit
Now that invocation semantics are fixed and permanent, we need some cleanup of the code base to round out support. It's also time to get some things in place that are part of the fallout from it. The 'invoke the current return continuation' op apparently got lost in the blowup. That needs to go in. return is a simple, if unimaginative name, and that works for me. The wording of PDD 03 implies that some of the registers can be populated by the op making the invocation, rather than explicitly setting them. In this case specifically the method/sub PMC, the object, and the return continuation. Leo brought this up but it got lost in the firestorm. We've already got invoke Px and invokecc Px along with the plain invoke and invokecc, but method calls aren't as well-rounded, and tail-call versions would be useful as well. With that out of the way, we should address exceptions. There's some support in now, but it's primitive, and needs updating. Exceptions are fairly straightforward. Setting an exception handler creates a exception continuation and pushes it onto the control stack. If an exception is thrown the control stack is walked looking for an exception handler and, when one is found, its continuation is invoked, at which point we transfer control to the exception handler. The following parameters are passed into the handler (and this I'm up for discussion on): *) Language throwing exception *) The subsystem throwing the exception *) The exception severity *) The exception code *) An object which the exception handler can do whatever it wants with Exceptions are not, by default, resumable. That's problematic with some classes of exception, so we aren't doing that. (Not, mind, that you can't throw a continuation to resume to as part of your object you throw at the exception handler) I'd like pushing exception handlers to remain simple -- the current system is almost OK. What I'd like it to change to is: push_eh label with popping the top exception handler being: pop_eh I'm up for better names, too. When pushing an exception handler we generate a return continuation with the resume address being the label rather than the next op. This is more restrictive than passing in a generic invokable object as the exception handler, but given that all the exception handling schemes in all our languages of interest do lexically scoped exceptions I think we're better off with the restriction. Allowing random invokables as exception handlers feels like it's likely to have some subtle security or consistency problems associated with it, though I can't give a good example at the moment. (Plus it can be overridden by a language that actually cares by having the exception handler invoke a random continuation) Still, forcing all exception objects to be created by parrot gives us some guarantees of control, so that's fine. Throwing exceptions should be: throw language, subsystem, severity, code, object And I'm definitely unsure what types language and subsystem should be. (I can see either ints or strings here) severity and code should both be ints, and object is a PMC. Rethrowing the current exception status should be done with the rethrow op, which takes no parameters. As part of this I want to reserve one language identifier for parrot-internal exceptions which can *not* be thrown by bytecode. The only way for these to propagate is if C code throws them or they're rethrown by an exception handler which can't handle them. This'll be used for exceptions we don't want to be fakeable. (I'm not sure we'll need any, but if we do I want the option) If bytecode does throw them it becomes a security exception instead, and we may well up and kill the interpreter. With pushing and popping exception handlers, we're going to want to pop off several exception handlers at once, in some cases an unknown number of handlers. As such I want to reintroduce something we talked about a very long time ago: stack marks. This allows us to do: pushmark 12 . . random code here . popmark 12 where everything put on the control stack after the pushmark identifier is popped off by the corresponding popmark identifier. Marks can nest, and it's the code's responsibility to not reuse labels. popmark will *not* cross routine boundaries, so the control stack will never be cleared out past the point where it was when the sub/method/whatever was entered. (Note that exception handlers don't count as subs here) That can be an exception in itself if we want, and probably ought to do. Now, with this, there's one other thing we need to talk about, and that's scope exit actions. We promised this a number of years back (and now I feel very old) and need to deliver on them. A scope exit action is put in place on the control stack with: pushaction Psub and when the action is popped off the stack because of a popmark, walking the stack back looking for exception
Re: Exceptions, sub cleanup, and scope exit
On Thu, 2004-11-18 at 13:36 -0500, Dan Sugalski wrote: I'd like pushing exception handlers to remain simple -- the current system is almost OK. What I'd like it to change to is: push_eh label with popping the top exception handler being: pop_eh I'm up for better names, too. The push_ is okay but eh is meh. push_handler seems better, though handler is terribly generic. If the documentation and comments use it consistently only for exceptions, though, it could work. -- c
Re: Exceptions, sub cleanup, and scope exit
On Thu, Nov 18, 2004 at 11:37:54AM -0800, chromatic wrote: On Thu, 2004-11-18 at 13:36 -0500, Dan Sugalski wrote: I'd like pushing exception handlers to remain simple -- the current system is almost OK. What I'd like it to change to is: push_eh label with popping the top exception handler being: pop_eh I'm up for better names, too. The push_ is okay but eh is meh. push_handler seems better, though handler is terribly generic. If the documentation and comments use it consistently only for exceptions, though, it could work. Throwcatch, so push_catcher ? I guess the HLL compiler needs to ensure that for every push the control flow will always pass through a matching pop. Otherwise that'll be another hard to debug situation. Couldn't (doesn't) parrot have a stronger concept of a 'scope' (as in perl5's scope.c) so that scope-exit cleanup can be automated and reliable? Tim [ignore me if I'm talking nonsense]
