Re: Events and JIT
Jeff Clites [EMAIL PROTECTED] wrote: But if the event dispatch thread is setting some flag for the target thread to detect, it's going to need to lock (or something similar) to make sure that the value of this flag is visible to other threads. Yep, that's true. The event thread puts an event into the interpreters task queue. This is done by locking the task queue, inserting the entry and unlocking it. This sequence ensures, that the queue entry with the event hanging off it, will be visible to the target interpreter. ... So that could mean a lock inside of every invoke No not AFAIK. Inserting the event into the task queue should already be enough, to make it visble for the target interpreter. So now we are probably quadratic with the size of the segment. (Patching N locations leads to N times where we un-patch all N locations.) I would say O(2*N) max. Yet another idea: The event thread could set some piece of memory (e.g. the JITed instruction stream) to PROT_NONE and catch the SIGSEGV. Some read/write barrier schemes for inkremental GC are using this approach. JEff leo
Macro preprocessor (was: cvs commit: parrot/imcc imcc.l)
Harry Jackson [EMAIL PROTECTED] wrote: What are the requirements on the pre-processor. $ perldoc imcc/docs/macros.pod $ $(EDITOR) imcc/imcc.l should provide all necessary information. leo
Re: [PATCH] Re: Register stacks organization
Luke Palmer [EMAIL PROTECTED] wrote: Leopold Toetsch writes: I'm working on yet another scheme at the moment, with a special op that pushes and clears simultaneously, to see if I can avoid copying altogether! It costs another indirection in the core registers, That would be a major change especially for prederefed and JIT code too. But we'll see what the benchmarks say. Yep. Fixed in included patch. Thanks, applied. - the COW flags is only reset on one user of 2 provocing even more unneeded ocpies. Again, I don't think this is hurting us. If there's another copy, it's in a continuation. We got that problem on *all* stacks, not only register frames used in Continuations. Luke leo
Re: Problem during make test
Chromatic [EMAIL PROTECTED] wrote: (gdb) bac This is the main thread, that has suspended itself during exit processing. #0 0x0ff976a4 in __pthread_sigsuspend () from /lib/libpthread.so.0 ... #3 0x0fd0a694 in exit () from /lib/libc.so.6 (gdb) thread 2 That's the thread-manager thread. It has sent the event thread a sig_cancel signal and is now waiting on the event thread to terminate: #1 0x0ff9c128 in waitpid () from /lib/libpthread.so.0 #2 0x0ff958d0 in pthread_handle_exit () from /lib/libpthread.so.0 #3 0x0ff94c90 in __pthread_manager () from /lib/libpthread.so.0 (gdb) thread 3 And finally the event handler thread, which seems to hang in waiting for the condition. WTF it doesn't get the cancel signal, sent by the thread-manager? #0 0x0ff976a4 in __pthread_sigsuspend () from /lib/libpthread.so.0 #1 0x0ff973e0 in __pthread_wait_for_restart_signal () from /lib/libpthread.so.0 #2 0x0ff93f9c in [EMAIL PROTECTED] () from /lib/libpthread.so.0 #3 0x101d7614 in queue_wait (queue=0x10279e30) at src/tsq.c:159 #4 0x1009c968 in event_thread (data=0x10279e30) at src/events.c:349 and the fact that the attached patch seems to fix things. I don't expect that it's correct. I don't know, why it seems to fix things. Parrot_new_terminate_event() places a stop the run-loop event into this interpreter's task queue. It doesn *not* effect the hanging event thread. But it seems to trigger something in an odd way, so that threads can make some progress and terminate finally. Really strange. I don't not even know, if your inserted line is executed, as it seems that exit() was called somewhere else. OTOH some lines later (interpreter.c:1143) the main thread tells the event thread to terminate by Parrot_kill_event_loop(), which pushes an event into the event queue. This also signals the waiting event thread, that something arrives, so it should wake up and finally terminate the event handler thread. I think, a better solution is to just call Parrot_exit() instead of exit(), so that Parrot_kill_event_loop() is run. leo
Re: Problem during make test
Harry Jackson [EMAIL PROTECTED] wrote: RHAS 2.1 dev edition Harry, do you still see these hanging parrot programs? chromatic, do you run a DeadRat (sorry) linux too? RedHat is well known to provide b0rken patches to otherwise running software. Could you try to up/down/side-grade *libpthread* (*not* glibc, at least if its separate). Harry Jackson leo
[DOCS] CVS version $Id strings
Some files have CVS version $Id strings, some don't. While tidying up the documentation I'm visiting every file. I can either: 1) add them when missing 2) remove them when present 3) do nothing I was inclined to (1) until I reflected that it did preserve a relation between local and repository versions. Say one has a number of different check outs of the distribution, then the $Id strings might come in handy to distinguish between them. So in the end I incline to (2). Does anyone have strong feelings either way? Mike
Re: Vtables organization
Luke Palmer wrote: Benjamin K. Stuhl writes: Other than the special case of :readonly, can you give me an example of when you'd need to, rather than simply writing a PMC class that inherits from some base? I'm having trouble thinking of an example of when you'd want to be able to do that... After all, since operator overloading and tying effectively _replace_ the builtin operations, what more does one need? Well, other than Constant, we need to be able to put locking on shared PMCs. We'd like to add a debug trace to a PMC. We could even make any kind of PMC sync itself with an external source on access, though that's a bit of a pathological case. Indeed, all of this can be done, however, by subclassing Ref. I think the reason this isn't sufficient is that we want to change the actual PMC into this new variant when it is possibly already in existence. Like my Csupplant was evilly trying to do. Perhaps there's a way to get that working safely... The issue is that the PMC's original vtable assumes (and should, IMHO be _able_ to assume) that it has total control over the PMC's data, so there is nowhere in the PMC to put a lock or a handle to an external source or anything. So you'd either need a Ref of some sort anyway or a global lookup table, which seems to be an even worse idea. Debug tracing, though, is a good question... I hate to pass an extra pointer to every vtable call just for that, though... -- BKS
[perl #25129] IO Buffer Test
# New Ticket Created by Stefan Lidman
# Please include the string: [perl #25129]
# in the subject line of all future correspondence about this issue.
# URL: http://rt.perl.org/rt3/Ticket/Display.html?id=25129
Hello
This is a test for the buffer bug that
was discussed on p6i a few days ago.
It does not remove the test file it makes
because I do not know a good way to do this.
/Stefan
output_is('CODE', OUTPUT, Buffer test);
set S0, buffTest
open P1, S0
set I0, 0
set I2, 1
LOOP:
gt I0, I2, DONE
set S1, I0
concat S1, S1,
print P1, S1
inc I0
mod I6, I0, 20
unless I6, NEW_L
branch LOOP
NEW_L:
print P1, \n
branch LOOP
DONE:
print P1, \n
close P1
PART_2:
open P1, S0
set I0, 0
LINE:
readline S1, P1
unless S1, SUCCESS
chopn S1, 1
NEXT_NR:
length I1, S1
le I1, 1, LINE
set S2,
SPLIT:
substr S3, S1, 0, 1
substr S1, 0, 1,
eq S3, , GOT_NR
concat S2, S2, S3
branch SPLIT
GOT_NR:
set I1, S2
ne I0, I1, FAILED
inc I0
branch NEXT_NR
FAILED:
print Failed\n
branch EXIT
SUCCESS:
print Successful\n
EXIT:
end
CODE
Successful
OUTPUT
Re: Docs and releases
On Tue, 13 Jan 2004, Paul Cochrane wrote: This also gives me an opportunity to mention to anyone with more time (and possibly ability) than me, that parrot is having problems on LinuxPPC. The specifics are: - parrot hangs on t/op/arithmetics when doing make test - make gives an implicit declaration of posix_memalign, but doesn't seem to have problems beyond this - a lot of warnings about implicitly truncating to unsigned type - a lot of warnings about discarding qualifiers (I don't know how much people worry about these kinds of things, but I found with the software I'm working on atm, that implicity truncation to unsigned was a problem found only on LinuxPPC, but not on LinuxX86, something to do with differences between the gcc's) Wordsize errors are one common type of error that show up on PPC (and SPARC) more readily than on x86, due to byte-order issues. When reporting problems, it's often a good idea to include the ./myconfig file in the parrot build directory -- it includes information about the variable types and sizes used in building parrot. These sizes may depend on your gcc version, how it was configured, and how it was called. -- Andy Dougherty [EMAIL PROTECTED]
Re: [perl #25129] IO Buffer Test
Stefan Lidman [EMAIL PROTECTED] wrote: readline S1, P1 The implementation of readline looks rather bogus. It allocates a huge (64K) string, sets that to zero, and the calls read, which calls readline. This is really suboptimal. It should probably call PIO_buf_readline or such directly, which could return a newly created string of the needed size. WRT the PANIC: interpreter-memory_allocated = 412528944, I don't know yet, why. Printing statistics after the readline makes the PANIC go away but still: Successful *** Parrot VM: Dumping GC info *** Total memory allocated = 384226672 DOD runs = 547 Collect runs = 11 leo
Various IMC Questions
Trying to get the tcl interpreter working with all the changes in the past few months: I used to be able to say: .pcc_sub _dumper prototyped .param PMC original ... but now PMC isn't a valid type anymore. Is there another way to get an IMC variable name tied to a PMC of indeterminate type? (or am I stuck with the .param $P1 syntax?) Also, .pcc_sub _main prototyped .param var argv used to work (causing argv to get populated properly with the command line arguments, when _main was the first sub defined). var isn't a valid type anymore either. I don't see anything in ./imcc/docs that covers command line arguments. Is the right solution to use one of the array types in the declaration here? (and, if so, which one?) Regards. -- Will Coke Coledawill at coleda dot com
Re: Various IMC Questions
At 11:56 AM -0500 1/19/04, Will Coleda wrote: Trying to get the tcl interpreter working with all the changes in the past few months: I used to be able to say: .pcc_sub _dumper prototyped .param PMC original ... but now PMC isn't a valid type anymore. Is there another way to get an IMC variable name tied to a PMC of indeterminate type? (or am I stuck with the .param $P1 syntax?) Use pmc. Lowercase. That works. Also, .pcc_sub _main prototyped .param var argv used to work (causing argv to get populated properly with the command line arguments, when _main was the first sub defined). var isn't a valid type anymore either. I don't see anything in ./imcc/docs that covers command line arguments. Is the right solution to use one of the array types in the declaration here? (and, if so, which one?) Once again, pmc is what you want. -- Dan --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: Various IMC Questions
Le Mon, Jan 19, 2004 at 11:56:28AM -0500, le valeureux mongueur Will Coleda a dit: Trying to get the tcl interpreter working with all the changes in the past few months: I used to be able to say: .pcc_sub _dumper prototyped .param PMC original ... but now PMC isn't a valid type anymore. Is there another way to get an IMC variable name tied to a PMC of indeterminate type? (or am I stuck with the .param $P1 syntax?) Also, .pcc_sub _main prototyped .param var argv used to work (causing argv to get populated properly with the command line arguments, when _main was the first sub defined). var isn't a valid type anymore either. I don't see anything in ./imcc/docs that covers command line arguments. Is the right solution to use one of the array types in the declaration here? (and, if so, which one?) In both case you should use Cpmc instead of CPMC, respectively Cvar. -- stef Regards. -- Will Coke Coledawill at coleda dot com
Re: Various IMC Questions
At 11:56 AM 1/19/2004 -0500, Will Coleda wrote: Trying to get the tcl interpreter working with all the changes in the past few months: I used to be able to say: .pcc_sub _dumper prototyped .param PMC original ... but now PMC isn't a valid type anymore. Is there another way to get an IMC To clarify the answers the others sent. PMC never was valid, but imcc just assumed if the variable type was not one of int|float|num|string|pmc or a valid PMC classname, then it must be a pmc. I sent out a notification that I was adding a stricter type check and so variations such as PMC, var, etc. would no longer work. Sorry for the inconvenience. -Melvin
Re: [perl #25129] IO Buffer Test
Stefan Lidman [EMAIL PROTECTED] wrote: [ VM Panic ] SPLIT: substr S3, S1, 0, 1 substr S1, 0, 1, The mem_allocate happens to be triggered by above string_replace, which does unmake_COW. During that GC is blocked, so we are allocating increasing blocks of memory until the PANIC. leo
Re: [perl #25129] IO Buffer Test
Stefan Lidman [EMAIL PROTECTED] wrote: [ and yet another f'up ] I've now comitted a fix for unmakeCOW not to turn off GC. So the test runs fine now with limited memory. *But*: Total memory allocated = 532480 DOD runs = 9680 Collect runs = 9660 Collect memory = 1388032592 ^^ Its horribly slow. /Me thinks that's the readline b0rkeness, I summarized ealier. Thanks Stefan again for this great test leo
Calling conventions, IMC
(Thanks for the quick response on the last email)
I have another problem. I have a 405-line file (containing a single
.pcc_sub) that is being included from my main .imc.
I get:
error:imcc:pcc_return not inside pcc subroutine
in file 'lib/interpret.imc' line 397
included from 'tcl.imc' sub '__interpret' line 116
...
Though the file contains:
1 .pcc_sub __interpret prototyped
2# An array of commands to interpret.
3.param PerlArray orig_commands
...
392 DONE:
393.debug(final retval is ')
394.debug(retval)
395.debug('\n)
396.pcc_begin_return
397.return retval
398.pcc_end_return
(I know the macro support is going away soon, I'm just trying to get
things working with the current snapshot.)
So, I /am/ inside a pcc subroutine when I .return.
I'm trying to slim this down into a smaller test, but thought I'd post
this in the meantime.
Regards.
--
Will Coke Coledawill at coleda
dot com
Re: Calling conventions, IMC
At 12:55 PM -0500 1/19/04, Will Coleda wrote:
Though the file contains:
1 .pcc_sub __interpret prototyped
2# An array of commands to interpret.
3.param PerlArray orig_commands
...
392 DONE:
393.debug(final retval is ')
394.debug(retval)
395.debug('\n)
396.pcc_begin_return
397.return retval
398.pcc_end_return
Is there a .end at the end of this?
--
Dan
--it's like this---
Dan Sugalski even samurai
[EMAIL PROTECTED] have teddy bears and even
teddy bears get drunk
Re: Calling conventions, IMC
(has a .end) Yup.
Managed to shrink it down to:
.pcc_sub __interpret prototyped
newsub P1, .Sub, __interpret
.pcc_begin_return
.return 0
.pcc_end_return
.end
It appears that creating a Sub that refers to the sub you're currently
in is the trigger. If I change the sub name to foo, it complains
about not finding foo. Comment it out, it compiles.
Sorry I didn't narrow down to a test case first -- after months of not
touching this, I wasn't sure how long it would take me. =-)
On Monday, January 19, 2004, at 01:13 PM, Dan Sugalski wrote:
At 12:55 PM -0500 1/19/04, Will Coleda wrote:
Though the file contains:
1 .pcc_sub __interpret prototyped
2# An array of commands to interpret.
3.param PerlArray orig_commands
...
392 DONE:
393.debug(final retval is ')
394.debug(retval)
395.debug('\n)
396.pcc_begin_return
397.return retval
398.pcc_end_return
Is there a .end at the end of this?
--
Dan
--it's like
this---
Dan Sugalski even samurai
[EMAIL PROTECTED] have teddy bears and even
teddy bears get drunk
--
Will Coke Coledawill at coleda
dot com
Re: Problem during make test
On Mon, 2004-01-19 at 04:11, Leopold Toetsch wrote: Harry, do you still see these hanging parrot programs? chromatic, do you run a DeadRat (sorry) linux too? Nope, none here. I can try a different pthread library though. -- c
question about global labels in imcc
Hi. In IMCC, how can I get the address of a label in a different compilation unit? Is there a way to do this at compile time? I've looked through the latest docs but according to imcc/docs/parsing.pod, global labels get stored in global variables that you must look up at run time (yuck). It seems that newsub does some magic regarding this, as .sub _main # This doesn't work set $I1, _baz print _baz: print $I1 print \n _shindig: # But this does newsub $P1, .Sub, _baz get_addr $I1, $P1 print _baz (fancy): print $I1 print \n exit 0 .end .sub _main2 _baz: exit 0 .end outputs _baz: -1 _baz (fancy): 136929824 Is there any way I can get that magic without having to actually create a new .Sub object? thanks, - TOGoS __ Do you Yahoo!? Yahoo! Hotjobs: Enter the Signing Bonus Sweepstakes http://hotjobs.sweepstakes.yahoo.com/signingbonus
Start of thread proposal
I've not gotten into the technical bits yet. That's next, but rip this apart first. =head1 DEFINITIONS So we can all talk about things the same way, the following definitons apply. Some of these are drawn from the POSIX thread spec, and as such we should have a translation section at the end. =over 4 =item THREAD An OS level thread. If that makes no sense, neither will any of the rest of this, in which case I recommend picking up Programming with POSIX Threads by Dave Butenhof, and coming back when you have. =item MUTEX This is a low level, under the hood, not exposed to users, thing that can be locked. They're non-recursive, non-read/write, exclusive things. When a thread gets a mutex, any other attempt to get that mutex will block until the owning thread releases the mutex. The platform-native lock construct will be used for this. =item LOCK This is an exposed-to-HLL-code thing that can be locked. Only PMCs can be locked, and the lock may or may not be recursive or read/write. =item CONDITION VARIABLE The sleep until something pings me construct. Useful for queue construction. Not conceptually associated with a MUTEX. (POSIX threads require this, so we're going to hide it there behind macros and/or functions) =item RENDEZVOUS POINT A HLL version of a condition variable. =item INTERPRETER Those bits of the Parrot_Interp structure that are absolutely required to be thread-specific. This includes the current register sets and stack pointers, as well as security context information. Basically if a continuation captures it, it's the interpreter. =item INTERPRETER ENVIRONMENT Those bits of the Parrot_Interp structure that aren't required to be thread-specific (though I'm not sure there are any) IPLUS anything pointed to that doesn't have to be thread-specific. The environment includes the global namespaces, pads, stack chunks, memory allocation regions, arenas, and whatnots. Just because the pointer to the current pad is thread-specific doesn't mean the pad Iitself has to be. It can be shared. =item INDEPENDENT THREAD A thread that has no contact IAT ALL with the internal data of any other thread in the current process. Independent threads need no synchronization for anything other than what few global things we have. And the fewer the better, though alas we can't have none at all. Note that independent threads may still communicate back and forth by passing either atomic things (ints, floats, and pointers) or static buffers that can become the property of the destination thread. =item SHARED THREAD A thread that's part of a group of threads sharing a common interpreter environment. =back =head1 REQUIREMENTS =head2 Supported Models =over 4 =item POSIX threads The threading scheme must be sufficient to support a POSIX share-everything style of threading, such as is used in perl 5's pthread model, as well as the thread models for Ruby and Python. =item Process-type' threads The scheme must also support the perl 5 iThreads threading model. In this model no data is shared implicitly, and all sharing must be done on purpose and explicitly. It very much resembles the Unix fork-process-with-shared-memory-segment model, not a surprise as it was originally developed with =back =head2 Guarantees =over 4 =item No Crashes The interpreter guarantees that no user program errors of any sort will crash the interpreter. This includes threading problems. As such, synchronization issues (where multiple interpreters are accessing the same shared data) must not crash the interpreter or corrupt its internal state. =back =head2 Assumptions =over 4 =item System memory allocation routines are threadsafe We are assuming that the memory allocation system of the base OS is threadsafe. While some of the C runtime libraries are notoriously thread dangerous, memory allocation code generally is threadsafe, and we'll assume that on all platforms. (Though we will, in general, manage our own memory) =back =head1 Proposal The proposal is as follows: =over 4 =item All global state shall be protected by mutexes Straightforward enough. This allows for independent threads to coexist without threatening the state of the proces. =item Multiple independent interpreters will be allowed Once again, straightforward enough. With threadsafe protected global state, there's no issue here. =item Only one OS thread in an interpreter at once While there is no requirement that any interpreter be tied to an underlying OS thread, under no circumstances may multiple OS threads use a single interpreter simultaneously. =item A Stop-and-copy communication method will be provided Parrot will provide a function to make a call into another interpreter and wait for that call to complete. This call may pass in data and have data returned to it. The interpreter making the call will block until the call is complete. The data passed in as parameters will be copied into the called interpreter, and any return values will be copied back into
Re: question about global labels in imcc
At 12:16 PM -0800 1/19/04, TOGoS wrote: Hi. In IMCC, how can I get the address of a label in a different compilation unit? Don't. The only way to transfer control to code in a different compilation unit (arguably in a different sub in the same compilation unit) is via invoke or one of its relatives. -- Dan --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: Start of thread proposal
At 3:16 PM -0500 1/19/04, Dan Sugalski wrote: I've not gotten into the technical bits yet. That's next, but rip this apart first. First rip -- I left a section unfinished. It should be: =item Process-type threads The scheme must also support the perl 5 iThreads threading model. In this model no data is shared implicitly, and all sharing must be done on purpose and explicitly. It very much resembles the Unix fork-process-with-shared-memory-segment model, not a surprise as it was originally developed with emulation of Unix's fork system in mind. -- Dan --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Number formatting
Okay, plan is this: 1) We will adopt ICU's formatting template for numeric formats 2) We'll roll our own to start with. #1 makes sense--we have to use something, and since we're linking in ICU but not postgres it makes sense to use the ICU format. I've some qualms about it since the numeric formatting seems a bit sub-par (no star or user-specified fill characters, no CR/DB or parens for alternate positive/negative notation) but it'll do, I think. At least it lets us skip nasty locale things. #2 is in part because we're not linking in ICU at the moment (it doesn't even build on latest-sarge debian installs) and in part because I don't want to yank in all of ICU just to be able to turn 45523.4 into $45,523.40. The op will be: format Sx, [INP]y, [SP]z The initial cut will likely not be at all locale-aware, but the docs will promise that it is. -- Dan --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Base string transforms
Time to add these in. I'm going to be adding in the following to the chartype vtable: upcase downcase titlecase to_chartype To transform things to upper-case, lowercase, titlecase (initial capital, basically), and to transform the string to an arbitrary different chartype. I'd like to get some alternative chartypes and encodings into parrot as soon as we possibly can, to make sure that we can actually handle things without resorting to unicode in all cases. (Which probably means I need to grab O'reilly's CJKV Information Processing off my shelf and start in on it--if anyone's got a better reference (or one that covers different languages) I'd be happy enough to have it...) -- Dan --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: Calling conventions, IMC
Will Coleda [EMAIL PROTECTED] wrote: 1 .pcc_sub __interpret prototyped 2# An array of commands to interpret. 3.param PerlArray orig_commands Please note that between .pcc_sub and .param nothing is allowed, not even a comment. leo
Re: Calling conventions, IMC
Will Coleda [EMAIL PROTECTED] wrote: (has a .end) Yup. Managed to shrink it down to: .pcc_sub __interpret prototyped newsub P1, .Sub, __interpret .pcc_begin_return .return 0 .pcc_end_return .end Two problems: You are overriding the already existing Sub with that name and 2) you destroy your return continuation, which could be already in P1. leo
Re: Start of thread proposal
DS == Dan Sugalski [EMAIL PROTECTED] writes: DS =item All shared PMCs must have a threadsafe vtable DS The first thing that any vtable function of a shared PMC must do is to DS aquire the mutex of the PMCs in its parameter list, in ascending DS address order. When the mutexes are released they are not required to DS be released in any order. why the ascending address order to grab mutexes? is this to help solve deadlocks? uri -- Uri Guttman -- [EMAIL PROTECTED] http://www.stemsystems.com --Perl Consulting, Stem Development, Systems Architecture, Design and Coding- Search or Offer Perl Jobs http://jobs.perl.org
Re: Start of thread proposal
Dan's thread proposal mentions: =item Automatic PMC sharing will be provided When a PMC is placed into a container which is shared (including lexical pads and global namespaces) then that PMC will automatically be marked as shared. It is acceptable for this to trigger an exception if for some reason a PMC should not be shared between interpreters. PMCs are, by default, not shared. This avoids sharing overhead for PMCs which are only used as temporaries and not shared. (Note that this is dangerous, and may end up not being done, due to the sharing of continuations) I don't know why this is dangerous. A continuation is a data structure just like an array or a hash. When you share it, everything inside it gets shared, too. For a continuation, this could be an awful lot of stuff, but it's the safe way. Luke
Re: Start of thread proposal
At 4:37 PM -0500 1/19/04, Uri Guttman wrote: DS == Dan Sugalski [EMAIL PROTECTED] writes: DS =item All shared PMCs must have a threadsafe vtable DS The first thing that any vtable function of a shared PMC must do is to DS aquire the mutex of the PMCs in its parameter list, in ascending DS address order. When the mutexes are released they are not required to DS be released in any order. why the ascending address order to grab mutexes? is this to help solve deadlocks? Yes. The recommendation I've always seen for deadlock avoidance is to always have all your code grab its mutexes in some fixed order. With source, it's generally recommended that you grab mutex variables in lexical order. Since we're all binary we need a different order, and ascending addresses are reasonably simple to do. -- Dan --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: Start of thread proposal
At 2:44 PM -0700 1/19/04, Luke Palmer wrote: Dan's thread proposal mentions: =item Automatic PMC sharing will be provided When a PMC is placed into a container which is shared (including lexical pads and global namespaces) then that PMC will automatically be marked as shared. It is acceptable for this to trigger an exception if for some reason a PMC should not be shared between interpreters. PMCs are, by default, not shared. This avoids sharing overhead for PMCs which are only used as temporaries and not shared. (Note that this is dangerous, and may end up not being done, due to the sharing of continuations) I don't know why this is dangerous. A continuation is a data structure just like an array or a hash. When you share it, everything inside it gets shared, too. For a continuation, this could be an awful lot of stuff, but it's the safe way. True, but the danger part is if we don't mark everything grabbed by a continuation as shared by default. If we do, we might as well mark everything as shared, as there'll be less overhead. -- Dan --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: Calling conventions, IMC
I didn't expect the code to be very usable, merely compilable. =-) If I want to call myself from myself, how do I do that then? For anything else, I do: .local Sub foo_sub newsub foo_sub, .Sub, __foo .pcc_begin prototyped .arg $S1 .pcc_call foo_sub tellmeagainwhyineedthislabel: .result $S1 .pcc_end If I can't explicitly create foo_sub if I'm in the middle of __foo, how do I recurse using calling conventions? And, in my defense, all this code worked a few months ago. =-) On Monday, January 19, 2004, at 01:54 PM, Leopold Toetsch wrote: Will Coleda [EMAIL PROTECTED] wrote: (has a .end) Yup. Managed to shrink it down to: .pcc_sub __interpret prototyped newsub P1, .Sub, __interpret .pcc_begin_return .return 0 .pcc_end_return .end Two problems: You are overriding the already existing Sub with that name and 2) you destroy your return continuation, which could be already in P1. leo -- Will Coke Coledawill at coleda dot com
[perl #25144] Unable to create self-referencing .Sub
# New Ticket Created by Will Coleda # Please include the string: [perl #25144] # in the subject line of all future correspondence about this issue. # URL: http://rt.perl.org/rt3/Ticket/Display.html?id=25144 The attached .imc file fails to compile with: borked.imc Description: Binary data error:imcc:pcc_return not inside pcc subroutine in file 'borked.imc' line 4 Commenting out the newsub allows it to compile. -- Will Coke Coledawill at coleda dot com
Re: Calling conventions, IMC
Will Coleda writes: I didn't expect the code to be very usable, merely compilable. =-) If I want to call myself from myself, how do I do that then? For anything else, I do: .local Sub foo_sub newsub foo_sub, .Sub, __foo .pcc_begin prototyped .arg $S1 .pcc_call foo_sub tellmeagainwhyineedthislabel: .result $S1 .pcc_end If I can't explicitly create foo_sub if I'm in the middle of __foo, how do I recurse using calling conventions You could use P0, which holds the subroutine object. Like using _ in Perl 6. Luke
RE: Start of thread proposal
Dan Sugalski wrote: [A] copying collector is generally untenable in a threaded environment. Can you elaborate upon the reasoning behind this statement? The first thing that any vtable function of a shared PMC must do is to aquire the mutex of the PMCs in its parameter list, in ascending address order. When the mutexes are released they are not required to be released in any order. Note that this locking strategy cannot avoid deadlock if the user is allowed to acquire these locks--HLL locks must be altogether different beasts from automatic PMC locks. That's okay. Just a design consequence worth noting for everyone. -- Gordon Henriksen IT Manager ICLUBcentral Inc. [EMAIL PROTECTED]
RE: Start of thread proposal
At 5:32 PM -0500 1/19/04, Gordon Henriksen wrote: Dan Sugalski wrote: [A] copying collector is generally untenable in a threaded environment. Can you elaborate upon the reasoning behind this statement? Sure. For a copying collector to work, all the mutators must be blocked, and arguably all readers should be blocked as well. (I think they need to be, otherwise you may be accessing reclaimed and reused memory) That means if we keep a copying collector we need to have everything that accesses strings or pmcs to get a lock on the GC before every access of every string or PMC. A touch excessive, I think. :) The first thing that any vtable function of a shared PMC must do is to aquire the mutex of the PMCs in its parameter list, in ascending address order. When the mutexes are released they are not required to be released in any order. Note that this locking strategy cannot avoid deadlock if the user is allowed to acquire these locks--HLL locks must be altogether different beasts from automatic PMC locks. That's okay. Just a design consequence worth noting for everyone. Oh, arguably it can't avoid deadlock at all, what with vtable methods having access to the full environment. I can live with deadlocks, only because there's no real alternative. -- Dan --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
RE: Start of thread proposal
Dan Sugalski wrote: For a copying collector to work, all the mutators must be blocked, and arguably all readers should be blocked as well. True of non-moving collectors, too. Or, let me put it this way: non- copying *GC* (the sweep or copy phase) can be threadsafe, but the mark phase is never threadsafe. The method in which marking is not threadsafe is a bit more pathological (i.e., it's not the common case as it is with the copying collector), but a standard tracing DOD cannot be correct when competeting with mutators. It WILL collect non- garbage (those are MUTATORS, remember), and the result WILL be Heizenbugs and crashes. Some of what I've written up addresses why. It's pretty simple to demonstrate a single case to prove the point, but I don't feel like re-creating the ASCII art right now. :) I'll send that section when I get out of the office. parrot will have to be able to suspend all threads in the environment. Unfortunate, but really quite unavoidable. Oh, arguably it can't avoid deadlock at all, what with vtable methods having access to the full environment. I can live with deadlocks, only because there's no real alternative. But PMC implementations have to fall inside of the trusted environment, so that's not really a failure. Of course uncooperative code can break a cooperative algorithm. :) -- Gordon Henriksen IT Manager ICLUBcentral Inc. [EMAIL PROTECTED]
Re: Start of thread proposal
On Tue, 20 Jan 2004 10:45, Dan Sugalski wrote;
Yes. The recommendation I've always seen for deadlock avoidance is
to always have all your code grab its mutexes in some fixed order.
Yes; otherwise, you need to back off and start again, if one lock
acquisition fails.
Consider these functions; for the purpose of this example, lexical
lock ordering is implied;
func1($, $, $, $, $);
func2($, $, $, $, $);
func3($, $);
So long as the locks are ramped up from the lowest to the highest,
there is no chance of func1 acquiring a lock to be held by func2 (eg,
$), if that other function already has one of the shared
dependancies (eg, $).
All well and good. But, what about recursive locks?
ie
sub func1($one is locked, $two is locked, $three is locked) {
for my $x ($one, $two, $three) {
func2($x.property) if $x.property;
}
}
sub func2($eins is locked) {
if ($eins.property) {
func2($eins.property, { });
}
}
$aaa.property = { };
$bbb.property.property = $aaa;
$ccc = { };
if (thork()) { # fork a thread
# thread A
func1($aaa, $bbb, $ccc);
} else {
# thread B
func2($bbb.property);
}
OK, the execution order that causes the deadlock is:
1. Thread B - func2() acquires a lock on the $bbb.property PMC.
2. Thread A - func() acquires a lock on $aaa, $bbb, $ccc.
3. Thread A - func2() acquires a lock on $aaa.property, which
returns quickly
4. Thread A - func2() blocks waiting for a lock on $bbb.property,
held by func2() in thread B
5. Thread B - func2() blocks waiting for a lock on
$bbb.property.property, held by func() in thread A.
So, there are still possibilities for deadlocks, as the non-linear
nature of subroutine calls screws up your nice lock ramping.
In summary, acquiring mutexes in a defined order as a means to avoid
deadlocks only works when you are acquiring them all up front. If you
are acquiring any later, and you detect a deadlock (ie, a loop of
threads blocking on locks held by each other), you *must* be able to
tell one of them to ramp off to holding no locks at all. ie,
ROLLBACK :).
The bugger is, winding back execution state automatically to when you
last started acquiring locks is probably an intractable problem from
an interpreter's perspective...
Sounds like a job for an exception to me ;-).
for (1..50) {
eval {
func_that_acquires_first_lock($args);
};
last unless $@ and $@ =~ m/mutex deadlock/i;
}
--
Sam Vilain, [EMAIL PROTECTED]
When I sell liquor, its called bootlegging; when my patrons serve it
on Lake Shore Drive, its called hospitality.
AL CAPONE
PDB [Was: Re: Calling conventions, IMC]
Ok. I've converted my code over to do this[1]. I am now able to compile tcl.imc, but all my code fails to run, ala: bash-2.05a$ make exit ../../../parrot ../tcl.pbc exit.tcl branch_cs: illegal resume offsetmake: *** [exit] Error 1 So, I'm trying to track down where this badness is happening: bash-2.05a$ ../../pdb tcl.pbc Parrot Debugger 0.0.2 (pdb) run examples/exit.tcl Restarting And it hangs. if I hit ^D, it closes. This smells like it's ignoring the command line argument and running the pbc with no args, which forces it to read tcl from STDIN. (...dig... yup, that's what it's doing.) So, how can I convince pdb to pass along the args to my bytecode? Also, if I do: bash-2.05a$ ../../pdb tcl.pbc Parrot Debugger 0.0.2 (pdb) load tcl.imc (pdb) list 1 100 I get 32 1/2 lines of output before getting a segfault. (I've attached the crash log) list by itself works fine. There doesn't seem to be a specific X for list 1 X where it consistently segfaults, but anything 12 seems to do a pretty good job. Regards. crash.log Description: Binary data [1] On Monday, January 19, 2004, at 05:16 PM, Luke Palmer wrote: Will Coleda writes: If I can't explicitly create foo_sub if I'm in the middle of __foo, how do I recurse using calling conventions You could use P0, which holds the subroutine object. Like using _ in Perl 6. Luke -- Will Coke Coledawill at coleda dot com
Re: Calling conventions, IMC
At 03:16 PM 1/19/2004 -0700, Luke Palmer wrote: Will Coleda writes: I didn't expect the code to be very usable, merely compilable. =-) If I want to call myself from myself, how do I do that then? For anything else, I do: .local Sub foo_sub newsub foo_sub, .Sub, __foo .pcc_begin prototyped .arg $S1 .pcc_call foo_sub tellmeagainwhyineedthislabel: .result $S1 .pcc_end If I can't explicitly create foo_sub if I'm in the middle of __foo, how do I recurse using calling conventions You could use P0, which holds the subroutine object. Like using _ in Perl 6. I'll commit a fix shortly. -Melvin
Memory corruption
Here's another obnoxious test case. I started to try to strip it down, but it starts working again if I even delete nonsense lines from a subroutine that is never called. And I'm working on something else and not at all in the mood to re-learn how to debug parrot internals. It turns out that I don't get the crash when running JITted, so I think I'll just do that for now. So, in case anyone is curious (hi leo!), attached is a 56KB (9KB gzipped) imc file. It crashes on a memcpy inside compact_pool (triggered by new_hash). b-buflen is obviously corrupted. Using -G to disable garbage collection (does that work?) doesn't seem to help matters at all. Deleting the __setup sub at the end of the file makes the problem go away. (Note that __setup is never actually called, and the body of the routine is irrelevant other than its length.) dead.imc.gz Description: GNU Zip compressed data
Re: cvs commit: parrot/imcc/t/syn pcc.t
On Jan-15, Melvin Smith wrote:
At 11:20 AM 1/15/2004 +0100, Leopold Toetsch wrote:
Melvin Smith [EMAIL PROTECTED] wrote:
For some reason 1 test in pcc.t is failing (the nci call)
Off by one error caused by:
-for (j = 0; j 4; j++) {
+for (set = 0; set REGSET_MAX; set++) {
As most loops inside Parrot use the former scheme, I'd rather keep it
then switching to an (it seems) error prone variant set = REGSET_MAX
I like my version because it is self-documenting.
I think these are #defines, but for enums I always use the pattern:
enum {
CLR_BLUE,
CLR_RED,
CLR_VOMIT_GREEN,
NUM_COLORS/* or CLR_COUNT, or CLR_ENTRIES, or ... */
};
for (i = 0; i NUM_COLORS; i++) ...
So how about a REGSET_SIZE?
Re: Start of thread proposal
At 12:58 AM + 1/20/04, [EMAIL PROTECTED] wrote: =item MUTEX This is a low level, under the hood, not exposed to users, thing that can be locked. They're non-recursive, non-read/write, exclusive things. When a thread gets a mutex, any other attempt to get that mutex will block until the owning thread releases the mutex. The platform-native lock construct will be used for this. Will this be macroised to hide the platform native implementation from the main body of the code? Yes. The sleep until something pings me construct. Useful for queue construction. Not conceptually associated with a MUTEX. (POSIX threads require this, so we're going to hide it there behind macros and/or functions) Could you elaborate on the nature of what would constitute a ping? POSIX has a function cond_signal (and cond_broadcast, which is similar). What happens is a thread grabs the condition variable and goes to sleep, and sleeps until another thread does a cond_signal, which then wakes it up. If there are multiple threads sleeping on the condition variable, only one is woken. (cond_broadcast wakes them all up) =item POSIX threads The threading scheme must be sufficient to support a POSIX share-everything style of threading, such as is used in perl 5's pthread model, as well as the thread models for Ruby and Python. Thankyou:) No problem. It's the model I prefer. :) =item Process-type' threads The scheme must also support the perl 5 iThreads threading model. In this model no data is shared implicitly, and all sharing must be done on purpose and explicitly. It very much resembles the Unix fork-process-with-shared-memory-segment model, not a surprise as it was originally developed with Pseudo forks? Yeah. On Win32, when Perl forks it starts a new thread and clones the interpreter and its contents. If you later do an exec in that thread it starts a new process and grabs hold of it. Definitely a clever trick. An interesting extract from the Windows Services for Unix docs. The Interix subsystem shows improvements in all aspects of performanceranging from 30 percent improvements in fork and exec performance... It's sales pitch, but I'm trying to verify it (and build parrot using it). If it works, cool. We can add environment probing at configure time. I worry some about the stability and evil involved there (I've a good idea how much work it is to weld fork into a process with a fundamentally different process model) but that's Microsoft's problem, not ours. =item System memory allocation routines are threadsafe We are assuming that the memory allocation system of the base OS is threadsafe. While some of the C runtime libraries are notoriously thread dangerous, memory allocation code generally is threadsafe, and we'll assume that on all platforms. (Though we will, in general, manage our own memory) Is there any likelyhood that memory allocation will be hidden behind macros at two levels: - ALLOCPOOL() for allocating large chunks of memory (ppols) that are later sub-allocated (and managed) by: - SUBALLOC() for sub allocating within a pool We'll generally hide behind the memory.c routines, if for no other reason than to allow the embedder to override the memory functions. We need to define that at some point... =item Each interpreter will have a unique id This ID will be independent of the process or OS thread, and will be constant across the lifetime of the interpreter. Interpreter IDs Imay be reused as interpreters are destroyed and recreated, and as such are only guaranteed valid while an interpreter is in use. The provision of method(s) to obtain the native TIDs/HANDLES would make life for those writing implementation specific extensions easier. TIDs, definitely. It'll be a sysinfo or interpinfo function. There's the issue of interpreters binding to different threads, but we'll deal with that. I'm not sure what the HANDLE is, but explain it and we can work something out. :) -- Dan --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
IMC returning ints
I did a cvs update, and it looks like imcc doesn't properly return
integers anymore from nonprototyped routines. Or maybe it never did,
and the switchover from nonprototype being the default to prototyped
is what triggered it (because I had to add some explicit
non_prototyped declarations, although I suspect they are incorrect.)
Test patch is attached, test case is:
.pcc_sub _main
$P0 = newsub _L_closure2
$I0 = 17
.pcc_begin non_prototyped
.arg $I0
.pcc_call $P0
L_after_call7:
.result $I1
.pcc_end
after_call:
print returned
print $I1
print \n
end
.end
.pcc_sub _L_closure2 non_prototyped
.param int value
.pcc_begin_return
.return value
.pcc_end_return
.end
? imcc/tc
Index: imcc/t/syn/pcc.t
===
RCS file: /cvs/public/parrot/imcc/t/syn/pcc.t,v
retrieving revision 1.31
diff -p -u -b -r1.31 pcc.t
--- imcc/t/syn/pcc.t20 Jan 2004 01:50:47 - 1.31
+++ imcc/t/syn/pcc.t20 Jan 2004 01:58:28 -
@@ -1,6 +1,6 @@
#!perl
use strict;
-use TestCompiler tests = 34;
+use TestCompiler tests = 36;
##
# Parrot Calling Conventions
@@ -96,6 +96,60 @@ CODE
10
20
30
+OUT
+
+output_is('CODE', 'OUT', non-prototyped int return);
+.pcc_sub _main
+ $P0 = newsub _L_closure2
+$I0 = 17
+ .pcc_begin non_prototyped
+ .arg $I0
+ .pcc_call $P0
+L_after_call7:
+ .result $I1
+ .pcc_end
+after_call:
+print returned
+print $I1
+print \n
+end
+.end
+
+.pcc_sub _L_closure2 non_prototyped
+ .param int value
+.pcc_begin_return
+.return value
+.pcc_end_return
+.end
+CODE
+returned 17
+OUT
+
+output_is('CODE', 'OUT', prototyped int return);
+.pcc_sub _main
+ $P0 = newsub _L_closure2
+$I0 = 17
+ .pcc_begin prototyped
+ .arg $I0
+ .pcc_call $P0
+L_after_call7:
+ .result $I1
+ .pcc_end
+after_call:
+print returned
+print $I1
+print \n
+end
+.end
+
+.pcc_sub _L_closure2 prototyped
+ .param int value
+.pcc_begin_return
+.return value
+.pcc_end_return
+.end
+CODE
+returned 17
OUT
##
Re: Start of thread proposal
On Monday, January 19, 2004, at 06:37 , Gordon Henriksen wrote:
Dan Sugalski wrote:
For a copying collector to work, all the mutators must be blocked,
and arguably all readers should be blocked as well.
True of non-moving collectors, too. [...]
Some of what I've written up addresses why. [...] I'll send that
section when I get out of the office.
Consider this simple object graph:
Root set = { A, B }
[ A=NULL]
[ B=C ] --- [ C=]
Collection begins in thread 1 and marks A as reachable before being
preempted by thread 2:
[xA=NULL]
[ B=C ] --- [ C=]
Thread 2 sets A to C, and nullifies B before being preempted again by
thread 1:
[xA=C ] --- [ C=]
[ B=NULL]
Thread 1 marks B as reachable:
[xA=C ] --- [ C=]
[xB=NULL]
The mark phase complete, thread 1 sweeps:
[ A=???] --- ???
[ B=NULL]
C was not marked reachable (although it was) and was thus erroneously
collected, leaving a dangling pointer. This problem applies equally to
copying and mark-sweep collectors.
Gordon Henriksen
[EMAIL PROTECTED]
