Re: register allocation
At 11:22 AM 8/9/2004 -0400, Dan Sugalski wrote: At 1:13 PM +0200 8/8/04, Leopold Toetsch wrote: Leopold Toetsch <[EMAIL PROTECTED]> wrote: You can verify this step by running -v: $ parrot -v inv_mod.imc 2>&1 | grep symb build_reglist: 5783 symbols allocate_non_interfering, now: 2205 symbols It really should help. It did. I'm not sure how long the build ran (the last time I checked it'd racked up over 150 minutes of CPU time on a 2GHz dual-processor system, and may have run up to about 180 CPU minutes) but it finished. Now we need to cut down the runtimes just a touch. :) DSWEPIC Dan Stop Writing Evil Pathological Intermediate Code. -Melvin
Re: register allocation
At 10:54 AM 8/7/2004 -0400, Dan Sugalski wrote: At 12:45 PM +0200 8/7/04, Leopold Toetsch wrote: Sean O'Rourke <[EMAIL PROTECTED]> wrote: [EMAIL PROTECTED] (Leopold Toetsch) writes: The interference_graph size is n_symbols * n_symbols * sizeof(a_pointer). This might already be too much. 2) There is a note in the source code that the interference graph could be done without the N x N graph array. Any hints welcome (Angel Faus!). Since that is my comment I'll comment. When I wrote the allocator, I oriented it around a basic block, and I figured a basic block would never have more than a few hundred symbols, so the N x N array was the fastest possible data structure for keeping the interference data. I'm not sure how many symbols we are talking about here, it would help if someone threw out a number before I go and make the wrong statement. I'm pretty sure that it would help if imcc had two modes of symbol analysis. Routine wide register allocation ends up with better quality code, but at the cost of compilation time and possible cases that can't be handled with the current design. If imcc sees too many symbols, it should try to break the sub down into basic blocks for interference analysis, but that also adds additional code requirements for when you put the basic blocks together and figure out loads, spills and reuse across basic blocks. There were a lot of things that I had planned for imcc that just never materialized. It is still very primitive when it comes to symbol analysis, even with all the work Leo has done. -Melvin
Re: Starting to make things final
At 05:13 PM 8/3/2004 -0700, Gregor N. Purdy wrote: Did I miss the creation of the compiler-writer list? I need to figure No, we are still holding our breath (and turning blue, purple, green, ...) Btw, I'll poke the Cola rewrite I have here and see where it stands. It gathered a bit of dust in the past 6 months. -Melvin
Re: Optimizations for Objects
At 08:57 AM 3/19/2004 +0100, Leopold Toetsch wrote: Dan Sugalski <[EMAIL PROTECTED]> wrote: > At 10:38 PM +0100 3/18/04, Leopold Toetsch wrote: >> >>Which brings up again my warnocked question: How can return >>continuations get reused? > Works like this. (No pasm, but it should be obvious) I was aware of Leo's RetContinuation because our initial implementation using a full Continuation was deemed a little heavy. As long as the RetContinuation honors Copy On Write, there should be no problem unless the RetContinuation is stored and referenced "outside" the original scope (or escapes the closure of the sub call or full continuation). The problem could be that some code is not honoring COW and it is not copying things before using. -Melvin
Re: imcc concat and method syntax
At 02:42 PM 3/13/2004 -0800, Steve Fink wrote: > > currently, the pir line > > S5 = S5 . 'foo' > > produces > > error:imcc:object isn't a PMC > > > > concatenation with . seems to be gone > > i cannot think of a good replacement for it > This should be fixable with some lexer or parser tweaking. Well, Perl 6 uses ~ . I think that would be a fair replacement: > > S5 = S5 ~ 'foo' We don't have the same ambiguity problems in PIR that we do in Perl6 since PIR is very simple and uniform. foo.baz is a method or member access foo . baz is concatenation For an intermediate language for multiple front end compilers 1) Strict rules are just fine 2) Eventually compilers will simply use the AST API and skip the textual mode. I tend to use the 'concat' op in my own code anyway. So I'll abide by Leo's or Melvin's ruling. Explicit concat works for me too. Parsing PIR is not too complex, so I'm not worried about a lexer kludge here or there. I prefer to keep the . for concat. -Melvin
Re: Dates and Times
Hi Brent, Welcome back to p6i. ;) At 08:12 PM 3/9/2004 -0800, Brent \"Dax\" Royal-Gordon wrote: On a platform with a halfway decent JIT, a pure-PASM implementation could be as fast as an op-based one, given liberal use of the non-PMC Agree. Besides, how fast does your date handling really need to be? I mean, *really*? Are you formatting eleventy billion dates in a tight loop or something? I actually have Perl programs that parse many many millions of billing records per day (with 2 dates per record) for certain wireless companies. Sometimes, if the customer wants to do an audit, we have to process over a month's worth, so we are bound by the actual execution time of the Perl script and the access time of a Sleepycat (Berkeley DB) database. The Perl programs must be able to scale with call/message volume, and right now the only thing we can do to improve it is put faster processors on it. (We have 8-way boxes with 64GB RAM, so 4GB hashes work just fine, but 1.5 hrs per day is still very finite when you have to baby sit scripts for a week to give customers the answers they want to know). Granted, I could rewrite this stuff in C, but we typically modify these things on very short notice and Perl gives us the flexibility to react quickly. So, when we are discussing dates, I am one very interested party. -Melvin PS: Sorry I'm so vague about the numbers. The customers are very sensitive about those numbers and I could get in trouble, but lets say they are in the billions for a rather small time sample.
Re: Dates and Times
At 03:52 PM 3/9/2004 -0800, Edward S. Peschko wrote: On Tue, Mar 09, 2004 at 04:21:24PM -0500, Gordon Henriksen wrote: > "Not an opcode" doesn't mean "balkanized." There is a parrot/stdlib > directory. fair enough, but then where does the distinction lie? Why put gmtime, et al. in opcodes? As well as addmonth? If you are optimising for simplicity and , it surely makes sense to put these in the standard library. If you are optimising for speed, then it makes sense to put them in opcodes. I don't think optimising for "X" is the reason. Parrot should have concise, necessary, complete opcode primitives upon which anything else can be built. Date parsing can be done all "with" opcodes, but please not "inside" opcodes. If we cannot provide a decently performing VM that makes people want to write stuff in bytecode (or compiled to bytecode) we have failed anyway. It all comes down to how fast one is versus the other. If everything is going to be built in the standard library based off of bytecode then there will be a performance hit. How severe, I don't know, but definitely a performance hit. This is subjective reasoning, though. If we want to talk performance hits, we should know where we stand. When bytecode isn't full of PMC thrashing, and actually uses the low level Ix and Sx registers where possible, the JIT works extremely well and the performance beats Perl5 by several orders of magnitude in many cases. Hopefully the "hints" in Perl6 will help us write better libraries, but some of it may have to be written in IMC or slightly higher level language (on the equivalent of C, compilable to bytecode). > > But how many times are you going to need to parse formats > > like '3 weeks from next Wednesday?'. > > This sort of creeping featuritis is why date formatting and especially > parsing do NOT belong as opcodes. It's too big a problem to solve in the I agree. But I don't see the 'creeping featuritis' that you see. As well as the memory imprint problems that you see. Is parrot going to link with libc? Yes, Parrot will link with libc. Regardless of what we implement as opcodes, there will never be a single solution that fits all, and it will never, ever end. Just look at Linux. -Melvin
Re: Dates and Times
At 11:37 AM 3/3/2004 -0500, Dan Sugalski wrote: I'm torn here as to what to do. On the one hand, it's supremely tempting to punt and not have parrot do a darned thing with the time and leave it to library code to handle it. On the other, CPAN is littered with the carcasses of time and date modules. On the third hand, at least some of the date handling stuff is provided by the underlying C runtime library (there's a lot of interesting stuff in the C89 spec) so it seems silly to reimplement something we're guaranteed to have around anyway. What I'm thinking we may want to do is provide a minimal interface--turn the time integer into a string rep or split out into an array, something like: Simple primitives are good. My stuff dealing with dates is usually hand written based on top of the localtime and gmtime calls in the standard Time:: module bundled with Perl anyway. Anything fancier requiring a CPAN download rarely gets used on production boxes because a lot of administrators don't mind installing a pre-built Perl package, but they shy away from downloading and compiling a ton of add-on modules (hey, they can't all be Perl-trained). I know I've been on this soapbox in the past but... I hope, with Parrot, people will get away from a common practice of writing Perl modules in C with a binary interface. Unless the module HAS to be in C for API interface (complex math computations or performance sensitive stuff like encryption and compression where 10% matters), I prefer a pure bytecode implementation for ease of deployment. Some people write both pure Perl and C implementations of modules; to get around that, I hope we can provide a low level, optimizable "system" programming language for Parrot (or put enough hints in Perl6) so people can write fast implementations without requiring a real C compiler. -Melvin
Re: Inconsistent Parrot / IMCC behavior
At 08:01 PM 2/28/2004 -0800, Gregor N. Purdy wrote: I made the change, and now I get consistent results. I'll check that in. I am still not clear, though, on why we wouldn't have the same failure in all cases. I'd think these should be equivalent: * Running parrot on 'foo.imc' * Running parrot on 'foo.pasm' generated from 'foo.imc' * Running parrot on 'foo.pbc' generated from 'foo.pasm' You would think it would be straightforward. Its not, due to the way we (Leo and I) mixed up IMCC and how it handles things, especially the lexer. The problem is definitely a bug, and I think we've made an error in trying to mix too many features and lexing/parsing modes into the same compiler. Its become too aggravating to maintain, at least for me. Leo is welcome to maintain v1 all he wants ;) The rewrite of IMCC and will handle _one single flavor_ of language. All syntax rules will apply the same, regardless of if it sees PASM ops or PIR/IMC ops. There won't be a PIR -> PASM translation phase, either. (I will probably relax the idea of compilation unit to allow simple streams of bytecode, rather than require a wrapping sub). Not that it helps you now, but I understand your frustration. -Melvin
Re: [perl #27003] bytecode (header?) problem in tru64/alpha
At 03:56 PM 2/24/2004 -0500, Andrew Dougherty wrote: On Tue, 24 Feb 2004, Jarkko Hietaniemi wrote: > > PackFile_unpack: Not a Parrot PackFile! > Magic number was [0x4c524550] not [0x013155a1] > Parrot VM: Can't unpack packfile t/native_pbc/integer_1.pbc. > error:imcc:main: Packfile loading failed That looks rather similar to what I get on SPARC. Here are three variations I just got: Something has definitely gone wrong. My initial test cases with byteorder handling were done with x86 32-bit, Sparc 32-bit and Sparc 64-bit, and I was able to load bytecodes between the 3, however, packfile has been through some considerable changes since the byteordering support. -Melvin
Re: Objects and time
At 05:09 PM 2/23/2004 +0100, Leopold Toetsch wrote: WRT feature freeze: I'd say: Starting from Tue, 24th 8.00 GMT no more feature patches *should* go in, *except* objects. Basically that means: everyone will get really quiet and we will all watch Dan. >:) -Melvin
Re: Objects and time
At 10:29 AM 2/23/2004 -0500, Dan Sugalski wrote: So, the question--shall we do objects and maybe miss the Feb 29th release, or do the Feb 29th release and do objects for the next release? As I think I'm only a little while off (maybe a day or so) from getting it working, I'm tempted to take a miss on the nice date in favor of a release with Good Stuff in it, but I'm flexible there. Nice dates are amusing, but of no real use. I say postpone the release. -Melvin
Re: Parrot Tetris with SDL bindings
At 04:07 AM 2/20/2004 +0100, Jens Rieks wrote: Hi all, here is a first alpha version of my upcoming tetris example for parrot. It is a good demonstration that parrot is already very powerful. Very cool. Great work. Assembling the sources to a single tetris.pasm seems to not work, "parrot tetris.pasm" then quits with error:imcc:Label '@pcc_sub_call_86' already defined in file 'tetris.pasm' line 1330 Am I doing something wrong, or is it a bug? imcc incorrectly uses line numbers to generate labels. Try this and see if it fixes your problem (its a temporary hack, it will cause incorrect line number error reporting, but should give you unique labels throughout). -Melvin Index: imcc.l === RCS file: /cvs/public/parrot/imcc/imcc.l,v retrieving revision 1.84 diff -u -r1.84 imcc.l --- imcc.l 9 Jan 2004 08:53:07 - 1.84 +++ imcc.l 20 Feb 2004 04:43:20 - @@ -797,7 +797,7 @@ frames = frame; /* XXX: Switch the filename */ -line = 1; +/*line = 1;*/ yy_switch_to_buffer(yy_create_buffer(file, YY_BUF_SIZE)); }
Re: [PATCH] IO fixes for Win32
At 11:57 AM 2/19/2004 -0800, Goplat wrote: Failed Test Stat Wstat Total Fail Failed List of Failed imcc/t/syn/file.t1 256121 8.33% 11 t/pmc/env.t 3 768 63 50.00% 3 5-6 t/pmc/perlarray.t1 256261 3.85% 26 t/pmc/sub.t 3 768493 6.12% 36-38 t/pmc/tqueue.t 2 512 52 40.00% 2-3 2 tests and 69 subtests skipped. Failed 5/94 test scripts, 94.68% okay. 10/1320 subtests failed, 99.24% okay. Not exactly success, but it's just the same failures I got before applying it. At least the IO stuff succeeded this time (imcc/t/syn/file.t fails because of shell issues). Works for me, applied, thanks. -Melvin
Re: [PATCH] IO fixes for Win32
At 10:02 AM 2/19/2004 -0800, Goplat wrote: --- Melvin Smith <[EMAIL PROTECTED]> wrote: > >Where is the hassle? It's just a few lines of code to check windows > >version. It's easier to code that than to make another configure option. > > Then submit a patch. Okay. (attached) Very good, thank you sir. I see you also addressed another outstanding issue with the negative return val to read (into an unsigned). Did you run the test suite and get successes? If so, I will apply. -Melvin
Re: Objects: Now or forever (well, for a while) hold your peace
At 01:34 PM 2/19/2004 -0500, Dan Sugalski wrote: At 10:21 AM -0800 2/19/04, Steve Fink wrote: On Feb-19, Dan Sugalski wrote: At 7:30 PM -0500 2/18/04, Simon Glover wrote: > One really pedantic comment: wouldn't it make sense to rename the > fetchmethod op to fetchmeth, for consistency with callmeth, tailcallmeth > etc? Good point. I'll change that, then. D yo reall wan t repea C's infamou "creat" (mis)spellin? Admittedl, i i no ver ambiguou i thi cas, becaus ther ar alread s man letter, bu stil, tw extr letter i a smal pric t pa... Ok, Ok, point well taken. :) I'll go for the longer version all around. I'm tossing you both in a lake ASAP. -Melvin
Re: [PATCH] IO fixes for Win32
At 09:27 AM 2/19/2004 -0800, Goplat wrote: --- Dan Sugalski <[EMAIL PROTECTED]> wrote: > At 12:40 AM +0300 2/18/04, Vladimir Lipsky wrote: > >From: "Goplat" <[EMAIL PROTECTED]> > > > >> --- Vladimir Lipsky <[EMAIL PROTECTED]> wrote: > >> > From: "Goplat" <[EMAIL PROTECTED]> > >> > > >> > > flags_to_win32 sets fdwShareMode to FILE_SHARE_DELETE, which is not > >> > > supported in win98 > >> > > >> > A fix for that should be windows version specific and needs support > >> > of the config subsystem. > >> > >> If you did that, a version compiled on NT wouldn't work on 9x. I'd say > >> most 9x users don't compile perl themself, they just download a binary > >> from ActiveState (who use NT)... > > > >Then there should be different binaries for different versions > > Yeah, I'm actually coming to that conclusion. Yes, it's sort of > easier to keep a single binary around from a packaging standpoint, > it's a hassle for everyone else because of it. Where is the hassle? It's just a few lines of code to check windows version. It's easier to code that than to make another configure option. Then submit a patch. I have no problem with that scenario. The code in question is mine, but I don't have the ability to play with older versions of Windows. I develop on XP, 2000, Linux and Solaris, and I really don't claim to be much of a Windows programmer, outside of Winsock. -Melvin
Re: Build broken due to missing inet_aton on Solaris 8
At 05:16 PM 2/12/2004 +0100, Leopold Toetsch wrote: Melvin Smith <[EMAIL PROTECTED]> wrote: > I'm not sure why Leo changed it, but I'll put it back. I'm not sure either ;) Must have been in one of the cleanup or such patches, I had put in. Sorry, No problem. I haven't really taken a survey of which OS have which nowadays. inet_pton is ipv6 enabled but older OSes will surely not have it. -Melvin
Re: Build broken due to missing inet_aton on Solaris 8
At 10:26 AM 2/12/2004 -0500, Andrew Dougherty wrote: A fresh checkout of parrot won't build for me due to the missing inet_aton symbol on Solaris 8. My perl5 configuration correctly records $Config{d_inetaton}=undef, but io_unix.o unconditionally expects inet_aton. cc -o parrot -L/usr/local/lib -R/usr/local/lib imcc/main.o \ blib/lib/libparrot.a -lsocket -lnsl -ldl -lm -lpthread -lrt Undefined first referenced symbol in file inet_aton blib/lib/libparrot.a(io_unix.o) ld: fatal: Symbol referencing errors. No output written to parr I know I only develop on Linux and Solaris. Linux can use either but Solaris needed inet_pton. I'm not sure why Leo changed it, but I'll put it back. Leo do you have an OS that does not have inet_pton? -Melvin
Re: RT Cleanup
At 02:00 PM 2/9/2004 -0500, Michal Wallace wrote: On Mon, 9 Feb 2004, Melvin Smith wrote: > My take on it is, since it is an intermediate language, we don't need > ability to have keywords as variables. Compilers can generate all > variables with names like $T01JHGJ_001 That can make it kind of nasty to debug compilers. I'm trying to move pirate towards generating names that match the actual variables (though I do still use _000 counters) Which can make it nasty, not having keywords as variables or naming them weird things? I assume you meant the latter, I agree, and my example was probably overexaggerated. You could simply mangle the high-level symbol name so if your language allows you to use 'if' as a variable, do $_if. Take your pick of the thousands of possibilities. I will at least commit a patch so IMCC will immediately flag the declaration of an illegal symbol name rather than allowing it and then getting confused later. Would it be valuable if all variables had (or could have) a $ in front of them? The ones that match /\$[PINS]\d+/ would still have implicit types, and anything else would require a .local or .global for the type declaration. I know how I want to answer the question, but I'd be lying if I said I had actually put a lot of thought in it. Any solution can be made to work: Possibilities: 1) Mangling scheme - potential issues with various languages as well as potential issues with importing external symbols unless we all agree on the same scheme. 2) Quoting symbol names - pretty much works in all cases but is a little annoying for hand written code I'll think more on the issues before giving an opinion since I'm sure there are some people that have already thought on it more than me; Larry, for instance. -Melvin
Re: RT Cleanup
At 07:26 PM 2/5/2004 -0500, Will Coleda wrote: Melvin: Here's a warnocked imcc issue for you: int main () { int if = 1; if (if) { if = 0; } } do you have a patch? I think its a nice to have so If you sourself provide a nice patch guess it would be applied :) I thought I replied to this one. My take on it is, since it is an intermediate language, we don't need ability to have keywords as variables. Compilers can generate all variables with names like $T01JHGJ_001 I could maybe cleanup the error handling of some of these cases by checking the legality of the variable name at the time of declaration. -Melvin
Mail troubles
Hopefully this makes it to p6i list this week. It seems with the recent worm activity, some ISPs have locked down mail servers even more. I have replied to several personal emails (WRT Parrot) and they are bouncing for various reasons, one of which is because my new ISP is on the DUL blacklist, and my mail server is getting rejections. (Not to mention my mail has been getting lost on the way to p6i as well) If you aren't getting replies from me (Tim & Cory for example), I'm trying to resolve the issue. -Melvin
Re: Threads... last call
At 11:45 PM 1/28/2004 -0500, Gordon Henriksen wrote: On Wednesday, January 28, 2004, at 12:53 , Melvin Smith wrote: At 12:27 PM 1/23/2004 -0800, Damien Neil wrote: Java Collections are a standard Java library of common data structures such as arrays and hashes. Collections are not synchronized; access involves no locks at all. Multiple threads accessing the same collection at the same time cannot, however, result in the virtual machine crashing. (They can result in data structure corruption, but this corruption is limited to "surprising results" rather than "VM crash".) But this accomplishes nothing useful and still means the data structure is not re-entrant, nor is it corruption "resistant", regardless of how we judge it. It does accomplish something very useful indeed: It avoids the overhead of automatic locking when it isn't necessary. When *is* that locking necessary? To a second order approximation, ***NEVER.*** Pardon me but I've apparently lost track of context here. I thought we were discussing correct behavior of a shared data structure, not general cases. Or maybe this is the general case and I should go read more backlog? :) -Melvin
Re: Threads... last call
Pardon me, I am trudging through all this thread backlog and have been trying not to post to add to the bandwidth. At 12:27 PM 1/23/2004 -0800, Damien Neil wrote: An existence proof: Java Collections are a standard Java library of common data structures such as arrays and hashes. Collections are not synchronized; access involves no locks at all. Multiple threads accessing the same collection at the same time cannot, however, result in the virtual machine crashing. (They can result in data structure corruption, but this corruption is limited to "surprising results" rather than "VM crash".) But this accomplishes nothing useful and still means the data structure is not re-entrant, nor is it corruption "resistant", regardless of how we judge it. -Melvin
Re: Calling conventions, IMC
At 03:16 PM 1/19/2004 -0700, Luke Palmer wrote: Will Coleda writes: > I didn't expect the code to be very usable, merely compilable. =-) > > If I want to call myself from myself, how do I do that then? > > For anything else, I do: > > .local Sub foo_sub > newsub foo_sub, .Sub, __foo > > .pcc_begin prototyped > .arg $S1 > .pcc_call foo_sub > tellmeagainwhyineedthislabel: > .result $S1 > .pcc_end > > > If I can't explicitly create foo_sub if I'm in the middle of __foo, how > do I recurse using calling conventions You could use P0, which holds the subroutine object. Like using &_ in Perl 6. I'll commit a fix shortly. -Melvin
Re: Various IMC Questions
At 11:56 AM 1/19/2004 -0500, Will Coleda wrote: Trying to get the tcl interpreter working with all the changes in the past few months: I used to be able to say: .pcc_sub _dumper prototyped .param PMC original ... but now PMC isn't a valid type anymore. Is there another way to get an IMC To clarify the answers the others sent. PMC never was valid, but imcc just assumed if the variable type was not one of int|float|num|string|pmc or a valid PMC classname, then it must be a pmc. I sent out a notification that I was adding a stricter type check and so variations such as PMC, var, etc. would no longer work. Sorry for the inconvenience. -Melvin
Re: Optimization brainstorm: variable clusters
At 04:26 PM 1/15/2004 -0700, Luke Palmer wrote: Melvin Smith writes: > My email was concerned with storing/retrieving multiple variables > with a single lookup, not with hinting to the compiler. Okay. Can you show an example of this optimization. I'd rather see it in a HLL talking about PIR, as opposed to in PIR itself. Sure, although what I am talking about is something that the compiler would insert itself, not something we would expose at the HLL. But anyway, a typical module with package variables (syntax may vary) might do: package MyLDAP; import LDAPLib; # Package variables string top = "dn=foo, cn=bar, co=blah"; string user = "melvin"; string password = "abcd"; sub do_something { do_something_else(top, type, user); } Assume that the 3 variables are commonly used in this package (in various routines). Assume that there are other variables and constants that are private to this package. Or forget package altogether, they might be globals. The Parrot translation of the sub do_something() has to fetch the 3 package variables every invocation. .sub _do_something .local string top .local string user .local string password top = fetch "MyLDAP::top" user = fetch "MyLDAP::user" password = fetch "MyLDAP::password" _do_something_else(top, user, password) .end This would translate to bytecode that assigns 3 registers to the variables. Each invocation those registers get initialized by a fetch (hash lookup). For the simple case, I'd like IMCC to look at the package (all subs) and compute a use-count of certain variables, pre-package the N most common (4, 8?) into a register block, and restore that block of registers at the entrance of the routine, rather than doing the 3 fetch lookups. # Beware pseudo-ops :) _do_something: fetch P29, "MyLDAP::top" fetch P30, "MyLDAP::user" fetch P31, "MyLDAP::password" _do_someting_else(P29, P30, P31) # Assume the fetch worked for the top 4 regs Changes to: _do_something: fetchreggroup "MyLDAP::g1"# 1 lookup and you get all 3 _do_someting_else(P29, P30, P31) # Assume the fetch worked for the top 4 regs, # so P28 just gets clobbered with a NULL I can see some potential problems to solve with regards to some languages where variables are dynamic and can be "undefined", such as Perl6, but the optimization would certainly work for constants in all languages. The only problem with Perl6 would be if a global or package variable's address changed after it was stored in the register group at bytecode load time, (which could probably happen). Anytime we cache something dynamic, we have to make sure the caches know about changes. I think that is where notifications might help. For constants it is easy. IMCC might say, "this routine requires us to intialize at least 3 registers with a constant value, lets make it into a register block" This may be a premature optimization, but for certain cases I think its pretty nifty. -Melvin
Re: Optimization brainstorm: variable clusters
At 06:13 PM 1/15/2004 -0500, Melvin Smith wrote: At 10:02 PM 1/15/2004 +0100, Elizabeth Mattijsen wrote: At 15:51 -0500 1/15/04, Melvin Smith wrote: Comments & questions welcome. Why am I thinking of the "register" keyword in C? I have no idea and I can't see the relationship. :) I just realized my response sounded crass, and wasn't meant to be. I welcome comments, I just didn't understand what relation you were getting at. Feel free to point it out to me. The context: Jonathan was asking about importing constants at runtime and/or constant namespaces. Dan and I were discussing the issues and how routines with lots of package globals or constants would spend a significant part of their time retrieving symbols. Jonathan did not want compile time constants, Dan did not want "importable" constants that mutate the bytecode at runtime, so I was trying to come up with a compromise, ugly as it may be. Weird optimizations are ok in my book if they can be hidden in a back-end compiler. -Melvin
Re: Optimization brainstorm: variable clusters
At 10:02 PM 1/15/2004 +0100, Elizabeth Mattijsen wrote: At 15:51 -0500 1/15/04, Melvin Smith wrote: IMCC could analyze a module, decide if the optimization makes sense, and place commonly used values (constants or variables) in a pre-packaged register frame. (or more than 1) This is done at compile / load time of course. If it were all constants, compile time works, but for PMCs and Strings it would have to be built at loadtime. Upon invoking a busy routine, it _might_ be more efficient (since we already save register frames anyway) to initialize the upper frame (top 16 registers) with a pre-built register set. It might also be more useful to have it more granular than 16, maybe in 4s or 8s. By doing life analysis and some weighting, IMCC might be able to turn multiple symbol lookups into 1. Comments & questions welcome. Why am I thinking of the "register" keyword in C? I have no idea and I can't see the relationship. :) With Parrot's architecture, we have overhead of storing and retrieving globals, lexicals and package variables by _name_. This was a design choice, but it has ramifications. C has none of this overhead since it does virtuall all of its symbol resolution at compile time or load time (except in the case of dlopen and company). My email was concerned with storing/retrieving multiple variables with a single lookup, not with hinting to the compiler. -Melvin
Optimization brainstorm: variable clusters
While sitting on IRC with Dan and Jonathan discussing how to optimizer a certain construct with how we handle globals/package variables, etc. I came to the conclusion that it would be valuable to not have to fetch each and every global, lexical or package variable by name, individually, but instead fetch them in clusters (4-16 at a time). We already have register frames which are saved and restored very efficiently. For instances where a module has package variables and globals, there exists an overhead upon repeatedly invoking a sub in one of those modules. I believe it would be quite handy to store register frames off for random access (not on the pad stack). The access could either be by name, or stashed in a PMC and held on the stack itself. IMCC could analyze a module, decide if the optimization makes sense, and place commonly used values (constants or variables) in a pre-packaged register frame. (or more than 1) This is done at compile / load time of course. If it were all constants, compile time works, but for PMCs and Strings it would have to be built at loadtime. Upon invoking a busy routine, it _might_ be more efficient (since we already save register frames anyway) to initialize the upper frame (top 16 registers) with a pre-built register set. It might also be more useful to have it more granular than 16, maybe in 4s or 8s. By doing life analysis and some weighting, IMCC might be able to turn multiple symbol lookups into 1. Comments & questions welcome. -Melvin
Re: IMCC v1 call for bug list and feature freeze
At 01:08 PM 1/15/2004 -0500, Dan Sugalski wrote: At 11:05 AM +0100 1/15/04, Leopold Toetsch wrote: Melvin Smith wrote: Feel free to checkout branch imcc1final to test with. Could you be a bit more verbose about that. I've now checked out the branch "imcc1final", which switched the whole tree to use that sticky tag. Into which branch should changes to non-imcc files go? And of course I don't understand, why you splitted the tree now, while bug-fixes for imcc1 aren't in. I'm glad he has split things off--IMCC desperately needs gutting and replacing (the code was never meant to be production code (which I knew when I said bring it in :) and there's *far* too many big uncommented sections filled with single-character variable names), but I'd rather not break things as they currently stand. A branch is exactly what's called for in this circumstance, so there's remote version control and backup, and folks who're interested can see what's going on. And as I said, this branch is not the major rework branch. Maybe I wasn't clear. imcc1final is for collection of all the final bug fixes _for_ imcc1 (that we know of) before we move on to major rework (imcc2). The major rework will be _after_ I merge imcc1final back to the tree. Then you can freeze it at will when you do the parrot 0.1 freeze. So, no cause for alarm. :) -Melvin
Re: IMCC v1 call for bug list and feature freeze
At 10:27 AM 1/15/2004 -0500, Melvin Smith wrote: At 11:05 AM 1/15/2004 +0100, Leopold Toetsch wrote: Melvin Smith wrote: Feel free to checkout branch imcc1final to test with. The command should be: cvs checkout -r imcc1final parrot Could you be a bit more verbose about that. I've now checked out the branch I would not checkout the branch into the current working directory, although it shouldn't hurt. All you have to do is switch your working Actually, eep no. What I said was wrong, sorry. It is _not_ a good idea to check out a branch into a working directory unless you already know the changes are compatible, because the changes aren't ready to actually be merged, and might just cause tons of conflicts (not the case here, though as they are minor). Unless I merge really quickly back to the trunk, I'll have a bit of work to do when I get ready to merge. Hopefully the changes will all be in imcc/* and there won't be any conflicts. -Melvin
Re: IMCC v1 call for bug list and feature freeze
At 04:17 PM 1/15/2004 +, Jonathan Worthington wrote: From: "Melvin Smith" <[EMAIL PROTECTED]> > I'd like to get imcc1 working as correct as possible and frozen > within a couple of weeks, then I'll start on the really major rework > for imcc2 (including all the deprecation that is going to make everyone > mad at me.) ;) Any chance of a list of what is being deprecated and what the major changes/new additions will be? Give me time to dig back through my p6i email and collect some of it. Some I remember, but some I don't. -Melvin
Re: IMCC v1 call for bug list and feature freeze
At 10:28 AM 1/15/2004 +, you wrote: On Thu, Jan 15, 2004 at 02:21:56AM -0500, Melvin Smith wrote: > I'd like to get imcc1 working as correct as possible and frozen > within a couple of weeks, then I'll start on the really major rework > for imcc2 (including all the deprecation that is going to make everyone If the next parrot release is (going to be) planned for mid Feb then it might be good to freeze v1 sooner to give more time for the impact of the rework to be absorbed. Or perhaps make v2 available sooner as "imcc2" and then rename them both (imcc->imcc1, imcc2->imcc) once imcc2 is stable. Tim [who may be talking nonsense]. Hope you don't mind me cc-ing this back to the list. This is exactly what I was planning to do. I expect for a while people won't want to target v2 so there will be a stable standby until such time that it matures and we deem to migrate Parrot tests to it (there will be a few minor breakages of syntax). -Melvin
Re: IMCC v1 call for bug list and feature freeze
At 11:05 AM 1/15/2004 +0100, Leopold Toetsch wrote: Melvin Smith wrote: Feel free to checkout branch imcc1final to test with. The command should be: cvs checkout -r imcc1final parrot Could you be a bit more verbose about that. I've now checked out the branch I would not checkout the branch into the current working directory, although it shouldn't hurt. All you have to do is switch your working branch back to the main parrot trunk. So far, my changes are only local to imcc/. Also you don't have to check it out; you could do a diff between the main and the branch and review the patch. "imcc1final", which switched the whole tree to use that sticky tag. Into which branch should changes to non-imcc files go? And of course I don't understand, Changes to non-imcc files should go into the main tree as usual. I'm just using the branch to move forward on imcc at my own pace and make the changes public while doing it. As I understand branches, this is exactly what they are for. It would probably help to get a document with cvs hints, specifically about branches. why you splitted the tree now, while bug-fixes for imcc1 aren't in. You must have missed my commit. I committed a revision with the branch that fixes a couple of issues with prototyped subs, specifically a couple of local test cases that I was using that were failing. -Melvin
Re: cvs commit: parrot/imcc/t/syn pcc.t
At 11:20 AM 1/15/2004 +0100, Leopold Toetsch wrote: Melvin Smith <[EMAIL PROTECTED]> wrote: > > For some reason 1 test in pcc.t is failing (the nci call) Off by one error caused by: > -for (j = 0; j < 4; j++) { > +for (set = 0; set < REGSET_MAX; set++) { As most loops inside Parrot use the former scheme, I'd rather keep it then switching to an (it seems) error prone variant "set <= REGSET_MAX" I like my version because it is self-documenting. Thanks for pointing out the bug though. -Melvin
IMCC v1 call for bug list and feature freeze
I'm freezing imcc as version 1 except for bug fixes. I'm working on fixing the PCC code emitter before freezing it. Besides the bugs concerning non-scalability of the register allocator, I'm interested in any bug reports (for IMCC only) that may have been Warnocked until now. I'd like to get imcc1 working as correct as possible and frozen within a couple of weeks, then I'll start on the really major rework for imcc2 (including all the deprecation that is going to make everyone mad at me.) ;) Feel free to checkout branch imcc1final to test with. The command should be: cvs checkout -r imcc1final parrot -Melvin
Re: run-once code
At 09:31 AM 1/14/2004 -0800, David Storrs wrote: On Wed, Jan 14, 2004 at 10:59:52AM -0500, Melvin Smith wrote: > I think Perl6 will allow a hint like so: > > my int $max_reached; > > The important thing is that $max_reached is used simply as a conditional, > and you don't pass it to a routine or otherwise use it in a way to cause it > to be promoted to a PMC What uses would cause it to be promoted to a PMC? Any use that doesn't allow a native type. Currently: passing it to a non-prototyped routine or storing it into a hash/list might trigger this. It depends on how well the optimizer handles hints. Since the optimizer could decide it only needs to create a temporary PMC for the uses in question, but leave the base variable as a native. If the optimizer is too aggressive it might decide that it is less expensive to just declare it as a PMC, so that is where we have to decide how serious we take hints. :) If the hint we are talking about becomes a directive, then the compiler just does what it is told. Any expression needing a coercion will get a temporary created, but all simple uses of $max_reached (increment, evaluation, conditional tests) will still work with a native int register, and this is probably what you want in this case. On the flip side, for a standard declaration: my $max_reached; (no hint) it should be possible with data-dependency analysis to decide $max_reached should be a native, without the hint. We shall see. -Melvin
Re: run-once code
At 10:16 PM 1/13/2004 -0700, Luke Palmer wrote: David Storrs writes: > Given this code: > > if ( some_expensive_lookup_function() >= $MAX_RECORDS ) { >mark_that_we_have_reached_max_records(); >return; > } > > After I enter that block once, I never want to evaluate the condition > again--I want the code to completely disappear from the bytecode (or, > at least, be jumped around). How would I do that in Perl 6? Hmm... my $max_reached; sub mark_that_we_have_reached_max_records() { $max_reached = 1; } If you use a hint in Perl6 to tell Parrot that $max_reached needs to be a native type you'll probably get near C-speed conditional tests with the JIT. (The JIT is many x faster than current Perl5 conditionals, but the reason we don't rave about it more often is that languages like Perl6 lose a lot of what the JIT provides since we have to create most variables as PMCs) I think Perl6 will allow a hint like so: my int $max_reached; The important thing is that $max_reached is used simply as a conditional, and you don't pass it to a routine or otherwise use it in a way to cause it to be promoted to a PMC (which is much heavier than a native). Currently the back-end compiler (imcc) doesn't coerce native types to a PMC, but it will eventually. If you use a hint, and simply set and test the conditional, the compiler should be able to allocate the conditional as a pure integer register, test it, and skip the PMC promotion. Then your worry about getting the test removed from the bytecode would be needless as you'd probably only waste a couple of JITed CPU cycles. -Melvin
Re: Questions about abstract pmcs
At 09:55 PM 1/12/2004 +0100, Stéphane Payrard wrote: On Mon, Jan 12, 2004 at 03:16:50PM -0500, Dan Sugalski wrote: > Which brings up a question. What's the difference between .local and .sym? > -- Currently, there is none. So I went for the shortest: grep -n -e LOCAL imcc.l imcc.l:181:".sym" return(LOCAL); imcc.l:206:".local"return(LOCAL); Its a relic. I had planned to make .sym usable at varying scope levels (as opposed to .local). I've now come to the conclusion that .sym is not very descriptive and I will probably use .global and other more specific names rather than changing .sym In any case, its there now and will probably stay for imcc hackers who prefer variety. :) -Melvin
Re: Mr Parrot's Neighborhood
At 11:18 PM 1/12/2004 -0500, Michal Wallace wrote: On Mon, 12 Jan 2004, Luke Palmer wrote: > A continuation is one snapshot -- it never changes, it never runs. > To invoke the continuation is to take you back to that snapshot and > start running from there. To invoke it a second time is exactly > like invoking it the first time. Thanks. I'd heard this a million times but putting it this way made it click for me. One important addition: While continuations are snapshots of execution context (execution path and variables), they are not snapshots of values. References to globals or lexicals will be restored as the snapshot, but their values can change. -Melvin
Re: cvs commit: parrot/imcc imcc.l
At 07:36 PM 1/9/2004 +0100, Leopold Toetsch wrote: Melvin Smith <[EMAIL PROTECTED]> wrote: > At 09:01 AM 1/9/2004 +0100, Leopold Toetsch wrote: > Which is why I hoped people would pitch in and help fix the tests. Its not tests only. There's already production code out there using Parrot - ask Dan or read his blog. And of course macros are used not only in tests, they are in library code and so on. Dan is smart enough not to arbitrarily update his tree with a current unstable CVS shapshot that is temporarily in flux. You don't update "production" code everytime a new revision appears in CVS. You wait until a release. In between releases, I would like to have opportunity to break things, honestly, while we are still in the alpha stage. > This is alpha code, and my approach is to do what needs to be done > and force us to deal with it sooner than later. That's ok. But we also had some priorities nailed down. First is to fix current bugs and to finally implement the pdd03 changes. I know, that one bug is related to macros, but changing the generated labels from line-numbers to an incremented counter wil very likely fix that too. I understand, but in real life I might get time to work on a small mod, and large mods sometimes have to wait for a larger "coding window." I think its time we create a branch for imcc2 mods. That way we can work in parallel on these things at a more granular pace and not affect Parrot for extended periods of time. We can then backpatch some of the changes as soon as they work rather than waiting to merge the branch (such as dup labels and PCC stuff) but the branch will be available for people to experiment with. I'll create the branch as soon as I get some significant stuff to commit, maybe this weekend. -Melvin
Re: cvs commit: parrot/imcc imcc.l
At 08:44 PM 1/9/2004 +, Harry Jackson wrote: Melvin Smith wrote: But, if you want macros to stay, let them stay. So, are they staying, staying for now or not sure yet? I have a few hundred lines of imcc that uses macros and if they are being removed then I would rather change now and save myself some pain later. I think we are going to have an alternative to C<.constant> builtin to IMCC, the best suggestion seems to be C<.define> As far as macros, I guess we will keep them for the near future. I think its time to create a new branch for imcc development, so lets see how things work out. It may be that we can keep them in some form. One thing to consider: when we put an API on top of IMCC, macros won't translate; they are only for the text version. -Melvin
[CONGRATS] Dan for 1st commercial use of Parrot :)
:)
Re: cvs commit: parrot/imcc imcc.l
At 09:01 AM 1/9/2004 +0100, Leopold Toetsch wrote: Melvin Smith <[EMAIL PROTECTED]> wrote: > IMCC macros are gone. Volunteers feel free to reimplement a > pre-processor (outside of IMCC) using the macro expansion code > that was removed from IMCC. Melvin, that's not the way to go. We can remove them from the lexer, when we have an external substitute. Just removing it breaks a lot of existing code. Failing tests may hide other problems and so on. Which is why I hoped people would pitch in and help fix the tests. This is alpha code, and my approach is to do what needs to be done and force us to deal with it sooner than later. But, if you want macros to stay, let them stay. -Melvin
Re: [RFC] Parrot runtime include files and .constant macros
At 08:58 AM 1/9/2004 +0100, Leopold Toetsch wrote: Melvin Smith <[EMAIL PROTECTED]> wrote: > This also means .pasm files won't be able to .include these anymore, > you'd have to use IMC. Why not just make .const work in both modes? Because pure PASM doesn't currently use type names. .const expects a type of (int|string|num) and isn't the same thing as .constant -Melvin
Re: weird bug w/new imcc syntax
At 09:26 PM 1/8/2004 -0500, Michal Wallace wrote: I love the new syntax for calling functions! Thanks Melvin!!! And... here's a weird bug. :) The following code fails with the message "No Entries on UserStack!" But, if you delete either/both of the empty comment lines and it works fine. :) Thanks for the report. I'll take a look. I'm trying to get back in Parrot mode after buying a house right before Christmas, moving, then catching the flu and getting behind at my day job. -Melvin
[RFC] Parrot runtime include files and .constant macros
Since all of the Parrot includes are .pasm and are using the old .constant directive, which was a macro expansion in the IMCC lexer, and I've removed macros from IMCC, I have a pending patch to parrot_include.pl and all of the parrot header files to change it to generate .imc include files rather than .pasm for runtime/parrot/include This also means .pasm files won't be able to .include these anymore, you'd have to use IMC. I didn't commit it because I'm not so sure removing .constant is the right thing to do, but I'm also pretty sure I don't like maintaining IMCC as the "reference" Parrot assembler. Sure, I like that it can process PASM instructions, but I'm not a fan of the other things (macros, etc.). I don't like supporting 2 "language modes" in the same lexer/compiler. Its confusing enough for beginners to remember that they can use C<.constant> in a .PASM file, but in .IMC they have to use C<.const>. And, oh by the way, .constant is done by the lexer, .const is handled by the compiler. Yuck. Anyone with opinions are asked to give their 2 cents here. -Melvin
[COMMIT] Remove IMCC macros (tons of tests broken now)
As planned, macros have been removed from IMCC. The downside is that this just revealed scads of instances where people were using macro expansion in the tests, especially the .constant directive. One particular problem is runtime/parrot/include/*.pasm These will need to be changed to .imc files and use the global .const directive. I don't have the energy to go through and fix them all myself but I'll do what I can. Hopefully others have some bandwidth. If it becomes too painful, I can revert the patch in a day or so; but hopefull we can live through it. :) -Melvin
Re: Archive tarball?
At 09:25 AM 1/8/2004 -0600, Jonathan Scott Duff wrote: On Thu, Jan 08, 2004 at 07:48:46AM -0700, Luke Palmer wrote: > If worse comes to worst, you can always ask me. I manage to keep the > largest amount of the language in my head with the most time available > to answer questions :-) Oh no, now *everybody* will be asking you stuff. :-) Are we taking periodic backups of Luke? =) -Melvin
Re: Continuations don't close over register stacks
At 06:37 PM 1/7/2004 -0700, Luke Palmer wrote: Leopold Toetsch writes: > Jeff Clites <[EMAIL PROTECTED]> wrote: > > On Jan 7, 2004, at 1:46 AM, Leopold Toetsch wrote: > >> That part is already answered: create a buffer_like structure. > >> *But* again register backing stacks are *not* in the interpreter > >> context. > > > I don't understand what you are getting at. They are not physically > > part of Parrot_Interp.ctx, but it holds pointers to them, right? > > No, they were in the context but aren't any more. > > > ... So, > > they need to be copied when the context is being duplicated. Is that > > your point, or are you trying to say that they are not _logically_ part > > of the context, or are not supposed to be? > > Exactly the latter: > That was AFAIK a design decision, when Dan did introduce CPS. At this > time register backing stacks went out of the continuation or whatelse > context - IIRC did Dan commit that to CVS himself. In which case I feel obliged to contest that decision. The register backing stacks are as much a part of the current state as the program counter is. I tend to agree, but maybe Dan can explain. I looked back at the CVS history and when I put continuations in, I did originally have register stacks in the Parrot_Context (although they weren't yet garbage collected). Dan since reverted that and put them back to the top level interpreter object. It seems to me they should be part of the context structure or continuations become very messy and make save/restoring of register pads almost useless in combination. -Melvin
Re: Continuations don't close over register stacks
At 04:41 PM 1/6/2004 -0700, Luke Palmer wrote: I want these things to be garbage collected, but DOD doesn't trace the buffer. I can't seem to find a way to mark the frames without making the chunks into PMCs (yuck). Is there a way to do this? I was about to answer your question when I saw your followup where you answered it yourself. Thats probably the way I would do it. A continuation can know how to mark everything it closes over, and it doesn't have to be a PMC. The brute force method is to copy the whole pad stack as soon as a register frame is pushed or popped. As long as the stack is tree based (where multiple copies of a set of frames can point to the same parent), you only need to copy the current chunk, not the whole stack, although most of the samples I ran at the time never used more than a single register pad stack chunk (I think it was 16 frames per chunk) so its probably an unnecessary short-cut, just copy all chunks. The downside to our implementation is in the return continuation case. The common case is to create the continuation that you plan to return with, and you already know there will be no need for copy-on-write in most cases because typically the execution path will return using that continuation, and that will then become the "main" execution context. The original interpreter context and all its associated stacks that existed at the time of the snapshot will usually immediately be readonly orphans as soon as you activate the return continuation (unless you saved the initial main context first). It'd be more optimal to skip marking COW altogether in certain cases. I think I've just confused myself so I'm sure I lost everyone. The short of it is: the general case of closing over everything and setting COW on everything for each return continuation is inefficient, because the pattern becomes: 1) Freeze continuation B (mark all stacks COW in A, stacks that B references) 2) Activate return continuation B (replaces old interpreter context/continuation A) 3) Continue execution which inevitably does stack modification and causes a COPY + CLEAR COW bit on everything in B's now private copy. B's private copy usually becomes the new "permanent" interpreter context, until the next continuation (C, etc.) is activated. Taking return continuations over and over has the effect of repeatedly making the main context readonly. Without reference counting (ugg) I'm not sure how else to implement continuations correctly and achieve any sort of shortcut to skip copying things. As I said to Dan when I first patched them in; hopefully someone smarter than me would come along and do it better/faster, I only care that it actually _works_, and for now, it doesn't, completely. Also, good papers on VM design with continuations seemed to be rare 2 yrs ago and I expect they still are. Now that I've taken you on a 2 mile tangent, back to the topic. I'd be happy to see your patch. -Melvin
Re: Continuations don't close over register stacks
At 07:53 AM 1/6/2004 -0700, Luke Palmer wrote: Aren't continuations supposed to close over the register stacks? In this code: I should hope to get 42, but instead I get "no more I frames to pop." They're not very good continuations if you have to treat them just like an address stack! Currently the Copy-On-Write bit isn't being honored on the register pad stacks, so restoretop (Parrot_pop_*()) is ignoring the fact that the stack it is dealing with is a readonly copy (taken by the continuation). They are being marked, though, so its 1/2 complete. I didn't finish the continuation implementation at the time, but I had assumed someone had during my absence. (How is that for passing the blame? :) ) -Melvin
Re: This week's summary
At 09:30 PM 1/5/2004 +, Piers Cawley wrote: Melvin Smith <[EMAIL PROTECTED]> writes: > At 07:55 PM 1/5/2004 +0100, Lars Balker Rasmussen wrote: >>The Perl 6 Summarizer <[EMAIL PROTECTED]> writes: >> > people's salaries will depend on Parrot. I confess I wouldn't be >> > surprised if, by the end of the year, we haven't seen the full >> > implementation of at least one of the big non-Perl scripting languages >> > on top of Parrot. >> >>I'm confused, are you optimistic or pessimistic in that last sentence? > > Knowing Piers, I would guess: optimistic. :) Have we met? You're right though. Unless you count our chats on IRC, no. I can deduce that much from IRC and summaries. We do read them, you know. :) -Melvin
Re: This week's summary
At 07:55 PM 1/5/2004 +0100, Lars Balker Rasmussen wrote: The Perl 6 Summarizer <[EMAIL PROTECTED]> writes: > people's salaries will depend on Parrot. I confess I wouldn't be > surprised if, by the end of the year, we haven't seen the full > implementation of at least one of the big non-Perl scripting languages > on top of Parrot. I'm confused, are you optimistic or pessimistic in that last sentence? Knowing Piers, I would guess: optimistic. :) -Melvin
Re: Generating optimized PIR?
At 10:59 AM 1/5/2004 -0500, Dan Sugalski wrote: Optimized for speed, at least. I'm finding that large subs seem to give imcc headaches. I'm not sure if it's O(n^2) or O(2^n) headaches, but definitely issues. At the moment I've got parrot churning on some pir code and it's taking quite a while. Final time tally: real41m46.978s user21m24.300s sys 0m41.080s Ack. I expected this would happen. Most probably it is register-coloring/spilling. I'm a little rusty on the register colorer; I do know the first version I wrote was not very fast for large numbers of registers, but I believe Leo had improved on it a bit. I'd really like to see the piece of code so I can do some profiling. Ended with a missing symbol error, of course, just to rub it in a bit. vm_stat reports a lot of zero-fill page allocation (about 1600 4K pages/sec) though the memory footprint isn't growing to match, so that might indicate at least something of the problem. (For all I know there's some sort of degenerate GC issue somewhere, depending on how imcc does its memory allocation and management) That could be IMCC repeatedly allocating/freeing its interference graphs for each basic block, but I'm not positive. I know IMCC's being redone, and we're nowhere near close to optimized, but I think it'd be worth it to get a handle on what sorts of things are likely to trigger off exponential time compiles when fed to IMCC. I'm assuming that it's due to a combination of sub size (about 61K of source in one sub) and locals needing coloring (132) but it'd be nice to know for sure. There are several tests I can think of that we should include in IMCC. 1) A large number of locals with a very long code segment, without branches or labels. This would tests large graph coloring without lots of basic blocks. 2) A large number of locals _with_ the normal amount of branches and labels. This would test the life analysis on a large number of basic blocks. 3) A small number of locals with variants of 1 & 2 above for branching/labels. Any chance of getting the code sample? -Melvin
Re: IMCC keyed crasher
At 12:27 PM 12/30/2003 -0500, Dan Sugalski wrote: IMCC bus errors (at least on OS X) when presented with the construct: set $P0[$I1], Params[$I1] This little test program triggers it for me: .sub _MAIN prototyped .local Array Foo .local Array Bar set Foo[1], Bar[1] .end IMCC also doesn't like the construct: Foo[1] = Bar[1] but that's a separate issue. Yep. Traditionally, there is not an instruction of the form above in intermediate or assembly languages, Parrot is really special. :) Funny, I get the following on this test program: error:imcc:op not found 'set_p_p__' (set<3>) in file 'dan.imc' line 6 -Melvin
Re: Strangeness with '.sub' in macros
At 05:45 PM 12/30/2003 +0100, Bernhard Schmalhofer wrote: Hi, I have been playing around with 'libpcre' for Parrot m4. For some reason I couldn't compile two regular expressions in the same PIR script. I created a sample C program and that worked like it should. It looks like the error has nothing to do with 'libpcre'. So I boiled down my code to a small test script. When a macro contains a '.sub' call, and that macro is used twice, then I get a 'memory error'. [EMAIL PROTECTED]:~/devel/Parrot/parrot/languages/m4> ../../parrot t/regex/macro_used_twice.imc Speicherzugriffsfehler Could sombody test the attached script on another machine? I'm working here on a Linux laptop: Two answers here. 1) We probably have a bug in macro expansion 2) Dan has declared that macros are coming out of IMCC. (Mainly because I told him I don't want them and they make the lexer more complicated than I want it to be.. and.. macros are less useful for compilers than people hand-writing IMC code) I prefer macros to be a part of a preprocessor tool external to the builtin IMC language. If enough people really have strong opinions on macros, now is the time to voice it, because otherwise I'm removing them. -Melvin
Re: PMC registry
At 07:38 PM 12/28/2003 -0500, Dan Sugalski wrote: At 7:19 PM -0500 12/28/03, Matt Fowles wrote: Dan Sugalski wrote: At 3:27 PM -0500 12/28/03, Matt Fowles wrote: Leopold Toetsch wrote: I'd use a custom hash with the PMC address being the key[1]. /Me thinks, it doesn't help, when a PMC gets registered multiple times - its always the same address - removing it multiple times is fine, the first succeeds, following fail silently, they do nothing. On a side note, couldn't this be used for the explicit root set augmentation version of DOD that was discussed? If you're speaking of what I think you are (my memory sucks) then this is exactly that. If you're not speaking of what I think you are, then no it isn't, but you can probably use it for that. :) I am, but there is a little more. The explicit root set augmentation was suggested as a way to avoid stackwalking. Ah, OK. No, this isn't that. There's far too much overhead involved for it to be reasonable for that purpose. I know people really hate the idea of walking the system stack, but it really isn't evil, and is definitely a much more stable and safe alternative to the gyrations that'd otherwise be needed. Reading my backlog from the holidays and flu. My 2 cents. It isn't guaranteed that disabling stack walking is going to work since there may be other places (even in Parrot internals) where someone is holding onto a PMC pointer while a DOD runs (besides extension code). The stack walking was added to help fix the infant mortality problem, which was way before extension API. -Melvin
Re: Threads
At 10:07 AM 12/23/2003 +0100, Elizabeth Mattijsen wrote: I think I agree with you in spirit, that we should have high expectations for Parrot and hopefully make the scripting languages that we are running more realistic as all-around programming languages. Eh, I think you should cross out the "hopefully". It is Larry's intention for Perl 6 to be a language for the next 30 years (IIRC). I doubt he meant it as just another "scripting" language. Lets not be unrealistic. Certain language features inherently make compile time (and run-time) optimizations hard or impossible. Perl 6 happens to have quite a handful of them so it is no surprise to me that we find we always seem to be a few cycles shy of certain traditional static languages. I see Parrot as an effort to close the gap between Perl/Python/Ruby and C#/Java on the exact same hardware, since significant advancements in execution speed by the Java and C# implementations are pretty unlikely. (They are already pretty fast) I think you are thinking too short term in that respect. ;-) Don't assume that because my opinion differs from your own that it means I think in less grand scale than you do. :) It is much too early to decide if a particular optimization towards single-threadness necessarily hurts anything and there is certainly no reason to assume Parrot/Perl6 won't run threaded applications well just because Perl5's ithreads don't. -Melvin
Re: Threads
At 11:59 PM 12/22/2003 +0100, Elizabeth Mattijsen wrote: At 14:14 -0500 12/22/03, Dan Sugalski wrote: At 1:49 PM -0500 12/22/03, Josh Wilmes wrote: I think it might be good to get started on regretting that as soon as possible ;-) Still, at the moment, so far as I can tell, most perl, python, and ruby programs that are run are run single-threaded, and as such that mode ought to have the fastest run. I wonder whether that is a good reason. Parrot originally came out of a desire to make a Perl 6 engine. In which threads, co-routines, More than that. Separating the runtime from the language front-end has been key in my book. events or whatever, are an integral part of the system. What Perl, Python and Ruby programs do now, should carry less weight than what all of these systems, and who knows what other languages that want to build on top of Parrot, want to be able to do in the future. I don't see how you can decide to attribute less "weight" to what Perl, Python and Ruby programs do now. Most programmers I know rank speed-of-prototyping and reduced line-count as their favorite things about the aforementioned languages. People will not migrate from Perl 5 to Perl 6 because it will be able to run the same application faster. If that iis the goal, you get bigger hardware and your done with much less fear of migration problems. People _will_ want to move from Perl 5 to Perl 6 because of: Lots of companies are still developing on the same hardware they were before Y2K. I'd bet they would welcome a Perl implementation that got them 3-4x speedup for free for new code. I will agree with you on legacy Perl apps, as most of my clients are exactly as paranoid as you describe. - good threads support - good signalling support - good event handling support All of these more or less need a system that can quickly switch context. I think it is unbalanced to regard threads & context-switching as the most important aspect when dealing with languages that do not lend themselves to optimization and static compilation. If you really want reliable signal handling (to the point of worrying about context switch overhead, we begin to sound like we want a real-time OS and language), then you (at least currently) don't write it in Perl/Python and _especially_ not Ruby. You write it in C/C++/Java/Objective-C, take yer pick. I think we need a change of mindset. Instead of seeing threaded programs as the special case, we would need to see that the single threaded program is the special case. See how many people use POE for event handling, and through what hoops (Perl) Tk needs to jump through to get proper event handling. I don't think we are of the mindset that threading is so special and rare that we will force threading programs to suffer in pain. I don't see optimization like that. When Dan says we optimize for a common case, it doesn't mean the uncommon case won't be greatly improved as well. I expect any implementation of Parrot to run single & multi-threaded programs many times faster than the Perl5 engine. I think I agree with you in spirit, that we should have high expectations for Parrot and hopefully make the scripting languages that we are running more realistic as all-around programming languages. I see Parrot as an effort to close the gap between Perl/Python/Ruby and C#/Java on the exact same hardware, since significant advancements in execution speed by the Java and C# implementations are pretty unlikely. (They are already pretty fast) -Melvin
Re: ParrotIO objects
At 11:02 AM 12/22/2003 -0700, Cory Spencer wrote: The program I'm writing is passing a ParrotIO object about to various functions (the ParrotIO object being either stdin or a file handle to a regular file). Each function can read as many bytes as it likes from the descriptor. There are times, however, where I might decide that I don't want the byte I just read, and need to push it back onto the stream. This brings up (I think) two ways around the problem: 1) I'd like to be able to peek ahead at a byte, before actually consuming it. ie) Yes, Parrot will get an "unget" and "peek". Now that the buffering layer is mostly working it should be quite easy. I've had a bad bout with the flu for the last 7 days. As soon as I can sit up again I'll see about patching it in. -Melvin
Re: pdd03 and method calls
At 10:42 PM 12/17/2003 +0100, Leopold Toetsch wrote: While playing with calling threaded subs, I came along a thing which I think might be suboptimal: pdd03 states that the method PMC should go into P2. This doesn't really play with Perl5 <-> Perl6 interoperbility IMHO. Perl5 methods are plain subs, where the first param is the object. I dunno, if Ponie will ever use ParrotClass/ParrotObject, but I'm sure, that calling Perl6 methods should work (and vv). So me thinks that the method PMC should be the first PMC argument living in P5. sub meth { my ($self, $arg1) = @_; #P5P6 ... Comments? Makes sense to me for non-prototyped methods. How 'bout: Non-prototyped: pass object in P5 array with rest of arguments Prototyped: pass in P2 as in current spec -Melvin
Re: Unexpected error...
At 02:00 PM 12/12/2003 -0700, Cory Spencer wrote: Can anyone tell me why the following code: .sub _main .local PerlUndef val val = new PerlUndef _foo("bar", val) end .end .sub _foo .param string v1 .param pmc v2 .pcc_begin_return .return 1 .pcc_end_return .end Would produce the following output: cog:~/parrot test.imc wrong param count in file '(unknown file)' near line -1 It seems to be related to passing the PerlUndef as the seoncd parameter... When using prototyped subs and mixing native types with PMC types as arguments, IMCC will get confused. Its because it isn't correctly checking the prototypes. The other problem is that when you use the shorthand version of sub calling (_foo(a,b)), IMCC assumes prototyped. It stems mainly from 2a below. I recommend: (1) Using non_prototyped subs and explicit .pcc_call directives, rather than the shorthand version for now. Until I get some fixes committed you have a couple of other options: (2) Don't mix PMCs and native types in argument lists if you use prototyped subs (2a) If you use PMCs, then IMCC won't coerce a constant argument into a PMC yet. _foo("bar", val) will pass "bar" as a plain string type, even if the prototype expects a PMC. This is a bug. I expect to have a large revision soon that will address lots of issues in IMCC, but it will most likely be in a new branch (imcc2) so as not to cause complete chaos with the "semi-stable" version. Then I will try to backpatch some of those fixes into the current IMCC. -Melvin
Re: Macros, PIR, and PASM
At 06:06 PM 12/11/2003 -0500, Dan Sugalski wrote: Folks, As IMCC's in some flux and likely to get gutted and reworked, the question of macros has come up. (They cause some grammar issues) So, to make life easier: Parrot's built-in PIR and PASM parsing modules do *not* need to do macros. (Though they do need to do .const things) Macro assemblers are for people, not compilers, and as such a separate tool (which can be a loadable compiler later if someone wants) is appropriate in this circumstance. So... As long as there's a macro-assembler preprocessing program that understands the macro system and spits out macro-expanded PIR and PASM, the macro facilities built into IMCC can get tossed out. (Time for someone to write Macro-Parrot.pl, methinks) There is already a nifty macro processor in imcc.l that probably should be salvaged and possibly moved to a separate lib. -Melvin
RE: [CVS ci] object stuff
At 03:05 PM 12/11/2003 -0500, Gordon Henriksen wrote: Melvin Smith <[EMAIL PROTECTED]> wrote: > my $foo = Oracle::Instance::DEV1::db_block_buffers; > > The namespace lookup in Oracle::Init checks the Oracle config > parameters which is external code. > > All sorts of neat possibilities. :) It is truly remarkable the lengths that Perl programmers seem to be willing go to in order to hide a function call or obscure the existence of an object. :) I'm not an advocate of the feature, but it is novel. Tying is a requirement of Perl, or so Dan has told me. Now that I think of it, I'm not sure I've seen any official requirement for tying namespaces, but it is probably orthogonal to tying classes. You might not want to take my example seriously as I'm not a Perl6 authority. -Melvin
Re: [CVS ci] object stuff
I think a heirarchy is a good idea for namespacing in general. I've always wanted to be able to tie namespaces in Perl 5. It would only make sense that if I tie Foo::, that Foo::anything:: would also go through that tie to get the anything:: stash. What do you mean by "tie" here? Are you talking about namespace aliasing, so that I can alias "Foo" to "A::B::C::D::E", so that I can say "Foo::bar" rather than "A::B::C::D::E::bar"? If so, it seems that this would work with or without a hierarchical structure. Definitely useful, though. Tying a namespace will involve allowing a namespace to basically hide its implementation, have a custom lookup routine, back-store its symbols, etc. I might tie a namespace to an Oracle database, so a lookup is really a callout to database code. my $foo = Oracle::Instance::DEV1::db_block_buffers; The namespace lookup in Oracle::Init checks the Oracle config parameters which is external code. All sorts of neat possibilities. :) -Melvin
Re: Namespaces
At 11:57 AM 12/11/2003 -0500, Dan Sugalski wrote: That does, though, argue that we need to revisit the global access opcodes. If we're going hierarchic, and we want to separate out the name from the namespace, that would seem to argue that we'd want it to look like: find_global P1, ['global', 'namespace', 'hierarchy'], "thingname" That is, split the namespace path from the name of the thing, and make the namespace path a multidimensional key. Or I suppose we could just punt and toss the special global access entirely and make the global namespace a hash 'o hashes hanging off the interpreter and access it like any other variable, but that makes local obscuration of the namespace somewhat difficult and I'd rather not for right now. This'd be the time to weigh in on it, folks... I expect we need multiple options and let compiler writers figure out which ones they need. (see **) 1) Lookup a fully qualified name in the top level namespace (true globals, Foo::Baz::i) 2) Lookup a fully qualified name in a specific namespace (namespace handle in PMC or string) 3) Upward lookup of a fully qualified name starting at a specific namespace (I'm in Baz, resolve j = lookup in Baz, lookup in Baz->parent, lookup in Baz->parent->parent) Difference between 2 and 3 is that 2 does not recurse upwards to resolve the name. **We may also want variations of non-qualified name lookups (optimized not to worry about tokenizing the identifier). That's off the top of my head from having written a couple of toy compilers. Keyword = toy Of course we'll need forms tailored to object instances that will also have to deal with the semantics of inheritance. -Melvin
RE: [CVS ci] object stuff
At 11:34 AM 12/10/2003 -0600, Robert Eaglestone wrote: Quoth Melvin Smith: >It be a bit friendlier to make the scope resolution operator something ^^ ACK >that at least 1 or 2 languages use as their own already; then all the rest >still have to mangle. Uh oh, time to vote? Voting for myself for having the most consecutive posts with bad grammar. -Melvin
Re: [CVS ci] object stuff
At 12:16 PM 12/10/2003 +, Tim Bunce wrote: *{"Foo\0Bar\0Baz"}->{var}; or *{"Foo\0Bar\0Baz\0var"}; [snip] I think Dan was proposing the first and that's fine. I think the second would be a mistake. Using a character that won't collide with HLL has a disadvantage in the general case: 1) ALL qualfied names (not some) have to be translated/mangled. What is the benefit, then, if all HLL compilers have to mangle it anyway? It be a bit friendlier to make the scope resolution operator something that at least 1 or 2 languages use as their own already; then all the rest still have to mangle. Benefits: -Some languages don't have to translate, so they win. -Parrot hackers don't have to write 'Foo\0Bar\0Baz'! -Compiler hackers have enough headaches without worrying about handling strings with embedded null characters. -Melvin PS: I still haven't seen Dan say if he has some other neat cheat trick in mind for using \0. I can see that it would be pretty fast for tokenizing Foo\0Bar\0Baz into its components given C strcpy semantics.
Re: [CVS ci] object stuff
At 01:37 AM 12/10/2003 -0700, Luke Palmer wrote: Dan Sugalski writes: > At 05:14 PM 12/5/2003 +0100, Leopold Toetsch wrote: > > set I2, P1["Foo\x00i"] # I1 == I2 > > > >gets currently the attribute idx (0) of "$Foo::i". > >Q: Should the assembler mangle the "Foo::i" to "Foo\0i" > > I don't like it either, but the alternative is to impose an external > hierarchic structure. Which we certainly could do, though it may make > some things more difficult. > > I'm more than willing to entertain arguments for and against the > hierarchic structure, but we're closing in on the point where we fix > the scheme and don't change it any more. I'm not sure mangling is a good idea: think about anonymous classes. If we assign a name to each lexical class, then there are problems when one instance of a class inherits from another instance of the same class. That is: sub make_class(Class $parent) { return class is $parent { has $.stuff; } } my $class1 = make_class(Object); my $class2 = make_class($class1); So, if you name that lexical class _class_0001 or something, "_class_0001\0$.stuff" refers to both attributes. I don't think so. I'm definitely not up to your level regarding Perl6 but I believe the example you give translates to: 1st call return class is Object { has $.stuff # Object::_class_0001::$stuff } 2nd call return class is "_class_0001" { has $.stuff # Object::_class_0001::_class_0002::$.stuff } No collision. $class2 has 2 $stuffs but at different class scopes. I'm also not disagreeing, I prefer a hierarchical approach as well but I need to take time to formulate a good argument for Dan. If you generate a name for each instance of a lexical class, you're setting yourself up for some major runtime cost -- concatenating the class name to the attribute on each access (or memory cost if you want to cache). This is why I don't like the non-hierarchical approach. There are, of course, disadvantages. It would be slower for cases where $something::or::other are acessed a lot, as it's three lookups in the absence of cache. Also, it's not as easy to tell where a symbol table I think this is a Perl6 problem more than a Parrot problem since no matter how many optimizations we add to Parrot, Perl6 may or may not be able to use them due to the dynamic nature it has. I definitely want to tie namespaces though. Me 2. -Melvin
Re: Incorrect scoping of constants in IMCC
At 07:58 AM 12/10/2003 +0300, Vladimir Lipsky wrote: From: "Melvin Smith" <[EMAIL PROTECTED]> > At 04:20 PM 12/9/2003 -0500, Dan Sugalski wrote: > >which is .included) and visible through the rest of the compilation unit. > > > Parser will not allow .const outside of a compilation unit and make it > global to the > compilation. Hmm... What do you mean by a compilation unit? a file? Will it be global to all compilation units or rather to the one where the .const was defined? 0x4C56 Right now a unit is a subroutine. Eventually it might also mean class, package, etc. I used a confusing term there. For a single compile "session", meaning all includes/imports, a global constant will be visible. -Melvin
Re: Incorrect scoping of constants in IMCC
At 10:04 PM 12/9/2003 -0500, Melvin Smith wrote: AWhen someone gets a chance to patch this one up, I'd much appreciate it. Fixed. Parser will not allow .const outside of a compilation unit and make it global to the compilation. .const inside a .sub will be local to the sub only (no change there). Typo. not=now "Parser will NOW allow .const outside of a compilation unit" -Melvin
Re: Incorrect scoping of constants in IMCC
At 04:20 PM 12/9/2003 -0500, Dan Sugalski wrote: Just ran across a bug in IMCC. The .const directive is incorrectly available only within a .sub/.end block. Silly. (And wrong) That makes it very difficult to usefully use constants--generally they're defined at the top of a file (or in a file which is .included) and visible through the rest of the compilation unit. When someone gets a chance to patch this one up, I'd much appreciate it. Fixed. Parser will not allow .const outside of a compilation unit and make it global to the compilation. .const inside a .sub will be local to the sub only (no change there). -Melvin
Re: Missing branch instructions?
At 11:52 AM 12/9/2003 -0500, Dan Sugalski wrote: >may not branch to OK. (There's no requirement that high-level comparisons >require a PMC to be equal to itself) Although committing such a confusing PMC to Parrot is certainly questionable. -Melvin
Re: [CVS ci] object stuff
At 05:14 PM 12/5/2003 +0100, Leopold Toetsch wrote: set I2, P1["Foo\x00i"] # I1 == I2 gets currently the attribute idx (0) of "$Foo::i". Q: Should the assembler mangle the "Foo::i" to "Foo\0i" Something about this embedded \0 character bugs me. I know its what Dan has in the design doc but it just seems too cryptic. -Melvin
[COMMIT] A few imcc tweaks
As discussed in the last IMCC RFC, I've committed a few of the changes. IMCC will now generate an error if the register type is unknown rather than just assign it a PMC register. P reg types are pmc, object, or a valid classname. Use pmc or object for "generic" P register to defer type checking. I think a few compilers will now be broken but the patches should be very simple to fix them. Also allow sub names without _ prepended, and allow @ to start labels. Tweak the fixup code a bit to use flags rather than looking for _ in symbol name. Added parser stubs for .global to grammar. -Melvin
Re: Determining PMC memory addresses
I don't think there was ever a consensus about opcode naming. It seems that we need this but can you give an example of where you are using it, just to give me some context to think with? -Melvin Cory Spencer <[EMAIL PROTECTED]> 12/03/2003 11:18 AM To: [EMAIL PROTECTED] cc: Subject:Re: Determining PMC memory addresses > We're already using 'eq' to perform equality testing, and in the interests > of maintaining a consistent design I would choose to stick with something > eq-related as opposed to changing it to 'same'. > > eqaddr/eqval? eq_addr/eq_val? eq_address/eq_value? So just to follow up on this thread, was there any preference at all for the name of the opcode performing equality testing of PMC memory addresses? I needed the functionality for the code I'm working on, so I've created a patch implementing an "eq_addr" opcode... Regards, Cory
Re: [RfC] Testing for null
We should have 1 recommended way for testing NULL registers. If we support get_bool() then lets make sure it works for REAL NULL pmc registers as well as PMCNULL. If not, code will appear to work correctly on a "safe" core but will seg fault on some other. Also, I see no reason not to use PMCNULL in all cores now. -Melvin Leopold Toetsch <[EMAIL PROTECTED]> 12/03/2003 10:43 AM Please respond to lt To: [EMAIL PROTECTED] (Juergen Boemmels) cc: [EMAIL PROTECTED] Subject:Re: [RfC] Testing for null Juergen Boemmels <[EMAIL PROTECTED]> wrote: > Hi, > I'm curently playing around with open calls returning a PMCNULL > instead of a half valid IO-Object. But the main problem with that is > that there is currently no way for the byte-code to detect such a > case. C tests for PMCNULL too and is usable. > * A PMCNULL has false semantics. This may be done by letting > get_boolean return FALSE, or by adding a test to if_p_ic op: > if(!PMC_IS_NULL($1) && $1->vtable->get_boolean($1)) Or the null.pmc gets a valid get_bool() vtable slot returning 0. OTOH when PMCNULL is a real NULL, this will fail then. > * Have a special op for this if_null and unless_null which exlusively > test for NULL. Probably better. > boe leo
Re: [RfC] Testing for null
At 03:46 PM 12/3/2003 +0100, Juergen Boemmels wrote: I'm curently playing around with open calls returning a PMCNULL instead of a half valid IO-Object. But the main problem with that is that there is currently no way for the byte-code to detect such a case. The if and unless ops call the get_boolean vtable method which throws an internal exception. As we can create a PMCNULL with the null_p op there should at least be a way to test for a PMCNULL. Two possible solutions come to my mind * A PMCNULL has false semantics. This may be done by letting get_boolean return FALSE, or by adding a test to if_p_ic op: if(!PMC_IS_NULL($1) && $1->vtable->get_boolean($1)) * Have a special op for this if_null and unless_null which exlusively test for NULL. My intention for PMCNULL was to catch invalid interpreter state, or invalid bytecode; cases which used to give us core dumps. Explicitly returning PMCNULL and allowing tests on PMCNULL may or may not fit into this, depending on your point of view. I'm not even certain what my view of it is. Allowing ANY valid test on PMCNULL seems to muddy the water by allowing it to actually be used. The question is: should we allow a NULL register test and would this replace Undef as the de facto way to return "nothing". Probably the answer is "yes" but I'd like to hear Dan's take on it. -Melvin
Re: [RFC] IMCC pending changes request for comments
At 10:37 AM 12/3/2003 +0100, Leopold Toetsch wrote: Melvin Smith <[EMAIL PROTECTED]> wrote: > 1) Currently typenames are not checked except with 'new ' As Dan layed out WRT PMC enums, we might have to defer type checking to runtime. That is for core PMCs we do strict type checking, other types get resolved at runtime. All I care is that IMCC checks that it is a valid class if the class is named, otherwise use 'pmc' to defer checking but at least use the correct register set. Even given what Steve said about no aliases, I'm considering keeping an "object" alias to "pmc" for potential future cases where semantics may diverge. > C<.local identifier> and C<.param identifier> both assume > that anything for type other than (int|float|string) is a PMC. Yep. Some remarks: .local ident collides with the macro syntax for local labels. So when doing something here I'd resolve that conflict too and allow only: .sym ident # ::= int|float|string|pmc Since macros are the less common case I'll have a look to see which needs to change to resolve the conflict, macros or IMC. I'm not sure but: .newsym ident could be handy too, compiling to new Px, <.Type> Ok. And while at it, imcc should know of lexicals. E.g. when the register allocator is out of regs, spilling lexicals to an array isn't necessary - they already have a permanent store in the lexical pad. The same thing goes for globals. I've been trying to get my head around both lately. > 4) No implicit global labels with the label :== IDENTIFIER COLON syntax. That's not easy. While its clear that a .sub is global there are more. PASM mode comes to my mind and some nasty one: imcc/t/syn/eval_3 ff. I need to think on this one some more as well. Thanks for the comments. -Melvin
Re: Symbolic vs Named variable register allocation
At 03:01 PM 12/3/2003 +0100, Leopold Toetsch wrote: Pete Lomax <[EMAIL PROTECTED]> wrote: > The following demonstrates that $I1 and .local int i map to the same > register in the output pasm code: Yep. The problem seems to be the backward branch. When you put the "test" sub after the "end" op, its working fine. At quick glance I would guess IMCC assumes that the "call test" branches to a basic block that doesn't overlap with the callee basic block. Much (most) of the flow analysis code was done before we adopted new calling conventions. I'll have a look to see if there a quick fix that would allow the code snippet sample to be legal. -Melvin
Re: [CVS ci] pmc compiler 2nd edition
At 11:10 AM 12/3/2003 +0100, Leopold Toetsch wrote: I've put in a really long pending patch. The upcoming vtable changes are simplified a lot, as e.g. null.pmc is fully generated now. Currently line numbers in generated .c files are wrong and disabled. Fixes welcome. I think we still have 2 versions of some things (ops2c2 and pmc2c2 spring to mind). I vote that you go ahead and remove the old versions, even if it temporarily breaks something, if you are confident in the new ones. -Melvin
Re: [RFC] IMCC pending changes request for comments
At 07:59 PM 12/2/2003 -0800, Steve Fink wrote: On Dec-02, Melvin Smith wrote: > > 1) Currently typenames are not checked except with 'new ' I would vote for no aliases at all. I propagated the existing uses of ".local object" in the Perl6 compiler and introduced several more uses, but that was only because I wasn't sure at the time whether it was intended to (now or in the future) have slightly different That would be my fault for not providing documentation with the compiler. Do you really want to generate the extra unused code at the end of all the subroutines? I don't think you want to autodetect whether the code is guaranteed to return. Good point. Its easy to detect returns if the code uses high level IMC directives, but in cases where it is all low-level PASM, it could get a little troublesome. It would also add a few dead instructions here and there. Nix that idea. .sub _gorble () returns (int r) r = 5 .end We may need something like this anyway for prototyped subs. Currently we are only prototyping the parameters, not the returns; not quite orthogonal. Thanks for the comments, -Melvin
Re: Objects!
At 10:52 AM 12/2/2003 -0500, Dan Sugalski wrote: Single-inheritance objects seem to be done. The code can use a lot of abuse and cleanup, and we need actual tests to make sure that it functions as it should, but they're in. Let me start the abuse. *(cough)* examples would be nice ;) Ok, abuse over. I am just happy we are moving again. *) Creating new objects involves calling the ->init vtable entry *on the class*. Because of this each class gets a custom vtable where the init method has been swapped out for one (from objects.c) that creates a new object instead. So do you want an init op added to object.ops or is there something I am missing. I did toss in 'isa', although the implementation is wrong as you said. -Melvin
[RFC] IMCC pending changes request for comments
In an effort to improve its error reporting ability and internal maintainability, I'm considering the following changes; well number (1) is already decided, but I need feedback from compiler maintainers before doing so. 1) Currently typenames are not checked except with 'new ' C<.local identifier> and C<.param identifier> both assume that anything for type other than (int|float|string) is a PMC. This was not intended to be permanent behaviour, but I never added the proper checking. Patching my local copy has uncovered a few places where people have used the undocumented "feature". The downside to the "feature" is lazy error checking and confusing messages from IMCC that have nothing to do with the error. New hackers write ".local integer i" and then waste time wondering why IMCC keeps saying "unknown parrot op foo_p_p" because integer really isn't a type, what they meant was (int). I see in a lot of tests the use of C and C as types in IMC code. I believe Jako and Perl6 compilers use C as well. I'm ok with just adding those 2 as aliases for C register, but I only want to add aliases that are appropriate. The feature of defaulting to PMC is going away, so get your requests in now. I also want to keep the aliases to a minimum. From now on any declarations in IMCC will require either an IMC register type or a predefined classname or PMC. Else, just use C to mean generic. Comments from the compiler maintainers (especially the Perl, Forth & Jako guys)? 2) In the absence of some sort of return instruction, subs currently just run off the end of their code and continue merrily. This feature really isn't useful as far as I can see because it is not supported to assume any ordering between compilation unit, which a sub _is_. It is easier to just assume a sub returns by the active convention. I'd like to just be able to write void subs as: .sub _foo print "hello\n" .end 3) Strict prototyping mode shortcut (backwards compatible of course): As usual, shortcuts are for hand-hackers, but its easier to debug either way. .sub _baz (pmc p, int i) ... .end Same as: .sub _baz protoyped .param pmc p .param int i ... .end 4) No implicit global labels with the label :== IDENTIFIER COLON syntax. Currently labels beginning in _ are global. I'd like to remove the implicit aspect of it. sub definitions will create global labels, as might other directives, but there will no longer be a requirement that subs start with _ in order to get the correct behaviour. We might still want global labels declarable, somehow, so I'm open to suggestions, but I don't see the need, really. A couple of these suggestions might seem uncalled for. Honestly I am trying to tighten up things and officialize IMCC behaviour so expect more soon. Some changes might cause minor breakage to existing compilers but that is just growing pain. -Melvin
Re: [RfC] Fix assigned PMC enum_class_xxx
At 10:40 AM 11/28/2003 +0100, Leopold Toetsch wrote: As outlined some time ago, when ops.num made it into the core, we need fix assigned PMC class enums too. (Changed class enums invalidate existing PBC files). 1) lib/Parrot/PMC.pm is the canonical source of PMC class => enum mapping. 2) the class enums should be numbered so that "base" classes come first and dependent classes later. 3) If we keep the current scheme for Const$class generation and switching, we have to use this numbering scheme: default ... implicitely #0 odd enums ... plain class even enums ... Const variant of class 4) Where config/* now have $pmc_names, %pmc_types is used. 5) Adding a new class starts with editing PMC.pm. If the current numbering can not be kept, PBC_COMPAT gets an entry too and thus invalidates existing PBCs. 6) The interactive step of selecting PMCs is IMHO unneeded. Its error prone, rarely used PMCs can be dynamically loaded. 7) We probably have to reserve some slots for Perl5-classes Comments welcome An additional thought. I feel there is no need to expose the enumerated values to user-code (bytecode or native methods). Looking up PMCs isn't really any different than looking up other symbols and could be fixed up at load time. This does away with any ordering or numeric range reservation issues. If I recall, Java stores class references as UTF encoded strings of the full path to the class [java/lang/foo] -Melvin
Re: Why are .sub and compilation unit one and the same thing?
At 08:10 PM 12/1/2003 -0700, Cory Spencer wrote: > However, if giving up IMCC's register allocator is worth gaining > the extra control of PASM, by all means do it, however I'm all ears > on suggestions for IMCC for features. *hint* In that case, I don't suppose it would be possible for IMCC to allow function calls in an "if expr goto LABEL" statement, ie: ... if _some_test() goto LABEL ... ... I for one would find that particularly useful. ;) That is probably pushing the bounds of "intermediate code" towards "high-level" land. :) Since IMC is supposed to be a language for compilers (not humans) to emit, what you really need is a nice fat high level language. Now why don't we have one of those handy. ;) -Melvin
Re: Some PIR "How do I?" questions
At 01:14 PM 11/27/2003 -0500, Dan Sugalski wrote: At 5:38 PM + 11/27/03, Pete Lomax wrote: On Thu, 27 Nov 2003 09:52:10 -0500, Melvin Smith <[EMAIL PROTECTED]> wrote: At 12:02 PM 11/27/2003 +, Pete Lomax wrote: Perl6 already does interpolation without special support from IMCC. I'll rephrase. Is there anything knocking about which would help with eg: printf (pFile, "Amount %12.3f [%-10.10s]\n",balance,name); Or do I have to rip the string apart character-by-character, then throw all my variables about in registers, cutting, chopping and padding them into shape, and then dumping them in iddy little bits? Depends entirely on two things: 1) Whether we want printf-style functionality as part of the core 2) Whether you want to use it I think the answer to #1 is a big yes. You can answer #2 as you need. :) To me, variable interpolation means high-level language interpolation using the high-level symbol name. printf isn't the same. I'd call it "format interpolation", if that makes sense. 1) interpolation by the symbol name we don't need at the Parrot/IMC level, and this was what the FAQ addressed. I could be wrong, though. 2) printf/sprintf - we do need it (and implemented in C) since it is a staple and is the reasonable hook for HLL implementors to do interpolation without having to write a special native method or PMC for each language. -Melvin
Re: Why are .sub and compilation unit one and the same thing?
Hi Pete, Looks like what you really need is a good way for IMC to handle: 1) Globals 2) Package (or file local) variables 3) Class definitions (with class "locals" or fields) All of these are planned, right now the only equivalent to 'local int a' in your code sample is a global variable. Hopefully very soon I will have addressed the missing features. However, if giving up IMCC's register allocator is worth gaining the extra control of PASM, by all means do it, however I'm all ears on suggestions for IMCC for features. *hint* -Melvin Pete Lomax <[EMAIL PROTECTED]> 11/29/2003 07:51 AM To: [EMAIL PROTECTED] cc: Subject:Why are .sub and compilation unit one and the same thing? Am I missing a trick here, thinking it would be better to allow eg: .imcc .local int a .sub _get_a return a .end .sub _set_a restore a .end .endimcc
Re: Some PIR "How do I?" questions
At 12:02 PM 11/27/2003 +, Pete Lomax wrote: Perl6 already does interpolation without special support from IMCC. That's nice for it. Where do I go crib from? ? -Melvin
Re: I don't like failed tests
It my fault. I was messing around with labels. I have a pending patch to clarify some more things in the API so for now we just need to disable the imcc/t/imcpasm tests. Global labels get fixup and are loaded into global table by packfile. The local labels don't need to be globals. -Melvin Juergen Boemmels <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 11/25/2003 01:02 PM To: Perl6 Internals <[EMAIL PROTECTED]> cc: Subject:I don't like failed tests Hi, the test in imcc/t/imcpasm/{pcc,optc}.t are currently failing. First I thought this was a problem with my setup because most tinderbox were green. It took me a while until I found out that they also fail, but you need to look in the logs to see it. Is there a bug in the tinderbox-code? Anyway: imcc generates code were the generated labels start with _@ instead of @ like the test suggests. Leo says [1] that labels should not start with an _ so the obvious (attached) patch might be wrong. bye bö [1] http://groups.google.com/groups?selm=200311190952.hAJ9qm812051%40thu8.leo.home imcc_test.diff has been removed from this note on November 25, 2003 by Melvin Smith
Re: PIO_eof
At 05:03 PM 11/25/2003 +0100, Juergen Boemmels wrote: Melvin Smith <[EMAIL PROTECTED]> writes: > In PIO_open the ParrotIO object never gets created if there is > an error condition. But PIO_open returns a valid IO-PMC which just has a NULL in its data segment. Maybe it should return PMCNULL if the open fails. It should. I was working on other things so I didn't loot too close. BTW: None of the PIO_* functions check if the incoming PMC is really a IO-PMC. Maybe its a good idea to add glib-like return_if_fail like assertion in the beginning of the functions. This reduces the speed but makes debugging easier. Toughts? They should do exactly that. Speed differences of 3-4 cycles are lost in the noise of IO calls anyway. > PIO_eof() returns true if the PMC has a null IO object or if > the PIO_F_EOF flag is set. > > >Is the "io == NULL" testing prohibited in Parrot embedding system? > > > >0x4c56 > > Currently it isn't prohibitied. That is the only way to detect an error > from PIO_open* at this time. No, even that is wrong. PIO_open returns a PMC with NULL data. So the test would be PMC_data(io) == NULL, but I think the better idea is to return PMCNULL. When I said "io" I meant ParrotIO, not PMC, but I see how it gets confusing. I think PMCNULL should be NULL for all PMCs, at least for user visible APIs. PMC internals (hash/array implementations, etc) don't necessarily have to use the convention, but I know my time has been better spent now that we get "NULL PMC" exceptions rather than seg faults. It'll just be a slow process to guarantee all of Parrot conforms. > If someone convinces me that it is better to return a ParrotIO object > with error flags set rather than skip creating it, I might consider it. No, I think we need to rethink the wrapping technology (which I introduced). Something like: struct parrot_io_t { PObj pobj; UINTVAL flags; PIOHANDLE fd; ... }; But this needs to allocate garbage-collected memory of sizeof(struct parrot_io_t) instead of sizeof(PMC). I don't know if something like that is already possible. Furthermore there are issues Yes, I'm pretty sure it is possible with current API. I'm occupied elsewhere with IMCC, classes and Cola rewrite, sorry I haven't been more help on IO. I'm normally ok with what you suggest, so have a go at it if you get free time, since you are the active IO maintainer. If you commit something that doesn't work well, we just change it, no harm done. Functional-Kludge is always better than Non-Existence in my book. -Melvin
Re: Some PIR "How do I?" questions
At 07:18 PM 11/24/2003 +0100, Jerome Quelin wrote: Dan Sugalski wrote: > On Mon, 24 Nov 2003, Jerome Quelin wrote: > > * How does one launch an exception? trap an exception? > > * How does one create a class? instanciate an object? > With the exception of the second bullet item, these are generic > parrot FAQs rather than IMCC/PIR specific ones, so ought to be put in > a different FAQ or doc section... The last 2 bullet items will also be specific to IMCC. The syntax for defining a class will trigger IMCC to construct and freeze the class with the bytecode at compile time which means it wont generate any Parrot instructions. That is how it currently does constant subs. Also, exceptions in IMCC will certainly be a briefer syntax, maybe even try/catch. I think the real question is: do we want to promote imcc as _the_ target language for compilers targetting parrot? The answer is yes. If the answer is yes (as I thought it was), then I don't think we need a pasm faq... We still need one but I do not want to maintain it, myself. Simon Cozens already wrote some stuff that could be used as a start. Of course there is also Perl6 Essentials. :) We DO need a Parrot Programming FAQ. I propose the IMCC faq functions as just that. We can still have the current Parrot FAQ, which is great at answering generic questions for people who haven't swallowed the blue pill yet. (Or was it the red pill, I vaguely remember Dan in a black trenchcoat and glasses, and guns.) -Melvin
Re: Some PIR "How do I?" questions
At 06:49 PM 11/24/2003 +0100, Jerome Quelin wrote: Jerome Quelin wrote: > Melvin Smith wrote: > > I just checked in an intial IMCC faq (parrot/imcc/docs/imcfaq.pod) > Great job. Attached you'll find some corrections for typos. And of course I forgot the patch file. Here it is. Thanks, applied! Also I added another sample that might interest compiler writers that may be new to Parrot/IMCC syntax. It just shows one approach to compiling a class with fields, methods and instantiating them. The syntax is a workaround for the lack of explicit class/object support in Parrot. -Melvin
Re: Some PIR "How do I?" questions
At 06:48 PM 11/24/2003 +0100, Jerome Quelin wrote: Melvin Smith wrote: > I just checked in an intial IMCC faq (parrot/imcc/docs/imcfaq.pod) Great job. Attached you'll find some corrections for typos. Thanks. BTW Robert has linked the HTML FAQ to the site. You can get to it from the Resources tab, or just go straight to http://www.parrotcode.org/faq/imcc I much prefer reading it in a browser. > Feel free to send me more questions for the FAQ. Ok, here are some more (some questions are easy - but a faq should also have some easy items -, others are not yet possible, but anyway): * How does one retrieve arguments given on the command-line? * How does one call a function? With arguments? With a variable number of arguments? Get the return value(s) of a sub? * How does one read from a file? stdin? unbuffered stdin? * How does one create socket and network stuff? * How does one launch an exception? trap an exception? * How does one create a class? instanciate an object? I'll get to work on these. Look for something tonight or tomorrow. Some of this refreshes my memory on things I need to implement. -Melvin
Re: PIO_eof
At 11:45 AM 11/24/2003 +0300, Vladimir Lipsky wrote: Hi everyone! In t/src/io.c, specifically test 9 and 10, I wonder if the PIO_eof check up works anywhere; because I didn't manage to find any place where the PIO_F_EOF flag is set up when the PIO_*_open function fails neither in io_stdio.c, io_unix.c nor io_win32.c In PIO_open the ParrotIO object never gets created if there is an error condition. PIO_eof() returns true if the PMC has a null IO object or if the PIO_F_EOF flag is set. Is the "io == NULL" testing prohibited in Parrot embedding system? 0x4c56 Currently it isn't prohibitied. That is the only way to detect an error from PIO_open* at this time. If someone convinces me that it is better to return a ParrotIO object with error flags set rather than skip creating it, I might consider it. It is probably more likely that we simply need documentation. -Melvin
Re: Some PIR "How do I?" questions
At 03:18 PM 11/21/2003 -0500, Dan Sugalski wrote: These could use some documenting (and yes, I know the answer to many) for future use for folks generating PIR. (Hint, hint -- documentation is a good thing) *) How do I declare an externally visible subroutine? *) How do I store a global variable *) How do I load a global variable *) How do I call an external function *) How do I get the sub pmc for a sub declared later (or earlier) in the file? *) How do I (or even can I) have a file-scoped variable? (like .local, only for all code in the file) I just checked in an intial IMCC faq (parrot/imcc/docs/imcfaq.pod) Hopefully we'll get it linked up in HTML format to the web site asap. The FAQ is small, but it at least answers all of the above. I did not go in too much depth about subs because I've got a wave of changes to commit soon, so I want to hold off on documenting syntax that is about to be deprecated. Feel free to send me more questions for the FAQ. -Melvin
Re: Bytecode portability and word/int sizes
At 01:07 PM 11/23/2003 +0100, Leopold Toetsch wrote: Melvin Smith <[EMAIL PROTECTED]> wrote: > At 11:34 PM 11/22/2003 +0100, Leopold Toetsch wrote: > Ix regs are for: > 1) Fast integer stuff > 2) Iteration (increment variables) > 3) Conditional checks > 4) Branching and holding addresses > 5) Indexing arrays and strings While the operands clearly are of wordsize (opcode_t) I doubt that it makes sense to say, that arrays or strings can hold max-INTVAL items. Maybe not, but the point is that you want the efficient wordsize for your INTVAL in those cases (ie. _not_ 64-bit on 32-bit architecture, or vice-versa). > All of this works out fine if we use the native size, and none of this > should be penalized (for example if we decided to use "default" 64-bit > INTVAL registers > on a 32-bit CPU, this is penalization -- which I am NOT proposing). The question is: Do we want to support a configuration with 32-bit opcode_t and 64-bit INTVALs? Perl5 has 64-bit IV support, but it doesn't has native data types. Perl6 may have both. The languages, we are running will define, if we need such a configuration. The languages don't get to decide for Parrot, though. Parrot should guarantee existence and size of INTVAL and HUGEINTVAL registers. How a HLL maps to Parrot registers (or PMCS) is its own business. To your 1st question, I don't think we should support any case where sizeof(opcode_t) != sizeof(INTVAL). Both of these should be the most efficient wordsize, so on each platform they should match. The 2 cases you are describing would be: a) 64-bit INTVAL on 32-bit platform (suboptimal) & 32-bit opcode_t (ok) b) 64-bit INTVAL on 64-bit platform (ok) & 32-bit opcode_t on 64-bit platform (suboptimal) Neither of these is a valid configuration IMO. **We should always guarantee both opcode_t & INTVAL will be optimal for speed. We never have to guarantee HUGEINTVAL will be. -Melvin
Re: Bytecode portability and word/int sizes
At 10:14 PM 11/22/2003 -0500, Melvin Smith wrote: At 11:34 PM 11/22/2003 +0100, Leopold Toetsch wrote: Melvin Smith <[EMAIL PROTECTED]> wrote: > to override it, it is not supported to choose INTVAL > OPCODE, though > the inverse is. So storing it in the header is probably redundant, unless > we change the rules. Your conlusion above is just reverse. The whole machinery relies on My conclusion isn't reverse, it is the current state of Parrot, is it not? Arg, you are correct, I must be dyslexic. I re-read my original note, even after you pointed it out and still read it in reverse. How embarassing. :( I hope you did get the basic idea of what I "thought" I was saying, since we were actually agreeing, even if I was convinced that we were not. -Melvin