This week's Perl 6 Summary
The Perl 6 Summary for the week ending 20030209 Welcome to the latest Perl 6 summary, your handy cut out and keep guide to the goings on in the crazy world of Perl 6 design and development. It's been a rather quiet week this week; only 75 messages in perl6-internals and a mere 57 in perl6-language. So, at least it's palindromic. We start off, as is traditional, with perl6-internals The 2004 Performance challenge Dan announced that he'd made a bet with Guido van Rossum that Parrot would be faster at executing a pure python benchmark of some sort to be determined. The details of the challenge will be announced at OSCON 2003 and the results at OSCON 2004. Dan stands to lose $10 and a round of beer for the Zope/Pythonlabs folks. (Dunno how many of them there are though...). We don't know what he stands to win yet, but I'd hope 'beers from each of the Zope/Pythonlabs folks' are included. For some reason nobody commented on this. http://makeashorterlink.com/?K2C815C63 More Parrot Objects Jerome Quelin wondered how Parrot objects would handle property inheritance. Dan pointed out that properties don't get inherited and Jerome realised he meant to ask about attribute inheritance. Attributes *are* inherited but are mostly invisible to any methods but the methods of the class that defines the attributes (though they will be accessible (presumably through reflection)). In another subthread, we got confused by multimethods. http://makeashorterlink.com/?O6D816C63 http://makeashorterlink.com/?O4E822C63 -- Multimethods Bytecode Metadata James Michael DuPont wanted to know what had been decided about Bytecode metadata and outlined the things that he'd like to know about a given bytecode. Leo Tötsch reckoned that what James wanted was either in the bytecode right now, or was handleable by the framework that was in place. He pointed James to docs/parrotbyte.pod in the Parrot distribution. Further discussions centred on the *amount* of metadata and whether this would affect loading speed and efficiency, or get in the way of the desired 'mmap and go' principle. Jürgen Bömmels pointed out that we also had to be capable of passing meta data from a compiler 'through' IMCC and on to the final bytecode. There was also a touching reunion between James Michael DuPont and Gopal V. Ah... http://makeashorterlink.com/?E41925C63 Multi programming language questions Phil Hassey has been lurking, fascinated on the internals list for a couple of months. This week he broke the silence by asking a couple of questions about cross language function dispatch, string compatibility and global scopes. For instance, PHP, Common Lisp and others are case insensitive about functions. Which is okay when you're calling such a function from a case sensitive language, but can be tricky if you call out from a case insensitive to a case sensitive language. Dan thought that there wasn't going to be much that could be done about this problem (at least, not transparently) but seems to think that the other issues raised shouldn't be too problematic. http://makeashorterlink.com/?R32931C63 Random questions David popped up and, after remarking on the amount of progress Parrot had made since he last looked at it, had a few questions about various bits and pieces. Leo and Dan provided a bunch of good answers. http://makeashorterlink.com/?X63912C63 A Scheme for extending core.ops Leo Tötsch seems to have got bored of 'just' being the Patch Monster and has been producing some discussion documents proposing all sorts of useful tricks for improving the design/speed/extensibility of Parrot's internals. This week he turned his attention to core.ops. His plan involves a way of reducing the size of core_ops, improving cache locality and generally solving a raft of problems I didn't even know Parrot had. His new scheme allows for a small core.ops, easy extension and no execution speed penalty for non core ops. As is usual with Leo's stuff, the scheme came with code. Gregor had a bunch of observations and suggestions and he and Leo thrashed out a slightly modified scheme for going forward. http://makeashorterlink.com/?P54921C63 Week of the alternative runloops Leo Tö offered a couple of different core runloops this week. First up was the Computed Goto Prederefed (CGP) runloop which, essentially combined a two runloop optimization techniques to provide what can only be described as massive speedups. The -O3 compiled version ran parrot_mops 6 times faster than the basic 'fast_core' and more than 3 times faster than the previous fastest core. People were impressed. A few days later, Leo reached into his bag of tricks and pulled out the CSwitch runloop that did
This weeks' summary
The Perl 6 Summary for the week ending 20030202 Welcome to the second Perl 6 summary of the Copious Free Time era and already I've broken the 'mailed out by Monday evening' promise. There were reasons however, mostly to do with going down to London to do the paperwork for my redundancy stuff. So, I'm emending the promise to 'mailed out by logical Monday evening'. Starting with perl6-internals (which was pretty quiet this week, only 76 messages compared to perl6-language which tipped the scales with 162) Parrot Objects (noun, not verb) Piers Cawley worried about class private attributes and asked it it was still going to be possible to write an object serializing tool that wouldn't require tons of Perl class overloads. Dan said that it should still be possible. http://makeashorterlink.com/?R34F24653 The packfile patches, an ongoing saga. Leo Tötsch spent the week working on the packfile format and Parrot's tools for manipulating it. Various internals folks provided feedback, pointers to standards and other handy feedback. http://makeashorterlink.com/?V25F31653 http://makeashorterlink.com/?G47F21653 http://makeashorterlink.com/?W28F35653 http://makeashorterlink.com/?O19F21653 Securing Parrot ASM Thomas Whateley has been thinking about how to allow Parrot to run untrusted code in a sandbox, and proposed a system where code would provide a list of required rights and Parrot would guarantee that only those privileges were provided. Matthew Byng-Maddick pointed everyone at the RFC he'd written along these lines and raised the issue of what happens when you call out to a module that's written natively, which he described a 'brick wall'. Others were less sure that this was a showstopper. Somewhat annoyingly my mailer seems to have had a bad moment and lost some messages, including one from Dan in which he outlined his vision for 'safe mode' and gave some pointers to the VMS docs in this area. http://makeashorterlink.com/?W3AF52653 http://dev.perl.org/rfc/353.pod http://makeashorterlink.com/?D2BF12653 -- Dan's outline Parrot Developer World Map Leon Brocard announced the Parrot Developer World Map, which plots the locations of all the Parrot developers who sent him their location details on a map of the world. A cursory glance at the map would seem to imply that a good place for a Parrot Developers' face to face meeting would be a Caribbean island. Gregor neatly one upped Leon with his interactive SVG version. http://www.astray.com/parrot/worldmap/ http://makeashorterlink.com/?O2CF12653 Coroutine context patch Jonathan Sillito found some problems with the coroutine code, so he wrote a test exposing the errors, patched things to fix the errors and added a documentation patch to be on the safe side. Go Jonathan. Steve Fink applied the patches. Jonathan also had a question about naming, and wondered if anyone would be interested in a howto for compiler writers. Nobody commented on this, but the howto sounds like a great idea to me. http://makeashorterlink.com/?T4DF21653 Multiple code segments and the interpreter Leo Tötsch noted some problems with interpreter-code and multiple code segments, suggested a way forward and asked for comments. Nicholas Clark commented, wondering about the potential for memory leaks but Leo seemed to think that that was covered. http://makeashorterlink.com/?Q3EF15653 Parrot run loop problems Leo has also been thinking about Parrot run loops and reckons we're doing it wrong. So he proposed a different way of doing things, working through its impact on the various kinds of Parrot core. Steve Fink liked Leo's idea, but Jason Gloudon was less sure. http://makeashorterlink.com/?T1FF53653 Meanwhile, in perl6-language More Array Behaviours Michael Lazzaro is working on documenting Perl Arrays and asked for confirmations and clarifications on various areas of Array behaviour. One subthread morphed into a discussion of when to use is properties and when to use but properties. It turns out that the 'is is compile time, but is runtime' rule of thumb is only a rule of thumb. Damian says that a better distinction is 'but is a property on a value, is is a property on a referent'. Variables, arrays, opaque objects, hashes etc are referents and therefore get is properties. Values are thing things that are held in variables, arrays, opaque objects etc, and they get but properties. http://makeashorterlink.com/?O20021753 Spare Brackets Martin wondered if it would be possible to use [] to do both hash and array lookups. Short answer No. Which isn't to say the thread didn't take off like a runaway train. After far too many messages we reached the same 'No' conclusion that Damian
This week's Perl 6 summary
The Perl 6 Summary for the week ending 20030119 Summary time again, damn but those tuits are hard to round up. Guess, what? perl6-internals comes first. 141 messages this week versus the language list's 143. Objects (again) Objects were still very much on everyone's mind as the discussions of Dan's initial thoughts about objects in Parrot continued. Jonathan Sillito put up a list of questions about Dan's original message which Dan answered promptly. Down the thread a little Dan mentioned that he hoped Parrot's objects would serve, reasonably unmodified for a bunch of languages (ie, he hoped that there wouldn't be a requirement for PythonRef/Attr/Class/Object etc), Chris Armstrong thought that, given what Dan had outlined so far, that wouldn't be straightforward. Dan thanked him for throwing a spanner in the works, asking for more details which Chris provided. Meanwhile Jonathan had some supplementary questions... Hmm... doing this blow by blow will take forever. Suffice to say that details are being thrashed out. At one point Dan's head started to spin as terminology mismatches started to bite, leading Nicholas Clark to suggest an entirely new set of terms involving houses and hotels (but with some serious underpinnings). http://makeashorterlink.com/?Y21952033 -- thread root, from last week. http://makeashorterlink.com/?M52912033 -- Jonathan's questions http://makeashorterlink.com/?A23912033 -- Chris throws a spanner http://makeashorterlink.com/?Z44922033 -- Nicholas tries for a monopoly on silliness Optimizing and per file flags Nicholas Clark wrote about requiring the ability to adjust compiler optimization flags on a per file basis (brought up by Dan on IRC apparently) and proposed a scheme. Quote of the thread (and quite possibly the year so far): When unpack is going into an infinite loop on a Cray 6000 miles away that you don't have any access to, there isn't much more you can do. Thanks for that one Nick. http://makeashorterlink.com/?I15965033 The draft todo/worklist Dan posted his current todo/worklist, which he described as reasonably high level, and a bit terse. I particularly liked the last entry Working Perl 5 parser. Surprisingly, there was very little discussion, maybe everyone liked it. http://makeashorterlink.com/?L56921033 Parrot Examples Joe Yates asked if we could add a helloworld.pasm to parrot/examples/assembly. Joseph Guhlin wondered what was so special about print Hello, world\n end that it would need a file of its own (though he did forget the end in his post, and segfaults are not really what you want in sample code). http://makeashorterlink.com/?E27912033 Thoughts on infant mortality (continued) Jason Gloudon posted a wonderfully clear exposition of the problems facing anyone trying to implement a portable, incremental garbage collector for Parrot which sparked a small amount of discussion and muttering from Dan about the temptation to program down to the metal. http://makeashorterlink.com/?X38931033 Operators neg and abs in core.ops Bernhard Schmalhofer posted an enormous patch adding neg and abs operators to core.ops. There were a few issues with the patch so it hasn't gone in yet and an issue with what underlying C functions are available reared its head too. http://makeashorterlink.com/?L29951033 The eval patch Leo Tötsch seems to have spent most of the week working on getting eval working and he opened a ticket on rt.perl.org to track what's happening with it. The response to this can be summarized as 'Wow! Fabulous!'. Once more, for Googlism, Leopold Toetsch is my hero. http://makeashorterlink.com/?I5A922033 Pretty Pictures Mitchell N Charity posted some pretty pictures that he'd generated with doxygen and graphviz. Most of the responses to this suggested he use different tools. Ah well. http://makeashorterlink.com/?B2B921033 Solaris tinderbox failures Andy Dougherty created an RT ticket for the Solaris tinderbox, which have been failing with the delightfully useful 'PANIC: Unknown signature type and wondered if things could be fixed up to be a little more informative. Apparently it was as issue with Leo's recently checked in eval patch. So Leo fixed it. http://makeashorterlink.com/?D1C925033 Parrot compilers Cory Spencer wondered about how the current compilers that target parrot work, noting that they seem to be duplicating a good deal of labour, and wondered if anyone had worked on a gcc like framework with a standardized Abstract Syntax Tree (AST). Everyone pointed him at IMCC. Gopal V also pointed out that, given the variety of implementation languages (C, Perl, Parrot...)
This week's Perl Summary
The Perl 6 Summary for the week ending 20030112
... and we're back. Yup, it's summary time again. We'll dive straight in
with perl6-internals (as if you expected anything else).
More thoughts on DOD
Leopold Tötsch posted a test program showing the effects of PMC size and
timing of garbage collection and allocation and suggested ways of
improving the GC system based on the conclusions he drew from its
results. Leo, Dan and Mitchell N Charity discussed this further and
tried a few different approaches to try and improve performance (though
Mitchell did worry about premature optimization). Work in this area is
ongoing.
http://makeashorterlink.com/?Y3EB16513
The Perl 6 Parser
Dan asked about the current state of the Perl 6 parser, wanting to know
what was and wasn't implemented and wondered about adding the Perl 6
tests into the standard parrot test suite. Sean O'Rourke and Joseph F.
Ryan both gave a summaries of where things stood. Joseph also suggested
a few refactorings of the parser to deal with the fluidity of the
current spec (almost all the operators have changed symbols since the
parser was first written for instance).
http://makeashorterlink.com/?I6FB35513
LXR - Source code indexing
Last week I said that Robert Spier had 'started work on getting a
browseable, cross referenced version of the Parrot source up on
perl.org'. What actually happened was that Robert asked Zach Lipton to
do the work. This week Zach delivered the goods which, I must say, look
fabulous.
I'm sure that if someone were to extend LXR so it had a better
understanding of .pasm, .pmc, .ops and other special Parrot source types
then the community would be very grateful indeed. I know I would.
http://makeashorterlink.com/?K10C23513 -- Announcement
http://tinderbox.perl.org/lxr/parrot/source
Thoughts on infant mortality
Piers Cawley offered what he thought might be a new approach to dealing
with the infant mortality problem which got efficiently shot down by Leo
Tötsch. Which led to further discussion of possible answers, and it
looks like Leo's proposed solution involving a small amount of code
reordering and early anchoring will be the one that's tried next. All
being well it won't require walking the C stack and hardware register
set, which can only be a good thing.
Later on, Leo asked if it'd be okay to check in his work so far on
redoing the GC because he was up to 15 affected files and was starting
to worry about integration hell. Steve Fink wasn't sure about one of his
changes, so Leo checked everything else in.
http://makeashorterlink.com/?K41C21513
http://makeashorterlink.com/?S62C23513
Objects, finally (try 1)
Last week I mentioned that Leon Brocard's wishlist for the next Parrot
iteration included Objects. This week Dan posted his first draft of what
Parrot Objects would and wouldn't be able to do. The eleventh entry on
Dan's list (Objects are all orange) seemed to be custom made to please
Leon. There was a fair amount of discussion (of course), but the
consensus was positive.
http://makeashorterlink.com/?S13C31513
The benchmarking problem
Nicholas Clark crossposted to p5p and perl6-internals to discuss the
problems of benchmarking parrot against perl 5. One of parrot's aims is,
of course, to go faster than perl 5. The problem is, how do you measure
'faster'? Nicholas has been working on making perl 5 go faster and was
distressed to find out that using 'perlbench' a patch of his went 5%
faster, 1% slower, 0% and 1% faster, depending on what machine/compiler
combination he ran the benchmark on. Leo Tötsch commented that he'd
found performance varying by over 50% in a JIT test, depending on the
position of a loop in memory. Andreas Koenig commented that he'd come to
the conclusion that bugs in glibc meant that there was little point in
benchmarking Perl at all if it was built with a glibc older than version
2.3 (apparently malloc/free proved to be gloriously erratic...) I'm
afraid not much was actually resolved though.
http://makeashorterlink.com/?L64C22513
Meanwhile, in perl6-language
The discussion of Variable types vs Value types continued from the
previous week. Dan opined that Arrays weren't necessarily objects, which
brought forth squawks from Piers Cawley who pointed out that being able
to do:
class PersistentList is Array {
method FETCH ($index) {
...
}
...
}
would be much nicer than tying a value in the Perl 5ish fashion. Dan
reckoned that delegation would probably be enough which, IMHO seemed to
miss the point. Various other people chimed in to, essentially, tell Dan
that he was wrong, but I'm not sure Dan agreed with them.
Meanwhile, in a
This week's summary
The Perl 6 Summary for the week ending 20030105 Hello and welcome to the first summary of 2003, welcome to the future. This summary covers 2 weeks, but they've been quietish what with Christmas and the New Year. So, starting as usual with perl6-internals A pile of patches to the Perl 6 compiler Joseph F. Ryan submitted a bunch of patches to the Perl 6 mini compiler, (found in the languages/perl6 subdirectory of your friendly neighbourhood parrot distribution) mostly implementing the the semantics for string and numeric literals discussed on perl6-documentation. Garbage Collection headaches Heads have been put together in an attempt to get Parrot's Garbage Collection system working efficiently and accurately (no destroying stuff before anyone's had a chance to use it, dammit!) It appears that there's still a good deal of head scratching to be done in this area (the chaps over on the LL1 list are wondering why we aren't just using the Boehm GC system...) I freely admit that GC makes my head hurt (especially as, in my current Perl 5 project I'm busy implementing mark and sweep collection for a persistent object store whilst also making sure that my random assortment of circular data structures has weakened references in just the right places so that stuff gets destroyed but only when it *should* be destroyed... Boy, am I looking forward to Perl 6 and not having to worry about this stuff ever again...) but I I'll have a go at summarizing the issues. The main problem appears to be that of 'Infant mortality', an issue that I will now attempt to explain. All the objects in memory can be represented as nodes in a graph, and the pointers between those objects can be represented as edges in that graph. The process of garbage collection involves taking a subset of those nodes (the rootset) and freeing (or marking as freeable) all those nodes in the graph which are not reachable from the rootset. Now, consider a function that sets up a new PMC, specifically a PMC that contains another PMC. The first step is grab the memory for our new PMC. Next we create the contained PMC, a process which allocates more memory... and there's the rub. Garbage Collection can get triggered at any point where we go to allocate more memory; unless the *containing* PMC is reachable from the rootset then it will get freed at the point. And that leads to badness. So the Infant Mortality problem can also be thought of as the problem of rootset maintenance. Which is, in theory, simple; just treat all C variables as members of the rootset. However, in practice it isn't that simple, mostly because hardware registers complicate the issue. Steve Fink offered an overview of the issues and some of the possible approaches to dealing with them, which sparked a fair amount of discussion amongst those who understood the issues. http://makeashorterlink.com/?K2FE52303 http://makeashorterlink.com/?Y20F32303 -- Steve's overview Variable/value vtable split Leo Tötsch posted a summary of where we stand on doing the variable/value vtable split, suggesting that he wanted to start feeding in patches soon. Mitchell N Charity supplied a handy dandy 'context' post with links to appropriate articles, and he and Leo did a certain amount of thrashing out of issues. http://makeashorterlink.com/?B11F21303 http://makeashorterlink.com/?G12F32303 Parrot gets another new language Ook! Jerome Quelin offered an implementation of the latest silly language, Ook! which can be thought of as brainf.ck for Librarians. Due to insanity, the Ook! compiler is implemented in Parrot assembly, and emits parrot assembly too, which led Jerome to ask for an 'eval' opcode. Which Leo promised to supply. And which Dan specced out in PDD6. All of which led Leo to comment that, for all these languages are toys, they do seem to be driving the implementation of important bits of Parrot. Nicholas Clark reminded everyone that a zcode interpreter would be another good thing to have a crack at because it would require a couple of other really useful bits of Parrot functionality. Ook! is now in the core. http://makeashorterlink.com/?R53F23303 Returning new PMCs David Robins wondered what was the resolution about creating and returning a new PMC in PMC ops that take a PMC* dest parameter. He and Dan discussed it back and forth and it became apparent that Dan really needs to get Parrot Objects defined... http://makeashorterlink.com/?Q24F21303 Fun with PerlHash Jerome Quelin noticed that you couldn't delete an item from a PerlHash. Leo fixed it. Jerome later asked how one could retrieve the keys of a PerlHash in Parrot assembly and wondered if there was a way to traverse a hash. Sadly the
This week's summary
The Perl 6 Summary for the week ending 20021208 Another Monday evening. Another summary to write. Starting, as is becoming tediously predictable, with perl6-internals. Another JIT discussion Toward the end of the previous week, Leopold Tötsch posted something about the latest round of changes to the JIT core. Daniel Grunblatt was concerned that the current JIT wasn't doing the right thing when it came hardware register allocation and wanted to remove a some conditional logic. Leo didn't agree at first, but became convinced and Daniel's requested change was applied. http://makeashorterlink.com/?P4AB24AB2 Fun with IMCC Lots of things happened with IMCC this week: * David Robins posted a list of minor niggles (For instance, it turns out you can't ret early from a .sub) and suggested some remedies. Leo Tötsch mentioned that the IMCC Cabal (which would consist of Melvin Smith, Sean O'Rourke, Angel Faus and Leo if there were a Cabal. But, as everyone knows, There Is No Cabal) have been discussing several of these issues. http://makeashorterlink.com/?Q1BB22AB2 http://makeashorterlink.com/?W1CB43AB2 * Art Haas had problems building IMCC, apparently bison didn't like the imcc.y file. Leo tracked down the problem (at the second time of asking, I think he might be slipping) and checked in a working fix. * Leo Tötsch made a pile of changes to IMCC to eliminate clashes between Parrot's PASM language and IMCC's PIR syntax, which had made it hard to mix the two. Full details of the changes are in his post. Gopal V wondered if there was any way of feeding code to IMCC beyond simply writing to a file and running IMCC. He'd had to make a bunch of changes to the IMCC files that he used, and wondered if there was a Better Way. Actually he didn't so much wonder as propose the aforementioned Better Way, lifting ideas from DotGNU's treecc. He and Leo discussed things, worked out an interface and Gopal went off to implement something (Yay Gopal!) http://makeashorterlink.com/?Y1DB21AB2 http://makeashorterlink.com/?A6EB42AB2 * Steve Fink posted a patch implementing a first cut at namespace support in IMCC. He wasn't at all sure that what he'd implemented was the Right Thing, but it supplied what he needed for the time being (if that makes sense) in the regex engine. Leo reckoned that it looked okay, and promised to apply it if nobody hollered. He also pointed out some problems with the current regex implementation to do with reentrancy and memory leakage. It turns out that Steve was working on languages/regex rather than the rx_* ops, which are the ones that have the problems. http://makeashorterlink.com/?P2FB16AB2 * Gregor N. Purdy had some problems with IMCC's syntax, a fragment of code that he thought should work fell over in a heap. Both Mr. Nobody and Leo pointed out that IMCC expects subroutines, and you should wrap your code in a .sub/.end pair. Once Gregor had that straight he posted a Jako program and the IMCC code he thought the Jako compiler should generate from it and asked for any feedback before he went to change the compiler. Leo Tötsch provided some (I assume) useful feedback. A little later Gregor posted again, he was still having problems with IMCC not quite behaving as he wanted for the Jako compiler. He and Leo thrashed it out over a few messages and, to cut a long story short, IMCC looks like it won't be changing. I'm not sure whether Gregor is happy about this... http://makeashorterlink.com/?X30C22AB2 http://makeashorterlink.com/?J51C32AB2 * Mr Nobody posted a patch to get IMCC to compile under windows. Apparently the OUT label clashes with something in the windows header files. The patch got applied. * Gregor N. Purdy got a little confused by how IMCC generates PASM code, and posted some sample code, interspersed with questions. Which Leo answered. It's worth looking at this; it shows off the kind of optimization that IMCC gets up to. http://makeashorterlink.com/?E22C25AB2 http://makeashorterlink.com/?Y53C22AB2 PMCs are the thing Dan announced that he's finally stopped waffling and frozen the PMC structures `modulo the odd twiddling to it.' He's added a pmc.ops file, and has started adding in ops to manipulate PMC internals. Leo asked for some clarifications, got some, and then wondered what the final 'Parrot Object' will look like. http://makeashorterlink.com/?Z34C23AB2 logical_not issue David Robins is having fun with logical_not and Ruby. The issue is that *all*
