Re: Input / Output encoding filters.
Just to add my 2 cents here, I've always felt that there should be basic primitives provided, and the HLL can take care of the rest. Technically the low-level IO routines don't have to know about encoding or compression schemes at all. A VM provides the building blocks, and you can add whatever abstractions you want on top. Obviously there is another camp that feels that everything needs to be implemented in low level C and special PMCs for each HLL and funky APIs everywhere, and those are probably the same guys that think Parrot needs to worry about the issue you are discussing. I'm not in that camp, which is the reason I just watch nowadays. To me, the elegance of the pure VM has been lost on Parrot. -Melvin "Its 2006, do you know where your opcodes are?" At 06:01 PM 2/21/2006, Martin D Kealey wrote: I'm a bit slow coming back to this, sorry. It seems that "seek" is used in two ways: * returning to some previously identified point (including the start or end of the file) * moving a given number of characters you want to move relative to a known location Clearly you can always do the first, just by using the underlying byte offset without regard for the encoding. If you have a fixed number of bytes per character then you can trivially do the second as well. But if you have a variable-length encoding then you have to read through the byte stream to get to the position you want; this might or might not be desirable depending on the characteristics of the underlying stream. Furthermore it makes (some) sense always to be able to seek *forwards* -- even on a tty device -- but not backwards. So my suggestion is that we change the interface to "seek", and have separate parameters for the "previously known position" and the "character offset". The latter is obviously just an integer, but the first is a black-box token -- maybe a PMC, but more likely a mangled integer -- to ensure that the two args are distinguishable. (Please excuse me as I discuss this in terms of a HLL rather than Parrot...) In other words, change this: $fpos = $io.tell(); $io.seek(SEEK_SET, $fpos) to this: ... $io.seek($fpos, 0) or for brevity, just this: ... $io.seek($fpos) Now SEEK_SET, SEEK_CUR and SEEK_END just become special cases of "previously known positions". And I'm tempted to say that they should be spelt "0", "undef" and "-1" respectively. Thence it's fairly straightforward for the units of "seek" to be whatever you find convenient: counting whole records, or lines of text, or whatever. Clearly this needs to be discussed in p6-lang, but having separated the two parameter types, the filter can decide which it can implement, and how. -Martin
Re: Shootout Updates Posted
Also, unless things have changed, multiple op dispatch cores are built in standard Parrot. I'm behind on my reading the list, so someone will correct me if I missed it, but last time I worked with the code there were 5 cores, just for experimental reasons. Ideally, for production the config will choose one safe core as well as the JIT which results in a much smaller binary and resident size. -Melvin At 05:23 AM 12/20/2005, Leopold Toetsch wrote: On Dec 20, 2005, at 4:01, Joshua Hoblitt wrote: I'm surprised that Parrot's memory usage is so high relative to other platforms. It looks like a hog even relative to Java. Has anyone taken a serious look at where all that memory is going? Memory usage as well as startup time aren't optimized at all. One thing comes to my mind immediately: the MMD_table takes more than 1 MB and setup time is in prominent position in profile logs. -J leo
Re: Copyrights
I originally wrote IMCC was a separate module, and I even released it on CPAN. For a long time it lived in languages/imcc but Leo rolled it in at some point. Copyrighting my code does not limit my ability to contribute it to a project under a dual copyright. I have never heard what the final decision was about copyrights, but at some point it was a choice of Artistic, LGPL, or some other. The only license I really have a problem with is the plain GPL, which I don't think Perl uses anyway. When someone makes a decision, or informs me of the existing policy, I'll be happy to commit a header that licenses all of those files to the Perl Foundation under whatever copyright they choose. -Melvin At 10:46 AM 7/3/2005, Will Coleda wrote: I was under the impression that for any code to be included in the parrot repository, that the copyright had to be assigned to the Perl Foundation. But there are exceptions to this throughout the repository, even in core files like imcc/main.c (Copyright Melvin Smith). We have a file with a Microsoft copyright in the repository. What's the official policy? Is it documented somewhere in the repository? (If not, can we get it documented?) Regards.
Re: Attack of the fifty foot register allocator vs. the undead continuation monster
At 01:16 PM 6/12/2005, MrJoltCola wrote: At 07:15 AM 6/12/2005, Chip Salzenberg wrote: 1) As far as variable lifetime, the brute-force method would assume lifetime windows (du-chains) from the first definition of each variable to the last function call in a basic block. Horrible for optimization. I said that wrong. If a standard du-chain intersects the window between first function call and last function call in a basic block, you probably have to extend the du-chain to the last call in the block. -Melvin
Re: Attack of the fifty foot register allocator vs. the undead continuation monster
At 07:15 AM 6/12/2005, Chip Salzenberg wrote: Therefore, register allocation must allow for implicit flow of control from *every* function call to *every* function return ... or, more precisely, to where *every* continuation is taken, including function return continuations. Yes. But for casual readers, you might want to qualify that with: "For a given basic block..." Just a couple of thoughts to add: 1) As far as variable lifetime, the brute-force method would assume lifetime windows (du-chains) from the first definition of each variable to the last function call in a basic block. Horrible for optimization. 2) For languages without eval() it is possible simply to do compile time analysis of routines and associate an AST attribute with whether the routine captures or invokes a continuation, and for languages with Continuation Passing Style calls, you can analyze whether the function does anything but use Return Continuations. This would give the register allocator enough information to properly break down the basic blocks into correct DU-chains and allocate with confidence. However, given Parrot's intended target languages, I feel that (2) is going to be rarely applicable. -Melvin
Re: Regarding Google's Summer of Code 2005
At 01:40 PM 6/3/2005, Matt Fowles wrote: > I have been working and thinking about improvements to the Parrot/IMCC > optimizer for a while now. Implementing SSA is definitely at the top > of my list, because it simplifies a lot of optimizations and makes > some others possible. The biggest challenge is to graft such a big > change onto a working system, with small changes at a time. I know that Melvin Smith (aka Mr JoltCola) was planning on a full scale rewrite of imcc. His plan involved SSA. I don't know what came of that; I think that he ran out of tuits... I ran out of tuits and interest, though I still lurk just to see whats going on. Parrot started moving in a direction I didn't like and I got tired of arguing to justify my patches. 2 yrs ago I just went back to doing private research projects on my own. From my point of view, the Parrot development community lacks adequate sense of humor and spirit of comradery for me to participate. I decided just to work on my Cola language and write a specific VM for it with the lessons I learned from Parrot. I still like register based VMs (memory transfer machines) but by working solo I can play with ideas like infinite register numbers and a more uniform, compact, RISC like instruction set encoding. My only goal is self education, so this works out better for me. Good luck guys, -Melvin
Re: [PATCH] make PIO_fdopen return NULL on incorrect flags
Actually no, from the PIO routines you should return a PMC that has a null handle, or that was my original intent. I think I was considering changing new_io_pmc() to return something like ParrotUndef in that case but never did. The very first version of the IO routines actually did return NULL for every error, but it was decided that wasn't the cleanest way. If you look at the revision history of io.c, revision 4673, I commented: >> Bug fixes. We should not be returning NULL for io PMCs, that generates an exception now.. A PMC should be returned with a NULL ParrotIO * for valid eof/bool tests. << As this was 2003, I honestly don't remember all of the discussion. There might be a record of it between Dan and me in the archives, however, chances are it was on IRC. -Melvin At 07:44 AM 5/5/2005, Leopold Toetsch wrote: Luke Palmer <[EMAIL PROTECTED]> wrote: > This patch makes PIO_fdopen return NULL when you give it flags it > doesn't understand. I gathered that this was the correct behavior > from the test that I untodid. > Luke Thanks, applied. leo
Re: parrot and refcounting semantics
At 01:10 PM 4/28/2005, Dan Sugalski wrote: At 5:57 PM +0200 4/28/05, Robin Redeker wrote: On Wed, Apr 27, 2005 at 03:43:32PM -0400, Dan Sugalski wrote: At 5:40 PM +0200 4/27/05, Robin Redeker wrote: The expense is non-trivial as well. Yeah, it's all little tiny bits of time, but that adds up. It's all overhead, and useless overhead for the most part. Yes, but do we know whether refcounting is really slower than a garbage collector in the end? Yes, we do. This is a well-researched topic and one that's been gone over pretty thouroughly for the past twenty years or so. There's a lot of literature on this -- it's worth a run through citeseer for some of the papers or, if you've a copy handy in a local library, the book "Garbage Collection" by Jones and Lins, which is a good summary of most of the techniques, their costs, drawbacks, and implementation details. Research indicates that in theory, garbage collection can even be faster than explicit memory management. One paper I remember in particular was done by IBM using various GCs for the JVM. -Melvin
Re: [RFC] .local, .syn, etc.
At 04:52 PM 4/18/2005, chromatic wrote: On Mon, 2005-04-18 at 14:44 +0200, Leopold Toetsch wrote: > Yep. As a first step, I'd redefine this to be C<.label>: > >.macro SpinForever (Count) > .label $LOOP: dec .COUNT# ".label $LOOP" defines a local label. >branch .$LOOP # Jump to said label. >.endm Can you only use .label inside of macros? (I don't know how useful it is outside of macros, but it's a question.) Currently yes (assuming you made .local = .label) because the macros are handled and expanded in the lexer only before the parser gets hold of it, so IF my memory serves, there is no token in the parser for anything inside a macro. I tried to yank macros out once and there was much screaming and wailing. :) -Melvin
Re: Parrot and the web (PHP?)
At 03:49 PM 4/13/2005, BÁRTHÁZI András wrote: I'm not a UNIX guru, but I don't know an easily installable solution for the problem. I would like to run just one Apache, and would like to run Perl as an Apache module. Chroot I think is not a solution for it. Running the script as CGI or running as much Apaches as much client you have is not a solution for me and for a lot of people. PHP offer an easy way to solve this problem. You obviously are talking about a web hosting environment with multiple applications and customers. I did not mean that chroot() was the solution, it is however part of the solution on the UNIX environment if you want proper security. Perl was the most famous web development environment some year ago, today PHP is that. I think one of I disagree. How do you support that blanket statement? I don't have any customers using PHP, they either use Java (some J2EE container, Oracle, BEA, Websphere, Tomcat) or they use Perl (CGI or mod-perl), etc. I agree with your statement if you are talking about small web-site hosting. 30 bucks a month for a website and you host 100 sites on a single shared server. PHP has a large share of that market, but for medium to large complexity apps, specifically commercial enterprise apps, PHP has very little presence. The marketing dollars are all behind Java and .NET. I guess it just goes to show we all swim in different ponds. Anyway, I did not mean to start a tangent, I want to see PHP run well on Parrot so we can agree on that. -Melvin
Re: Parrot and the web (PHP?)
At 02:33 PM 4/13/2005, Dan Sugalski wrote: At 8:25 PM +0200 4/13/05, BÁRTHÁZI András wrote: An other question is, that how can you tell to the platform, to limit these features, maybe non-modifiable environment variables and command line parameters can be the ways of it. For that you need a full-blown quota and privilege system. Luckily there are plans for one. :) As far as boxing a VM into a sub-directory, etc. UNIX (chroot) and VMS make this a breeze since the mechanisms are builtin to the OS, it is Windows where all the work has to be done. Maybe Windows has matured since the last time I looked at this sort of thing, but most sys admins I know still prefer to run their JVMs, app servers, etc. in a UNIX environment just for this reason. Solaris 10 just took it to a new level with "zones" although there have been similar patches out there for BSD and Linux for a long time. -Melvin
Re: imcc/ directory
At 06:57 PM 4/11/2005, Matt Diephouse wrote: Now that IMCC is a core part of Parrot, I'd like to see the imcc/ directory go away. I'd be willing to spend some time trying to prepare some patches (it'd be a good way to become more familiar with the source), but I have a few questions first: (1) Is this desirable? Technically IMCC should be separate I think mild separation that still exists is a good thing. IMCC does not actually execute anything, it is a "registered" front-end that handles PIR and PASM. You could add another front-end called PIR2 if you wanted. It just so happens that IMCC is the default front-end. (3) Should all the headers in imcc/ be moved to include/parrot/? (4) Should all the files be moved to src/? You should be able to build a standalone VM that will run pre-compiled bytecodes without including the full parser/compiler, IMO. I'd prefer to see the separation continue. Regards, -Melvin
Re: Blocks, continuations and eval()
At 10:03 AM 4/8/2005, wolverian wrote: To get to the real topic: In Perl 6, the generic solution to fix this (if one wants to fix it) seems, to me, to be to add a .eval method to objects that represent scopes. I'm not sure if scopes are first class values in Perl 6. Are they? How do you get the current scope as an object? Are scopes just Code objects? On #perl6, theorbtwo wasn't sure if .eval should be a method on coderefs or blocks. Is there a difference between the two? I always hated this about Ruby; there seems to be no practical value to the separation. Also, are blocks/coderefs/scopes continuations? Should .eval be a method in Continuation? I'm having a bit of trouble following you, but I can tell you that the VM portion treats continuations as well as lexical scopes or pads as first class Parrot objects (or PMCs). I cannot say how much Perl6 will expose to the high level language. Can you tell me what your idea of a "scope" is? I'm thinking a continuation, and if that is what you are thinking, I'm thinking the answer to your question is yes. -Melvin
Re: Monthly Release Schedule
At 12:32 PM 4/7/2005, Chip Salzenberg wrote: According to Peter Sinnott: > I set up a tinder server a couple of weeks ago. > Not sure if anyone else looks at it. > > http://unlinked.vm.bytemark.co.uk/tinder//parrot/status.html I understand that tinderbox is an automated system for test builds with result collation. Is there any need for a tinderbox central point being "official" (e.g. perl.org machine), or will a random server (no offense Peter :-)) do just as well? No security implications, etc? It was on parrotcode or dev.perl.org at some point. Maybe that can be reused? -Melvin
Re: Monthly Release Schedule
At 03:21 AM 4/7/2005, Leopold Toetsch wrote: MrJoltCola <[EMAIL PROTECTED]> wrote: > At 06:24 PM 4/6/2005, Chip Salzenberg wrote: >> * What platforms are required for release? I'd guess that we'd get >> almost of all of our developers (and users, for that matter) with: >> >> darwin >> linux-x86-gcc3.* >> win32-ms-cl > You should round that out with 64-bit Sparc. You are volunteering for running smoke tests regularly ;-) Patches to PLATFORMS are equally welcome. Which I still do periodically. I don't know when/why Sparc ever fell out of the mix. This was the reference platform for the byteordering patches. We should add linux-x86_64-gcc3.* to the list of primary targets. But we should make sure that these targets are really tested, at best with an official tinderbox setup again. Where did Tinderbox go anyway? I don't mind running a tinderclient at all. I can tell you now Sparc / GCC is broken for most due to our broken Configure. Our config pulls out the params that were used to build Perl with, and this is invalid because most Sparc folks are running a pre-built Perl and GCC binary that was built on a distributor's system. So for example, even though the system will have both Perl and GCC, Perl's config will say it was built with Sun's CC compiler. Configure fails and has to be hand hacked to work, or you have to build your own Perl to get the config before you build Parrot (not too tough, but should not be required). -Melvin
Re: Monthly Release Schedule
At 10:28 PM 4/6/2005, Chip Salzenberg wrote: According to MrJoltCola: > At 06:24 PM 4/6/2005, Chip Salzenberg wrote: > > * What platforms are required for release? I'd guess that we'd get > >almost of all of our developers (and users, for that matter) with: > > > > darwin > > linux-x86-gcc3.* > > win32-ms-cl > > You should round that out with 64-bit Sparc. How many Sparc developers have we got? I'm not sure, one or two, but I don't count as I'm not active atm, I just commentate. :) On related questions: Could you unpack for me the status of 32-bitness vs. 64-bitness in the Sparc world? Are all (most of) the chips 64-bit, and for how long? The kernels? The apps? All of the UltraSparc chips are 64-bit, which makes 64-bit Sparc about a decade old now, 1995 I think (since early Solaris versions, 2.5 at least). Virtually all of Sparc/Solaris is 64-bit nowadays, but there are quite a few hobbyists with pre-Ultra boxes still running out there. Anyway, Solaris supports 32-bit and 64-bit apps. The kernel is 64-bit. Companies like Oracle are starting to phase out releases for 32-bit versions of their products. 4 out of the 7 boxes in my office are 64-bit. All of my customers run big monsters (8-16 cpus with 64-128GB RAM) and as best I can tell Sun is still ruling the RISC/UNIX world even though the UltraSparc isn't the fastest per CPU. Its a darn fun chip to program for, though. However, I think the next big thing is Solaris on Opteron (64-bit) for 2-4 CPU midrange. I have told Leo and others that I will provide access to Solaris 10 on UltraSparc II and III processors when I get the time to build a zone on one of my boxes. I'll also probably make a Sun compiler available. I wish we could get tinderbox back up and running. I know there is a ton of Linux and OS X out there, but most mission critical apps are still running on Solaris, HPUX or AIX in my business, though I am seeing a slow adoption of Redhat Enterprise Linux. -Melvin
Re: Monthly Release Schedule
At 06:24 PM 4/6/2005, Chip Salzenberg wrote: * What platforms are required for release? I'd guess that we'd get almost of all of our developers (and users, for that matter) with: darwin linux-x86-gcc3.* win32-ms-cl You should round that out with 64-bit Sparc. -Melvin
Re: [perl #34669] [TODO] Make imcc.l compatible with modern flex
At 04:00 PM 4/4/2005, via RT wrote: # New Ticket Created by Chip Salzenberg # Please include the string: [perl #34669] # in the subject line of all future correspondence about this issue. # https://rt.perl.org/rt3/Ticket/Display.html?id=34669 > The creators of flex were rather mean to go and change the internal API without even incrementing the major version. But that's what they did. Now Parrot doesn't work with current flex. We should make imcc.l (and any other lexers) compatible with modern flex versions. Frankly I've never really like flex, but as long as we're using it, let's use the current version. Only reason I chose flex was that it is the only one I was able to compile on Win32 as well as UNIX. If you know of something better let me know because I'd love to use it for my current and future projects. I'm using flex++ on my current compiler because its nice to be able to easily instantiate multiple lexers without hacking the plain C flex, however lately I'm having second thoughts because it forces me to link in iostream which is huge. I think I'm going back to STDIO. -Melvin
Re: [Fwd: a warning and a failure for parrot in Tru64]
At 06:33 AM 4/1/2005, Nick Glencross wrote: Having never had access to a Tru64 system, does that mean that parrot is compiled 64 bit? Two initial comments: * This is a platform that we've not had a chance to test on, so I'm grateful to see it tested on a new platform. It was hoped that it would work, but to stop the test failing it was supposed to skip them until formally tested. Why did the skipping not work though? What would the intsize attribute be (from config_lib.pasm I guess) Not true. We've done successful compiles before on Tru64. Maybe as of 0.0.6 or something but we were using Compaq's testdrive system and 64-bit gcc. At one time Parrot dir run on both 64-bit Sparc and Alpha and I tested all these when doing the byte-ordering code. * Secondly, any chance I can get access to other platforms, such as this one, to troubleshoot the problem? If not, perhaps I can instrument the code and have the results sent back to me... Check out and see if testdrive is still available. I don't even remember the IPs or account info, nor do I remember the URL to go to but you can get free shell accounts if they still offer it. Also, I had set up an S/390 account at IBM for Daniel way back when and did a succesful compile there, but we never got the JIT written and it fell off the map. -Melvin
Re: Parrot bytecode reentrancy
At 05:57 PM 3/31/2005, Nigel Sandever wrote: Is Parrot bytecode reentrant? Yes. That is, if I want to have two instances of a class in each of two threads, will the bytecode for the class need to be loaded twice? No, just once. Also, will it be possible to pass objects (handles/references) between threads? Yes, otherwise threads are no more useful than processes. -Melvin
Re: Helping The Paranoid Parrot on Windows
At 04:44 PM 3/28/2005, Ron Blaschke wrote: MrJoltCola wrote: > At 04:08 PM 3/28/2005, Ron Blaschke wrote: >> > On the one hand, IMCC doesn't really help you _run_ code, so I'm not >> > inclined to see it part of libparrot. On the other hand, I haven't >> > grokked the entire code base organization, so I could be Greatly >> > Missing The Point. >>I have left things as-is while getting linkage correctly. That is, >>most of IMCC is currently part of libparrot (only F is not). >>In the beginning, I thought IMCC is part of the parrot executable, but >>reality left me puzzled, so I thought I'll leave the decision to you. > IMCC is integral for Parrot in order to eval/compile new PASM and PIR at > runtime. If you ONLY want to support precompiled bytecodes you > can get by without it but for quite some time now IMCC _is_ the > Parrot front end. > It might make sense to provide separate libparrotvm without dynamic eval > capability so you can leave out the actual compiler and register allocator > and have it loaded on demand (libparrotimcc or some such). > This would help save memory in the default case. So, I guess the yacc stuff - yylex, yyparse, yyin and such - should be hidden somewhere inside IMCC, behind an interface, which would then be part of libparrot interface? Yes, I think so. Of course I always wanted to write a pure parrot form of the whole IMCC compiler, but nowadays Parrot is so reliant on C that it seems a pointless exercise to go "Pure Parrot" until the whole community gets on-board. -Melvin
Re: Helping The Paranoid Parrot on Windows
At 04:08 PM 3/28/2005, Ron Blaschke wrote: > On the one hand, IMCC doesn't really help you _run_ code, so I'm not > inclined to see it part of libparrot. On the other hand, I haven't > grokked the entire code base organization, so I could be Greatly > Missing The Point. > On the gripping hand, if you'd like to experiment with putting IMCC > into libparrot, bison's "-p prefix" option seems relevant: "Rename the > external symbols used in the parser so that they start with prefix > instead of yy." Maybe "-p Parrot_imcc_"? I have left things as-is while getting linkage correctly. That is, most of IMCC is currently part of libparrot (only F is not). In the beginning, I thought IMCC is part of the parrot executable, but reality left me puzzled, so I thought I'll leave the decision to you. IMCC is integral for Parrot in order to eval/compile new PASM and PIR at runtime. If you ONLY want to support precompiled bytecodes you can get by without it but for quite some time now IMCC _is_ the Parrot front end. It might make sense to provide separate libparrotvm without dynamic eval capability so you can leave out the actual compiler and register allocator and have it loaded on demand (libparrotimcc or some such). This would help save memory in the default case. -Melvin
Re: Passing on the hat
At 06:55 PM 3/21/2005, Chip Salzenberg wrote: According to Dan Sugalski: > As such, I'd like to say a big thanks to Chip Salzenburg who's agreed > to take the hat. I thank you for your kind words, and for giving me the opportunity again to work long hours and explain difficult and arbitrary design decisions to enthusiastic contributors. :-) And *all* of us thank you for the last four and a half years. I'm glad you'll still be around; your work certainly will be. Congrats Chip, I can't actually follow what Dan said since I did not receive the original message from Dan, all I see is replies. Was this a reply to a private message or is the mailing server doing funny things again? I'm just curious about how Dan is doing since I haven't talked to him in a while. :) -Melvin
Re: Parrot questions
At 09:47 AM 3/11/2005, theUser BL wrote: (with the languages Nice and Groovy) and .net, but written esecialy for scripting-languages. True, Parrot is slanted toward dynamic scripting languages that recompile and eval themselves on the fly, but it does provide low-level registers and features that nice, statically bound languages without continuation passing style can call. I've already done this with a demo Java like language that only uses the low level numeric and string registers and uses the faster jump/bsr instructions rather than the continuation passing style. If you read my IMCC faq, you'll also see how I kludged OOP without using Parrot's dynamic class support. It results in very fast code that is well-JITable, compared to what Perl6 compiles to. Point is: You can do it how you want to, but if you want to cooperate with other languages on the Parrot VM, you'll need to use the Parrot conventions at least on the boundaries of your modules. But I see two problems: 1. the license Perl is licensed under the GPL and Artistic license. Python is licensed under the Python-license and Ruby is licensed under the GPL and its own license. So, it would be nice, if Parrot would be licensed under the GPL, Artistic license, Python-license and Rubys own license. I don't think, that Python would be public its package with something included, which is under an other license then the Python-license. Moot. You can write commercial compilers that target the Parrot VM and distribute it as you see fit. On top of that, if you don't use anything but what Parrot provides in its base PMC classes, you don't even have to distribute a runtime lib for your special language. 2. Parrot is written by Perl-people for Perl I think it would be better, if there existing a group of 2 Perl people, 2 Python people and 2 Ruby people or so, which disigned _together_ a VM for scripting languages and then support all this VM. The VM is designed and working, the VM boys have done the bulk of their job. Parrot can currently do things that the JVM cannot. Parrot's support library needs a lot of work, but this has never stopped compiler writers in the past. :) It is time for the compiler writers to start pulling the load. I have red, that you create yourself a Python for Parrot. But that wille be not more supported by the Python-people like IronPython or Jython I think. It would be better to working with the Python and Ruby people together. And letting Parrot independent of any language. Perl is probably the heaviest influence in the scripting world. I'd wait and watch what happens when Perl6 is for real and we start doing releases for the Parrot VM and see how many people follow. -Melvin
Re: .HL_language pragma (was: Proposed steps towards the next release 0.2 and beyond.)
At 07:50 AM 3/7/2005, Leopold Toetsch wrote: Roger Browne <[EMAIL PROTECTED]> wrote: > On Mon, 2005-03-07 at 11:01 +0100, Leopold Toetsch wrote: >> 4.1.1) implement a language pragma in assembler, e.g.: >> >> .HL_language Python Sounds good to me except that I would prefer you remove the HL_ and just call it .language to be consistent with other directives. Incidentally this is what .NET uses as well except its implementation uses GUID instead of a string, and it also allows the compiler vendor and source document type to be included in the directive. Dan and I talked about this a couple of years ago on IRC. Originally I wanted to stay away from GUIDs as I think they are overkill, and had considered allowing ASCII: .language Perl6 .language Perl6:ThePerlFoundation But..., this requires this section in the bytecode to be variable sized, so there might be some merit to using GUIDs, especially for vendors. I think to keep PIR "convenient" for hackers I want to accept both strings and 32 bit integers for each. If a string, it must be already registered in IMCC and we can probably reserve the first 1024 for "pre-registering" but IMCC still would be able to accept integers. In any case, it needs to be fixed width in the bytecode header. We need to revisit the bytecode format. Last time through, I put an "Opcode type" at some offset, per Dan's request. We store a magic number for that. I'm pretty sure we can use this field to identify the language using whatever is provided in the .language pragma, but I can't speak for Dan. I think he might have had different plans for opcode type. -Melvin
Re: Calling C/C++ library routines
At 01:27 PM 2/25/2005, vlad florentino wrote: Is there now, or will there be in the future, any way to call C/C++ library routines from within Parrot? For example, a mysql, pcre or libcurl library. Either static or dynamic. C yes. C shares objects are dynamically loadable by Parrot. C++? Not directly, you'll need to wrap any C++ APIs in "extern C" stubs to get at them from Parrot and make sure you stuff any object pointers into a PMC. It has been done, an example is the C++ Qt Toolkit. -Melvin
Re: Q: about Parrot assembly code
At 12:57 PM 2/25/2005, Steve Coleman wrote: constructs that could not be logically mapped from other CPU's into Parrot? Does Parrot assume/use many high level constructs not found in real processors? Some CPU translations, like from CISC to RISC, are clearly easier than the reverse, but other types would prove much more difficult, like those with specialized register windows and DSP vector processors. I doubt there is much that could not be mapped to Parrot, but Parrot does things that would be impossible to map directly back to a CPU. Parrot is a mix of low, medium and high level instructions. We have ops that are jitted to a single instruction or two on a CPU, but we have many more ops that are large routines that would have to be translated to an API call requiring run-time environment support, continuations, co-routines, various other odd things which require garbage collectable stacks and execution contexts. You should be able to easily map your external APIs to Parrot. It has a native interface, so for example if you have an Oracle app that uses libclntsh.so, you'll simply need to translate any calls to the Parrot Native Interface by generating the appropriate stub for the Parrot side. For extended processor functions, you'd need to implement them or JIT them. It may be simpler to support JIT-only for your project. When you talk about DSP processors, you are out of my range. Register windows ala Sparc could be implemented easily, and has been discussed periodically, though we chose a different calling convention. Regarding mapping from Parrot, you'd need to build a libparrot and link to your new environment for dispatch of those things that don't translate. I'm not sure why you'd want to do this, though. We don't have any application base to speak of. :) Understanding what can and can not be translated to and from the virtual Parrot CPU would be the key to my looking at this possibility any deeper, but I could also forsee the possibility of some future type of IDE tools being used to create and debug other applications visually through Parrot as an intermediary between languages and runtime bytecode. Translation of programs (e.g. Perl<=>Python<=>.Net) would also be possible. There has been some successful work done by various people if you dig back through the archives. I was able to successfully translate some Java to Parrot, as did someone else; who it was, I forget. Someone else has been playing with Z-machine lately I think. Cheers, -Melvin
Re: [perl #34258] [TODO] Here documents for PIR
At 11:48 AM 2/25/2005, Bernhard Schmalhofer wrote: MrJoltCola wrote: At 03:21 AM 2/25/2005, Leopold Toetsch wrote: MrJoltCola <[EMAIL PROTECTED]> wrote: > I feel that this feature is for higher level languages. [ snip ] > ... PIR is for compilers, not people, My impression was that the target for compilers should be a yet non-existent abstract syntax tree. Though I'm somewhat confused about the distinction between parse trees and AST. PIR and PAST could be considered as being just some languages. Why don't they take the some route, and generate an AST that gets compiled into PBC? However most compilers I have seen so far are generating PIR and go from there. Text mode IR is easier to debug with the naked eye than an AST. ASTs are great if you have supporting tools to browse the tree, dump things out and translate things back to IR. Virtually all compilers start with an IR and mature into AST usage, but many successful compilers are built with completely separate layers. See Muchnicks's work at Sun Microsystems. He has been the main influence on me as a compiler writer. PIR is foremost Parrot's primary assembly language. If it were for compiles only, it wouldn't have needed "a = b + c" in the first place, the "add" opcode is doing the same. It filled a need at the time I wrote it, because there was pretty much nothing else to write code in except PASM and Jako. It was supposed to be a thin layer (register allocation and instruction choosing and subs) over PASM. It is certainly possible to write code in PIR. But it is even more certain that this is no fun. Adding HERE documents remedies an immediate need, especially for playing with compilers and test scripts. I agree. Why not write a pre-processor that will handles things like this and then merge them forwards towards a front-end compiler rather than backwards into the mess of IMCC? Trust me, I know the IMCC code, and it is BAD BAD BAD. And I fully share the blame with Leo and Angel, bless their hearts. :) > Nowadays we have a hoard of little toy compilers. Which language of that hoard should be promoted to become THE standard language for writing test scripts and the Parrot standard library? PIR is too hard on people, and a scripting language like Ruby, Perl or Python is propably to hard on Parrot. It propably shouldn't a newly invented language either, because all language designers are having fun with designing Perl6. You echo my feelings, PIR is too hard on people, and I feel that Parrot's growth suffers for it, not because of PIR, but because that is all we use. I'm appalled that 3 years after I submitted IMCC to the project, everyone is still writing in PIR. Parrot need's its own version of "C" or "C#" to empower more people to contribute. If every new feature for Parrot is thought of in terms of PIR & IMCC, then IMCC will just become more of a tangled mess that nobody wants to touch, and high-level development will continue to drag as it has for years. How about ECMAScript? It has an specification, it has objects and no pointers. No complaints from me, I don't vote much anyway, but I have strong feelings regarding this project. The mixing of the VM team and the compiler team is a mistake. It is a problem for commercial development, much less open-source. One pumpking making decisions for such a wide scope doesn't leave much room for alternate points of view and unimpeded movement. This is why I urged Dan long ago to form a parrot-compilers list and split the project into separate decision making teams. Speaking of Dan, where is he? -Melvin
Re: [perl #34258] [TODO] Here documents for PIR
At 03:21 AM 2/25/2005, Leopold Toetsch wrote: MrJoltCola <[EMAIL PROTECTED]> wrote: > I feel that this feature is for higher level languages. [ snip ] > ... PIR is for compilers, not people, PIR is foremost Parrot's primary assembly language. If it were for compiles only, it wouldn't have needed "a = b + c" in the first place, the "add" opcode is doing the same. It filled a need at the time I wrote it, because there was pretty much nothing else to write code in except PASM and Jako. It was supposed to be a thin layer (register allocation and instruction choosing and subs) over PASM. Nowadays we have a hoard of little toy compilers. PIR has nice constructs for function and method calls. I don't see any reason, why people shouldn't write PIR code directly. A HERE document syntax allows to write more readable code. ...and adds another straw to the Camel's back of IMCC maintainability ...and encourages people to stay complacent and keep writing PIR code by hand, which was the main point of my note. -Melvin
Re: [perl #34258] [TODO] Here documents for PIR
This should actually be titled "Where are all the compilers?" - I haven't ranted in a couple of years, so I'm due. Ranting is nothing more than broadcasting my emotions from a soapbox but it is so fun, I love to do it. Let me respectfully give my opinion. In no way am I criticizing your suggestion of here documents, nor am I ranting at the originator of the suggestion. I feel that this feature is for higher level languages. I did not even want macros, but there was a lot of griping when I removed them, so I relented and Leo reversed the patch. What really is the problem here is the lack of a decent alternative. It is not PIR that is lacking features. PIR, by definition should lack features. It stands for "Pick-your-P-word Intermediate Representation." What is missing is a good implementation language that compiles to PIR. PIR is for compilers, not people, and I don't want to see PIR turn into "PIRL". What I always wanted to do was implement a C subset for Parrot, and write all "low-level" things in that language. C is very good for close translation to underlying "hardware" and it doesn't take a full-time think-tank to implement a compiler for it. I'd like to see thin drivers written for things (ala JDBC Thin SQL drivers), with VERY limited use of native interfaces. I actually have a pretty nice compiler sitting in my archives, but got too busy with life to finish it. The little "Cola" language that is in languages right now is a hack job that was a quick prototype to see if people would actually write C like code for Parrot. Cola got very little interest, so I shelved my work on it and IMCC2. Years after I dumped much work into Parrot, I've seen no progress or support from other language camps (Python, Scheme) and Perl 6 seems to be such a daunting goal to implement, I'm afraid we won't a full, commercial-free implementation for years. (I've written a couple of fully featured compilers in a matter of weeks, and I won't even begin to tackle Perl6 unless someone offered to fund me full-time). Heck, what happened to the Pony project? Is there anybody out there? Or did we all find that we were happier with Perl 5 than we realized. Or were we so slow with Perl 6 that everyone had time to swallow the .NET pill? I feel better now, -Melvin At 04:53 PM 2/24/2005, via RT wrote: # New Ticket Created by Bernhard Schmalhofer # Please include the string: [perl #34258] # in the subject line of all future correspondence about this issue. # https://rt.perl.org/rt3/Ticket/Display.html?id=34258 > Hi, in the near, or far, future there will be test scripts and compiler input in PIR. For that it would be nice, if long text doesn't have to be crammed into a single line. So some kind of HERE document syntax is needed for PIR. Suggestions for syntax elements are welcome. '<<' is already taken for the left shift operator. CU, Bernhard -- /* [EMAIL PROTECTED] */ DSL Komplett von GMX +++ Superg�nstig und stressfrei einsteigen! AKTION "Kein Einrichtungspreis" nutzen: http://www.gmx.net/de/go/dsl
Re: Native support for GNU R
At 01:40 PM 2/9/2005, Aaron Sherman wrote: On Wed, 2005-02-09 at 12:08, [EMAIL PROTECTED] wrote: > I read that you can provide support (in Perl 6) for most languages that > parsers have been written for. As it appears to me, however, the languages > that you are mainly interested in are substitutes (competitors) to Perl. You're mistaken about a core item here. First off, Perl 6 may have This is the reason we should have changed the list name to parrot-internals years ago, or spawned a new one. Nowadays I'm not as interested in the Perl6 language as I am the potential to run a lot of languages on a common VM. -Melvin
Re: [perl #34002] [PATCH] 'const' for ParrotIOLayerAPI instances
They are well defined at the moment, but the author's intent of the API was to have them mutable. -Melvin At 01:18 PM 2/1/2005, Leopold Toetsch wrote: MrJoltCola <[EMAIL PROTECTED]> wrote: > Layer and layer API members may be changed at runtime. Yes, the current > structure > members are all static, but they don't have to be. Please note: only the layer API was changed. > I would reverse this patch. A xxx_layer_api seems rather const to me, constituted from a bunch of well defined functions. A different set of functions would create a different API - probably. > -Melvin leo
Re: [perl #34002] [PATCH] 'const' for ParrotIOLayerAPI instances
Layer and layer API members may be changed at runtime. Yes, the current structure members are all static, but they don't have to be. I would reverse this patch. -Melvin At 06:26 AM 1/31/2005, via RT wrote: # New Ticket Created by François PERRAD # Please include the string: [perl #34002] # in the subject line of all future correspondence about this issue. # https://rt.perl.org/rt3/Ticket/Display.html?id=34002 > All instances of ParrotIOLayerAPI could be declared as 'const' in the IO subsystem. I hope that it's an improvement (better & safer code) and not a problem for future feature. François Perrad
Re: [COMMIT] imcc moves out of languages
Thanks Robert for the fixup. Nothing like throwing a grenade to get people scrambling. -Melvin -Original Message- From: Robert Spier <[EMAIL PROTECTED]> Sent: Oct 23, 2003 1:33 PM To: [EMAIL PROTECTED] Subject: Re: [COMMIT] imcc moves out of languages > > So much for preserving repository history. > Yep. Is the history gone, or are you able to do something for it? I've reattached the history to the files. All should be well. Depending on when you last updated, you _may_ see weird warnings on your next update -- they should be harmless. -R
Re: Parrot: maximizing the audience
On Thu, 5 Sep 2002 10:27:05 -0700 Steve Fink <[EMAIL PROTECTED]> wrote: >Speaking of Sub/Coroutine/Continuation, right now we *really* need >someone who pretends to understand this stuff to take a look at >Jonathan Sillito's patches and do something with them. Or give him >commit privs, or something. Hey those are my three headed babies you are talking about! :) Sorry, I've been preoccupied and I must have missed Jonathan's patch. I will have a look. -Melvin
Re: Off-list discussions, was Re: imcc hack for perl6 regexes
>Can I respectfully request that you guys make a lot more of your >discussions public? languages/imcc and languages/perl6 are very major >components, and they have been very little discussed on-list. imcc Sure, I have no problem with it. At one time someone suggested making a separate list for Parrot compilers, so I took it as a hint that maybe we were spamming. >didn't bother me that much because I thought mainly just Melvin was >working on it (I spend enough time listening to the voices in my head To be fair, Angel has been doing most of the work lately. Work has had me pinned. I'm hoping with Sean's first cut at Perl6, we can identify weakness and requirements concerning an intermediate language, and re-architect when we are done with the "use cases." >perl6 seems like something the wider community just might be >interested in hearing about... ;-) You _would_ think so, wouldn't you? :) Personally I've been a little disappointed in the involvement(interest) of late. -Melvin
Re: imcc hack for perl6 regexes
On Wed, 21 Aug 2002 18:02:51 +0200 Angel Faus <[EMAIL PROTECTED]> wrote: >I think we all agree that since parrot can have dynamic oplibs (and core >parrot has hundreds of ops), imcc needs some way to directly express them. >The idea of having parrot ops be included as such, and imcc directives be >prepended with "." looks fair to me. Ok, then this point is settled. >a) imcc becomes like parrot assembly, but with virtual registers and a few >administrative ops (for calls and such), that start with "." > >or > >b) imcc becomes like parrot assembly, but with virtual registers, a few >administrative ops, and some convenient infix notation like for stuff like "a =1" or "b = a + 1", No. :) c) imcc becomes a "middle" level language. I never meant it to be an assembler. In practice intermediate languages provide other constructs such as aggregate type definition that are not available in the assembler. type i : packed int[32] type r : record { foo : int, bar : string } This is not assembler. This is medium level computer language in my book. You could even see things like ..pragma classes_are_hashes or something that would tell the compiler that aggregates should be implemented as standard "Perl" hashes in this piece of code, whereas others might want "packed" aggregates with no runtime introspection capability. >the usage of parrot ops "as is" is good for us, because we don't need to >invent, develop, and document, another language. It lets us focus on making >imcc do well a few tasks (ahem) such as register allocation and >language-neutral optimitzations. But I'm willing to invent and develop another language. And it should be a lot richer than just an assembler with a register allocator and spiller. :) >I am not a priori against imcc diverging from assembly. But I think the only >good reason for divergence should be making the task of the language compiler >simpler. But that was my whole reason for writing it in the first place. I did not want to target the assembler directly. -Melvin
Re: Stack performance issue
On 02 Jul 2002 16:35:02 +0100 Tom Hughes <[EMAIL PROTECTED]> wrote: >I've done that for the register stacks, and I'll do the same for the >other stacks unless somebody spots a flaw in my logic and points out >that the GC will catch it... No, your logic is correct, stacks are still outside of the GC. I'm working on fixing this so I can finish co-routines and continuations correctly. Right now if you play around with continuations you'll get leakage. That is the reason I left stacks allocated at their high water mark until we fix GCed chunks. Meant to finish it this weekend, but life has a way of interrupting; I should be able to commit it this week. -Melvin
Re: Half measures all round
On Tue, 4 Jun 2002 10:06:44 -0700 (PDT) Larry Wall <[EMAIL PROTECTED]> wrote: >It's really not that difficult to run two interpreters in the >same process. I already made Perl and Java run together nicely. Agree. >Scaffolding is supposed to be ugly. You wouldn't believe how ugly >the transitional form between Perl 4 and Perl 5 was, when half the The difference in software branches and highways is that software can be built completely off-site until it's reasonably functional, and then moved in on the back of a semi-trailer. There is no reason why we shouldn't produce a "Purely P6" virtual machine, parser, etc. first, then start building the scaffolding. -Melvin
Re: [COMMIT] Added preprocessor layer to newasm.
On Tue, 28 May 2002 01:19:25 -0400 Jeff <[EMAIL PROTECTED]> wrote: >newasm now handles constants, macros, and local >labels within. Here's a Great work! >expansion. Also, they don't expand >recursively. '.constant FOO >"blah"n.constant BAR "Hey, .FOO"' won't do what >you want, sadly. Thats exactly what I want. I don't think the assembler should do any sort of interpolation with string constants at all. >that's what I'll work on next. When Simon first committed it, I tested newasm and noticed 2-3x speedup on assembly speed. Is this still the case? I've been running some tests with executing Cola on the fly, and the assembly phase (slow) is the big bug in the soup right now, however I've been using the old assembler. PS: Thanks (Simon and Jeff) for a lot of hard work on newasm. -Melvin
Re: [netlabs #629] [PATCH] Memory manager/garbage collector -major revision
On Tue, 28 May 2002 12:50:06 +0200 Jerome Vouillon <[EMAIL PROTECTED]> wrote: >I propose the following alternate guidelines. > > > STRING * concat (STRING* a, STRING* b, STRING* c) { >PARROT_start(); >PARROT_str_params_3(a, b, c); >PARROT_str_local_2(d, e); > >d = string_concat(a, b); >e = string_concat(d, c); >PARROT_return(e); > } If you search the archive, you'll find I've already proposed this type of solution in 2 different flavors and both times the consensus has gone back to setting flags. It seems we are going around in a circles. I also agree its funny we are worrying about performance when its not apparent the allocation overhead even has anything to do with the current performance problem. -Melvin
[PATCH] PIO - more stuff
Changes: -Minor layer cleanups -Win32 layer added (mostly stubs for now) but will be using the Win32 API and company rather than the unix-ish fake ones. -stdin/stdout/stderr Win32 specific code added. Rather than use 0,1,2, Microsoft says use GetStdHandle(), etc. so that stuff is there. Soon we should be able to start using these rather than fprintf(stderr, ...) and remove stdio linkage. New files: io/io_win32.c Renamed io_os.c -> io_unix.c -Melvin diff -urN tmp/parrot/MANIFEST parrot/MANIFEST --- tmp/parrot/MANIFEST Mon Jan 7 08:37:51 2002 +++ parrot/MANIFEST Wed Jan 9 00:06:25 2002 @@ -116,8 +116,9 @@ include/parrot/unicode.h interpreter.c io/io.c -io/io_os.c io/io_stdio.c +io/io_unix.c +io/io_win32.c io/TODO jit.c jit/i386/core.jit diff -urN tmp/parrot/Makefile.in parrot/Makefile.in --- tmp/parrot/Makefile.in Tue Jan 8 12:24:29 2002 +++ parrot/Makefile.in Wed Jan 9 00:07:14 2002 @@ -74,7 +74,7 @@ CHARTYPE_O_FILES = chartypes/unicode$(O) chartypes/usascii$(O) -IO_O_FILES = io/io$(O) io/io_os$(O) io/io_stdio$(O) +IO_O_FILES = io/io$(O) io/io_stdio$(O) io/io_unix$(O) io/io_win32$(O) INTERP_O_FILES = global_setup$(O) interpreter$(O) parrot$(O) register$(O) \ core_ops$(O) core_ops_prederef$(O) memory$(O) packfile$(O) stacks$(O) \ diff -urN tmp/parrot/include/parrot/io.h parrot/include/parrot/io.h --- tmp/parrot/include/parrot/io.h Thu Jan 3 17:37:37 2002 +++ parrot/include/parrot/io.h Wed Jan 9 00:11:55 2002 @@ -57,9 +57,11 @@ #define PIO_F_READ 001 #define PIO_F_WRITE 002 #define PIO_F_APPEND004 +#define PIO_F_TRUNC 010 #define PIO_F_FILE 100 #define PIO_F_PIPE 200 #define PIO_F_SOCKET400 +#define PIO_F_CONSOLE 0001000 /* A terminal, we can linebuf */ #define PIO_F_LINEBUF 001 #define PIO_F_BUF 002 #define PIO_F_MALLOC004 /* Buffer malloced */ @@ -88,11 +90,21 @@ unsigned char * next; /* Current read/write pointer */ }; + +#ifdef WIN32 +typedef HANDLE PIOHANDLE; +#else +typedef int PIOHANDLE; +#endif + + struct _ParrotIO { -INTVAL fd; /* Low level OS descriptor */ -INTVAL mode; /* Read/Write/etc. */ -INTVAL flags; /* Da flags */ -off_t filepos;/* Current real file pointer*/ +PIOHANDLE fd; /* Low level OS descriptor */ +UINTVAL mode; /* Read/Write/etc. */ +UINTVAL flags; /* Da flags */ +off_t fsize; /* Current file size*/ +off_t fpos; /* Current real file pointer*/ +off_t lpos; /* Last file position */ ParrotIOBuf b; /* Buffer structure */ ParrotIOLayer * stack; /* ParrotIOFilter * filters; */ @@ -101,7 +113,7 @@ struct _ParrotIOLayer { void* this; /* Instance specific data */ const char * name; -INTVALflags; +UINTVAL flags; ParrotIOLayerAPI* api; ParrotIOLayer * up; ParrotIOLayer * down; @@ -121,11 +133,12 @@ /* Others to come */ -extern ParrotIOLayerpio_os_layer; -extern ParrotIOLayerpio_stdio_layer; -#ifdef WIN32 -/*extern ParrotIOLayer pio_win32_layer; */ +#ifndef WIN32 +extern ParrotIOLayerpio_unix_layer; +#else +extern ParrotIOLayerpio_win32_layer; #endif +extern ParrotIOLayerpio_stdio_layer; /* This is list of valid layers */ extern ParrotIOLayer* pio_registered_layers; @@ -152,7 +165,7 @@ INTVAL (*Pushed)(ParrotIOLayer * l, ParrotIO * io); INTVAL (*Popped)(ParrotIOLayer * l, ParrotIO * io); ParrotIO * (*Open)(theINTERP, ParrotIOLayer * l, -const char * name, const char * mode); +const char * name, UINTVAL flags); ParrotIO * (*Open2)(theINTERP, ParrotIOLayer * l, const char * name, const char * mode, int perm); @@ -163,7 +176,7 @@ const char * name, const char * mode, DummyCodeRef *); ParrotIO * (*FDOpen)(theINTERP, ParrotIOLayer * l, -INTVAL fd, const char * name); +PIOHANDLE fd, UINTVAL flags); INTVAL (*Close)(theINTERP, ParrotIOLayer * l, ParrotIO * io); size_t (*Write)(theINTERP, ParrotIOLayer * l, @@ -208,15 +221,16 @@ extern struct PMC *
ParrotIO: Patch attached this time :)
Oops. The patch helps. -Melvin diff -urN tmp/parrot/MANIFEST parrot/MANIFEST --- tmp/parrot/MANIFEST Tue Jan 1 11:58:13 2002 +++ parrot/MANIFEST Tue Jan 1 16:40:01 2002 @@ -79,6 +79,8 @@ include/parrot/trace.h include/parrot/unicode.h interpreter.c +io/io.c +io/io_os.c jit/i386/core.jit jit/i386/lib.jit jit/i386/string.jit diff -urN tmp/parrot/Makefile.in parrot/Makefile.in --- tmp/parrot/Makefile.in Tue Jan 1 15:11:57 2002 +++ parrot/Makefile.in Tue Jan 1 15:19:24 2002 @@ -73,12 +73,14 @@ CHARTYPE_O_FILES = chartypes/unicode$(O) chartypes/usascii$(O) +IO_O_FILES = io/io$(O) io/io_os$(O) + INTERP_O_FILES = global_setup$(O) interpreter$(O) parrot$(O) register$(O) \ core_ops$(O) core_ops_prederef$(O) memory$(O) packfile$(O) stacks$(O) \ string$(O) encoding$(O) chartype$(O) runops_cores$(O) trace$(O) pmc$(O) key$(O) \ platform$(O) ${jit_o} resources$(O) -O_FILES = $(INTERP_O_FILES) $(CLASS_O_FILES) $(ENCODING_O_FILES) $(CHARTYPE_O_FILES) +O_FILES = $(INTERP_O_FILES) $(IO_O_FILES) $(CLASS_O_FILES) $(ENCODING_O_FILES) +$(CHARTYPE_O_FILES) ### diff -urN tmp/parrot/core.ops parrot/core.ops --- tmp/parrot/core.ops Tue Jan 1 12:22:54 2002 +++ parrot/core.ops Tue Jan 1 16:22:35 2002 @@ -73,12 +73,21 @@ +=item B(p) + +Close IO stream $1. + =item B(i|ic) Close file opened on file descriptor $1. =cut +inline op close(p) { + PIO_close(interpreter, (ParrotIO*)($1->data)); + goto NEXT(); +} + inline op close(i|ic) { fclose((FILE *)$1); goto NEXT(); @@ -111,6 +120,10 @@ +=item B(p, s|sc, s|sc) + +Open file named $2 with Perl style mode string specified in $3, and +create the IO stream PMC into $1. =item B(i, s|sc) @@ -124,6 +137,18 @@ =cut +inline op open(p, s|sc, s|sc) { + ParrotIO * io; + STRING *path = $2; + STRING *mode = $3; + io = PIO_open(interpreter, path->bufstart, mode->bufstart); + if(!io) { +/* FIXME: Handle error */ + } + $1 = new_io_pmc(interpreter, io); + goto NEXT(); +} + inline op open(i, s|sc) { if (! ($1 = (INTVAL)fopen(($2)->bufstart, "r+"))) { perror("Can't open"); @@ -222,6 +247,8 @@ =item B(i|ic, p) +=item B(p, s|sc) + Print $2 to the file on $1. (0, 1, and 2 correspond to stdin, stdout, and stderr) @@ -247,6 +274,16 @@ goto NEXT(); } +op print(p, s|sc) { + ParrotIO * io; + STRING *s = $2; + io = (ParrotIO*)($1->data); + if (s && string_length(s) && io) { +PIO_puts(interpreter, io, (const char *)s->bufstart); + } + goto NEXT(); +} + op print(p) { PMC *p = $1; STRING *s = (p->vtable->get_string(interpreter, p)); @@ -325,6 +362,21 @@ } goto NEXT(); } + + + + +=item B(s|sc) + +Print $1 to standard output stream + +op puts(s|sc) { + STRING *s = $1; + if (s && string_length(s)) { +PIO_puts(interpreter, pio_stdout, (const char *) s->bufstart); + } + goto NEXT(); +} diff -urN tmp/parrot/include/parrot/io.h parrot/include/parrot/io.h --- tmp/parrot/include/parrot/io.h Mon Sep 17 21:16:59 2001 +++ parrot/include/parrot/io.h Tue Jan 1 16:38:23 2002 @@ -3,17 +3,221 @@ * CVS Info * $Id: io.h,v 1.1 2001/09/18 01:16:59 gregor Exp $ * Overview: - * IO operations header + * Parrot IO subsystem * Data Structure and Algorithms: * History: + * Originally written by Melvin Smith * Notes: * References: + * Some ideas and goals from Perl5.7 and Nick Ing-Simmons' work */ #if !defined(PARROT_IO_H_GUARD) #define PARROT_IO_H_GUARD -#define Init_IO(x) +#include "parrot/parrot.h" + +#ifndef SSIZE_MAX +#define SSIZE_MAX 8192 +#endif + +#ifndef STDIN_FILENO +# define STDIN_FILENO 0 +#endif + +#ifndef STDOUT_FILENO +# define STDOUT_FILENO 1 +#endif + +#ifndef STDERR_FILENO +# define STDERR_FILENO 2 +#endif + + +/* Average block size of most systems (usually varies from 2k-8k), + * later we can add some config to query it from the system at + * build time (struct stat.st_blksize maybe). + */ +#define PIO_BLKSIZE 4096 +#define PIO_LINEBUFSIZE 256 + + +enum { +PIO_TYPE_FILE, +PIO_TYPE_PIPE, +PIO_TYPE_SOCKET, +PIO_TYPE_MAX +}; + +enum { +PIO_BUFTYPE_NONE = 0, +PIO_BUFTYPE_FULL, +PIO_BUFTYPE_LINE, +PIO_BUFTYPE_MAX +}; + + +typedef struct _ParrotIOLayerAPIParrotIOLayerAPI; +typedef struct _ParrotIOLayer ParrotIOLayer; +typedef struct _ParrotIOFilter ParrotIOFilter; +typedef struct _ParrotIOBuf ParrotIOBuf; +typedef struct _ParrotIOParrotIO; + +/* This is temporary until subs/code refs are done..*/ +typedef void * DummyCodeRef; + + +struct _ParrotIOBuf { +size_t bufsize; +unsigned char * buf; +
ParrotIO : Please test this [PATCH] for breakage
All who have time, specifically Win32, please test this patch for compilation. I currently don't have a Win32 environment to test with. Anyone with non-Linux please test. What this patch does: -Implements initial IO stack framework (pushing/popping layers) -Implements an initial "layer" for the stack, the layer being a low level OS layer and pops it on at interp initialization. -Implements the initial "entry layer" in io.c -Implements the standard IO streams (STDIN, STDOUT, STDERR) in very simple form as Parrot IO PMCs (not stdio FILE * wrappers). See io.h (pio_stdin, etc.) -Adds 4 rudimentary calls to core.ops to test with open, close, print, puts puts() will print through ParrotIO.STDOUT instead of STDIO.STDOUT Note, I'm concentrating primarily on design right now, not completeness, there are a lot of "calls" to be added to the layer API. Once we like the design we will complete the OS layer and start working on a buffered layer, filters, async stuff as well as other layers, etc. I still haven't discussed with anyone whether UTF stuff makes sense as a layer or filter. It can be implemented as either/or. Files modified: include/parrot/io.h include/parrot/pmc.h core.ops MANIFEST Files added: io/ io/io.c io/io_os.c -Melvin
struct Parrot_Interpreter argument
Since so many functions pass around the interpreter, can we add a standard, short interpreter arg macro so I don't have to clutter the argument list of every function I write. theINTERP or something? -Melvin
IO filters
Can we start some dialogue about stream filters? What form they take, are we talking regular expressions, etc. -Melvin
Re: Allocators
Dan wrote: >># In lieu of a de-allocator for mem_allocate_aligned I vote >># we at least do something in the interim and I volunteer to >># help as soon as someone decides what it is! >> >>Maybe we can have a mem_free_aligned that somehow figures out what the >>starting address is. If we do that and document which data structures >>are aligned and which aren't, we shouldn't have a problem. > >You can't do that--there's no way to tell where the memory block really >started. Right, at least while everyone is allocating memory by multiple methods. Too bad free() is too dumb() :), some mem management libs I recall use some magic cookie values to validate, hidden in a negative offset of the chunk, but apparently we can't rely on that. > >Anyway, all the aligned stuff's a bit of a hack-up to make up for the lack >of arenas. It'll be fixed right reasonably soon. > >Dan Cheer! -Melvin
Re: Allocators
>The only thing that needs the allocated alignment is some of the internal >pieces--the stack chunks and register frames, really. Everything else can >use a plain malloc. Well, mem_allocate, rather, which can be a wrapper around malloc for now. > >Dan Maybe we can do this for now? -Melvin --- resources.c Mon Oct 22 15:27:05 2001 +++ resources.c.new Wed Dec 12 16:14:15 2001 @@ -23,7 +23,7 @@ } STRING *new_string_header(struct Parrot_Interp *interpreter) { - return mem_allocate_aligned(sizeof(STRING)); + return mem_sys_allocate(sizeof(STRING)); }
Allocators
In lieu of a de-allocator for mem_allocate_aligned I vote we at least do something in the interim and I volunteer to help as soon as someone decides what it is! I know GC is on the way but... At minimum we should be using plain malloc() until a better solution or the current one is finished, else right now we have a glaring memory leak for ops that need to destroy strings. Can't the current function be at least tweaked so it still rounds mem chunks but doesn't modify the starting address of the chunk. Is anyone actually even using the feature that this technique allows? -Melvin
IO patch version 2 - please comment
Here is a start for the IO skeleton. I implemented open, close, read, write, setbuf and some constants in io.h -Implements a simple ParrotIO PMC with null vtable -open() takes a string for mode which is Perl-ish (<, >, >>, |, +>, +<) Comments? Would the group prefer numeric constants here? -write() does block buffering or no-buffering, (still need line buffering)-read (currently raw read, unbuffered) -setbuf() allows buffer control although it doesn't allow a user supplied buffer at the moment (they are good sometimes because automatic buffers are faster) -close() just flushes the stream and closes Added a sample file.pasm that does a open/write/close/open/read test. TODO: Lots and lots. Standard in/out/err streams so we can ditch stdio. Read buffering, Line buffering, Pipes, Random access, filters, async stuff. I need feedback on this patch before I continue down the road. Thanks, Melvin io-v2.patch Description: Binary data
Re: mem_allocate_aligned
On Monday, December 10, 2001 10:44:09 AM Dan Sugalski wrote: > At 02:57 PM 12/10/2001 +, Simon Cozens wrote: >>On Sun, Dec 09, 2001 at 11:26:44PM -0500, [EMAIL PROTECTED] wrote: >> > it has to do with the fact that the memory allocator >> > adjusts the address before returning the chunk and free() >> > then gets confused. Does this seem to be the issue? >> >>That's exactly what it is! Damn, I'd been seeing this too, and >>decided it was best to just ignore this until we have GC. > >Yup. Sorry, I've known about this for a bit but kept forgetting. >(And putting it off for the GC too...) > >Mem_allocate_aligned is used more than it needs to be in spots, and >is >generally dodgy. And wasteful. Aye, thats what fresh eyes are for, to remind you of what you meant to do while doing 103224 other things. Anyway, I'll return to using malloc/free for my stuff then until someone gets time to get to the GC and allocators. With the IO stuff we could technically keep a private list of recycled IO objects and never free them at all. -Melvin
mem_allocate_aligned
While implementing one of my IO routines in core.ops I used free_string() on a string that was created ultimately with mem_allocate_aligned() and I get core dumps. I assume it has to do with the fact that the memory allocator adjusts the address before returning the chunk and free() then gets confused. Does this seem to be the issue? Or am I using it wrong... Here is the code that crashes AUTO_OP read(p, s, i|ic) { char *tmp; ParrotIO * io; STRING *s; INTVAL len = $3; string_destroy($2); <-- Seg fault [rest of code omitted] } The Parrot code is as follows: set S1, "/tmp/test.txt" set S2, ">>" open P1, S1, S2 write P1, "Testing...\n" close P1 set S2, "<" open P1, S1, S2 read P1, S1, 7 print S1 print "\n" close P1 end Thanks for any suggestions, -Melvin
Re: Re: IO call names
So for example, open in an int context does a raw open, open in a scalar or PMC context does a fancy open (buffered or whatever) and returns a IO object? Also, if you want the interface to be the same for all these ops, how do you want callbacks implemented? 1) Are we doing callbacks? 2) If so, I assume they are PMC/subroutines or however you guys call that (I'm still unclear on where we use PMCs). We can either define things like callbacks and filters now by starting an RFC if there isn't one already, or we can put those things on the back burner and at last start with a skeleton that at least does all of the synchronous stuff, etc. I can start an RFC draft if you like. More questions...(don't you love newbies!) Is there a goal to be completely stdio independant or are we providing a compile time option like Perl to override? The latter implies the ifdef jungle of PerlIO :) -Melvin Dan Sugalski <[EMAIL PROTECTED]> wrote: > At 01:19 PM 12/6/2001 -0500, Melvin Smith wrote:>Would we prefer to leave the >current system call names as-is (open,close >read,seek) as the direct call through versions and name >the IO routines pio_open, ... or go the route of Perl and do >sys_open, etc. for the raw system call versions and name the Parrot IO >API as the default names (open,close)? The intent's those opcodes are all there is for the I/O system presented to a parrot program, and the opcode functions do the right thing depending on things like filenames, flags passed to open, and the filters pushed on the I/O streams. Arguably open needs beefing, and we might need to define how filters get pushed, popped, spliced, listed, called, and generally work (well, OK, it'd be good if we told people what they needed to do... :) but doing: read P1, P2 should read a line/record regardless of whether the filehandle's open on a socket, a file, a sub-process, some sort of tied thingie, or a database, and it shouldn't matter if there are a dozen filters all twiddling the data between source and sink. That was my plan, at least. :) Dan --"it's like this"--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
PATCH: test_prog -d flag
Here we go... Just adds the debug flag for test_prog. -Melvin --- orig/parrot/test_main.c Mon Oct 22 21:00:01 2001 +++ parrot/test_main.c Wed Dec 5 18:13:03 2001 @@ -20,6 +20,7 @@ int bounds_checking; int profiling; int tracing; +int debugging; struct Parrot_Interp *interpreter; init_world(); @@ -36,6 +37,7 @@ bounds_checking = 0; profiling = 0; tracing = 0; +debugging = 0; while (argc > 1 && argv[1][0] == '-') { if (argv[1][1] == 'b' && argv[1][2] == '\0') { @@ -59,6 +61,13 @@ } argc--; } +else if (argv[1][1] == 'd' && argv[1][2] == '\0') { +debugging = 1; +for(i = 2; i < argc; i++) { +argv[i-1] = argv[i]; +} +argc--; +} } /* If we got only the program name, complain */ @@ -105,6 +114,10 @@ return 1; } +if (debugging) { +interpreter->flags |= PARROT_DEBUG_FLAG; +} + if (bounds_checking) { interpreter->flags |= PARROT_BOUNDS_FLAG; }