Re: More Array Behaviors
Damian Conway [EMAIL PROTECTED] writes: Dave Whipp wrote: OK, I've assimilated all that (though it still feels wrong). I think you are saying that of the following, the 4th is an error. my @d = @x but Foo; # error: no values involved in this assignment Correct. Although presumably this: my @d = @x »but« Foo; is okay. I think that the thing that confused is that Cis Vs Cbut distinction was introduced as compile-time vs run-time. It seems that this distinction is not relevant. Let's say it's an approximation to the true. Lets see if I apply this to objects. Objects are variables Yes. (they contain attributes). Yes. All access to objects is via (scalar) references: Yes. these references, being values, can have Cbut properties. Yes. When a method is invoked on an object, that method has an invocant, which enables the method to see the Cbut properties on that reference. Yes. If an object wants to have properties that apply to all references (to a given instance), then that property must be defined as an attribute of the object. Yes. At least, for the new opaque objects. Because there's no way to ascribe the necessary Cis property to them. Bugger. I was hoping that Perl 6 was going to make a Pixie like Object database easier to write; looks like I'm wrong. One of the things Pixie does is to attach its control data by magic to the object itself (rather than any particular variable pointing to it). This lets us do all sorts of useful stuff without invading the object since we know that it will carry our metadata around it. Please, consider making but properties attach to the target of a pointer rather than to the pointer itself. And if you don't want to do that, I'd say that there is a need to be able to specify some new kind of property that does attach to the target. -- Piers
Re: Spare brackets :-)
Damian Conway [EMAIL PROTECTED] writes: This may sound like a silly idea It's been suggested previously. Has anyone considered removing with the syntactic distinction between numeric and string indexing -- that is, between array and hash lookup? Yes. We rejected the idea. In particular, it would seem that %foo[$key] would be just as easy for the compiler to grok as %foo{$key} Sure. But then is this: $ref[$key] an array or hash look-up??? Decided at runtime? -- Piers
[CVS ci] packfile #3 - double/long double PBC support
This patch enables reading 12 byte long doubles on parrot with 8 byte doubles and vv. Writing long doubles via imcc/packout works too. The questions still remains: how portable are these formats? Still borken: assemble.pl leo
Re: occasioanl CVS hickups?
Robert Spier wrote: Odd. There's not enough information in the logs to figure out what's going on. (And the code shouldn't have this kind of failure mode.) If it keeps happening, please keep me in the loop. Did it again. This time w/o error message - it looked totally sane. From yesterdays commit (packfile #2) only the 2nd part arrived at perl.cvs.parrot. leo
Re: Compiling to Parrot
Hi there, didn't have time to reply earlier, had to do some research on Lua and had to get approval for the project, so couldn't let you know earlier. sorry about that. But now, I have it (the approval, that is) so I'll be implementing a compiler for Lua-parrot (most probably IMCC in between). thanks to the people who suggested/replied on my posting. Regards Klaas-Jan - Original Message - From: Dan Sugalski [EMAIL PROTECTED] To: K Stol [EMAIL PROTECTED]; [EMAIL PROTECTED] Sent: Tuesday, January 21, 2003 5:05 PM Subject: Re: Compiling to Parrot At 5:01 PM +0100 1/21/03, K Stol wrote: well, I think not, then I can't help it. What do you think about compiling Lua to parrot (IMCC)? I like the idea, and I don't think you'll see anyone else tackle it for a while. (And if that falls through, there's always LISP... :) From: Dan Sugalski [EMAIL PROTECTED] At 4:46 PM +0100 1/21/03, K Stol wrote: Well, I'd do it as a project for my Bachelor's, so I won't get permission to do such a project, if it already exists. Ah, that could be a problem. Will it be a problem if you start a project that someone else later also starts? From: Dan Sugalski [EMAIL PROTECTED] At 9:17 AM +0100 1/21/03, K Stol wrote: Hi there, A few weeks ago I posted something about a Tcl-parrot compiler, but Will Coleda already was working on such a project. It would be a as a final project for my bachelor's. But because such already exists, I'm looking for something else. If you're interested in doing a Tcl compiler, by all means, go ahead--I wouldn't let the fact that someone else is doing it stop you. The point of doing it is for the experience, which you'll get regardless of any other implementation. It's also distinctly possible that neither you nor Will will finish a full implementation (as it's likely a rather large undertaking for one person) but you'll each have part of it that can be merged together. -- Dan --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: Bytecode metadata
On Sun, 26 Jan 2003, James Mastros wrote: just define a new packfile section, SIGNATURE, that is defined to be a cryptographic signature of all sections previous to it in the file. I'm battling with this in another file format at the moment; if possible can we please *not* have it sensitive to its own location in the file? For example, an auto-dearchive zip-file has its index at the end of the file, so that the code can go at the front. It would be nice if the whole archive could be signed, rather than just the dearchiving code. My suggestion: make the signature define which other parts of the file it applies to, say with a list of region boundaries as byte addresses in the file; that way signature manipulation remains fairly simple, and it's not too hard to check that a given section is spanned by a signature. And you could have multiple signatures applying to different parts of the file (one to the zip archive, another to the unarchiver). And how is this going to interact with -T or whatever we're going to use? Under my suggested scheme, the data would be untainted if it's covered by a verified signature, and tainted if not. -Martin
Securing Parrot ASM
Hi, I've been thinking about how to run un-trusted code, without having to audit every line, or use some sort of sandbox, and was wondering if Parrot could provide a Mandator Access Control mechanism (ala SE Linux/Flask). When assembling Parrot, the assembler could either look in a file or a perl BEGIN type block containing a list of access requests along the lines of: syscall time read-write directory /tmp listen socket 80 connect socket 25 read-write file /etc/shadow These commands should be easy/quick to audit, could be easily generated by higher level language complier from similar directives in whichever language it is compiling, and parrot would guarentee that only these system priviledges were provided. As a sysadmin I'd certainly be gratefull to have a small block to audit and be certain of what a module/program could do to my system. If people think something like this would be usefull, I'd be more than happy to research this further and try to come up with some code Cheers, Tom
Re: Securing Parrot ASM
On Tue, Jan 28, 2003 at 11:41:14AM +, Thomas Whateley wrote: Hi, I've been thinking about how to run un-trusted code, without having to audit every line, or use some sort of sandbox, and was wondering if Parrot could provide a Mandator Access Control mechanism (ala SE Linux/Flask). When assembling Parrot, the assembler could either look in a file or a perl BEGIN type block containing a list of access requests along the lines of: syscall time read-write directory /tmp listen socket 80 connect socket 25 read-write file /etc/shadow If people think something like this would be usefull, I'd be more than happy to research this further and try to come up with some code It would be immensely useful, especially for online, distributed games, in addition to general sysadmin paranoia. I'm very interested in this. It reminds me of the capability systems. I'm admittedly pretty culeless when it comes to this stuff, but it's one of the reasons I really want Parrot to succeed. Of course, doing this on the parrot level brings up some interesting issues; how exactly do we define what needs to be restricted in case of untrusted code? I'm inclined to say All classes, because a white list is really the only sane way to do security, with defaults being classes that are deemed harmless. Also, I think that this should be made as dynamic as possible so different languages/applications can plug in to the security system to overload certain things (intentionally vague here). One other thing to think about is resource limits. It'd be nice to not require `ulimit' or whatever system-specific resource limitation mechanism, but rather rely on the parrot interpreter to baby-sit. Also, it'd make catching these resource-limit violations much more convenient; an exception could be raised (or, e.g., the rate at which bytecodes are executed could be throttled), rather than simply rudely killing the process. For what I want to do, it's not really required, and it's not really relevant to the type of security we're discussing here, but it would still be very, very useful. Some incoherent ramblings of mine about secure distributed code in virtual worlds: http://twistedmatrix.com/users/radix.twistd/Distributed_20Games A really cool research-project language, E, which is all about capabilities: http://erights.org/ CapDesk, a slightly irrelevant but very interesting design for capability UIs: http://www.combex.com/tech/index.html -- Twisted | Christopher Armstrong: International Man of Twistery Radix | Release Manager, Twisted Project -+ http://twistedmatrix.com/users/radix.twistd/
[perl #20584] [PATCH] packfile #5
# New Ticket Created by Leopold Toetsch # Please include the string: [perl #20584] # in the subject line of all future correspondence about this issue. # URL: http://rt.perl.org/rt2/Ticket/Display.html?id=20584 This is a first try to solve the packfile wordsize issues. Could people with 64 bit machines please test this. Also committing some test files to t/native_pbc would be appreciated. TIA, leo -- attachment 1 -- url: http://rt.perl.org/rt2/attach/49836/38410/a8b2d5/packfile5.patch --- parrot/include/parrot/packfile.hTue Jan 28 10:21:49 2003 +++ parrot-leo/include/parrot/packfile.hTue Jan 28 12:56:08 2003 @@ -319,11 +319,13 @@ opcode_t * PackFile_Constant_unpack_key(struct Parrot_Interp *interpreter, struct PackFile * pf, struct PackFile_Constant *, opcode_t * packed); -opcode_t PackFile_fetch_op(struct PackFile *pf, opcode_t *stream); +opcode_t PackFile_fetch_op(struct PackFile *pf, opcode_t **stream); -INTVAL PackFile_fetch_iv(struct PackFile *pf, opcode_t *stream); +INTVAL PackFile_fetch_iv(struct PackFile *pf, opcode_t **stream); FLOATVAL PackFile_fetch_nv(struct PackFile *pf, opcode_t **stream); + +char * PackFile_fetch_cstring(struct PackFile *pf, opcode_t **stream); void PackFile_assign_transforms(struct PackFile *pf); --- parrot/packfile.c Tue Jan 28 10:21:48 2003 +++ parrot-leo/packfile.c Tue Jan 28 14:26:02 2003 @@ -95,13 +95,16 @@ ***/ opcode_t -PackFile_fetch_op(struct PackFile *pf, opcode_t *stream) { -if(pf-fetch_op == NULL) -return *stream; +PackFile_fetch_op(struct PackFile *pf, opcode_t **stream) { +opcode_t o; +if (!pf-fetch_op) +return *(*stream)++; #if TRACE_PACKFILE == 2 PIO_eprintf(NULL, PackFile_fetch_op: Reordering.\n); #endif -return (pf-fetch_op)(*stream); +o = (pf-fetch_op)(**stream); +((unsigned char *) (*stream)) += pf-header-wordsize; +return o; } /*** @@ -116,10 +119,12 @@ ***/ INTVAL -PackFile_fetch_iv(struct PackFile *pf, opcode_t *stream) { +PackFile_fetch_iv(struct PackFile *pf, opcode_t **stream) { if(pf-fetch_iv == NULL) -return *stream; -return (pf-fetch_iv)(*stream); +return *(*stream++); +PIO_eprintf(NULL, PackFile_fetch_iv: Unsupported.\n); +exit(1); +return (pf-fetch_iv)(**stream); } /*** @@ -142,7 +147,7 @@ FLOATVAL f; HUGEFLOATVAL g; double d; -if(pf-fetch_nv == NULL) { +if (!pf-fetch_nv) { #if TRACE_PACKFILE PIO_eprintf(NULL, PackFile_fetch_nv: Native [%d bytes]..\n, sizeof(FLOATVAL)); @@ -171,14 +176,52 @@ return f; } +static opcode_t +fetch_op_mixed(opcode_t b) +{ +union { +unsigned char buf[8]; +opcode_t o1; +opcode_t o2; +} u; +opcode_t o; + +#if PARROT_BIGENDIAN +# if OPCODE_T_SIZE == 4 + /* wordsize = 8 then */ + fetch_buf_le_8(u.buf, (unsigned char *) b); + return u.o2; /* or u.o1 */ +# else + o = fetch_op_le(b);/* or fetch_be_le_4 and convert? */ + return o 32;/* or o 0x */ +# endif +#else +# if OPCODE_T_SIZE == 4 + /* wordsize = 8 then */ + fetch_buf_be_8(u.buf, (unsigned char *) b); + return u.o1; /* or u.o2 */ +# else + o = fetch_op_be(b); + return o 0x; +# endif + +#endif +} /* * Assign transform functions to vtable */ -void PackFile_assign_transforms(struct PackFile *pf) { +void +PackFile_assign_transforms(struct PackFile *pf) { #if PARROT_BIGENDIAN if(pf-header-byteorder != PARROT_BIGENDIAN) { pf-need_endianize = 1; +if (pf-header-wordsize == sizeof(opcode_t)) pf-fetch_op = fetch_op_le; +else { +pf-need_wordsize = 1; +pf-fetch_op = fetch_op_mixed; +} + pf-fetch_iv = fetch_iv_le; if (pf-header-floattype == 0) pf-fetch_nv = fetch_buf_le_8; @@ -188,7 +231,13 @@ #else if(pf-header-byteorder != PARROT_BIGENDIAN) { pf-need_endianize = 1; +if (pf-header-wordsize == sizeof(opcode_t)) { pf-fetch_op = fetch_op_be; +} +else { +pf-need_wordsize = 1; +pf-fetch_op = fetch_op_mixed; +} pf-fetch_iv = fetch_iv_be; if (pf-header-floattype == 0) pf-fetch_nv = fetch_buf_be_8; @@ -201,12 +250,16 @@ else if (NUMVAL_SIZE != 8 pf-header-floattype == 0) pf-fetch_nv = fetch_buf_le_8; /* XXX else */ +} # if TRACE_PACKFILE PIO_eprintf(NULL, header-byteorder [%d] native byteorder [%d]\n, pf-header-byteorder, PARROT_BIGENDIAN); # endif -} #endif +if (pf-header-wordsize != sizeof(opcode_t)) { +pf-need_wordsize = 1; +pf-fetch_op = fetch_op_mixed; +} }
Re: Spare brackets :-(
ECMAscript already tried this. Bad idea. If your hash keys happen to look like large numbers (e.g. you have 7-digit product codes) as soon as you store one of them, it says: Oh, this looks like a number, so we'll store it like an array and happily creates a million empty array entries for you. ~ John Williams On Tue, 28 Jan 2003 [EMAIL PROTECTED] wrote: This may sound like a silly idea but ... Has anyone considered removing with the syntactic distinction between numeric and string indexing -- that is, between array and hash lookup? In particular, it would seem that %foo[$key] would be just as easy for the compiler to grok as %foo{$key} but would mean that we could stop worrying about the precedence of postfix/infix {, and things like if %test { $count++ } would not require whitespace before the { to be disambiguated. I don't have a complete solution as anonymous array and hash construction would still need different syntaces, but has anyone else thought about this? - Martin
Re: Securing Parrot ASM
On Tue, Jan 28, 2003 at 11:41:14AM +, Thomas Whateley wrote: I've been thinking about how to run un-trusted code, without having to audit every line, or use some sort of sandbox, [snip] block to audit and be certain of what a module/program could do to my system. As author of http://dev.perl.org/rfc/353.pod, I thought somewhat about these issues, and eventually hit a rather hard brick wall. What happens when you link in some module that's written natively? Basically, my conclusion was that this was, unfortunately, still necessary, but once you do it, then all bets about restriction and security are off. If you can get around the necessity of that (and only allow things which are parrot-native, then you can control it). If people think something like this would be usefull, I'd be more than happy to research this further and try to come up with some code I suspect you'll end up hitting the same problems as I did, but if you want to do it in the situations where there is no linking allowed, then it's probably sane. MBM -- Matthew Byng-Maddick [EMAIL PROTECTED] http://colondot.net/
Re: Securing Parrot ASM
On Tue, Jan 28, 2003 at 02:11:39PM +, Matthew Byng-Maddick wrote: On Tue, Jan 28, 2003 at 11:41:14AM +, Thomas Whateley wrote: I've been thinking about how to run un-trusted code, without having to audit every line, or use some sort of sandbox, [snip] block to audit and be certain of what a module/program could do to my system. As author of http://dev.perl.org/rfc/353.pod, I thought somewhat about these issues, and eventually hit a rather hard brick wall. What happens when you link in some module that's written natively? Basically, my conclusion was that this was, unfortunately, still necessary, but once you do it, then all bets about restriction and security are off. If you can get around the necessity of that (and only allow things which are parrot-native, then you can control it). Hrm, maybe I just don't know what's going on, but I'm not sure why this is a problem. Couldn't call out to native functions or perhaps call out to native functions in this library or even call out to *this* native function be a capability? AFAIC (which means for the applications I'm interested in), any of the three are still Good Enough. I guess what I'm saying is, sure, you can't stop a native function (which was called from parrot code) from doing whatever it wants, but you can still prevent the parrot code from using that function in the first place (right?). -- Twisted | Christopher Armstrong: International Man of Twistery Radix | Release Manager, Twisted Project -+ http://twistedmatrix.com/users/radix.twistd/
Re: Securing Parrot ASM
On Tue, Jan 28, 2003 at 11:04:43AM -0500, Christopher Armstrong wrote: On Tue, Jan 28, 2003 at 02:11:39PM +, Matthew Byng-Maddick wrote: What happens when you link in some module that's written natively? Basically, my conclusion was that this was, unfortunately, still Hrm, maybe I just don't know what's going on, but I'm not sure why this is a problem. Couldn't call out to native functions or perhaps call out to native functions in this library or even call out to *this* native function be a capability? AFAIC (which means for the applications I'm interested in), any of the three are still Good Enough. Oh, yes, it is still a reasonable capability, but at the point that you allow that capability, you can forget any of the rest of them. (this is of course, ignoring any possible buffer overruns/format string/double-free or other types of vulnerability where you can change the executed code). I guess what I'm saying is, sure, you can't stop a native function (which was called from parrot code) from doing whatever it wants, but you can still prevent the parrot code from using that function in the first place (right?). Yes, but looking at the current Perl core, a large number of the day-to-day useful modules are written native (read: in C), so you end up losing there. That's not to say that future ones will have to be, but... In reality, however, the problem as I see it is that this is a capability which, once acquired overrides all others (wheras the others can be mutually orthogonal). MBM -- Matthew Byng-Maddick [EMAIL PROTECTED] http://colondot.net/
Re: Securing Parrot ASM
I've been thinking about how to run un-trusted code, without having to audit every line, or use some sort of sandbox, and was wondering if Parrot could provide a Mandator Access Control mechanism (ala SE Linux/Flask). I think that this is a great idea. When assembling Parrot, the assembler could either look in a file or a perl BEGIN type block containing a list of access requests along the lines of: syscall time read-write directory /tmp listen socket 80 connect socket 25 read-write file /etc/shadow Wouldn't it also help to add a chroot layer? In my mind, /etc/passwd should not even _exist_ to untrusted code. It should be chrooted to its own dir. Yes, I realize that one can concievable break out of a chroot, but this should be made really hard. Fred Ollinger
Re: Securing Parrot ASM
Pardon my ignorance on the whole issue but I'm just a lurker trying to understand enough to help out. =) I know security on parrot like this would be difficult, and this thread is specifically about securing PASM, but what about something like FreeBSD's 'jail' command built in? That way, even untrusted code could possibly be set(somewhere in the parrot configuration, compile time, command line, enviroment, etc...) to be jailed. The man pages are here: http://www.freebsd.org/cgi/man.cgi?query=jailapropos=0sektion=0manpath=FreeBSD+5.0-currentformat=html http://www.freebsd.org/cgi/man.cgi?query=jailapropos=0sektion=2manpath=FreeBSD+5.0-currentformat=html I don't know if this will help or not, but this discussion brought to my mind this. If anyone wants the source it should be available via the web or I can forward it to any who ask. Sorry for being so out of the loop. =/ --Joseph Guhlin http://www.josephguhlin.com/
Re: Spare brackets :-)
Sure. But then is this: $ref[$key] an array or hash look-up??? Decided at runtime? Doesn't help if $ref refers to a type that has both hash-like and array-like accessability. And that will be very common, since all Perl 6 regexes return such objects. Damian
Re: More Array Behaviors
At 8:46 AM + 1/28/03, Piers Cawley wrote: Bugger. I was hoping that Perl 6 was going to make a Pixie like Object database easier to write; looks like I'm wrong. One of the things Pixie does is to attach its control data by magic to the object itself (rather than any particular variable pointing to it). This lets us do all sorts of useful stuff without invading the object since we know that it will carry our metadata around it. Please, consider making but properties attach to the target of a pointer rather than to the pointer itself. And if you don't want to do that, I'd say that there is a need to be able to specify some new kind of property that does attach to the target. Given the current implementation (And I hear rumors that you lurk on the internals list, Piers... :) I think we can manage something like this. Might require a drop to assembly somewhere, perhaps hidden behind a function rather than with syntax, but it should be doable... -- Dan --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: Spare brackets :-)
At 8:47 AM + 1/28/03, Piers Cawley wrote: Damian Conway [EMAIL PROTECTED] writes: Sure. But then is this: $ref[$key] an array or hash look-up??? Decided at runtime? How? People use strings as array indices and ints/floats as hash indices, and count on autoconversion to Make It Work. -- Dan --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: More Array Behaviors
Piers Cawley wrote: Bugger. I was hoping that Perl 6 was going to make a Pixie like Object database easier to write; looks like I'm wrong. One of the things Pixie does is to attach its control data by magic to the object itself (rather than any particular variable pointing to it). This lets us do all sorts of useful stuff without invading the object since we know that it will carry our metadata around it. Please, consider making but properties attach to the target of a pointer rather than to the pointer itself. Errno. That's rather the whole point of Cbut properties [*]. However, if your class returns a reference to an object and that reference has the appropriate value properties ascribed to it (i.e. stuck on the *reference*), then any access through that reference (or through any subsequent copy of it) will see those properties. So your magic Pixie would look something like class Pixie { method new($class: %args) { return $class.SUPER::new(%args) but Magic('whatever'); } # etc. } Damian [*] People, we just *have* to find better names for these things! I'd suggest we henceforth call them value properties (for Cbut) and referent properties (for Cis).
Re: Spare brackets :-)
--- Dan Sugalski [EMAIL PROTECTED] wrote: At 8:47 AM + 1/28/03, Piers Cawley wrote: Damian Conway [EMAIL PROTECTED] writes: Sure. But then is this: $ref[$key] an array or hash look-up??? Decided at runtime? How? People use strings as array indices and ints/floats as hash indices, and count on autoconversion to Make It Work. On the one hand: Java/ECMA/J-script does it. All objects are associative arrays. All arrays can be associative, on demand. Ca[foo] = 1; Presto. Associative array. On the other hand: This gets dangerous really quickly, since Perl's autoconversion works differently. Specifically, since we treat things as strings unless they need to be otherwise rather than treating them as the type that they were when you created them. (Javascript doesn't have much in the way of I/O, so the act of getting data in is a bit of an effort, and that effort usually has the side effect of providing type data.) Writing a roulette game may get challenging: @colors[0] = Green; @colors[00] = Green; @colors[000] = Green; Do those get autoconverted to numbers? (They can, obviously. But they shouldn't.) This kind of thing points back at a discussion we had once before about more kinds of context -- meaning at the time numeric versus string versus ... whatever. I think that if we do this we'd better know more about what we're expecting versus what we're losing. On the losing side, the difference between lowercase-array and lowercase-hash is probably significant, performance-wise. Merging the array and hash notions may cost a lot of speed for a lot of people. On the gaining/expecting side is ... what? Freeing up curly braces? Improving the syntax? Other stuff not obvious to me right now? If we go that route, we could certainly include a new pure array type: my @trough = slop(); my PureArray @pa = ducks(); my PureHash @ph = dictionary(); =Austin
Re: More Array Behaviors
Dan Sugalski [EMAIL PROTECTED] writes: At 8:46 AM + 1/28/03, Piers Cawley wrote: Bugger. I was hoping that Perl 6 was going to make a Pixie like Object database easier to write; looks like I'm wrong. One of the things Pixie does is to attach its control data by magic to the object itself (rather than any particular variable pointing to it). This lets us do all sorts of useful stuff without invading the object since we know that it will carry our metadata around it. Please, consider making but properties attach to the target of a pointer rather than to the pointer itself. And if you don't want to do that, I'd say that there is a need to be able to specify some new kind of property that does attach to the target. Given the current implementation (And I hear rumors that you lurk on the internals list, Piers... :) Nah, that's someone else with the same name and writing style. I think we can manage something like this. Might require a drop to assembly somewhere, perhaps hidden behind a function rather than with syntax, but it should be doable... Breaths sigh of relief -- Piers
Re: More Array Behaviors
Damian Conway [EMAIL PROTECTED] writes: Piers Cawley wrote: Bugger. I was hoping that Perl 6 was going to make a Pixie like Object database easier to write; looks like I'm wrong. One of the things Pixie does is to attach its control data by magic to the object itself (rather than any particular variable pointing to it). This lets us do all sorts of useful stuff without invading the object since we know that it will carry our metadata around it. Please, consider making but properties attach to the target of a pointer rather than to the pointer itself. Errno. That's rather the whole point of Cbut properties [*]. However, if your class returns a reference to an object and that reference has the appropriate value properties ascribed to it (i.e. stuck on the *reference*), then any access through that reference (or through any subsequent copy of it) will see those properties. So your magic Pixie would look something like class Pixie { method new($class: %args) { return $class.SUPER::new(%args) but Magic('whatever'); } # etc. } [*] People, we just *have* to find better names for these things! I'd suggest we henceforth call them value properties (for Cbut) and referent properties (for Cis). Hmm... Here's Pixie's interface: my $pixie = Pixie.new.connect('dbi:foo:bar', $user, $pass); ... my $cookie = $pixie.insert($some_arbitrary_object); # Time passes, we forget everything but the value of $cookie my $pixie = Pixie.new.connect('dbi:foo:bar', $user, $pass); my $locked_object = $pixie.get_with_locking_strategy( $cookie, Strategy::ExclusiveLock.new ); When $locked_object is collected, Pixie unlocks it in the database. If, while $locked_object is in scope something else tries to fetch an object with the same cookie (from within the same thread) then Pixie hands 'em a reference to the same object. Right now Pixie keeps a handle on everything by attaching 'ObjectInfo' objects to every object that comes under its control so it can keep track of what's going on (it's a two way link, the object has a normal ref (via magic) to the objectinfo, which has a weakref back to the object itself, so when the object goes out of scope, the objectinfo goes out of scope too, triggering unlocking and other such stuff). So, if Pixie::get_with_locking_strategy looks something like: method get_with_locking_strategy( $self: $oid, Strategy $lock_strategy ) { return $self.cache_get($oid); CATCH Exception::NotInCache { my $flattened_object = $self.store.get_with_locking_strategy($oid, $lock_strategy); my $obj_info = ObjectInfo.new.set_pixie($self) .set_oid($oid) .set_lock_strategy($lock_strategy); my $real_obj = Storable::retrieve($flattened_object) but PixieInfo($obj_info); $obj_info.set_real_object($real_obj); $self.cache_insert($obj_info); return $real_obj; } } method cache_get( $oid ) { my $info = %.cache{$oid} // die Exception::NotInCache.new; return $info.real_object; } Will every copy of the value of $real_obj have the PixieInfo property? Am I going mad, or is order really that important? My initial idea was to do the Cset_real_object and Ccache_insert and then just do Creturn $real_obj but PixieInfo($obj_info), but then, if I understand you correctly, the Info object's back reference to the real object wouldn't have a PixieInfo property, which would in turn mean that cache_get's return value wouldn't have the property and Bad Things would happen. It seems counterintuitive that the order of operations in this case should be so important. Or am I missing something? -- Piers
More Array Behaviors (Take 2)
OK, here are the answers so far -- or more accurately, strawman interpretations of those answers that should be objected to if they're wrong. 1) Edge cases in array indexing: my int @a = (1,2,3); @a[0] # 1 @a[1] # 2 @a[2] # 3 @a[3] # undef (warning: index out-of-bounds) @a[2**128]# EXCEPTION: index is above max allowed index @a[ Inf ] # undef (warning: can't use Inf as array index) @a[ undef ] # 1 (warning: undefined index) @a['foo'] # 1 (warning: non-numeric index) @a[ NaN ] # EXCEPTION: can't use NaN as array index @a[-1]# 3 @a[-2]# 2 @a[-3]# 1 @a[-4]# undef (warning: index out-of-bounds) @a[-Inf] # undef (warning: can't use Inf as array index) 2) There is a platform-dependent maximum array size, ((2**32)-1 for 32-bit platforms.) Attempting to access an index outside that range throws an exception. Note that this applies to both 'real' and 'sparse' arrays. If these are incorrect statements, please correct me. --- The other two questions are less definitively answered, so far. I shall attempt to disambiguate them... MikeL
Re: More Array Behaviors (Take 2)
At 10:13 AM -0800 1/28/03, Michael Lazzaro wrote: OK, here are the answers so far -- or more accurately, strawman interpretations of those answers that should be objected to if they're wrong. I think some of this is incorrect which, because Damian thinks otherwise, will need some hashing out from Larry on how he wants perl arrays to behave. Because... 1) Edge cases in array indexing: my int @a = (1,2,3); @a[3] # undef (warning: index out-of-bounds) Or a real 0, since you said @a can only return integers. @a[2**128]# EXCEPTION: index is above max allowed index Except we can manage this internally, so I don't know that it's a problem @a[ Inf ] # undef (warning: can't use Inf as array index) I'd throw an exception here. @a[-4]# undef (warning: index out-of-bounds) @a[-Inf] # undef (warning: can't use Inf as array index) Or zero, since it's an int array. 2) There is a platform-dependent maximum array size, ((2**32)-1 for 32-bit platforms.) Attempting to access an index outside that range throws an exception. Note that this applies to both 'real' and 'sparse' arrays. But since we've got big(int|float|rat)s there's no real reason for that to be a problem. If you want to use 10**100**100 as an array index, you could just throw an awful lot of memory at us... -- Dan --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: More Array Behaviors (Take 2)
On Tuesday, January 28, 2003, at 10:20 AM, Dan Sugalski wrote: At 10:13 AM -0800 1/28/03, Michael Lazzaro wrote: 1) Edge cases in array indexing: my int @a = (1,2,3); @a[3] # undef (warning: index out-of-bounds) Or a real 0, since you said @a can only return integers. Thanks -- I stand corrected, you're almost certainly right. Arrays defined with a cell type that can't be undefined should never return undef, regardless of index. They should instead return the default for that type... int(0), num(0.0), str(''), whatever... I also agree @a[Inf] is worthy of an exception. Without objection, so documented. MikeL
Re: More Array Behaviors (Take 2)
On Tue, 28 Jan 2003, Dan Sugalski wrote: At 10:13 AM -0800 1/28/03, Michael Lazzaro wrote: @a[ Inf ] # undef (warning: can't use Inf as array index) I'd throw an exception here. @a[-4]# undef (warning: index out-of-bounds) @a[-Inf] # undef (warning: can't use Inf as array index) Or zero, since it's an int array. If you throw an exception for +Inf, you should throw it for -Inf too. ~ John Williams
Arrays: Default Values
There has been discussion of allowing a default value for array cells -- that is, one aside from Cundef or whatever the type-specific default is. Questions, in order of increased evilness: 1) What's the final decided syntax? Two possibilities: my @a is Array( default = 'foo' ); # attrib? my @a is default('foo');# property? 2) Assume the default value is a simple value, e.g. 'foo'. my @a is Array( default = 'foo' ); @a[5] = 'bar'; @a[4]; # 'foo' @a[5]; # 'bar' @a[6]; # 'foo' @a[-1];# 'bar' *NOTE!* @a[-3];# 'foo' @a[-10]; # 'foo' Correct? 2a) When a cell is explicitly re-undefined, does the default value take effect? my @a is Array( default = 'foo' ) = (1,2,3); @a[1] = undef; @a[1]; # undef, or 'foo'? STRAWMAN ANSWER: 'foo'. 2b) Primitive-typed arrays: Given the behavior of (2a), and the fact that primitive-typed arrays can't store undef, what happens here? my int @a is Array( default = 5 ); @a[0] = 0; @a[0]; # 0, or 5? @a[0] = undef; @a[0]; # 0, or 5? STRAWMAN ANSWER: 5, in both cases. So don't do that unless you mean it. 3) Can the default value be a closure, based on index location? my @a is Array( default = { $_ ** 2 }); STRAWMAN ANSWER: Yes, because it's cool. 3a) NOTE that closure-based defaults effectively render the array infinite. Therefore -- If the requested index is negative, what happens? @a[-5]; STRAWMAN ANSWER: The closure just gets a negative number as the requested index. It's up to you to make it work, if you want it to. 3b) Does an infinite array still get an exception thrown when trying to access an infinite [Inf] or [-Inf] index? STRAWMAN ANSWER: Yes, it does. MikeL
Re: Securing Parrot ASM
On Tue, Jan 28, 2003 at 10:39:33AM -0600, Joseph Guhlin wrote: Pardon my ignorance on the whole issue but I'm just a lurker trying to understand enough to help out. =) I know security on parrot like this would be difficult, and this thread is specifically about securing PASM, but what about something like FreeBSD's 'jail' command built in? That way, even untrusted code could possibly be set(somewhere in the parrot configuration, compile time, command line, enviroment, etc...) to be jailed. The man pages are here: The fun bit is getting information in and out of the jail. ie. you trust a module but not one of its derived classes Also, you might wish to note that both jail(2) and chroot(2) need to be called by the super-user. The reason you can't do a jail entirely within Parrot is that the moment you link to any native code, all bets are off, and you're reliant on what the kernel allows you to do. Parrot can no longer control it. It seems to me that the linking with native code is going to end up being one that most people switch on, because it will be necessary and/or useful in getting anything done. MBM -- Matthew Byng-Maddick [EMAIL PROTECTED] http://colondot.net/
RE: Securing Parrot ASM
Christopher Armstrong: # One other thing to think about is resource limits. It'd be nice to not # require `ulimit' or whatever system-specific resource limitation # mechanism, but rather rely on the parrot interpreter to # baby-sit. Also, it'd make catching these resource-limit violations # much more convenient; an exception could be raised (or, e.g., the rate # at which bytecodes are executed could be throttled), rather than # simply rudely killing the process. For what I want to do, it's not # really required, and it's not really relevant to the type of security # we're discussing here, but it would still be very, very useful. I don't see why Parrot couldn't do much of this. It can certainly audit allocations made through its own memory-allocation system, and with only a little help from the system it should be able to audit its processor usage as well (at least within Parrot bytecode). I'm not sure about disk space usage, but that's a pretty OS-level thing anyway. --Brent Dax [EMAIL PROTECTED] @roles=map {Parrot $_} qw(embedding regexen Configure) How do you test this 'God' to prove it is who it says it is? If you're God, you know exactly what it would take to convince me. Do that. --Marc Fleury on alt.atheism
RE: Securing Parrot ASM
Matthew Byng-Maddick: # It seems to me that the linking with native code is going to # end up being one that most people switch on, because it will # be necessary and/or useful in getting anything done. Then make sure that link in native code isn't a permission--link in native code library X is. That way the sysadmin or whoever can make sure that only harmless libraries get in. The control mechanism could be in Parrot_dlopen, which would make (well-behaved) native libraries subject to it too. --Brent Dax [EMAIL PROTECTED] @roles=map {Parrot $_} qw(embedding regexen Configure) How do you test this 'God' to prove it is who it says it is? If you're God, you know exactly what it would take to convince me. Do that. --Marc Fleury on alt.atheism
Re: Securing Parrot ASM
On Tue, Jan 28, 2003 at 09:24:20AM -0800, Brent Dax wrote: Christopher Armstrong: # One other thing to think about is resource limits. It'd be nice to not # require `ulimit' or whatever system-specific resource limitation # mechanism, but rather rely on the parrot interpreter to # baby-sit. Also, it'd make catching these resource-limit violations # much more convenient; an exception could be raised (or, e.g., the rate # at which bytecodes are executed could be throttled), rather than # simply rudely killing the process. For what I want to do, it's not # really required, and it's not really relevant to the type of security # we're discussing here, but it would still be very, very useful. I don't see why Parrot couldn't do much of this. It can certainly audit allocations made through its own memory-allocation system, and with only a little help from the system it should be able to audit its processor usage as well (at least within Parrot bytecode). I'm not sure about disk space usage, but that's a pretty OS-level thing anyway. Cool. I'm really only concerned about CPU and memory usage, as I'd never allow plain file I/O to my untrusted code -- just application-level APIs for doing specific things that might access the disk. -- Twisted | Christopher Armstrong: International Man of Twistery Radix | Release Manager, Twisted Project -+ http://twistedmatrix.com/users/radix.twistd/
Re: [Introspector-developers] Re: Bytecode metadata
If memory serves me right, James Michael DuPont wrote: Bah. That's parrot -o foo.o foo.pmc isn't it? And if we make C a parrot supported language we can even build parrot with parrot? Hmmm... bootstrapping 1. The gcc : I have %99 of the information about the function bodies of parrot c source code in rdf/xml. That could be fed to parrot. That would only be part of the issue ... generating stuff out of RDF AST's is only half of our trouble ... In fact , I think it would be much easier if someone managed to convert RTL into Parrot (a gcc backend) ... It won't be a hack like egcs-jvm since Parrot is already a register machine and has pointer instructions already ... 2. The pnet/C : there has been work done by Rhys to make a managed c compiler for pnet. Gopal has been working on a parrot bytecode emitter for Pnet. Well ... I haven't been working on parrot bytecode emitter ... It's just that we have a pm_codegen.tc treecc handler inside parrot which is stubbed up ... until I can do some kind of class generation for parrot, the C# AST cannot be used for anything useful. Also Dan was not so hot about having a C compiler for Parrot when he met Rhys on IRC... So I'm sticking to compiling C# to Parrot as an aim , and *maybe* Java as well ... Gopal -- The difference between insanity and genius is measured by success
Re: Securing Parrot ASM
On Tue, Jan 28, 2003 at 04:15:41PM +, Matthew Byng-Maddick wrote: On Tue, Jan 28, 2003 at 11:04:43AM -0500, Christopher Armstrong wrote: Hrm, maybe I just don't know what's going on, but I'm not sure why this is a problem. Couldn't call out to native functions or perhaps call out to native functions in this library or even call out to *this* native function be a capability? AFAIC (which means for the applications I'm interested in), any of the three are still Good Enough. Oh, yes, it is still a reasonable capability, but at the point that you allow that capability, you can forget any of the rest of them. (this is of course, ignoring any possible buffer overruns/format string/double-free or other types of vulnerability where you can change the executed code). I guess what I'm saying is, sure, you can't stop a native function (which was called from parrot code) from doing whatever it wants, but you can still prevent the parrot code from using that function in the first place (right?). Yes, but looking at the current Perl core, a large number of the day-to-day useful modules are written native (read: in C), so you end up losing there. That's not to say that future ones will have to be, but... In reality, however, the problem as I see it is that this is a capability which, once acquired overrides all others (wheras the others can be mutually orthogonal). I don't see why. Why, for example, would socket access (a C-implemented feature, certainly) allow access to everything else? Presumably, the built-in parrot[/perl/python/etc] functionality is `trusted' such that it only does what it *says* it does (after auditing and removing 100% of the bugs, of course ;), and you can give out caps to them on a per-func/obj/whatever basis. But maybe I'm totally misunderstanding what you're saying. Are you just assuming there are bugs in all native built-in functions that are exploitable to gain other capabilities? An example would help. Anyway, even if you're right about access to native functions, the way I'm thinking I'd use this would be to have a trusted program (i.e., my imaginary virtual world engine) running various untrusted scripts from around the world. With such a system, we'll just give the untrusted code access to application-level APIs (which themselves will be managed with capabilities, to facilitate differing trust levels) -- no need to give them access to such things as file I/O and whatnot. However, I guess socket access would be required, as I plan on having them executed in a separate process (with ulimits) and talk to the main system through an RPC (i.e., sockets) mechanism, with probably a chroot/jail thrown in for good measure, but this is an easy thing to audit and I can't see ever requiring more low-level access. -- Twisted | Christopher Armstrong: International Man of Twistery Radix | Release Manager, Twisted Project + http://twistedmatrix.com/users/radix.twistd/
Re: Securing Parrot ASM
On Tue, Jan 28, 2003 at 10:39:33AM -0600, Joseph Guhlin wrote: Pardon my ignorance on the whole issue but I'm just a lurker trying to understand enough to help out. =) I know security on parrot like this would be difficult, and this thread is specifically about securing PASM, but what about something like FreeBSD's 'jail' command built in? That way, even untrusted code could possibly be set(somewhere in the parrot configuration, compile time, command line, enviroment, etc...) to be jailed. The man pages are here: The fun bit is getting information in and out of the jail. ie. you trust a module but not one of its derived classes Also, you might wish to note that both jail(2) and chroot(2) need to be called by the super-user. If it's jail and something gets out then it's not a good jail. To me jail means what you see is what you get. If you trust code in the jail, then you should be ok. The reason you can't do a jail entirely within Parrot is that the moment you link to any native code, all bets are off, and you're reliant on what the kernel allows you to do. Parrot can no longer control it. This is probably out of my league. I'm sure your right on this one. It seems to me that the linking with native code is going to end up being one that most people switch on, because it will be necessary and/or useful in getting anything done. Well, that would mean two parrots, which I think is great. One w/ jail for untested code, one w/o. I see four levels of security: 1. chroot jail where / is the toplevel /parrot. If there needs to be a /usr, then it's /parrot/usr. 2. catching all calls and keeping them in line. For example, there are some opcodes that can't work in sandbox mode. 3. Manually checking over all libs that are linked to for safety. Only safe (those that can't be used to pick your way out of jail) go into /jail/lib. 4. Many things that I have not thought of, but are known to others. This is probably overly general and simplistic, but I just want to make sure that when we are talking about #2, people don't think that they mean #3 or heaven forbid #4. Fred Ollinger
Re: Securing Parrot ASM
Matthew == Matthew Byng-Maddick [EMAIL PROTECTED] writes: I guess what I'm saying is, sure, you can't stop a native function (which was called from parrot code) from doing whatever it wants, but you can still prevent the parrot code from using that function in the first place (right?). Yes, but looking at the current Perl core, a large number of the day-to-day useful modules are written native (read: in C), so you end up losing there. That's not to say that future ones will have to be, but... In reality, however, the problem as I see it is that this is a capability which, once acquired overrides all others (wheras the others can be mutually orthogonal). Obviously this is a problem; however, this is one of the easier things to solve in a system like Parrot. By constructing proxies to the native libraries that allow the insertion of checks and restrictions, untrusted code can be written to use the usual APIs without allowing them to assume the full power of the usual implementations. So, presumably, in the restricted environment, File would be replaced with RestrictedFile which only allows accessing files in a certain subdirectory, or owned by a particular user, or such; Socket would be replaced with RestrictedSocket which can only connect to certain hosts, or must prompt for permission to connect; and so on. One could even imagine something like a RestrictedCanvas in a GUI application, that allows the untrusted code to draw only on a small portion of the application's window. All this is very close to achievable in existing language; the *hard* part is preventing DoS attacks on CPU and memory. Parrot should make that doable as well, with the ability to run multiple interpreters in the same process. msg26694/pgp0.pgp Description: PGP signature
Re: Securing Parrot ASM
Brent == Brent Dax [EMAIL PROTECTED] writes: I don't see why Parrot couldn't do much of this. It can certainly audit allocations made through its own memory-allocation system, and with only a little help from the system it should be able to audit its processor usage as well (at least within Parrot bytecode). I'm not sure about disk space usage, but that's a pretty OS-level thing anyway. Shouldn't this be doable by starting a separate 'restricted interpreter' within the process that is given a limit on the number of bytecodes it can run and amount of memory it can allocate? As far as I can see, this would achieve the primary goals of restriction (throttling CPU/memory use). Filesystem usage can be controlled through a restricted file API, so that shouldn't be a very low-level concern. A system that might be worth examining is Darius Bacon's idel, a small VM designed for running untrusted code. http://www.accesscom.com/~darius/software/idel/ msg26695/pgp0.pgp Description: PGP signature
Re: Securing Parrot ASM
At 11:41 AM + 1/28/03, Thomas Whateley wrote: Hi, I've been thinking about how to run un-trusted code, without having to audit every line, or use some sort of sandbox, and was wondering if Parrot could provide a Mandator Access Control mechanism (ala SE Linux/Flask). Ah, I've been hoping to avoid this for a while for sheer, screaming lack of tuits, but... Here's the deal for 'safe mode'. (For background, as everyone in the Unix world seems to be happy reinventing security wheels, surf over to the VMS doc site at http://www.openvms.compaq.com/doc, and the VMS system security manual specifically at either http://www.openvms.compaq.com/doc/731FINAL/6346/6346PRO.HTM or http://www.openvms.compaq.com/doc/731FINAL/DOCUMENTATION/PDF/OVMS_731_SYS_SEC.PDF) Posits: *) We may potentially put in resource quotas. This includes time, memory, CPU usage, I/O requests, and I/O bytes *) We may be running untrustworthy code which will try and subvert the interpreter *) We may potentially put in access controls which restrict what code can do *) Embedders may want to intercept all file IO calls anyway So, here's the facilities we're going to build. *) There'll be an alternate set of opcodes that validate their parameters, so no sneaky trying to access register 554, or branching off the end of the world. *) There'll be a set of 'privileges' of some sort (call 'em capabilities or whatever) and to do various tasks will require that you have an appropriate privilege *) You will be able to load in external code (and I realize that there is the general Hit C and all bets are off rule). External modules will have the potential to have trust rights attached to them so we can mark C things that are actually safe as loadable, or loadable in certain circumstances *) Code segments can be granted temporarily elevated privs, presumably being done for code that's been audited and deemed safe *) Data will be marked as tainted if it comes from an external source for perl/ruby style taint checking -- Dan --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
PyCon
Hey, any Parrot hackers going to the Python convention at the end of March? http://python.org/pycon/. Price will be $150-$200. I'm very interested in meeting and discussing there :-) -- Twisted | Christopher Armstrong: International Man of Twistery Radix | Release Manager, Twisted Project -+ http://twistedmatrix.com/users/radix.twistd/
RE: [CVS ci] packfile #2
Leopold Toetsch wrote: - 8/12 byte float issues are still the same - are these formats really portable, or should we try to store ASCII equivalents? No? ? Because my knowledge here approaches zero, so I'm just aping information back at you from google searches and scanning documents. There are 3 standard binary IEEE 754 derived formats: Single Precision: 4 bytes Double Precision: 8 bytes Quadruple Precision: 16 bytes C99 and Modula 3 provide the recommended IEEE 754 bindings. Java requires it. A nice quote from William Kahan father of IEEE 754: Programmers seem unaware that IEEE 754 is a standard for their programming environment, not just for hardware. Portable Encoding of Floating-Point Values http://research.microsoft.com/~hollasch/cgindex/coding/portfloat.html What hardware supports IEEE 754: http://www.mscs.mu.edu/~georgec/IFAQ/casares1.html IEEE 754 Binary Floating Point William Kahan's documents: http://http.cs.berkeley.edu/~wkahan/ieee754status/ IEE 754 Homepage: http://grouper.ieee.org/groups/754/ Designing Language Support for IEEE 754: http://grouper.ieee.org/groups/754/meeting-materials/2001-10-18-langdesign.p df IEEE 754 Support in C99 http://grouper.ieee.org/groups/754/meeting-materials/2001-07-18-c99.pdf Nice Overview: http://research.microsoft.com/~hollasch/cgindex/coding/ieeefloat.html Nice Links: http://cch.loria.fr/documentation/IEEE754/ IEEE 854 Decimal Floating Point IBM's decNumber Library (non-free ANSI C impl of IEEE 854): http://www2.hursley.ibm.com/decimal/decnumber.html IBM's BigDecimal (open-source impl for Java of IEEE 854): http://www2.hursley.ibm.com/decimalj/ -- Garrett Goebel IS Development Specialist ScriptPro Direct: 913.403.5261 5828 Reeds Road Main: 913.384.1008 Mission, KS 66202 Fax: 913.384.2180 www.scriptpro.com [EMAIL PROTECTED]
Parrot developer world map
Last week I collected your data. This week I bring you pretty pictures: http://www.astray.com/parrot/worldmap/ So London would seem a good place for a Parrot developer day, as would California. I guess most people will be meeting up at Perl conferences anyway. What do people have in mind for such a thing? A room, at least one computer, an internet connection and a Plan? Are you looking to learn more about Parrot or do you have something more specific in mind? Leon ps feel free to send in your location if you've forgotten to: -- Leon Brocard.http://www.astray.com/ scribot.http://www.scribot.com/ ... Famous last words - Don't worry, I can handle it
[perl #20592] [PATCH] save and restore more of a coroutine's context (and more)
# New Ticket Created by Jonathan Sillito # Please include the string: [perl #20592] # in the subject line of all future correspondence about this issue. # URL: http://rt.perl.org/rt2/Ticket/Display.html?id=20592 A description of each attachment: 1) coroutine.t (which should be put in t/pmc/) exposes some errors in our coroutine code. 2) coroutine.patch fixes those errors by saving and restoring more of the coroutine's context. More specifically the user_stack, control_stack and pad_stack. 3) document.patch (some of this this was submitted before but not applied) adds documentation for the push_pad op to core.ops, which I somehow missed. It also brings pdd06_pasm.pod up to date. I do not like maintaining documentation in more than one place, so in pdd06_pasm.pod I put a terse (but correct) description along with a reference to the generated documentation in docs/core_ops.pod. Is that suitable??? And a couple of questions: 1) I was thinking of writing a couple of short how to documents aimed at compiler writers. Is there interest in something like this? 2) Which of the following names are consistent with our current naming convention? new_coroutine coroutine_new Thanks! -- Jonathan Sillito -- attachment 1 -- url: http://rt.perl.org/rt2/attach/49853/38427/66cf1f/coroutine.t -- attachment 2 -- url: http://rt.perl.org/rt2/attach/49853/38428/c9fd23/coroutine.patch -- attachment 3 -- url: http://rt.perl.org/rt2/attach/49853/38429/34393f/document.patch coroutine.t Description: coroutine.t coroutine.patch Description: coroutine.patch document.patch Description: document.patch
Re: Arrays: Default Values
Austin Hastings wrote: --- Damian Conway [EMAIL PROTECTED] wrote: my @a is default(666); print @a[2]; # prints 666 @a[4] = 1; print @a[2]; # now prints undef :-( [typo in third line corrected] I don't understand your example. Can you explain it again, using words of less than one syllable? If the scheme is (as someone was advocating) that the default is used only in place of elements that don't exist (as opposed to elements that are allocated but contain Cundef), then allocating a previously unallocated element has the potential to change the value it returns, even if that previously unallocated element is not initialized when allocated. So, assigning to @a[4], causes @a[0..3] to be created as well. Which, under a default-for-non-existent-elements-only policy causes @a[2] to stop returning the default. What's @[4]? Why does it (even if it's a typo from @a[4]) cause @a[2] to become defined? It doesn't. It causes it to exist. I understood that it was being advocating that only non-existent elements returned the default. This is why that's a bad idea. If it does become defined, why doesn't it get the default value -- no-one explicitly told it to be undef? Because the whole idea of a default value is that you *don't* put it in every element. The lack of a value in an element is what triggers the default value to be returned. And in Perl Cundef is how we signify the absence of a value. Damian
Re: Arrays: Default Values
On Tue, Jan 28, 2003 at 11:15:26AM -0800, Michael Lazzaro wrote: 2) Assume the default value is a simple value, e.g. 'foo'. my @a is Array( default = 'foo' ); @a[5] = 'bar'; @a[4]; # 'foo' @a[5]; # 'bar' @a[6]; # 'foo' @a[-1];# 'bar' *NOTE!* Um ... why? 2a) When a cell is explicitly re-undefined, does the default value take effect? my @a is Array( default = 'foo' ) = (1,2,3); @a[1] = undef; @a[1]; # undef, or 'foo'? STRAWMAN ANSWER: 'foo'. This makes sense to me. 2b) Primitive-typed arrays: Given the behavior of (2a), and the fact that primitive-typed arrays can't store undef, what happens here? my int @a is Array( default = 5 ); @a[0] = 0; @a[0]; # 0, or 5? 0, definitely. @a[0] was defined so the default doesn't come into play. @a[0] = undef; @a[0]; # 0, or 5? 5, because undefined things get the default value. 3) Can the default value be a closure, based on index location? my @a is Array( default = { $_ ** 2 }); STRAWMAN ANSWER: Yes, because it's cool. Heh. I'd want different syntax for default values vs. default generators though. 3a) NOTE that closure-based defaults effectively render the array infinite. Therefore -- If the requested index is negative, what happens? @a[-5]; STRAWMAN ANSWER: The closure just gets a negative number as the requested index. It's up to you to make it work, if you want it to. Sounds good to me. 3b) Does an infinite array still get an exception thrown when trying to access an infinite [Inf] or [-Inf] index? STRAWMAN ANSWER: Yes, it does. Based on 3a, I'd say that the closure should get Inf (or -Inf) and do the appropriate thing. In your example above, Inf**2 would be Inf, so the closure would return Inf (assuming I've got the Inf math right and it's not really NaN) -Scott -- Jonathan Scott Duff [EMAIL PROTECTED]
Re: Arrays: Default Values
Michael Lazzaro said: There has been discussion of allowing a default value for array cells -- that is, one aside from Cundef or whatever the type-specific default is. Questions, in order of increased evilness: 1 and 2 seem fine to me. 2a) When a cell is explicitly re-undefined, does the default value take effect? my @a is Array( default = 'foo' ) = (1,2,3); @a[1] = undef; @a[1]; # undef, or 'foo'? STRAWMAN ANSWER: 'foo'. Seems right to me. Anything else would be very confusing, I think. 2b) Primitive-typed arrays: Given the behavior of (2a), and the fact that primitive-typed arrays can't store undef, what happens here? my int @a is Array( default = 5 ); @a[0] = 0; @a[0]; # 0, or 5? 0. Being unable to store 0 would seem to be a major limitation. @a[0] = undef; @a[0]; # 0, or 5? An exception or 5. Maybe undefining an element could always set it to the default value. 3) Can the default value be a closure, based on index location? my @a is Array( default = { $_ ** 2 }); STRAWMAN ANSWER: Yes, because it's cool. No, because it's unnecessary. You can always do my $value = @a[$x] //= $x ** 2; or skip the = depending on how you are trading memory / speed. Yes, I know that just about everything is unnecessary to someone. To me, default values as a whole seem a little unnecessary, in fact. -- Paul Johnson - [EMAIL PROTECTED] http://www.pjcj.net
Re: Arrays: Default Values
--- Michael Lazzaro [EMAIL PROTECTED] wrote: 1) What's the final decided syntax? Two possibilities: my @a is Array( default = 'foo' ); # attrib? my @a is default('foo');# property? Since we want arrays (lowercase) to support them, too, it should be a property. my @a is default('foo'); 2) Assume the default value is a simple value, e.g. 'foo'. my @a is Array( default = 'foo' ); @a[5] = 'bar'; @a[4]; # 'foo' @a[5]; # 'bar' @a[6]; # 'foo' @a[-1];# 'bar' *NOTE!* @a[-3];# 'foo' @a[-10]; # 'foo' 1) What if I *WANT* an array with negative indices? Case in point, the compare interface returns negative/zero/positive results, so I may want to use those. (Granted, using the -1 as from-end is cool. But writing a hand-optimized table of index values for use in a Shell/Metzner sort is also cool, and depends on quickly getting the results of the carry bit.) 2a) When a cell is explicitly re-undefined, does the default value take effect? No. If I wanted that, I'd say my @a[5] = @a.default; my @a is Array( default = 'foo' ) = (1,2,3); @a[1] = undef; @a[1]; # undef, or 'foo'? STRAWMAN ANSWER: 'foo'. No, undef. OTOH, deleting @a[1] would reset it to default. 2b) Primitive-typed arrays: Given the behavior of (2a), and the fact that primitive-typed arrays can't store undef, what happens here? my int @a is Array( default = 5 ); @a[0] = 0; @a[0]; # 0, or 5? @a[0] = undef; This should cause a blip of some kind. If storing an explicit undef (as opposed to undef but 0 or C$v = undef; @a[0] = $v; there should be an exception. If storing an implicit undef: convert to int (IOW: 0) and emit a warning. @a[0]; # 0, or 5? STRAWMAN ANSWER: 5, in both cases. So don't do that unless you mean it. Again, this is wrong. Storing undef is storing undef. If it's an error, treat it as such. If it's not an error, then let me store it. If I want to store the default value, I have access to the default value and can store it. 3) Can the default value be a closure, based on index location? my @a is Array( default = { $_ ** 2 }); STRAWMAN ANSWER: Yes, because it's cool. No, because defaulting method may be independent of default value. It should be called default_method instead: my @a is default_method( {$_ ** 2} ); 3a) NOTE that closure-based defaults effectively render the array infinite. Therefore -- If the requested index is negative, what happens? @a[-5]; STRAWMAN ANSWER: The closure just gets a negative number as the requested index. It's up to you to make it work, if you want it to. Another question: If you ask for a value and get it, does the array grow? Or does that happen only on assignment? (I favor assignment, but if the closure isn't a pure function, what happens? Can we support differentiating between Cis cached subs (meaning: don't allocate storage for this array element) and not (meaning: you'll want to allocate storage, because this value isn't reproducible)? 3b) Does an infinite array still get an exception thrown when trying to access an infinite [Inf] or [-Inf] index? STRAWMAN ANSWER: Yes, it does. No, it doesn't. Like negative numbers in 3a, Inf gets passed to the default_method, which may itself throw the exception. But since Inf and NaN are floating-point things, they should be easy to recognize. (In fact, I'd say just provide a default sub and show people what it looks like.) ::Array.default_method := ::array.default_method := sub - ($this: $index) { die(Invalid array index) unless $index.isa('int'); # Not setting the array[index] here, since this is cached. return defined($this.default) ? $this.default : undef; } is cached; PS: If the .default changes, the cached values are wrong. How to I tell Perl to clear a function's return cache? Or do we just not cache the function and allocate memory on an index-by-index basis? =Austin
Re: More Array Behaviors (Take 2)
Michael Lazzaro wrote: OK, here are the answers so far -- or more accurately, strawman interpretations of those answers that should be objected to if they're wrong. 1) Edge cases in array indexing: my int @a = (1,2,3); @a[0] # 1 @a[1] # 2 @a[2] # 3 @a[3] # undef (warning: index out-of-bounds) @a[2**128]# EXCEPTION: index is above max allowed index I would have thought that for SparseArrays (which require index remapping anyway), BigInt indices should be allowed. @a[ Inf ] # undef (warning: can't use Inf as array index) I would have though the behaviour of out-of-range indices like @a[2**128] and @a[Inf] ought to be consistent. Hence I'd have expected an exception here. @a[ undef ] # 1 (warning: undefined index) @a['foo'] # 1 (warning: non-numeric index) @a[ NaN ] # EXCEPTION: can't use NaN as array index @a[-1]# 3 @a[-2]# 2 @a[-3]# 1 @a[-4]# undef (warning: index out-of-bounds) @a[-Inf] # undef (warning: can't use Inf as array index) Another exception, I'd have thought. 2) There is a platform-dependent maximum array size, ((2**32)-1 for 32-bit platforms.) Attempting to access an index outside that range throws an exception. Which is why I'd expect that CInf as an index should be fatal. Note that this applies to both 'real' and 'sparse' arrays. Given the implicit promotion of ints to BigInts everywhere else, this seems inconsistent. At least for SparseArrays, where indices are remapped anyway. Damian
Re: Arrays: Default Values
--- Austin Hastings [EMAIL PROTECTED] wrote: No, undef. OTOH, deleting @a[1] would reset it to default. Ere someone flames my for using a hash keyword in an array context: s/deleting/absquatulating (e.g., via pop, shift, or splice)/ =Austin
More Array Behaviors (Take 3)
Corrected in accordance with design team member feedback. These should be solid, now. Thanks much for the responses. 1) Edge cases in array indexing: my @a = (1,2,3); @a[0] # 1 @a[1] # 2 @a[2] # 3 @a[3] # def (warning: index out-of-bounds) @a[2**128]# def (warning: index out-of-bounds) @a[ Inf ] # EXCEPTION: can't use +Inf as array index @a[ undef ] # 1 (warning: undefined index) @a['foo'] # 1 (warning: non-numeric index) @a[ NaN ] # EXCEPTION: can't use NaN as array index @a[-1]# 3 @a[-2]# 2 @a[-3]# 1 @a[-4]# def (warning: index out-of-bounds) @a[-Inf] # EXCEPTION: can't use -Inf as array index Where def is whatever the type-specific default is, typically Cundef, C0, or C''. 2) There is NO platform-dependent maximum array size. If it's not a sparse array, you'll run out of memory long before you run out of indexes, but using bigints as indexes for sparse arrays is OK. MikeL
Re: More Array Behaviors
On Tue, Jan 28, 2003 at 09:17:36AM -0800, Damian Conway wrote: Errno. That's rather the whole point of Cbut properties [*]. [*] People, we just *have* to find better names for these things! I'd suggest we henceforth call them value properties (for Cbut) and referent properties (for Cis). Over on [EMAIL PROTECTED] there seemed to be confusion about attributes and properties, given that some languages use one where we use the other. I had a dig in a thesaurus, I suggested that chattels and virtues were interesting words that unambiguously describe respectively extra data you've attached to a thing, and an extra qualities you've given it. But that's still only one (controversial) word for properties, and we need two. And I don't really like chattels, whereasvirtues sits nicely with bless. I got one private reply, and summarised. But nothing further. Warnock's Dilemma? (Message ID [EMAIL PROTECTED] if that helps - I don't have an archive link). Nicholas Clark
Re: Arrays: Default Values
On Tue, Jan 28, 2003 at 12:30:41PM -0800, Austin Hastings wrote: --- Michael Lazzaro [EMAIL PROTECTED] wrote: my int @a is Array( default = 5 ); @a[0] = undef; This should cause a blip of some kind. If storing an explicit undef (as opposed to undef but 0 or C$v = undef; @a[0] = $v; there should be an exception. If storing an implicit undef: convert to int (IOW: 0) and emit a warning. @a[0]; # 0, or 5? I'm not sure. I think I like the idea of @a[0] = undef; being a blip, but undef @a[0]; resetting the value to the default. Conceptually perl5 already has a distinction between assigning undef to an aggregate, and passing an aggregate to the undef operator: $ perl -le '@a = %ENV; print scalar @a; undef @a; print scalar @a' 42 0 $ perl -le '@a = %ENV; print scalar @a; @a = undef; print scalar @a' 42 1 so it's not a great leap to extend this difference to scalar values. Although it may be one leap too far. Nicholas Clark
Re: Arrays: Default Values
On Tue, 2003-01-28 at 16:23, Leopold Toetsch wrote: Austin Hastings wrote: Another question: If you ask for a value and get it, does the array grow? Or does that happen only on assignment? ( Arrays (or hashes) don't grow on reading - never. Never say never. You're correct for pure reading like so: $x = @a[0]; But for less pure forms of reading: foo(@a[0]); auto-vivification will have to happen in some cases. e.g. if foo requires a lvalue parameter. You can't know if an actual write will happen, so you have to auto-vivify in order to pass a reference. Or did I miss something there? -- Aaron Sherman [EMAIL PROTECTED] This message (c) 2003 by Aaron Sherman, and granted to the Public Domain in 2023. Fight the DMCA and copyright extension!
Re: Arrays: Default Values
On Tuesday, January 28, 2003, at 01:14 PM, Damian Conway wrote: I'm not compelled by the counter-argument that this makes it impossible to store an Cundef in an array with a default. Because the whole point of an array having a default is to prevent those nasty out-of-range Cundefs from popping up in the first place. And thinking of defaults as mappings leads me to conclude that it's entirely reasonable to allow sub refs as default specifiers, as a means of creating computed arrays. But I'd expect that the property name is different in that case (probably Cis computed), so as to distinguish: The next (oft-asked) question is whether or not Cis computed denotes read-only, or if you can store to an Cis computed array. my @a is computed { $^index**2 }; @a[4] = 'something completely different'; And things like this would be downright hysterical: my @a is computed { $^index**2 }; pop @a; :-) MikeL
Re: Arrays: Default Values
Austin Hastings wrote: --- Austin Hastings [EMAIL PROTECTED] wrote: No, undef. OTOH, deleting @a[1] would reset it to default. Ere someone flames my for using a hash keyword in an array context: s/deleting/absquatulating (e.g., via pop, shift, or splice)/ What's wrong with Cdelete in an array context? If there's to be a distinction between storing an undef value and resetting to the default value -- and, having read Damian's thoughts on the matter, I'm almost convinced that there shouldn't be -- then I think it's much more obvious to distinguish Cdelete and Cundef rather than Cundef @a[1] and C@a[1] = undef. Smylers
Re: Arrays: Default Values
--- Michael Lazzaro [EMAIL PROTECTED] wrote: On Tuesday, January 28, 2003, at 01:01 PM, Nicholas Clark wrote: On Tue, Jan 28, 2003 at 12:30:41PM -0800, Austin Hastings wrote: --- Michael Lazzaro [EMAIL PROTECTED] wrote: my int @a is Array( default = 5 ); @a[0] = undef; This should cause a blip of some kind. If storing an explicit undef (as opposed to undef but 0 or C$v = undef; @a[0] = $v; there should be an exception. If storing an implicit undef: convert to int (IOW: 0) and emit a warning. Hmm. I don't have a strong preference either way, but I'm not sure why (given Cmy int @a): @a[ undef ] Cundef should be autoconverted to 0 with warning, but in: @a[0] = undef; Cundef should _not_ be autoconverted to 0, but instead trigger an exception. They're both in Cint context, unless we want to make a special int being used as an array index context that's different from normal Cint context. Seems like Cundef should be treated like 0 (plus warning) in numeric context either everywhere, or nowhere. (?) I agree. Warning: Using undef as array index would be a nice message - probably save a bunch of debugging. =Austin
Re: Arrays: Default Values
Nicholas Clark wrote: I'm not sure. I think I like the idea of @a[0] = undef; being a blip, but undef @a[0]; resetting the value to the default. That thought crossed my mind as well before I got to your message ... Conceptually perl5 already has a distinction between assigning undef to an aggregate, and passing an aggregate to the undef operator: ... however I'm unconvinced that that's the sort of distinction that should be encouraged. Smylers
Re: Arrays: Default Values
On Tue, Jan 28, 2003 at 04:07:17PM -0500, Aaron Sherman wrote: I think this debate is easier if you think of defaults as overriding and auto-vivification method on a container. Hmm. I don't :-) I think it is easier if you think of defaults as overriding undef. On Tue, 2003-01-28 at 14:47, Paul Johnson wrote: Michael Lazzaro said: 2a) When a cell is explicitly re-undefined, does the default value take effect? my @a is Array( default = 'foo' ) = (1,2,3); @a[1] = undef; @a[1]; # undef, or 'foo'? STRAWMAN ANSWER: 'foo'. Seems right to me. Anything else would be very confusing, I think. This would be very confusing to me. In fact, it seems WRONG to me unless we're saying that such arrays can simply never hold an undefined value... which again seems wrong. I think that seems right. my @a is Array ( default = 'foo' ); @a[1] = undef; Should yield the following, I would think: ('foo', undef, 'foo' x Inf) Though, obviously there's no such thing as @a[2], but if you reference it, that's how it would auto-vivify. If that's not the case, I need to get my head around why, since Perl *does* distinguish between defined and exists. But I wish it wouldn't for arrays. That only came about to support pseudo-hashes which are going / have gone away. Are you suggesting that hashes should also have a default? That would seem consistent and at least as useful as arrays having a default. -- Paul Johnson - [EMAIL PROTECTED] http://www.pjcj.net
Re: Spare brackets :-)
On Tue, Jan 28, 2003 at 09:24:50AM -0800, Austin Hastings wrote: --- Dan Sugalski [EMAIL PROTECTED] wrote: At 8:47 AM + 1/28/03, Piers Cawley wrote: $ref[$key] an array or hash look-up??? Decided at runtime? How? People use strings as array indices and ints/floats as hash indices, and count on autoconversion to Make It Work. On the one hand: Java/ECMA/J-script does it. That's nice, but Perl isn't Java/ECMAScript/JavaScript/JScript/C/C++/Pascal. It's Perl. Perl uses square brackets for arrays, and curly braces for hashes. Period. And Perl 6 will continue in the path of Perl 1..5, *not* in the path of some other broken syntax. If you have any questions about this, please refer to the 1st, 2nd, or 3rd editions of Programming Perl, or to any of the millions of Perl programmers who have that distinction hard-wired into their wetware. Z.
Re: Arrays: Default Values
On Tue, Jan 28, 2003 at 03:06:19PM -0800, Damian Conway wrote: Austin Hastings wrote: --- Damian Conway [EMAIL PROTECTED] wrote: my @a is default(666); print @a[2];# prints 666 @a[4] = 1; print @a[2];# now prints undef :-( [typo in third line corrected] I don't understand your example. Can you explain it again, using words of less than one syllable? If the scheme is (as someone was advocating) that the default is used only in place of elements that don't exist (as opposed to elements that are allocated but contain Cundef), then allocating a previously unallocated element has the potential to change the value it returns, even if that previously unallocated element is not initialized when allocated. So, assigning to @a[4], causes @a[0..3] to be created as well. Which, under a default-for-non-existent-elements-only policy causes @a[2] to stop returning the default. It doesn't have to be that way: $ perl -le 'sub e { print exists $a[shift] ? 1 : 0 } e 2; $a[4]++; e 2; e 4; delete $a[4]; e 2; e 4' 0 0 1 0 0 No, I don't know which side I'm arguing anymore :-) Actually, I do. I don't like exists on arrays. -- Paul Johnson - [EMAIL PROTECTED] http://www.pjcj.net
Re: Arrays: Default Values
Aaron Sherman wrote: auto-vivification will have to happen in some cases. e.g. if foo requires a lvalue parameter. You can't know if an actual write will happen, so you have to auto-vivify in order to pass a reference. Or did I miss something there? I think the idea is to use a special object which, if assigned to, will create the required element. Dave. -- http://dave.whipp.name
Re: Arrays: Default Values
So ... with the discussion of what if i really wanted to put an undef in there b/c it's not just that i haven't defined it but rather that it really isn't defined. I KNOW it's not defined, and i'm now explicitly saying it's undefined as opposed to before when i was implicitly suggesting that i didn't know what it was and used a default 'unknown' discussion ... What we really need is: @a[2] = undef but undef; or, possibly (more tongue-in-cheek-y) @a[2] = undef but seriously; so the ... property? would say you have a default, maybe, but i don't care. this is REALLY undef is that possible? aside from it being disturbing to write undef but undef :o --attriel (the first suggestion is serious, but the syntax is flawed; the second suggestion has better syntax but is tongue-in-cheek suggested ... )
Re: This week's Perl 6 Summary
On Tue, 2003-01-28 at 06:34, Piers Cawley wrote: Compiling to Parrot K Stol is looking for a final project for his Bachelor's degree and would like to implement some language targeting Parrot and asked for suggestions. Simon Wistow suggested PHP or Lua, Leon Brocard suggested Java (though Gopal V wasn't sure that was such a good idea just yet). Dan suggested that, just because someone else was already working on a TCL compiler it didn't mean it wasn't worth trying anyway, but apparently it would have been unlikely to get approval as a Bachelor's project. There was some further discussion of Lua and whether there would be any real point in implementing it and the thread died down before Mr Stol told anyone what he'd decided to do, which was rather a shame I thought. http://makeashorterlink.com/?R1C423D33 i suggest smalltalk (esp. Squeak!), or Mozilla's XUL. -banjo
Re: [CVS ci] packfile #2
Garrett Goebel wrote: Leopold Toetsch wrote: - 8/12 byte float issues are still the same - are these formats really portable, or should we try to store ASCII equivalents? No? ? Because my knowledge here approaches zero, so I'm just aping information back at you from google searches and scanning documents. There are 3 standard binary IEEE 754 derived formats: Single Precision: 4 bytes Double Precision: 8 bytes Quadruple Precision: 16 bytes Many thanks for the links (searching for this stuff is a pain, there are too many results ;-) Anyway: - are there platforms with quad precision floats out there? - or should 12 byte long doubles get converted to 8 byte IEEE doubles. leo
Re: Arrays: Default Values
attriel wrote: So ... with the discussion of what if i really wanted to put an undef in there b/c it's not just that i haven't defined it but rather that it really isn't defined. I KNOW it's not defined, and i'm now explicitly saying it's undefined as opposed to before when i was implicitly suggesting that i didn't know what it was and used a default 'unknown' discussion ... What we really need is: @a[2] = undef but undef; or, possibly (more tongue-in-cheek-y) @a[2] = undef but seriously; so the ... property? would say you have a default, maybe, but i don't care. this is REALLY undef is that possible? aside from it being disturbing to write undef but undef :o --attriel (the first suggestion is serious, but the syntax is flawed; the second suggestion has better syntax but is tongue-in-cheek suggested ... ) What about: undef @a[2]; or possibly even (although I would argue that undef should remain a unary operator only): @a[2].undef(); Joseph F. Ryan [EMAIL PROTECTED]