Re: Threaded Perl bytecode (was: Re: stackless python)

[Adam Turoff]

On Tue, Oct 24, 2000 at 10:55:29AM -0400, Chaim Frenkel wrote:
> I don't see it.
> 
> I would find it extremely akward to allow 
> 
>   thread 1:   *foo = \&one_foo;
>   thread 2:   *foo = \&other_foo;
> [...]
> 
>   copy the &foo body to a new location.
>   replace the old &foo body with an indirection
> 
> (I believe this is atomic.)

Actually, that shouldn't be awkward, if both threads have their own
private symbol tables.

In any case, that's a different kind of threading.   IPC threading
has nothing to do with bytecode threading (for the most part).
What you describe here is IPC threading.  Larry was talking about 
bytecode threading.  :-)

Bytecode threading is a concept pioneered in Forth ~30 years ago.  Forth
compiles incredibly easily into an intermediate representation.  That 
intermediate representation is executed by a very tight interpreter
(on the order of a few dozen instructions) that eliminates the need for 
standard sub calls (push the registers on the stack, push the params
onto the stack and JSR).

Forth works by passing all parameters on the data stack, so there's no
explicit need to do the JSR or save registers.  "Function" calls are done
by adding bytecode that simply says "continue here" where "here" is the
current definition of the sub being called.  As a result, the current 
definition of a function is hard-coded into be the callee when the caller
is compiled. [*]

The problem with 
*main::localtime = \&foo;
*foo = \&bar;
when dealing with threaded bytecode is that the threading specifically 
eliminates the indirection in the name of speed.  Because Perl expects
this kind of re-assignment to be done dynamically, threaded bytecodes
aren't a good fit without accepting a huge bunch of modifications to Perl 
behavior as we know it.  (Using threaded bytecodes as an intermediate
interpretation also confound optimization, since so little of the
context is saved.  Intentionally.)

Hope that clarifies things.

Z.


*: Forth is an interactive development environment of sorts and
the "current definition of a sub" may change over time, but the
previously compiled calling functions won't be updated after the
sub is redefined, unless they're recompiled to use the new definition.

It gets stranger than that if you probe more deeply.

Re: [not quite an RFC] shared bytecode/optree

[Simon Cozens]

On Tue, Oct 24, 2000 at 04:41:38PM -0700, Benjamin Stuhl wrote:
> It seems to me that one thing that the perl6 bytecode
> implementation _should_ do (in the interests of being light
> and fast, as well as meshing well with MT) is be
> position-independant.

Fancy offering a patch to RFC310?

-- 
If they can put a man on the moon, why can't they put them all there?

[not quite an RFC] shared bytecode/optree

[Benjamin Stuhl]

Firstly, by "bytecode" I mean a .pmc and by "optree" I mean
the perl6 VM's internal form that it goes through
executing.

It seems to me that one thing that the perl6 bytecode
implementation _should_ do (in the interests of being light
and fast, as well as meshing well with MT) is be
position-independant. What do I mean? That all direct
references to SV*'s or regexes or anything else in the
bytecode _and_ the optree should actually be handles of
some sort. This has several benefits:

1. Bytecode can just be mmap'ed or read in, no playing
around with relocations on loading or games with RVAs
(which can't be used anyway, since variable RVAs vary based
on what's been allocated or freed earlier).

2. (more importantly, IMHO) Bytecode and the optree are
shareable between threads. My primary reason for opposing
to the RFC proposing that modules must be reloaded in each
thread is the immense amount of memory that would be wasted
without bytecode/optree sharing.

3. With a good slab allocator and possibly some mprotect()
calls (and a good OS) bytecode/optree suddenly becomes
_completely_ shared between child processed. No more
needing to restart httpd and mod_perl6 because the mixing
of code and data has doubled the core usage of each
process!

I don't have the background to seriously argue
implementation, but I might suggest a "handle table" of
sorts which defines for each thread and CV which variable
goes with which handle. This sort of ties in with my
(vague) idea that CVs should carry around instructions for
building their scratchpad, rather than the pad itself (IOW,
scratchpads become purely part of the stack frame, rather
than the subroutine's carrier variable). This is all for
the purpose of reducing the required locking around
subroutine calls to nil or almost nil (perhaps one to make
sure that no-one's changed the subroutine out from under us
via eval("*foo = \&bar;"); or the like).

At any rate, I'm just spouting off ideas sparked by various
recent discussions (I probably need a higher blood sugar or
something). It's probably too early to seriously argue
technical merits, but on the other hand, basic VM design
can start before we know the precise grammar.

-- BKS

__
Do You Yahoo!?
Yahoo! Messenger - Talk while you surf!  It's FREE.
http://im.yahoo.com/

Re: Perl6 the platform-dependent bits...

[Dan Sugalski]

At 09:08 PM 10/24/00 +0100, Simon Cozens wrote:
>On Tue, Oct 24, 2000 at 02:52:53PM -0500, Garrett Goebel wrote:
> > > It's a good idea, but it really Isn't There Yet.
> >
> > Fair enough...
>
>Hey, I'm not Dan. There should have been big  tags around that
>previous mail.

Luckily for me, I am. (I think...:) C-- has some nifty ideas, but I don't 
think it's portable or solid enough to use.

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

RE: Perl6 the platform-dependent bits...

[Garrett Goebel]

From: John Porter [mailto:[EMAIL PROTECTED]]
> 
> Garrett Goebel wrote:
> > 
> > I'm sure you won't be surprised by this, but I recall John
> > Porter as being a C-- fan. Now why is he being mysteriously silent?
> 
> Nope, wasn't me.  Never heard of it until someone brought it 
> up earlier. I do admit, it sounds intriguing.

I should obviously do my homework better... sifting through the archives...
it was Kevin Scott and John Tobey that appeared to be interested in C--.

Re: Perl6 the platform-dependent bits...

[Simon Cozens]

On Tue, Oct 24, 2000 at 02:52:53PM -0500, Garrett Goebel wrote:
> > It's a good idea, but it really Isn't There Yet. 
> 
> Fair enough...

Hey, I'm not Dan. There should have been big  tags around that
previous mail.

-- 
God gave man two ears and one tongue so that we listen twice as much as
we speak.
-- Arab proverb

Re: Perl6 the platform-dependent bits...

[John Porter]

Garrett Goebel wrote:
> 
> I'm sure you won't be surprised by this, but I recall John
> Porter as being a C-- fan. Now why is he being mysteriously silent?

Nope, wasn't me.  Never heard of it until someone brought it up earlier.
I do admit, it sounds intriguing.


> I just wanted to mention it, as
> short-comings of C/C++ and platform dependence have been a recent topic. 

There's also Objective-C, and gcc already supports it.

-- 
John Porter

By pressing down a special key  It plays a little melody

RE: Perl6 the platform-dependent bits...

[Garrett Goebel]

From: Simon Cozens [mailto:[EMAIL PROTECTED]]
> 
> On Tue, Oct 24, 2000 at 12:54:26PM -0500, Garrett Goebel wrote:
> > Everyone seems to have their favorite laundry list of the 
> > failings of C, and an accompanying list of complex and/or
> > fragile solutions. 
> 
> I'm coming to the rapid conclusion that C-- is yours. 

;) It is more of a "wasn't there some neat new thing-a-ma-jig people were
talking about". I'm sure you won't be surprised by this, but I recall John
Porter as being a C-- fan. Now why is he being mysteriously silent?


> (complicated and fragile solution.) While Simon Peyton Jones
> is a storming guy and I appreciate his work a lot, I don't
> think that C-- is the answer for Perl. 

Fair enough.

 
> C-- doesn't have varargs. 

Is that a show-stopper?


> Or a decent, fast, portable implementation.

But it is possible there will be one before there will be a Perl 6. And if
people are seriously considering re-inventing wheels in C...  -Of course
adoption of an immature technology is definitely a recipe for missing time
schedules, and probably for other disasters.


> And we'd all have to learn C--.

Would we? Or would it be possible to use a mix of C--, C, and/or C++? Would
that result in a Frankenstein's monster or best of breed mutt?

 
> It's a good idea, but it really Isn't There Yet. 

Fair enough... I don't know enough to evaluate C-- or whether or not there
are ideas in there that should be co-opted. I just wanted to mention it, as
short-comings of C/C++ and platform dependence have been a recent topic.  I
don't know enough to debate the pro's and con's, but figure that other
interested persons might. Of course that assumes there are other interested
persons.

That said, the C-- folks appear to have been working on just these problems
for years... and might have ideas that can be borrowed even without jumping
on the bandwagon.

This is the point where I have to duck out of the thread... as I don't have
much more to contribute. My stack o' personal reading list books will take
at least another 3-5 years before I'll know enough to contribute much more
than collection and regurgitation of other people's ideas.

Garrett

Re: Perl6 the platform-dependent bits...

[Simon Cozens]

On Tue, Oct 24, 2000 at 12:54:26PM -0500, Garrett Goebel wrote:
> Everyone seems to have their favorite laundry list of the failings of C, and
> an accompanying list of complex and/or fragile solutions. 

I'm coming to the rapid conclusion that C-- is yours. (complicated and fragile
solution.) While Simon Peyton Jones is a storming guy and I appreciate his
work a lot, I don't think that C-- is the answer for Perl. 

C-- doesn't have varargs. Or a decent, fast, portable implementation. And we'd
all have to learn C--.

It's a good idea, but it really Isn't There Yet. 

-- 
All the good ones are taken.

RE: Perl6 the platform-dependent bits...

[Garrett Goebel]

From: Garrett Goebel
> From: Dan Sugalski [mailto:[EMAIL PROTECTED]]
> > (Though if someone comes up with a way to make the
> > platform-dependent bits really small and isolated I'm all for it)
>
[mention of C--]
> 
> From: Dan Sugalski [mailto:[EMAIL PROTECTED]]
> > I don't think so--we want perl to generate the executable
> > code directly. If we're generating an intermediate language
> > that then gets compiled, we might as well use C. (Even
> > though it's rather limited)

Hmm... I'm not asking for enlightenment, well maybe a few more shadows in
what is otherwise darkness ;)

I was thinking more along the lines of the implementation of the Perl
interpreter, etc. in the context of working around the failings of C. Not
the generation of Perl bytecode... So I'm probably wandering off topic
(Which is easy as I'm already having trouble *following* the topic).

Everyone seems to have their favorite laundry list of the failings of C, and
an accompanying list of complex and/or fragile solutions. It seems many of
the perl6-internals discussions are more or less: "How do we get around such
and such failing of C and the Standard Library when doing foo?"

I'm not claiming to understand this laundry list of C's short-comings... I'm
just regurgitating.

C Short-comings:
o  Stdio and asynchronous IO
o  Tail call optimization
o  Garbage collection w/ layout of stack frames
o  Memory Aliasing and instruction scheduling
o  Returning multiple values in registers from procedures
o  Misaligned memory accesses
o  Range checks on multi-way jumps
o  Data layout
o  code generator specific extensions to access stack,
   allocation, and environment pointers
o  [you're own personal list...]

How to solve some of these problems, such as replacing Stdio with ATT's sfio
library, etc. have already been discussed. Past discussions made mention of
C-- as well.

I don't know if it'd be possible to realistically consider using C-- at all.
Or if it'd be possible to cull ideas from C-- that could be used in the C
implementation or in conjunction with a C implementation. As I said, I'm 99%
ignorant on this topic. But I am fairly capable of regurgitating the C--
overview, and it does appear to touch on the subject of minimizing the Perl6
platform-dependent bits...

C-- Goals:
o  Interoperable: C-- <-> C
o  Portable assembler intended particularly for garbage-collected
   source languages
o  Blazingly fast w/ architecture independence considerations
o  Exploits existing code generators: gcc, VPO, 
o  Independent of any particular code generator. I.e., no 
   commitment to a particular backend
o  Largely independent of target architecture. Exception: word size
o  Human writable and readable
o  Not a distribution format
o  Is assembler, not full programming language

C-- Features:
o  Interfaces w/ C
o  Ordinary procedures
   x  procedures may return multiple values
   x  number and type of parameters must match definition
o  Weak/Minimal Typing
   x  Tells code generator
  -  How big it is
  -  What hardware resources to use, i.e. floats
   x  No provision for type security
o  Tail Calls
   x  Guarantees tail call optimization w/o stack growth
o  Local Variables
   x  Allow arbitrary number of LV's to be declared
   x  mapped to registers instead of stack slots if they'll fit
o  Local labels and gotos
o  Conditionals that omit range checks
o  Backend has complete control of stack
   x  Parameterized procedures and returns
   x  C-- implementation is free not to use a control stack
o  Detailed control of static and dynamic memory allocation
   x  Explicit flagging of mis-aligned or over-aligned accesses
   x  No support for varying Endian-ness
   x  No aliasing directive for pointers
o  Generalized support for GC, Debuggers, and Exception Handlers
o  Global registers
o  Arithmetic operations and conversions between data types
o  Separate compilation

And if you're still with me and interested, the C-- website can be found at
www.cminusminus.org

Re: Acceptable speeds (was Re: TIL redux (was Re: What will thePerl6 code name be?))

[Steve Fink]

Most of the time, perl's performance is a total non-issue for me. When
it is, I'm generally doing the same things as Dan (ie, things resembling
dd if=/dev/hda | perl -e ...). I posted some vague benchmarky ideas to
perl6-meta at one point. Here they are again:

-

You did ask at one point for some suggestions of other speed goals. So
here are example subgoals:

1. Methods are close enough in speed to subroutine calls (<=20% slower?)
that nobody will avoid using them for performance reasons.

2. A fairly tight loop with a subroutine call is N% faster.

6. Grepping for a variable-sized regular expression on a nonblocking
pipe is >N% faster (with incremental regular expression matching, N can
be superlinear.)

-

They're all too ill-defined to test, but maybe the first step is to
enumerate a bunch of general tasks where performance is likely to
matter. Let's see...:

1. Binary parsing
2. Sucking data off of disk and spewing it back out (presumably
modifying it on the way through)
3. Calling subroutines
4. Calling methods
5. Matching
6. Matching variable-length things while juggling buffers
7. use'ing big modules
8. Math
9. General parsing (something requiring a mixture of code and regexes)
10. Tk-style callback frenzies and mark/tag manipulation (?)
11. external (XS) code in tight loops
12. Lots of concurrent instantiations of things with big shareable stuff
(eg CGI.pm) (more of a space than speed issue, though the former tends
to turn into the latter)

If I were to write up a Perl5 benchmark addressing one or a few of
these, where should I put it?

(Crossposted to -internals)

Threads vs. Threaded Inlince code (was Re: Threaded Perl bytecode)

[Uri Guttman]

along the lines of a perl6 project code name, we run into the name
collision of process threads and threaded inline code (TIL). these 2
concepts have nothing to do with each other other than overloading the
same word. please be clear which one you are refering to when unless
your post is in an email thread (damn, that word again!) that makes it
obvious which meaning is intended.

uri

-- 
Uri Guttman  -  [EMAIL PROTECTED]  --  http://www.sysarch.com
SYStems ARCHitecture, Software Engineering, Perl, Internet, UNIX Consulting
The Perl Books Page  ---  http://www.sysarch.com/cgi-bin/perl_books
The Best Search Engine on the Net  --  http://www.northernlight.com

Re: Threaded Perl bytecode (was: Re: stackless python)

[Chaim Frenkel]

> "JvV" == John van V <[EMAIL PROTECTED]> writes:

JvV> If this is the case, the code underlying the treading would utilize normal 
functions to poll the concurrent event streams and programmers could 
JvV> choose between the threads and functions depending on their levels of comfort.

JvV> This is good for my hypothetical remote sensing devices because the byte code 
interpreter would be single threaded, presumably smaller, and 
JvV> a lot easier to test.

Different threads. I believe the original was Threaded Interpreter Code,
(think forth) i.e. using pointers (or direct machine calls) to other
body of code made up of pointers or a real piece of code.

You seem to be thinking of threaded execution.


-- 
Chaim FrenkelNonlinear Knowledge, Inc.
[EMAIL PROTECTED]   +1-718-236-0183

Re: Threaded Perl bytecode (was: Re: stackless python)

[John van V]

This thread seems to be winding down fttb**,  Judging from the below, perl 
multithreading [ I am guessing ] is simply syntatic sugar much the 
same as Koch C++ is a wrapper for regular cc compiler.

This sounds nice, 
   > Threading Perl bytecode would be nice about 98% of the time, and
   > should offer a speed increase.  
 
But from the operations industry perspective:

>  idioms like:
> *main::localtime = \&localtime_replacement;
~and~
>  if anything was overriden anywhere, no module code could be read in as bytecode, 

are issues esoteric enough to be avoided simply because of the cost of maintenance.  
Experience has also shown that an overdependance 
sophistication is a good way to wind up stranded waiting for a bugfix (I'm refering to 
financial ruin).

If this is the case, the code underlying the treading would utilize normal functions 
to poll the concurrent event streams and programmers could 
choose between the threads and functions depending on their levels of comfort.

This is good for my hypothetical remote sensing devices because the byte code 
interpreter would be single threaded, presumably smaller, and 
a lot easier to test.

John

** like I'm going to tell you ;)

Re: Threaded Perl bytecode (was: Re: stackless python)

[Chaim Frenkel]

> "AT" == Adam Turoff <[EMAIL PROTECTED]> writes:

>> Wouldn't just the appearance of *foo = \&other_foo, be enough to tell
>> the compiler to treat all foo's (or perhaps if there were some dataflow
>> analysis some region of code) to use indirection?

AT> You're forgetting eval "*foo = \&other_foo" and the like.  :-)

AT> And, if modules are threaded when they're bytecompiled, it's rather 
AT> difficult to intuit some random invocant's use of *foo = \&other
AT> (at runtime) to add the required indirection post hoc.

AT> Perl's expected behavior almost requires unthreaded bytecode.  Threaded 
AT> bytecode could work, if it's explicitly requested (thus making 
AT> *main::localtime = *foo = \&bar; a fatal error, at the user's request.)

I don't see it.

I would find it extremely akward to allow 

thread 1:   *foo = \&one_foo;
thread 2:   *foo = \&other_foo;

Rather, this style should be done via a variable indirection &{$foo}.

I don't see much of a speed hit. Until the *foo assignment is actually
done, all threading is directly to the body of &foo. When the *foo
assignment is done,

copy the &foo body to a new location.
replace the old &foo body with an indirection

(I believe this is atomic.)

And optionally, the indirection could be to a fixup routine, that would
adjust the caller to directly point at the new body.


-- 
Chaim FrenkelNonlinear Knowledge, Inc.
[EMAIL PROTECTED]   +1-718-236-0183

Re: Special syntax for numeric constants [Was: A tentative list of vtable functions]

[Dan Sugalski]

At 02:34 PM 10/24/00 +0100, David Mitchell wrote:
>Ken Fox <[EMAIL PROTECTED]> wrote:
> > David Mitchell wrote:
> > > Now of course if we have the luxury of deciding that core perl 'knows'
> > > about complex numbers, then of  the parser can be made to recognise ...
> >
> > The core doesn't need to know -- that was my point. All the core needs
> > is the basic constant folding rules _it_already_has_ combined with a
> > macro to define "i". When you "use complex" the macro would be folded
> > into the parser. The core doesn't need any special support (other than
> > decent macros... ;)
>
>Not being au fait with all RFCs atc, are there any concrete proposal
>for macros like this, ir is this just a hazy suggestion?

Probably just a hazy suggestion, though if complex numbers are part of the 
perl core, it'll know how to handle this particular case. (The "2i" thing)

> > > In summary: Perl shouldn't do interpetation of numeric literals, but
> > > should instead delegate it to the numeric class currently in scope.
> >
> > I agree with you. The complication is that there isn't *a* numeric class
> > in scope -- there are *many* numeric classes.
>
>Well, I was assuming that there would be *a* numeric class in scope
>- as defined be the innermost lexical 'use foo'.
>I assumed that Perl wouldn't be clever enough to know about all available
>numberic types and automatically chose the best representation; rather
>that it was the programmer's responsibilty via 'use' or some other syntax.

Numeric constants will probably fall into two classes--those perl's parser 
knows about and can convert to, and those it doesn't and just treats as 
strings.


Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: Special syntax for numeric constants [Was: A tentative list of vtable functions]

[David Mitchell]

Ken Fox <[EMAIL PROTECTED]> wrote:
> David Mitchell wrote:
> > Now of course if we have the luxury of deciding that core perl 'knows'
> > about complex numbers, then of  the parser can be made to recognise ...
> 
> The core doesn't need to know -- that was my point. All the core needs
> is the basic constant folding rules _it_already_has_ combined with a
> macro to define "i". When you "use complex" the macro would be folded
> into the parser. The core doesn't need any special support (other than
> decent macros... ;)

Not being au fait with all RFCs atc, are there any concrete proposal
for macros like this, ir is this just a hazy suggestion?

> 
> > In summary: Perl shouldn't do interpetation of numeric literals, but
> > should instead delegate it to the numeric class currently in scope.
> 
> I agree with you. The complication is that there isn't *a* numeric class
> in scope -- there are *many* numeric classes.

Well, I was assuming that there would be *a* numeric class in scope
- as defined be the innermost lexical 'use foo'.
I assumed that Perl wouldn't be clever enough to know about all available
numberic types and automatically chose the best representation; rather
that it was the programmer's responsibilty via 'use' or some other syntax.

> We could model numeric
> classes as a tower ala Scheme in which case a constant is given to the
> most fundamental class first and then the classes higher up the tower.
> The core might just implement this by throwing a compile-time exception
> that can be caught by the classes higher up the tower. (I think the only
> real problem here is handling large precision numbers -- everything else
> can be handled with macros and constant folding.)

I'm not familiar with Scheme, I'm afraid.