Re: RFC 38 (v1) Standardise Handling Of Abnormal Numbers

[Tim Jenness]

On 5 Aug 2000, Perl6 RFC Librarian wrote:

> This and other RFCs are available on the web at
>   http://tmtowtdi.perl.org/rfc/
> 
> =head1 TITLE
> 
> Standardise Handling Of Abnormal Numbers Like Infinities And NaNs
> 

Excellent idea.

> 
> =head1 REFERENCES
> 
>   Math::Complex

You might want to add PDL::Math (part of the PDL dist) to the references
since that already tries to handle NaN and related issues in a portable
way. The development version extends this by adding "bad" values to PDL
for those platforms that don't have NaN (bigger overhead than using NaN
but better than nothing).

-- 
Tim Jenness
JCMT software engineer/Support scientist
http://www.jach.hawaii.edu/~timj

PDL request

[Tim Jenness]

On Fri, 4 Aug 2000, Tuomas Lukka wrote:

> > Both of these are language related. Is ther really nothing in the
> > internals that we need to make pdl better?
> 
> One important thing that I don't have time to write an RFC for: allow XS
> to make dynamically loaded modules' jump tableable automatically.
> Currently, if you want to use another module's library or routines, you
> have to modify the other module to generate a jump table. Either sharing
> the symbols (via options) or allowing autogenerated jumptables would be
> great.
> 

I think that the Tk module suffers from the same problem so it might be
quite easy to make a case for it.

-- 
Tim Jenness
JCMT software engineer/Support scientist
http://www.jach.hawaii.edu/~timj

RFC 1 (v2) Implementation of Threads in Perl

[Perl6 RFC Librarian]

This and other RFCs are available on the web at
  http://tmtowtdi.perl.org/rfc/

=head1 TITLE

Implementation of Threads in Perl

=head1 VERSION

Maintainer: Bryan C. Warnock <[EMAIL PROTECTED]>
Date: 04 Aug 2000
Version: 2
Mailing List: [EMAIL PROTECTED]
Number: 1

=head1 ABSTRACT

Perl 6 should be built around threads from the beginning.

=head1 DESCRIPTION

Perl 5 attempted (with relatively good success) to implement threads
atop the current architecture.  It did, unfortunately, leave several gaps,
traps, and "features" in heavy concurrency uses.  These weaknesses could
be fixed if Perl was built with threading from the start.

All Perl programs are threaded.  Most just only have one.

=head1 MOTIVATORS

Impatience, Hubris, and Laziness, in that order.

=head1 IMPLEMENTATION

Attempt to build-in thread constructs for the internals, while allowing
a Thread module to safely and robustly add user thread constructs, while
not making things bad for the single-threaded folks.

=head2 SUMMARY OF IMPLEMENTATION

The summary is based on the current Perl 5 architecture.  As the internal
structure changes, like using vtables, the thread design will have to 
change.

=over 4

=item *

Create an additional pseudo-global stash, one per thread created, that is
local to that thread.  This stash would be the default space for non-
lexical variables.  C<$main::foo> == C<$foo> within one thread, while 
C<$main::foo> != C<$main::foo> in different threads.  There need be no way to 
specify the particular thread-space, as it should be visible only to the 
owning thread.

=item *

The Thread module should add a C keyword or function that explicitly
access a variable in the program-global stash.

C
C
C<$foo = global $main::foo;  # Localize this instance of $main::foo.>

=item *

The Thread module should, on inclusion, also set the optree flag that triggers
mutex locking on variables within the perl core itself.  (As differentiated
by a user-created and controlled mutex.)  This is to guarantee that the
above constructs will actually work - user created race conditions aside.

=item *

Populate the thread-space stash with the built-ins, vice the program global 
stash.  Very few of the built-ins are meaningless in this threaded construct,
most are truly independent, and those that aren't, like $^O, should probably
be read-only anyway.  

=back

=head2 IMPACT

=over 4

=item *

Impact on Perl on a non-thread-supporting architecture.  None.  (The mutex
locking code would be no-opped out, the Thread module would fail on inclusion,
preventing any of the global semantics from being invoked.  The thread
space would appear to the program to be a standard global stash.)

=item *

Impact on Perl built for non-threaded use.  None.  Same as above.

=item *

Impact on a single-threaded program under a multi-threaded Perl.  None, most
likely, for the above reasons.  (There would be an additional flag check, 
vice, I believe, automatic mutex locking under the current scheme.)

=item *

Impact on multi-threaded scripts under a multi-threaded Perl.  Some.  Mutex
locking would occur much as it does today.  Current Perl scripts, without 
the knowledge of global versus thread space would find data-sharing broken.
Threads have been declared experimental, and I believe the benefits of 
simplifying threads in general outweigh the heartache of those (who would
benefit) that would have to change their programs.  In addition, see the
notes about module inclusion below.

=item *

Impact on Perl 5.  Possible mutual compatibility between Perl 5 and Perl 6, 
with the exception of C and the sematics it would add.  See the
notes below about module inclusion. (Obviously, other changes to the language 
notwithstanding.)

=back

=head2 UNKNOWNS

=over 4

=item *

Probably the biggest unknown, and the one with the largest potential impact,
will be exactly how module inclusion will work with threads under Perl 6.

Currently, modules are parsed and interpreted at compile time in a global
scope.  Under the above architecture, this will populate the primary thread
by default, as secondary threads are a run-time issue.

So how do secondary threads C a module?  How does a module's symbols 
find themselves in the proper thread space, instead of cramming the primary
thread space at compile time, or, worse yet, completely undoing the entire
point of threads by making everything global?

Certainly, the global approach, by default, is not the desired solution, as
you now lose any ability to make your interface reentrant, unless it is
specifically designed and tested for thread use.  In which case, the module
would then need to C, which would then initiate multi-threading,
assuming the core and platform supported it, even if the original program
didn't.  Evil, evil, evil.

Another possibility, and another one I do not like, is that a thread 
inherits the entire stash of the parent thread.  Now, you either need to
duplicate the entire stash, or

Re: perl6-internals-gc sublist

[Nathan Torkington]

John Tobey writes:
> OK.  Ask, cancel that request.  Sorry.

In the future, it's best to address your list requests directly
to the working group chair.  They'll decide and ask Ask.  In fact,
asking Ask could probably be done offline.

Nat

RFC 38 (v1) Standardise Handling Of Abnormal Numbers

[Perl6 RFC Librarian]

This and other RFCs are available on the web at
  http://tmtowtdi.perl.org/rfc/

=head1 TITLE

Standardise Handling Of Abnormal Numbers Like Infinities And NaNs

=head1 VERSION

  Maintainer: Jarkko Hietaniemi <[EMAIL PROTECTED]>
  Date: 04 Aug 2000
  Version: 1
  Mailing List: [EMAIL PROTECTED]
  Number: 38

=head1 ABSTRACT

The handling of various abnormal numeric entities like infinities
(positive, negative), not-a-numbers (NaNs, various kinds of those,
signaling, non-signaling), epsilons (positive and negative) is left to
the native math libraries.  A more concerted effort to standardise the
behaviour of Perl across the platforms would be desirable.

=head1 DESCRIPTION

The abstract pretty much took care of it.

=head1 IMPLEMENTATION

In an SV have bits that mark whether the numeric aspect of the scalar
is abnormal and if so, what kind of abnormality are we dealing with.
The Perl math operations have to watch out for those bits and do The
Right Thing before the native math ops kick in, so that for example
infinity + anything-but-nan = infinity, anything + nan = nan, and so on.

Watching out for the abnormality bits may be a performance hit.  If
so, it may be possible during the configuration phase probe how well
the native math semantics obey the semantics we define for Perl, and
let the native math take care of where it and us agree.

=head1 REFERENCES

Math::Complex

RFC 36 (v1) Structured Internal Representation of Fi

[Perl6 RFC Librarian]

This and other RFCs are available on the web at
  http://tmtowtdi.perl.org/rfc/

=head1 TITLE

Structured Internal Representation of Filenames

=head1 VERSION

  Maintainer: Jarkko Hietaniemi <[EMAIL PROTECTED]>
  Date: 04 Aug 2000
  Version: 1
  Mailing List: [EMAIL PROTECTED]
  Number: 36

=head1 ABSTRACT

Wherever Perl internally uses filenames (in a very inclusive sense:
filenames, directory names, whatever) the components of the file name
should be stored in a platform-neutral structured format.

=head1 DESCRIPTION

Perl is great at munging data and in general Perl scripts are very
portable.  Howevere, before one gets to it one usually has to open
some files and the results have to put somewhere.  The naming of files
("files" here meant in the most inclusive possible sense, the Unix
sense where everything has a name: naming of persistent storage
objects, if you wll) is however one of the most unportable things
there is.

A filename may have directory part, filename part, suffixes, volume
names, version numbers.  The component separators are different and
there may be several even on a single platform.  There may be one or
more root directories, the notation for parent directory varies, and
so on.

=head1 IMPLEMENTATION

Perl core functions should pass filenames to and from each other as
structures that contain the components of filenames in a structure.
To be as future-proof as possible, the components could probably be in
Unicode.  For native use (when actually accessing the filesystem
layer) and for performance a ready-to-be-used concatenated
representation can be created on demand.  Because the filename is
already in parsed form, manipulation of the name is much easier and
cleaner than currently.  Conversion between the file naming
conventions of various operating systems would become almost trivial.

A vague possibility: the proposed internal format could be designed to
be flexible enough to present also URLs/URIs.

=head1 REFERENCES

  perlport manpage for discussion of filenames
  File::Spec
  File::Basename

Re: RFC 27 (v1) Coroutines for Perl

[Sam Tregar]

On 4 Aug 2000, Perl6 RFC Librarian wrote:

> This proposal introduces a new keyword to perl, "co", as a complement
> to "sub".  A subroutine is defined thusly:
> 
>   co foo { ... }
> 
> Coroutines can also be closures:
> 
>   my $x = co { ... }

Does this really require a new keyword?  Why not do it as a module:

   use Coroutine;

   my $foo = new Coroutine(sub => sub { ... });

   # to send
   print $foo "data\n";

   # to read
   my $val = <$foo>;

This also allows a much easier path to future expansion.  You can add new
options to new() easily - a priority option is something you
mentioned.  You could also setup martialling if such is required.

> Coroutines can be implemented in their traditional manner, using context
> switching, or as threads, but would hide the mundane details of threads
> from the programmer.  They would be more suited for producer/consumer type
> problems rather than massively parallel compuations, but I doubt there is
> much call for the latter to be implemented in perl.

And, of course, this wouldn't be the only threading primative offered by
Perl!

Other than those points, this RFC has my whole-hearted support.  I'd love
to see solid coroutine support in Perl.

-sam

Re: Imrpoving tie() (Re: RFC 15 (v1) Stronger typing through tie.)

[Dan Sugalski]

At 09:38 PM 8/4/00 -0400, Ken Fox wrote:
>Dan Sugalski wrote:
> >$foo = 12;
> >$bar = something();
> >$bar = $foo;
> >
> > could work out to:
> >
> >$foo = $bar = 12;
> >something();
>
>If $foo is a lexical variable and it hasn't been aliased then
>you might be able to do that optimization. The following is just
>as good and safer:
>
>$foo = 12;
>scalar(something());
>$bar = $foo;

Integer constant assignment may well be cheaper than scalar copies. You're 
right about the scalar bit, though, since it's not the same as void context.

> > Where if you tie, we have *no* idea what you're doing. This:
> >
> >tie $foo, SomeModule;
> >$foo = 12;
> >
> > could well reboot some server in outer mongolia for all we know. Tied
> > variable access counts as a function call, which smacks optimization all by
> > itself.
>
>Do we have to worry about pass-by-ref semantics and @_ assignments
>too? This is a weird edge case that I've never ran into. That's going
>to put a big damper on optimization with subs.

Probably not with tie, but with function calls in general, sure. We can do 
some flow control analysis on the subs and propagate it outwards so we 
might know, for example, that:

   sub foo {
 my (@vars) = @_;
 return scalar @vars;
   }

doesn't change its args or any globals, so can be safely optimized around 
to some extent.

>Actually, we have this problem without tie if we allow external
>code to register new primitive types. For example, we assume that +
>is commutative so that we can re-order expressions and fold constants.
>That would break a module that implemented some weird + that wasn't
>commutative.

Yup. It's an issue for things that implement any non-standard semantics for 
existing ops, especially if those ops are overridden at runtime so the 
optimizer doesn't know. It's one thing to mess with tied variables, its 
another entirely to make + behave differently.

I think we'll need to get a ruling from Larry at some point on this one.

>Anybody that does this has to define a new operator, i.e. if they
>use the standard operators, they have to live with the standard
>semantics. (This is why string concatentation should never use +!)

That works for me.

>I really need to start playing with a parser and dataflow analysis
>before thinking too much more about this. It might be that we need
>something *way* more strict than use strict before aggresive
>optimization is possible.

Well, we can still do some pretty aggressive things, but there will be some 
significant limits to code movement and such. Luckily (?) perl's not fast 
enough that rescheduling's a big deal--it's not like with machine code 
where you want to reorder to avoid cache misses and pipeline stalls and 
such. Hoisting code out of a loop, like so:

foreach (1..10) {
  $i = $_;
  print $i;
}

to

   foreach (1..10) [
 print $_;
   }
   $i = 10;

Or, I suppose, this:

   print "12345678910";
   $i=10;

will be a win, but when exactly assignments happen isn't nearly as big a 
deal. I think. Maybe.

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: Imrpoving tie() (Re: RFC 15 (v1) Stronger typing through tie.)

[Ken Fox]

Dan Sugalski wrote:
>$foo = 12;
>$bar = something();
>$bar = $foo;
> 
> could work out to:
> 
>$foo = $bar = 12;
>something();

If $foo is a lexical variable and it hasn't been aliased then
you might be able to do that optimization. The following is just
as good and safer:

$foo = 12;
scalar(something());
$bar = $foo;

> Where if you tie, we have *no* idea what you're doing. This:
> 
>tie $foo, SomeModule;
>$foo = 12;
> 
> could well reboot some server in outer mongolia for all we know. Tied
> variable access counts as a function call, which smacks optimization all by
> itself.

Do we have to worry about pass-by-ref semantics and @_ assignments
too? This is a weird edge case that I've never ran into. That's going
to put a big damper on optimization with subs.

Actually, we have this problem without tie if we allow external
code to register new primitive types. For example, we assume that +
is commutative so that we can re-order expressions and fold constants.
That would break a module that implemented some weird + that wasn't
commutative.

Anybody that does this has to define a new operator, i.e. if they
use the standard operators, they have to live with the standard
semantics. (This is why string concatentation should never use +!)

I really need to start playing with a parser and dataflow analysis
before thinking too much more about this. It might be that we need
something *way* more strict than use strict before aggresive
optimization is possible.

- Ken

relocatable perl, again (was Re: kpathsea)

[John Tobey]

[CC'd to perl6-build]

Dan Sugalski <[EMAIL PROTECTED]> wrote:
> At 03:25 PM 8/4/00 -0700, Russ Allbery wrote:
> >Simon Cozens <[EMAIL PROTECTED]> writes:
> >
> > > Sounds good. Here's a slight modification: perllib.db is a cache;
> > > lookups take place as normal, but then any new information is written
> > > into the cache.  The cache is invalided every $configurable period.
> >
> >Putting on my sysadmin hat, anything that a program wants to modify during
> >its normal course of operation and isn't a dotfile in the user's home
> >directory is inherently Evil and not to be tolerated if at all possible.
> 
> I fully agree. Not that I dislike the idea of figuring out file locations 
> at module install time instead of runtime, mind--I rather like that. (Do 
> the work once, rather than every darned time) make install, however, should 
> be the only thing that messes with these files.

How tough would this be?

# perl -MPerl::Mover -e 'copyperl "/home/me/local"'

Perl/Mover.pm does whatever's needed regarding binedits, relinking,
and whatnot.  Would this allow us to remove the discussion from
internals, at least?

-John

-- 
John Tobey, late nite hacker <[EMAIL PROTECTED]>
\\\   ///
]]] With enough bugs, all eyes are shallow.   [[[
///   \\\

Re: perl6-internals-gc sublist

[John Tobey]

OK.  Ask, cancel that request.  Sorry.
-John

Dan Sugalski <[EMAIL PROTECTED]> wrote:
> At 02:45 PM 8/4/00 -0400, John Tobey wrote:
> >I guess, more than establishing a working group, I'm hoping to siphon
> >GC debates out of the more general internals list, since a lot of
> >people love discussing GC at great length, and I don't expect that
> >debate to have much relevance to the rest of the internals.
> 
> I'd rather not do that, though. The perl lists are supposed to be rather 
> more targeted than that. (Not that I mind a good GC discussion, but the 
> point is to get an RFC for a workable scheme)
> 
>   Dan
> 
> --"it's like this"---
> Dan Sugalski  even samurai
> [EMAIL PROTECTED] have teddy bears and even
>   teddy bears get drunk
> 
> 

Re: PDL-P: indexing

[Jeremy Howard]

Joshua N Pritikin wrote:
> <...>
> Also, you can specify a non-default step size:
>
>   @pdl(1:9:2, 1:9:2);  # (1,1) (3,1) (5,1) (7,1) (9,1) (1,3) (3,3) ...
>
> Although I'm not sure how frequently custom step sizes are used in PDL
> code...
>
More generally, we need to be able to specify:
- Slices: non-default step sizes here allow n-dimensional tensors to be
handled gracefully
- Indirect indexes: eg. @a=(3,4,7); @b[@a] works as in perl now
- Masks: eg. @a=(0,0,1,1,0,0,1); @b->mask[@a] works as above

As you can see, in terms of the semantics, simple slice() and mask()
functions can on the face of it be used to generate the indirect index
approach that perl already supports.

Unless your array is of infinite size of course... maybe we don't want to
get into that right now...

There's a bit to be done in the implementation to make sure that real
numerical programs don't grind to a halt. For instance, it's not unusual to
want to say (this is not real code):
@a = sumover(@b[$INDEX::j]*@c[$INDEX::j])
where $INDEX::j is a column iterator, and @c and @b are matrices, and '*' is
component-wise multiplication. @a then becomes the sum of the component-wise
multiplications of columns in @b and @c.

Now what if @c and @b are sparse matrices? That means they're actually:
@c = @craw[@cindexes]; @b = @braw[@bindexes];
Then in every loop we're following the indirect index, doing a lookup to
find the right spot in column vector, and in the worst case generating a
temporary matrix of the size of @b and doing a redundant loop.

Perl has all the information it needs at compile time though to combine the
indirect index and the slice, and to avoid the redundant memory copy and
loop in the reduce (as I've mentioned previously on perl6-internals).

Sparse arrays/matrices/tensors are not rare. Transposing them is common too,
as is reducing along one dimension. Languages which don't deal with these
issues gracefully at compile time simply can't be used for numerical
programming (because they can be an order of magnitude slower by not doing
the optimisations I've mentioned).

As Karl Glazebrook described so eloquently, PDL currently jumps through all
kinds of hoops to do appropriate optimisations. However, because this all
has to be done in C, it's hard for us Mere Mortals so write our own
functions. It would be just _so_ fun to do all this in pure perl.

Re: Avoid memory copy and redundant loops in reduce/fold

[Jeremy Howard]

Ken Fox wrote:

> > $a = sum(@b*@c+@d)
>
> I'm not strong in math, but I do remember a bit about row and column
> vectors. Isn't @b*@c ambiguous? Shouldn't it normally be interpreted
> as a dot product, i.e. treat all vectors the same?
>
It depends on what perl6-language comes up with. Probably '*' will mean
component-wise product and 'x' will mean inner product (or visa-versa). I'm
talking about the component-wise product version.

> > The normal problem with this type of structure is that the previous
> > statement would create 2 array copies, and 3 loops for most compilers.
In
> > perl speak, it might look like:
> > $dummy1[$_] = $b[$_]*$c[$_] for (0..$#b-1);
> > $dummy2[$_] = $d[$_]+$dummy1[$_] for (0..$#dummy1-1);
> > $sum+=$_ for (@dummy2);
>
> A simpler rule would be to treat an array as an implicit stream
> whenever it's used in a scalar+reduce context. Then the following
> code would be generated:
>
>   $dummy1 = stream @b;
>   $dummy2 = stream @c;
>   $dummy3 = stream @d;
>   while ($dummy4 = $dummy1->next * $dummy2->next + $dummy3->next) {
> $sum += $dummy4;
>   }
>
Excellent! There are C++ libraries that use the stream approach, and they're
_really_ fast! But they're a pain to use. This is just the kind of extra
optimisation I was hoping that perl 6 could achieve.

> > $sum+=$b[$_]*$c[$_]+$d[$_] for (0..$#b-1);
>
> Shouldn't that be
>
>   $sum+=$b[$_]*$c[$_]+$d[$_] for (0..min($#b, $#c, $#d));
>
Actually, we're still trying to decide on perl6-language what this means. It
might be even be max(), with undef passed in for short arrays. I was just
being lazy to show an example.

> > Without this optimisation, array semantics become next to useless for
> > numeric programming, because their overhead is just so high.
>
> I think talking about array semantics instead of vector and matrix
> semantics is next to useless too, but I'm not a math guy.
>
I really mean 'tensor semantics', which is a generalisation of matrices to n
dimensions. But I was worried that non-math guys might see 'tensor' and
their brains would explode ;-)

Anyway, tensors are just a generalisation. Ideally, good slicing, masking,
and indirect indexing on arrays allow the user to easily create any kind of
multi-dimensional structure they want, including sparse structures. Then
it's just a case of creating wrappers to make them look like real tensors.

...As long as the overhead is stripped away at compile time, of course.

Re: ffcall GPL -> LGPL

[Nathan Torkington]

Bradley M. Kuhn writes:
> I wrote http://tmtowtdi.perl.org/rfc/13.pod to propose that we create a
> licensing working group.  It needs someone in authority to create the
> working group.  Unfortunately, I have yet to find out how the process works
> to create new top-level working groups.
> 
> Of course, it could be that no one has gotten time to figure out that
> procedure.  That's understandable.

Hi, that's me.  :-)  Still chewing on it.

Nat

Re: ffcall GPL -> LGPL

[Bradley M. Kuhn]

Tom Christiansen wrote:
> >No, it's a CPAN module.
 
> Oh, good.  Someone has a licensing list going--somewhere.

I wrote http://tmtowtdi.perl.org/rfc/13.pod to propose that we create a
licensing working group.  It needs someone in authority to create the
working group.  Unfortunately, I have yet to find out how the process works
to create new top-level working groups.

Of course, it could be that no one has gotten time to figure out that
procedure.  That's understandable.

-- 
Bradley M. Kuhn  -  http://www.ebb.org/bkuhn

 PGP signature

RFC 35 (v1) A proposed internal base format for perl

[Perl6 RFC Librarian]

This and other RFCs are available on the web at
  http://tmtowtdi.perl.org/rfc/

=head1 TITLE

A proposed internal base format for perl variables

=head1 VERSION

  Maintainer: Dan Sugalski <[EMAIL PROTECTED]>
  Date: August 04, 2000
  Version: 1
  Mailing List: [EMAIL PROTECTED]
  Number: 35

=head1 ABSTRACT

Perl needs to keep track of its data somehow. This is a possible how.

=head1 DESCRIPTION

This RFC describes the generic portions of perl's internal data
representation. This information is common across all of perl's
different data types.

This is similar to the structure used in perl 5, with one major
difference. Rather than having all the intellegence needed to use a
variable separate from that variable, this RFC embeds that information
into the variable itself. This allows for more efficient code to
access the vriables, and it lets us add in variable types on the
fly. This way perl doesn't, for example, have to know how to access an
individual element of an array of integers--it just asks the array to
return it a particular element. Code MUST use the vtable functions to
get or set values from variables. They MUST NOT directly access the data.

This base structure should be considered immobile, so it's safe to
maintain pointers to it. The data portion of a variable should be
considered moveable, and may be shuffled around if a variable changes
its type, or the garbage collector needs to compact the heap.

Implementation on various types (arrays, hashes, scalars) as well as
sub-types (integer scalars, string scalars, objects) is left to
another RFC.

=head1 IMPLEMENTATION

The base variable structure looks like:

struct {
  IV GC_data;
  void *variable_data;
  IV flags;
  void *vtable;
  void *sync_data;
}

The fields, in order, are:

=over 4

=item GC_data

Random data for the garbage collector, whichever one is used. This
could be a marker for M&S GC, or a refcount for refcount GC, or just
nothing at all if we get a really clever GC.

=item variable_data

Equivalent to perl5's sv_any pointer, this is a pointer to the actual
data structure for the variable. It may, in certain cases, be coopted
to hold the actual value. (This is likely the case for a scalar that
holds just an integer, where the native int size is equal to or
smaller than the native pointer size)

The actual structure that hangs off will depend both on the class of
variable (scalar, hash, array) and the type of that class (integer
array, integer scalar, filehandle, reference) and isn't specified here

=item flags

This field holds various flags that hold the status of the
variable. (Flags to be RFC'd later)

=item vtable

The vtable field holds a pointer to the vtable for a variable. Each
variable type has its own vtable, holding pointers to functions for
the variable. Vtables are shared between variables of the same
type. (All integer arrays have the same vtable, as do all string
scalars and so on)

vtable contents will be RFCd separately. All variables will share a
common set of functions, though scalars, arrays, and hashes will have
their own set of extensions on top of that.

=item sync_data

A pointer to a structure used for synchronization between async parts
of perl. This will probably be used mainly for threading, but may be
used to indicate interpreter ownership for variables shared between
interpreters, or other such things. If 0, the scalar is assumed to be
unshared and no synchronization is done.

=back

=head1 IMPACT ON THREADING

The sync_data B be used properly by perl when safe access to
data is needed. What scheme is used to synchronize is not dictated by
this RFC. It's assumed that whatever scheme is put in place will
perform properly.

For speed and simplicity, the functions in the vtable may assume that
proper synchronization has been done on the variable before they have
been called, and that they have exclusive and safe access. No
synchronization beyond this is guaranteed, so if the vtable functions
have extra synchronization needs they are assumed to take care of that
themselves.

=head1 IMPACT ON EMBEDDING

None. Generally embedding apps won't deal with actual perl data

=head1 IMPACT ON EXTENSIONS

None. Extensions get pointers to this structure, which as far as they
know is a magic cookie. (In fact the official perl term for the thing
handed to extensions is a Perl Magic Cookie, or PMC) Knowledge of the
internals is a no-no at this level.

=head1 IMPACT ON OP FUNCTIONS

Op functions have intimate knowledge of the internals and unrestricted
access. Therefore they're assumed to know what they're doing, and will
therefore heed the info in this RFC.

=head1 REFERENCES

sv.h

Re: kpathsea

[Dan Sugalski]

At 03:25 PM 8/4/00 -0700, Russ Allbery wrote:
>Simon Cozens <[EMAIL PROTECTED]> writes:
>
> > Sounds good. Here's a slight modification: perllib.db is a cache;
> > lookups take place as normal, but then any new information is written
> > into the cache.  The cache is invalided every $configurable period.
>
>Putting on my sysadmin hat, anything that a program wants to modify during
>its normal course of operation and isn't a dotfile in the user's home
>directory is inherently Evil and not to be tolerated if at all possible.

I fully agree. Not that I dislike the idea of figuring out file locations 
at module install time instead of runtime, mind--I rather like that. (Do 
the work once, rather than every darned time) make install, however, should 
be the only thing that messes with these files.

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: kpathsea

[Russ Allbery]

Simon Cozens <[EMAIL PROTECTED]> writes:

> Sounds good. Here's a slight modification: perllib.db is a cache;
> lookups take place as normal, but then any new information is written
> into the cache.  The cache is invalided every $configurable period.

Putting on my sysadmin hat, anything that a program wants to modify during
its normal course of operation and isn't a dotfile in the user's home
directory is inherently Evil and not to be tolerated if at all possible.

Bear in mind that site-wide Perl installations are going to be exported
read-only.

-- 
Russ Allbery ([EMAIL PROTECTED]) 

Re: RFC 32 (v1) A method of allowing foreign objects in

[Dan Sugalski]

At 05:33 PM 8/4/00 -0400, John Porter wrote:
> > perl punts. We require that the code we interface to defines a
> > dispatch function for each object, and we then just call the dispatch
> > function with enough information for it to Do The Right Thing.
> >
> > Perl will define some generic conversion
> > routines (to int, float, and string) which the foreign code is welcome
> > to override if it wishes.
>
>I wonder if this idea was (or could yet be) informed by CORBA IDL,
>or perhaps some other IDL, such as Xerox ILU?

Nope. It was something I really wanted not too long ago when writing some 
XS code to interface to a C++ library. (Ended up writing some extra C++ 
wrappers and hooking into DESTROY)

This is part of my "Make embedding perl everywhere a no-brainer" campaign. :)

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: RFC 32 (v1) A method of allowing foreign objects in

[John Porter]

> perl punts. We require that the code we interface to defines a
> dispatch function for each object, and we then just call the dispatch
> function with enough information for it to Do The Right Thing.
> 
> Perl will define some generic conversion
> routines (to int, float, and string) which the foreign code is welcome
> to override if it wishes.

I wonder if this idea was (or could yet be) informed by CORBA IDL,
or perhaps some other IDL, such as Xerox ILU?

-- 
John Porter

RFC 32 (v1) A method of allowing foreign objects in

[Perl6 RFC Librarian]

This and other RFCs are available on the web at
  http://tmtowtdi.perl.org/rfc/

=head1 TITLE

A method of allowing foreign objects in perl

=head1 VERSION

  Maintainer: Dan Sugalski <[EMAIL PROTECTED]>
  Date: August 02, 2000
  Version: 1
  Mailing List: [EMAIL PROTECTED]
  Number: 32

=head1 ABSTRACT

Currently it's a darned pain to deal with objects created in other
languages in perl. It would be very nice to be able to call a function
written in some other language, take the returned object, and treat it
as a perl object. (Which means making method calls and such on it)

=head1 DESCRIPTION

Everybody and their dog has written an object-oriented language at
this point, and it'd be nice to be able to use objects created by
those languages as if they were real perl objects. This would make
interfacing perl and OO languages[1] easier, and make things much
nicer for embedded and embedding code.

Since writing half a zillion perl2WhoKnowsWhat modules is going to be
unweildy, not to mention different for each and every object type,
perl punts. We require that the code we interface to defines a
dispatch function for each object, and we then just call the dispatch
function with enough information for it to Do The Right Thing.

>From the perl programmer's point of view, foreign objects and native
objects look the same. Perl will define some generic conversion
routines (to int, float, and string) which the foreign code is welcome
to override if it wishes.

=head1 IMPLEMENTATION

A scalar is made an object via a call into the perl library. The
scalar is marked as an object and stuck into a package. Attached to
the scalar is a pointer to the native object, a pointer to a generic
dispatch routine, and a pointer to the native destruction routine for
the object.

Native code would create the object thusly:

  TheObj *foo;
  SV *new_sv;
  foo = new TheObj("A parameter");
  sv = perl_new_sv();
  perl_make_sv_object(sv, "Some::Package", foo, &dispatch_routine,
&destroy_routine);

  perl_return(perl_make_ref(sv));

When you make a method call in perl, like so:

  $native_obj->foo($bar, $baz);

perl calls the underlying dispatch routine to do the actual work. The
dispatch routine has a function signature like so:

  int status = dispatch(void *native_obj, sv *perl_scalar, char *method_called,
int *num_args_in, perl_arg_stack *arg_stack,
int *num_args_out, perl_arg_stack *return_values);



=head1 REFERENCES

None I can think of

=head1 FOOTNOTES

[1] Like, say, C++ or Scheme 

Threaded Code

[Chaim Frenkel]

Here's a reference that I found to TILs.

http://www.complang.tuwien.ac.at/forth/threaded-code.html

Does anyone recall an article on Machine language threading? It
avoided the overhead of a dispatch loop by replacing the threading
pointers with actual machine calls. (Could it have been a Byte article?)

(Might have been back in the '70s)


-- 
Chaim FrenkelNonlinear Knowledge, Inc.
[EMAIL PROTECTED]   +1-718-236-0183

Re: RFC 27 (v1) Coroutines for Perl

[Dan Sugalski]

At 11:24 AM 8/4/00 -0800, Michael Fowler wrote:
>On Fri, Aug 04, 2000 at 03:13:24PM -, Perl6 RFC Librarian wrote:
> >   co foo { ... }
>
> >   my $x = co { ... }
> >
> >$y = <|foo>;
> >
> >print |$x "hello, world\n";
> >
> > Inside a coroutine, the meanings of "<>" and the default file
> > descriptor for print, printf, etc. are overloaded.
>
>Is it just me, or does this look like a tied filehandle with some extra
>syntactic sugar and a user-defined subroutine?

No, there's more than that. When the coroutine does a print, it stops. When 
the main code does a read from the filehandle, it stops and the coroutine 
picks back up. (Well, more or less)

It's a simple form of threading, sort of. Perl's actually in both the 
coroutine and the main code at the same time.

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: RFC 27 (v1) Coroutines for Perl

[Michael Fowler]

On Fri, Aug 04, 2000 at 03:13:24PM -, Perl6 RFC Librarian wrote:
>   co foo { ... }

>   my $x = co { ... }
> 
>$y = <|foo>;
> 
>print |$x "hello, world\n";
> 
> Inside a coroutine, the meanings of "<>" and the default file
> descriptor for print, printf, etc. are overloaded.

Is it just me, or does this look like a tied filehandle with some extra
syntactic sugar and a user-defined subroutine?

use Coroutine qw(co);

co foo { } # creates the filehandle "foo" in the current package,
   # and ties it to the Couroutine package

my $x = co { } # ties a filehandle to the Coroutine package and returns it

# Then everything else is a matter of using the filehandle as normal.

$y = ;

print $x "hello, world\n"

The PRINT method can select prior to calling the user-defined subroutine.


Michael
--
Administrator  www.shoebox.net
Programmer, System Administrator   www.gallanttech.com
--

Re: perl6-internals-gc sublist

[Dan Sugalski]

At 02:45 PM 8/4/00 -0400, John Tobey wrote:
>I guess, more than establishing a working group, I'm hoping to siphon
>GC debates out of the more general internals list, since a lot of
>people love discussing GC at great length, and I don't expect that
>debate to have much relevance to the rest of the internals.

I'd rather not do that, though. The perl lists are supposed to be rather 
more targeted than that. (Not that I mind a good GC discussion, but the 
point is to get an RFC for a workable scheme)

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: kpathsea

[Chaim Frenkel]

I don't think so. I think the db should reflect the installation. The
sysadmin should be able to uninstall something. Expecting a file and
not finding it should be a fatal error. And it shouldn't keep looking.

Hmm, some care will have to be taken for setuid, etc.



> "SC" == Simon Cozens <[EMAIL PROTECTED]> writes:

SC> (Chaim, please can you *not* CC me stuff on a list you know I read! It's
SC> not as if we don't get too much mail already from this. :)

SC> On Fri, Aug 04, 2000 at 12:19:30PM -0400, Chaim Frenkel wrote:
>> Then how about a perllib.db. Pre-digested search paths. What %INC would
>> look like if everything and the kitchen sink were loaded. No parsing, 
>> nothing but straight forward lookup code.
>> 
>> If not found, or an entry not found fall back.

SC> Sounds good. Here's a slight modification: perllib.db is a cache; lookups
SC> take place as normal, but then any new information is written into the cache.
SC> The cache is invalided every $configurable period.

SC> Of course, we can do this with Perl 5, but it would be kludgy. (Remember
SC> putting subrefs into @INC?)

-- 
Chaim FrenkelNonlinear Knowledge, Inc.
[EMAIL PROTECTED]   +1-718-236-0183

Re: Language RFC Summary 4th August 2000

[Damian Conway]

   > What I'm planning to RFC is a simple format() built-in (probably
   > in a pragma) very similar to the form() subroutine described in:

http://www.csse.monash.edu.au/~damian/TPC/2000/Autoformat/paper.html

Damian

Re: perl6-internals-gc sublist

[Dan Sugalski]

At 02:48 PM 8/4/00 -0400, John Tobey wrote:
>Dan Sugalski <[EMAIL PROTECTED]> wrote:
> > Yup, and I realized one of my big problems to GCs that move memory
> > (references that are pointers and such) really isn't, if we keep the
> > two-level variable structure that we have now. The 'main' SV structure
> > won't move, while the guts that the equivalent of sv_any points to can
> > without a problem.
>
>I certainly hope this data layout factoid is still subject to change.

Well, the layout of variables is subject to change, certainly, though 
playing games with pointer bits for data typing's likely right out. The 
second level of the variable doesn't actually have to exist--I don't mind 
if a scalar's data pointer is coopted to be an integer, say, or a float if 
its big enough.

Nothing's set in stone, so do feel free to put an RFC together for it. (I 
can be convinced... :)

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: Language RFC Summary 4th August 2000

[Damian Conway]

   > >> It definitely is, since formats do things that can't be done in modules.
   > >
   > >Such as???
   > 
   > Well, the easy binding of variables for later use. When one declares a 
   > format, variables in it are saved for later use without needing refs. 
   > Formats are sort of like a quote where the variables are referenced rather 
   > than interpolated when defined, with the values pulled (and possibly 
   > altered) when the quote is used.

What I'm planning to RFC is a simple format() built-in (probably in a pragma)
very similar to the form() subroutine described in:


I would envisage that the correct way to set up the kinds of
referenced-not-interpolated templates is by creating a subroutine or closure:

sub format1 {
format "> >>> [",
   $bullet,
   $text;
}

which you can then interpolate at need:

print format1;

Alternatively, one could make the format nicely parametric using higher-order
functions:

my $format2 = 
format "> >>> [",
  __, __;

and later:

print $format2->($magic_bullet, $special_text);

or partially fill it in:

my $format3 = $format2->($magic_bullet);

I'll get to work on the format RFC as soon as possible, so that you can
see what I mean.

   
   > :) I promised Kirrily that I'd race you in the RFC count...

That's hardly fair on you: I can just pull any crazy idea out of my head
whereas *you* have to be able to implement yours! :-)

Damian

Re: Language RFC Summary 4th August 2000

[Dan Sugalski]

At 04:00 AM 8/5/00 +1000, Damian Conway wrote:
>> It definitely is, since formats do things that can't be done in modules.
>
>Such as???

Well, the easy binding of variables for later use. When one declares a 
format, variables in it are saved for later use without needing refs. 
Formats are sort of like a quote where the variables are referenced rather 
than interpolated when defined, with the values pulled (and possibly 
altered) when the quote is used.

>> If they yank formats out (which is just dandy by me) that means
>> that some means of providing format's functionality needs has to
>> be designed in.
>
>I'm working on it.

:) I promised Kirrily that I'd race you in the RFC count...

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: perl6-internals-gc sublist

[John Tobey]

Dan Sugalski <[EMAIL PROTECTED]> wrote:
> Yup, and I realized one of my big problems to GCs that move memory 
> (references that are pointers and such) really isn't, if we keep the 
> two-level variable structure that we have now. The 'main' SV structure 
> won't move, while the guts that the equivalent of sv_any points to can 
> without a problem.

I certainly hope this data layout factoid is still subject to change.

-John

Re: perl6-internals-gc sublist

[John Tobey]

Dan Sugalski <[EMAIL PROTECTED]> wrote:
> This seems a bit premature, given that we haven't actually come up with 
> even a framework for an API, or hashed out much on the format of variables 
> or the interpreter structure. (Nor the threading or event stuff...)
> 
> Are you comfortable with a shorter term list (Say,two or four weeks) that 
> will work up several different GC RFCs, based on various constraints? If 
> so, I'm fine with it.

Sure, if that's your preference.

I guess, more than establishing a working group, I'm hoping to siphon
GC debates out of the more general internals list, since a lot of
people love discussing GC at great length, and I don't expect that
debate to have much relevance to the rest of the internals.

-- 
John Tobey, late nite hacker <[EMAIL PROTECTED]>
\\\   ///
]]] With enough bugs, all eyes are shallow.   [[[
///   \\\

Re: RFC 27 (v1) Coroutines for Perl

[Uri Guttman]

> "DS" == Dan Sugalski <[EMAIL PROTECTED]> writes:

  DS> Not gonna happen. Tk and signals, at the very least, will see to that. 
  DS> Coroutines are also an awfully limited threading mechanism, and I think 
  DS> they'd have the same problems that threads have (or the use of multiple 
  DS> processors in an SMP system would be shot), so it seems silly to limit 
  DS> threading to coroutines.

  DS> Might as well do 'em all, though the low-level contstructs (i.e. what we 
  DS> build) may overlap a bunch.

i *agree* very strongly. a clean low level design will support all the
flow methods equally well and with decent efficiency.

that is a major reason why i want to move all of those rfc's under the
flow one so we can properly address that low level design and language
changes to support them all. i should have my draft rfc done later
tonight and will post it. it has some missing sections you can fill in
like on co-routines and completions.

uri

-- 
Uri Guttman  -  [EMAIL PROTECTED]  --  http://www.sysarch.com
SYStems ARCHitecture, Software Engineering, Perl, Internet, UNIX Consulting
The Perl Books Page  ---  http://www.sysarch.com/cgi-bin/perl_books
The Best Search Engine on the Net  --  http://www.northernlight.com

Re: Language RFC Summary 4th August 2000

[Damian Conway]

   > It definitely is, since formats do things that can't be done in modules.

Such as???
   
   > If they yank formats out (which is just dandy by me) that means
   > that some means of providing format's functionality needs has to
   > be designed in.

I'm working on it.

Damian

Re: RFC 27 (v1) Coroutines for Perl

[Dan Sugalski]

At 01:17 PM 8/4/00 +0500, Tom Scola wrote:
> >[I think this belongs on the language list, FWIW, Cc'd there]
> >
> >I like this, but I'd like to see this, inter-thread queues, and events
>all
> >use the same communication method. Overload filehandles to pass events
>
> >around instead, so:
>
>I'm proposing that events and threads be dropped in lieu of coroutines.

Not gonna happen. Tk and signals, at the very least, will see to that. 
Coroutines are also an awfully limited threading mechanism, and I think 
they'd have the same problems that threads have (or the use of multiple 
processors in an SMP system would be shot), so it seems silly to limit 
threading to coroutines.

Might as well do 'em all, though the low-level contstructs (i.e. what we 
build) may overlap a bunch.

>  I view events and threads as low-level attempts to implement what coroutines
>express elegantly.

In some ways, sure. In others, no.

>There would be two perl run-times, one event-based and the other thread
>based, much like java green threads and native threads.

Ah. Don't think so. Code duplication is Evil, and I'd rather have a unified 
interpreter.

> > Invoking the coroutine could use another keyword,
> >perhaps invoke, that returns the filehandle the coroutine talks on. So:
>
> >
> >   $fh = invoke foo("1", "2", "3")
>
>As I tried to explain in the RFC, the "invoke" keyword be redundant.  Writing
>to, or reading from the filehandle would invoke the coroutine.

You're overloading filehandle syntax way too much, and parameter passing's 
nasty that way.

   $foo = <|coroutine "bar", "baz">;

Yech. Ugly.

> >This would necessitate the expansion of select to check for pending
> >events/coroutine writes/data, but that's likely to happen anyway, so...
>
>
>In my proposed event-based perl runtime, the select loop would be hidden
>from the programmer.  All I/O calls would be non-blocking and context 
>switching.

I meant select the perl construct, not select the low-level construct.

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: perl6-internals-gc sublist

[Dan Sugalski]

At 12:31 PM 8/4/00 -0400, Chaim Frenkel wrote:
> > "DS" == Dan Sugalski <[EMAIL PROTECTED]> writes:
>
>DS> This seems a bit premature, given that we haven't actually come up with
>DS> even a framework for an API, or hashed out much on the format of 
>variables
>DS> or the interpreter structure. (Nor the threading or event stuff...)
>
>Either the API or the internal representations should be worked on.
>They might be done in parallel.

Yup, and I realized one of my big problems to GCs that move memory 
(references that are pointers and such) really isn't, if we keep the 
two-level variable structure that we have now. The 'main' SV structure 
won't move, while the guts that the equivalent of sv_any points to can 
without a problem.

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: GC

[Gisle Aas]

Dan Sugalski <[EMAIL PROTECTED]> writes:

> At 01:35 AM 8/4/00 -0400, Ken Fox wrote:
> >Simon Cozens wrote:
> > > Ref counting isn't garbage collection.
> > > http://www.jwz.org/doc/gc.html
> >
> >Please explain that to Richard Jones and Rafael Lins who have
> >written a beautiful book surveying garbage collectors.
> 
> Title! Title, ISBN, and publisher, please!

"Garbage Collection - Algorithms for Automatic Dynamic Memory Management"
ISBN 0-471-94148-4
Publisher: John Wiley & Sons

Thoughts on moving GCs, from my Eiffel experience

[Raphael Manfredi]

I've implemented a set of GC algorithms that could move objects in
the ISE Eiffel runtime back in 1991-92.

The result was well worth the effort, but it was a can of worms.

:.,$s/Eiffel/Perl/g

Everything in the Eiffel world worked great, but Eiffel has provision
to call C routines, that can then callback Eiffel routines from C.
Since GC could trigger anywhere, one had to be careful.

Some C routines specially written with Eiffel in mind were told to
use the eif_access() macro (name may be not exact) to access Eiffel
objects, without caching the results, since it could change anytime
du to a GC cycle being run.

That did not work for C routine NOT written with that in mind
(e.g. a pure libc routine), so I added the ability to eif_freeze()
objects into memory, with the guarantee that the object would no
longer move.

But then, if object don't move, they pollute your generation-scavenging
algorithms, so you need to freeze objects elsewhere, in a place where
they will NEVER move again.

If you have a decent GC that can workaround cycles, you also need to
maintain a set of active roots in your object graph. That's usually
the root object (in Eiffel, it's an architected object, built by the
runtime on entry), plus the stack (local vars), plus "old objects
referencing new ones" for generation scavening, plus "remembered"
objects (objects given by Eiffel to the C side and no longer referenced
from any Eiffel object, but which must stay alive because the C side
can use them for feature calls).

Add threading to that.

What saved me was that the compiler was re-written from scratch, and
so was the run-time.  I started with a new "malloc", then the GC.
The two were tightly connected and needed to be aware of each other.
Then I tackled the CECIL (C -> Eiffel) and HECTOR (Eiffel -> C)
interfaces.  Everything else followed nicely.

I don't know how much of this applies to Perl6, but now you know
as much as I do on the subject. (well, almost...)

Raphael

Re: RFC 27 (v1) Coroutines for Perl

[Tom Scola]

>I don't see the need for a new keyword. An attribute should be fine
>
>   sub foo : coroutine { }
>
>Would do quite well. It would fit with the :method attribute, and
>a possible :lvalue attribute.

That would be fine.

>Anyway, isn't what you are proposing more of a generator/sink rather
>than a coroutine? I understood a coroutine to be a set of mutual
>calling functions/procedures.

The two things you described are the same thing.  Read the second reference
in my RFC:

Re: Imrpoving tie() (Re: RFC 15 (v1) Stronger typing through tie.)

[Dan Sugalski]

At 12:35 PM 8/4/00 -0400, Chaim Frenkel wrote:
> > "DS" == Dan Sugalski <[EMAIL PROTECTED]> writes:
>
>DS> The language semantics of tie strongly impact the internals. tie() is
>DS> basically a declaration that the rules are completely different (and
>DS> unknown at compile time) for the tied variable. Shoots down 
>optimization a
>DS> bunch, since access to a tied varible has to be treated as a function 
>call
>DS> rather than as access to data with known behaviours.
>
>Why?
>
>The vtbl for a tied variable would do all the work. Either the pointer
>is to a springboard into perl code, or internal code, or the XS
>replacement code.

It's more than that. If the optimizer does code flow, it can know that 
$foo's been assigned an integer, for example, and maybe even that that 
variable's never been changed, so this:

   $foo = 12;
   $bar = something();
   $bar = $foo;

could work out to:

   $foo = $bar = 12;
   something();

Where if you tie, we have *no* idea what you're doing. This:

   tie $foo, SomeModule;
   $foo = 12;

could well reboot some server in outer mongolia for all we know. Tied 
variable access counts as a function call, which smacks optimization all by 
itself.

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: RFC 27 (v1) Coroutines for Perl

[Tom Scola]

>[I think this belongs on the language list, FWIW, Cc'd there]
>
>I like this, but I'd like to see this, inter-thread queues, and events
all 
>use the same communication method. Overload filehandles to pass events

>around instead, so:

I'm proposing that events and threads be dropped in lieu of coroutines.
 I view events and threads as low-level attempts to implement what coroutines
express elegantly.

There would be two perl run-times, one event-based and the other thread
based, much like java green threads and native threads.

> Invoking the coroutine could use another keyword,
>perhaps invoke, that returns the filehandle the coroutine talks on. So:

>
>   $fh = invoke foo("1", "2", "3")

As I tried to explain in the RFC, the "invoke" keyword be redundant.  Writing
to, or reading from the filehandle would invoke the coroutine.

>
>This would necessitate the expansion of select to check for pending 
>events/coroutine writes/data, but that's likely to happen anyway, so...


In my proposed event-based perl runtime, the select loop would be hidden
from the programmer.  All I/O calls would be non-blocking and context switching.

Re: what will be in Perl6 ?

[Tim Bunce]

On Thu, Aug 03, 2000 at 08:21:38AM -0400, Joshua N Pritikin wrote:
> On Wed, Aug 02, 2000 at 11:40:09PM +0100, [EMAIL PROTECTED] wrote:
> > On Wed, Aug 02, 2000 at 10:57:27AM -0700, Larry Wall wrote:
> > > http://windows.oreilly.com/news/hejlsberg_0800.html
> > 
> > Impressive. Quite deeply impressive.
> 
> Careful!  Don't be overwhelmed by the marketing spin.

I wasn't.

> Don't underestimate the perl community.

I don't.

Tim.

Re: RFC 27 (v1) Coroutines for Perl

[Chaim Frenkel]

I don't see the need for a new keyword. An attribute should be fine

sub foo : coroutine { }

Would do quite well. It would fit with the :method attribute, and
a possible :lvalue attribute.

Anyway, isn't what you are proposing more of a generator/sink rather
than a coroutine? I understood a coroutine to be a set of mutual
calling functions/procedures.

I also see this as more appropriate to the -language group.

Hmm, would a 
return :here EXPR
handle the resumption issues?



PRL> =head1 DESCRIPTION

PRL> This proposal introduces a new keyword to perl, "co", as a complement
PRL> to "sub".  A subroutine is defined thusly:

PRL>   co foo { ... }

PRL> Coroutines can also be closures:

PRL>   my $x = co { ... }

PRL> Unlike coroutines as defined by Knuth, and implemented in laguages
PRL> such as Simula or Python, perl does not have an
PRL> explicit "resume" call for invoking coroutines.  Instead, perl uses a
PRL> mechanism that perl programmers are already familiar with: pipes.
PRL> Note that these are intra-process pipes, and would be implemented
PRL> differently than kernel pipes.

PRL> Coroutines are identified by a prefixed vertical bar.  To invoke a
PRL> coroutine, you write data to or read data from it.

PRL>$y = <|foo>;

PRL>print |$x "hello, world\n";

-- 
Chaim FrenkelNonlinear Knowledge, Inc.
[EMAIL PROTECTED]   +1-718-236-0183

Re: Imrpoving tie() (Re: RFC 15 (v1) Stronger typing through tie.)

[Chaim Frenkel]

> "DS" == Dan Sugalski <[EMAIL PROTECTED]> writes:

DS> The language semantics of tie strongly impact the internals. tie() is 
DS> basically a declaration that the rules are completely different (and 
DS> unknown at compile time) for the tied variable. Shoots down optimization a 
DS> bunch, since access to a tied varible has to be treated as a function call 
DS> rather than as access to data with known behaviours.

Why?

The vtbl for a tied variable would do all the work. Either the pointer
is to a springboard into perl code, or internal code, or the XS
replacement code.

We might be able to add a hint hook to the module (or the vtbl) that
would help optimization and the compiler.


-- 
Chaim FrenkelNonlinear Knowledge, Inc.
[EMAIL PROTECTED]   +1-718-236-0183

Re: perl6-internals-gc sublist

[Chaim Frenkel]

> "DS" == Dan Sugalski <[EMAIL PROTECTED]> writes:

DS> This seems a bit premature, given that we haven't actually come up with 
DS> even a framework for an API, or hashed out much on the format of variables 
DS> or the interpreter structure. (Nor the threading or event stuff...)

Either the API or the internal representations should be worked on.
They might be done in parallel.


-- 
Chaim FrenkelNonlinear Knowledge, Inc.
[EMAIL PROTECTED]   +1-718-236-0183

Re: Avoid memory copy and redundant loops in reduce/fold

[Chaim Frenkel]

> "DS" == Dan Sugalski <[EMAIL PROTECTED]> writes:

DS> At 06:51 AM 8/4/00 -0700, Benjamin Stuhl wrote:
>> This actually leads to a much more general question, namely
>> passing of arrays to functions. For ppcode at least and
>> probably any code using the perl API, it should be possible
>> and IMHO desirable to push the AV* (or equivalent), rather
>> than expanding the array and pushing each of its elements.
>> Furthermore, if we do this, it would make passing named
>> array arguments (sub foo (@baz, @qux) { ... }) much
>> simpler.

DS> There are some threading issues, but if we do iterators then we can 
DS> definitely use this shortcut--shifting off of @_ just iterates through each 
DS> array short-cut pushed onto the arg stack.

This would do away with the list/array duality. And an array return
@foo = &my_sub;
would be a couple of pointer swaps. Probably not a win, but a series
of array returns may not need to have copies made. Just leave the
array on the stack.

join ' ', grep(...), grep(), ...

and only join would need to iterate.

I think only @_ and the named prototypes would need to be aware of the
difference.

Though what about %hash on the stack? Hmm, not a problem @_ would
use the each iterator.


-- 
Chaim FrenkelNonlinear Knowledge, Inc.
[EMAIL PROTECTED]   +1-718-236-0183

Re: kpathsea

[Simon Cozens]

(Chaim, please can you *not* CC me stuff on a list you know I read! It's
not as if we don't get too much mail already from this. :)

On Fri, Aug 04, 2000 at 12:19:30PM -0400, Chaim Frenkel wrote:
> Then how about a perllib.db. Pre-digested search paths. What %INC would
> look like if everything and the kitchen sink were loaded. No parsing, 
> nothing but straight forward lookup code.
> 
> If not found, or an entry not found fall back.

Sounds good. Here's a slight modification: perllib.db is a cache; lookups
take place as normal, but then any new information is written into the cache.
The cache is invalided every $configurable period.

Of course, we can do this with Perl 5, but it would be kludgy. (Remember
putting subrefs into @INC?)

-- 
For true believers, LORD would be K\textsc{nuth} in TeX, and
L\textsc{amport} in LaTeX. Atheists prefer \phantom{LORD}. Agnostics
may need to use the ifthen package.
 - Chris Boyd, comp.text.tex

Re: kpathsea

[Chaim Frenkel]

Then let me propose (i.e. throw up the balloon and duck while everyone
shoots at it).

Then how about a perllib.db. Pre-digested search paths. What %INC would
look like if everything and the kitchen sink were loaded. No parsing, 
nothing but straight forward lookup code.

If not found, or an entry not found fall back.

Decision of searching in the installation tree if not found in the db
is debatable. But I would 'trust' the perllib.db. Otherwise there is
an installation problem.

Special entries could be made for finding site-lib, versioning.



> "SC" == Simon Cozens <[EMAIL PROTECTED]> writes:

SC> On Fri, Aug 04, 2000 at 09:13:44AM -0400, Chaim Frenkel wrote:
>> Well, the issue is how much time is spent opening directories and checking
>> for entries. Also on an NFS mounted file system, the directory has to be
>> re-requested.

SC> It may take just as long to parse the kpathsea configs, grovel through the
SC> lsr file and locate the thing you need. I'd have to benchmark it to be sure.

>> Actually, thinking about it. This may make automatic @INC adjustments.
>> All the core needs to find is the ls-R, that would give it what to search.

SC> Nope. kpathsea needs various config stuff to get it bootstrapped.

SC> It's an idea, and a reasonably sensible one, but I'm really not sure it's
SC> a win.

-- 
Chaim FrenkelNonlinear Knowledge, Inc.
[EMAIL PROTECTED]   +1-718-236-0183

Re: RFC 27 (v1) Coroutines for Perl

[Dan Sugalski]

At 09:04 AM 8/4/00 -0700, Nathan Wiger wrote:
> > I like this, but I'd like to see this, inter-thread queues, and events all
> > use the same communication method. Overload filehandles to pass events
> > around instead, so:
> >
> >my $thing = <$handle>;
> >
> > could read a record from a file, or get an event from the event queue, or
> > receive some data from another thread. (With perhaps <$foo> in an lvalue
> > context being the same as sending a message down the filehandle to
> > whatever's on the other end)
>
>I have my way with the new open(), this'll happen. See RFC 14. Version 2
>(which explores this in detail) should be up this afternoon.

I don't know that overloading open is appropriate for this. There's no 
reason that multiple different things return filehandles. It's a clever and 
keyword-conserving hack to have open do a zillion different things, but it 
might be better to have different creation/attachment methods for 
conceptually distinct things.

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: RFC 27 (v1) Coroutines for Perl

[Nathan Wiger]

> I like this, but I'd like to see this, inter-thread queues, and events all
> use the same communication method. Overload filehandles to pass events
> around instead, so:
> 
>my $thing = <$handle>;
> 
> could read a record from a file, or get an event from the event queue, or
> receive some data from another thread. (With perhaps <$foo> in an lvalue
> context being the same as sending a message down the filehandle to
> whatever's on the other end)

I have my way with the new open(), this'll happen. See RFC 14. Version 2
(which explores this in detail) should be up this afternoon.

-Nate

Re: RFC 27 (v1) Coroutines for Perl

[Dan Sugalski]

At 03:13 PM 8/4/00 +, Perl6 RFC Librarian wrote:
>Coroutines for Perl

[I think this belongs on the language list, FWIW, Cc'd there]

I like this, but I'd like to see this, inter-thread queues, and events all 
use the same communication method. Overload filehandles to pass events 
around instead, so:

   my $thing = <$handle>;

could read a record from a file, or get an event from the event queue, or 
receive some data from another thread. (With perhaps <$foo> in an lvalue 
context being the same as sending a message down the filehandle to 
whatever's on the other end)

Also, I'd rather use sub attributes rather than a new keyword. Something like:

   sub foo : coroutine($fh) {}

to declare the sub as a coroutine that talks on the filehandle $fh. (As far 
as the sub is concerned) Invoking the coroutine could use another keyword, 
perhaps invoke, that returns the filehandle the coroutine talks on. So:

   $fh = invoke foo("1", "2", "3")

would fire up the coroutine foo, pass it 1, 2, and 3 in @_, and return to 
the caller the filehandle that foo's synchronizing on. Expanding it to have 
separate in and out filehandles or something also wouldn't be unreasonable.

This would necessitate the expansion of select to check for pending 
events/coroutine writes/data, but that's likely to happen anyway, so...

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

RFC 27 (v1) Coroutines for Perl

[Perl6 RFC Librarian]

This and other RFCs are available on the web at
  http://tmtowtdi.perl.org/rfc/

=head1 TITLE

Coroutines for Perl

=head1 VERSION

  Maintainer: Tom Scola <[EMAIL PROTECTED]>
  Date: 04 Aug 2000
  Version: 1
  Mailing List: [EMAIL PROTECTED]
  Number: 27

=head1 ABSTRACT

Two motavations for the development of Perl 6 are to bring threading
and event-based programming to the core language.  Coroutines can be
implemented in either a threaded or an event-based manner, are at a
much higher level conceptually, and are much easier to program than
either of those methods.  As implemented in perl, coroutines use
programming methods that most perl programmers are already familiar
with, making the transition to them simple and natural.

=head1 DESCRIPTION

This proposal introduces a new keyword to perl, "co", as a complement
to "sub".  A subroutine is defined thusly:

  co foo { ... }

Coroutines can also be closures:

  my $x = co { ... }

Unlike coroutines as defined by Knuth, and implemented in laguages
such as Simula or Python, perl does not have an
explicit "resume" call for invoking coroutines.  Instead, perl uses a
mechanism that perl programmers are already familiar with: pipes.
Note that these are intra-process pipes, and would be implemented
differently than kernel pipes.

Coroutines are identified by a prefixed vertical bar.  To invoke a
coroutine, you write data to or read data from it.

   $y = <|foo>;

   print |$x "hello, world\n";

Inside a coroutine, the meanings of "<>" and the default file
descriptor for print, printf, etc. are overloaded.

Coroutines and pipelines would effectively turn Perl into a dataflow
programming language, providing yet another paradigm for Perl to
encompass.

=head1 IMPLEMENTATION

Coroutines can be implemented in their traditional manner, using context
switching, or as threads, but would hide the mundane details of threads
from the programmer.  They would be more suited for producer/consumer type
problems rather than massively parallel compuations, but I doubt there is
much call for the latter to be implemented in perl.

=head2 OPEN ISSUES

=over

=item B

There probably needs to be some mechanism for defining the priority of
a coroutine.

=item B

Is new syntax needed for passing references via pipelines?

=back

=head1 REFERENCES

=over

=item *

Donald Knuth, I

=item *

"Pipes -- Powerful and Elegant Programming Paradigm" 
http://www.softpanorama.org/Scripting/pipes.shtml

=item *

"Coroutines in Python" http://www.nightmare.com/~rushing/copython/

=back

Re: Language RFC Summary 4th August 2000

[Dan Sugalski]

At 08:41 AM 8/4/00 -0400, Bryan C. Warnock wrote:
>On Fri, 04 Aug 2000, Uri Guttman wrote:
> > > "s" == skud  <[EMAIL PROTECTED]> writes:
> >
> >   s> Up for grabs:
> >   s> -
> >   s> Formats out of core
> >
>
>Somehow, I missed this message.
>
>I don't think that's a language issue.  Whether Perl continues to
>support formats certainly is, but its location within Perl is more of
>an internals thing.

It definitely is, since formats do things that can't be done in modules. If 
they yank formats out (which is just dandy by me) that means that some 
means of providing format's functionality needs has to be designed in.

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: named parameters

[Dan Sugalski]

At 04:37 AM 8/4/00 -0600, Tom Christiansen wrote:
> >Point taken. But part of the goal was moving a lot of stuff out of CORE
> >and making Perl faster. New features are great, but we should figure out
> >whether or not they're truly CORE-worthy. Sticking it in a pragma like
> >strict helps to solve this issue, but as always TMTOWTDI.
>
>I find the notion that Perl will become appreciably faster simply
>by having fewer default features to be unlikely.

If we go the double-indirect opcode function route, cutting down on the 
number of opcodes may well help, as will careful grouping of the opcode 
functions so opcodes that execute in sequence have their function pointers 
close-by. This'll reduce cache misses, which is a good thing.

It also reduces the number of opcodes the optimizer (and the people writing 
the optimizer) need to deal with, which isn't a bad thing either. (Though, 
granted, I really doubt anyone'll spend any significant amount of mental 
effort on the localtime op)

Finally, it may make things slightly easier for folks writing 
JIT/compiler/translators for perl bytecode. If we yank the 'odd' ops out 
into separate functions, and the doodad writers provide an interface to the 
non-op functions, it means they don't have to worry about writing code to 
do localtime however we do it, they can just call our function.


Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: GC

[Dan Sugalski]

At 01:35 AM 8/4/00 -0400, Ken Fox wrote:
>Simon Cozens wrote:
> > Ref counting isn't garbage collection.
> > http://www.jwz.org/doc/gc.html
>
>Please explain that to Richard Jones and Rafael Lins who have
>written a beautiful book surveying garbage collectors.

Title! Title, ISBN, and publisher, please!


Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: Imrpoving tie() (Re: RFC 15 (v1) Stronger typing through tie.)

[Dan Sugalski]

At 03:59 PM 8/3/00 -0700, Nathan Wiger wrote:
> > Several people have requested strong typing as a feature, but have been 
> shot
> > down with reasons such as "it's un-Perl-like", with an added "it'll slow
> > everything down for those who don't want it".
>
>Definitely.

Maybe. And it may speed things up. And finally if we do the vtable 
implementation of variables, at worst it'll only impact those variables 
that are strongly typed.

> > Unfortunately, accessing and manipulating tied variables is incredibly 
> slow,
> > so improving their speed is a large part of making this proposal feasible.
>
>I think a lot of effort should be put into improving tie(). There are
>tons of advantages to tie, and it could be used in incredibly flexible
>ways in Perl6 as I see it. While this is mainly an internals issue, I
>think it should be pretty high on the priority list.

The language semantics of tie strongly impact the internals. tie() is 
basically a declaration that the rules are completely different (and 
unknown at compile time) for the tied variable. Shoots down optimization a 
bunch, since access to a tied varible has to be treated as a function call 
rather than as access to data with known behaviours.

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: Avoid memory copy and redundant loops in reduce/fold

[Dan Sugalski]

At 06:51 AM 8/4/00 -0700, Benjamin Stuhl wrote:
>This actually leads to a much more general question, namely
>passing of arrays to functions. For ppcode at least and
>probably any code using the perl API, it should be possible
>and IMHO desirable to push the AV* (or equivalent), rather
>than expanding the array and pushing each of its elements.
>Furthermore, if we do this, it would make passing named
>array arguments (sub foo (@baz, @qux) { ... }) much
>simpler.

There are some threading issues, but if we do iterators then we can 
definitely use this shortcut--shifting off of @_ just iterates through each 
array short-cut pushed onto the arg stack.

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: perl6-internals-gc sublist

[Dan Sugalski]

At 09:42 AM 8/4/00 -0400, John Tobey wrote:
>Oops, I guess this should be contingent on Dan Sugalski's approval.

This seems a bit premature, given that we haven't actually come up with 
even a framework for an API, or hashed out much on the format of variables 
or the interpreter structure. (Nor the threading or event stuff...)

Are you comfortable with a shorter term list (Say,two or four weeks) that 
will work up several different GC RFCs, based on various constraints? If 
so, I'm fine with it.

>John Tobey <[EMAIL PROTECTED]> wrote:
> > Ask, can we please have the following list:
> >
> > Name:   perl6-internals-gc
> > Chairs: John Tobey <[EMAIL PROTECTED]>
> > Ken Fox <[EMAIL PROTECTED]>
> > Deadline:   Conterminous with perl6-internals (currently 13
> >   October 2000)
> > Mission:Determine which garbage collection models will be
> >   supported.  Identify requirements and implications of
> >   such support.
> > Description:This list will discuss possible memory management
> >   frameworks and requirements they would place on API,
> >   language, and internals.
> >
> > Thanks in advance,
> > -John
> >
> >


Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: Avoid memory copy and redundant loops in reduce/fold

[Ken Fox]

Jeremy Howard wrote:
> $a = sum(@b*@c+@d)

I'm not strong in math, but I do remember a bit about row and column
vectors. Isn't @b*@c ambiguous? Shouldn't it normally be interpreted
as a dot product, i.e. treat all vectors the same?

> The normal problem with this type of structure is that the previous
> statement would create 2 array copies, and 3 loops for most compilers. In
> perl speak, it might look like:
> $dummy1[$_] = $b[$_]*$c[$_] for (0..$#b-1);
> $dummy2[$_] = $d[$_]+$dummy1[$_] for (0..$#dummy1-1);
> $sum+=$_ for (@dummy2);

A simpler rule would be to treat an array as an implicit stream
whenever it's used in a scalar+reduce context. Then the following
code would be generated:

  $dummy1 = stream @b;
  $dummy2 = stream @c;
  $dummy3 = stream @d;
  while ($dummy4 = $dummy1->next * $dummy2->next + $dummy3->next) {
$sum += $dummy4;
  }

> $sum+=$b[$_]*$c[$_]+$d[$_] for (0..$#b-1);

Shouldn't that be

  $sum+=$b[$_]*$c[$_]+$d[$_] for (0..min($#b, $#c, $#d));

> Without this optimisation, array semantics become next to useless for
> numeric programming, because their overhead is just so high.

I think talking about array semantics instead of vector and matrix
semantics is next to useless too, but I'm not a math guy.

- Ken

Re: Avoid memory copy and redundant loops in reduce/fold

[Benjamin Stuhl]

> The normal problem with this type of structure is that
> the previous
> statement would create 2 array copies, and 3 loops for
> most compilers. In
> perl speak, it might look like:
> $dummy1[$_] = $b[$_]*$c[$_] for (0..$#b-1);
> $dummy2[$_] = $d[$_]+$dummy1[$_] for (0..$#dummy1-1);
> $sum+=$_ for (@dummy2);
> (Sorry if this isn't very idiomatic perl--it's not really
> my native
> language.)
> 
> Progressive C++ numeric programming libraries like POOMA
> and Blitz++ use
> template meta-programming techniques to implement
> 'expression templates'.
> Templates are used to create the parse tree for these
> kind of array
> expressions at compile time, and the compiler then
> optimises out the extra
> loops and array copies to create something like:
> $sum+=$b[$_]*$c[$_]+$d[$_] for (0..$#b-1);
> 
> Without this optimisation, array semantics become next to
> useless for
> numeric programming, because their overhead is just so
> high. But writing
> numericly intensive programs without array semantics is
> messy--they become
> littered with control structures and loops (which is
> particularly
> unintuitive for mathmaticians used to the compact
> notation of mathematics).
> 
> So, could perl 6 do this optimisation (assuming that the
> array
> notation/folding stuff makes its way into the language)?
> Given that some
> amount of compilation or interpretation will presumably
> still be done at
> run-time, perhaps this is much easier in perl than in
> C++...

This actually leads to a much more general question, namely
passing of arrays to functions. For ppcode at least and
probably any code using the perl API, it should be possible
and IMHO desirable to push the AV* (or equivalent), rather
than expanding the array and pushing each of its elements.
Furthermore, if we do this, it would make passing named
array arguments (sub foo (@baz, @qux) { ... }) much
simpler. 
If a subroutine asks for @_, than we can go the old way and
push everything, but reducing the number of stack pushes on
list operators could be a major win.

-- BKS

__
Do You Yahoo!?
Kick off your party with Yahoo! Invites.
http://invites.yahoo.com/

Re: perl6-internals-gc sublist

[John Tobey]

Oops, I guess this should be contingent on Dan Sugalski's approval.
-John

John Tobey <[EMAIL PROTECTED]> wrote:
> Ask, can we please have the following list:
> 
> Name:   perl6-internals-gc
> Chairs: John Tobey <[EMAIL PROTECTED]>
> Ken Fox <[EMAIL PROTECTED]>
> Deadline:   Conterminous with perl6-internals (currently 13
>   October 2000)
> Mission:Determine which garbage collection models will be
>   supported.  Identify requirements and implications of
>   such support.
> Description:This list will discuss possible memory management
>   frameworks and requirements they would place on API,
>   language, and internals.
> 
> Thanks in advance,
> -John
> 
> 

Re: Language RFC Summary 4th August 2000

[Chaim Frenkel]

> "BCW" == Bryan C Warnock <[EMAIL PROTECTED]> writes:

BCW> On Fri, 04 Aug 2000, Uri Guttman wrote:
>> > "s" == skud  <[EMAIL PROTECTED]> writes:
>> 
s> Up for grabs:
s> -
s> Formats out of core
>> 

BCW> Somehow, I missed this message.

BCW> I don't think that's a language issue.  Whether Perl continues to
BCW> support formats certainly is, but its location within Perl is more of
BCW> an internals thing.

Not quite. It reflects up to the language. Are all non-core items, hard
coded into the language or are they able to be recognized as an already
installed module.

This would help avoid the proliferation of uses. And let perl find the
right use. The only need for use would then be to customize the behavior.


-- 
Chaim FrenkelNonlinear Knowledge, Inc.
[EMAIL PROTECTED]   +1-718-236-0183

perl6-internals-gc sublist

[John Tobey]

Ask, can we please have the following list:

Name:   perl6-internals-gc
Chairs: John Tobey <[EMAIL PROTECTED]>
Ken Fox <[EMAIL PROTECTED]>
Deadline:   Conterminous with perl6-internals (currently 13
October 2000)
Mission:Determine which garbage collection models will be
supported.  Identify requirements and implications of
such support.
Description:This list will discuss possible memory management
frameworks and requirements they would place on API,
language, and internals.

Thanks in advance,
-John

kpathsea

[Simon Cozens]

On Fri, Aug 04, 2000 at 09:13:44AM -0400, Chaim Frenkel wrote:
> Well, the issue is how much time is spent opening directories and checking
> for entries. Also on an NFS mounted file system, the directory has to be
> re-requested.

It may take just as long to parse the kpathsea configs, grovel through the
lsr file and locate the thing you need. I'd have to benchmark it to be sure.

> Actually, thinking about it. This may make automatic @INC adjustments.
> All the core needs to find is the ls-R, that would give it what to search.

Nope. kpathsea needs various config stuff to get it bootstrapped.

It's an idea, and a reasonably sensible one, but I'm really not sure it's
a win.

-- 
"If that makes any sense to you, you have a big problem."
-- C. Durance, Computer Science 234

Re: date interface (was Re: perl6 requirements, on bootstrap)

[Chaim Frenkel]

> "SC" == Simon Cozens <[EMAIL PROTECTED]> writes:

SC> On Thu, Aug 03, 2000 at 09:12:45PM -0400, Chaim Frenkel wrote:
>> And while were here, does anyone understand kpathsea?

SC> Yes.

>> Would it be a win. I think it would.

SC> There's been some amount of talk on one of the sekrit cabal TeX
SC> mailing lists about getting rid of it. I can't see where you'd
SC> want to use it. To locate modules and shared objects? No, far
SC> better (and probably actually faster) to search a list of paths as
SC> we're doing right now. It also means you don't have to trust all
SC> the users and packages out there to update their Perl lsrs every
SC> time something changes.

Well, the issue is how much time is spent opening directories and checking
for entries. Also on an NFS mounted file system, the directory has to be
re-requested.

As for keeping hte ls-r updated. That would be the job of the installation
process. perl core would be taken care by install. site-lib (or its
replacement) would be handled by makemaker's replacement.

All other directories would be handled via the fallback to normal searching.

Actually, thinking about it. This may make automatic @INC adjustments.
All the core needs to find is the ls-R, that would give it what to search.
The rest would be PERL[56]?LIB and family.


-- 
Chaim FrenkelNonlinear Knowledge, Inc.
[EMAIL PROTECTED]   +1-718-236-0183

Avoid memory copy and redundant loops in reduce/fold

[Jeremy Howard]

I'm a bit unsure of where to put this... it's a question of implementation,
but of a feature that is still being discussed still in perl6-language...
Let me know if this is the wrong forum or the wrong time...

There's been a lot of discussion about reduce/fold on perl6-language. The
details are still being decided (and Damian Conway is doing an RFC on this
and similar topics), but it's likely that something like this will be
doable:
$a = sum(@b*@c+@d)
where the lists are all 1-dimensional arrays, here, so $a ends up as the sum
of {the component-wise multiplication of @b and @c, added component-wise to
@d}.

The normal problem with this type of structure is that the previous
statement would create 2 array copies, and 3 loops for most compilers. In
perl speak, it might look like:
$dummy1[$_] = $b[$_]*$c[$_] for (0..$#b-1);
$dummy2[$_] = $d[$_]+$dummy1[$_] for (0..$#dummy1-1);
$sum+=$_ for (@dummy2);
(Sorry if this isn't very idiomatic perl--it's not really my native
language.)

Progressive C++ numeric programming libraries like POOMA and Blitz++ use
template meta-programming techniques to implement 'expression templates'.
Templates are used to create the parse tree for these kind of array
expressions at compile time, and the compiler then optimises out the extra
loops and array copies to create something like:
$sum+=$b[$_]*$c[$_]+$d[$_] for (0..$#b-1);

Without this optimisation, array semantics become next to useless for
numeric programming, because their overhead is just so high. But writing
numericly intensive programs without array semantics is messy--they become
littered with control structures and loops (which is particularly
unintuitive for mathmaticians used to the compact notation of mathematics).

So, could perl 6 do this optimisation (assuming that the array
notation/folding stuff makes its way into the language)? Given that some
amount of compilation or interpretation will presumably still be done at
run-time, perhaps this is much easier in perl than in C++...

Re: named parameters

[Ken Fox]

"Bryan C. Warnock" wrote:
> It seems to me that whether something is CORE-worthy is really a
> decision for the internals team.  Whether something should be a valid,
> meaningful construct in Perl, regardless of the implementation, is the
> focus of the language team.

The CORE of the Perl language is the syntax and semantics available
without any additional modules.

The CORE of the perl interpreter is libperl.so or just "perl" in a
static executable.

The CORE of the Perl distribution is whatever comes in perl.tar.gz.

Those are decisions for perl6-language, perl6-internals and perl6-build,
basically in that order. If perl6-language says something is CORE,
then perl6-internals has to live with it. The cases where something is
CORE in the language, but not in the interpreter (like "use strict")
makes dealing with Perl trickier. That's probably just me talking like
an embedder though.

- Ken

Re: Language RFC Summary 4th August 2000

[Bryan C . Warnock]

On Fri, 04 Aug 2000, Uri Guttman wrote:
> > "s" == skud  <[EMAIL PROTECTED]> writes:
> 
>   s> Up for grabs:
>   s> -
>   s> Formats out of core
> 

Somehow, I missed this message.

I don't think that's a language issue.  Whether Perl continues to
support formats certainly is, but its location within Perl is more of
an internals thing.

-- 
Bryan C. Warnock
([EMAIL PROTECTED])

Re: named parameters

[Bryan C . Warnock]

Internals added to cc:

On Fri, 04 Aug 2000, Nathan Wiger wrote:

> Point taken. But part of the goal was moving a lot of stuff out of CORE
> and making Perl faster. New features are great, but we should figure out
> whether or not they're truly CORE-worthy. Sticking it in a pragma like
> strict helps to solve this issue, but as always TMTOWTDI.

> BTW, I use CORE to mean "perl" and not "perl.tar.gz".

Hmmm, is that the generic "we"?

(It seems to me that whether something is CORE-worthy is really a
decision for the internals team.  Whether something should be a valid,
meaningful construct in Perl, regardless of the implementation, is the
focus of the language team.  Perhaps a better job of making this
distinction will keep the language list from its current bloat, as skud
pointed out.  The language list keeps migrating from "why" to "how".)

 -- 
Bryan C. Warnock
([EMAIL PROTECTED])

Re: RFC: Foreign objects in perl

[John Tobey]

Ken Fox <[EMAIL PROTECTED]> wrote:
> John Tobey wrote:
> > I think we are trying to accommodate any of several GC systems to be
> > selected amongst in future.
> 
> Then the Perl API needs to allow for the GC to move objects. If that
> can't happen, the majority of interesting collectors can't be used.

Yes, the Perl API needs to be extended for any of this to work.  At a
minimum, we'll need a Perl_mark(pTHX) or similar.

> > Ken Fox <[EMAIL PROTECTED]> wrote:
> > > Did you ever have this problem [cycles between systems] or is it
> > > contrived?
> > 
> > I personally guarantee the issue will arise.  :-)
> 
> Your finalizer doesn't catch cycles. Did you ever have problems with
> it not reclaiming garbage? A collector can still be useful even if it
> can't find all garbage. (Conservative collectors can't for example.)

Right, it won't catch cycles containing Perl structures, the same as
in Perl.

I suspect separating GC discussion out from this list will do more
good than harm, but in a 5-minute search I haven't found the formally
correct way (if there is one) to suggest a perl6-internals-gc list.

-- 
John Tobey, late nite hacker <[EMAIL PROTECTED]>
\\\   ///
]]] With enough bugs, all eyes are shallow.   [[[
///   \\\

Re: named parameters

[Tom Christiansen]

>Point taken. But part of the goal was moving a lot of stuff out of CORE
>and making Perl faster. New features are great, but we should figure out
>whether or not they're truly CORE-worthy. Sticking it in a pragma like
>strict helps to solve this issue, but as always TMTOWTDI.

I find the notion that Perl will become appreciably faster simply
by having fewer default features to be unlikely.  For example,
cutting down the syscall keywords from (insert random number...)
40 to 10 will certainly have no notable effect, either in compile-time,
run-time, or, perhaps surprising to many but to the best of
recollection soundly proven, very much in the way of disk image
size either.

Until you know *exactly* what it is that is "slow" and why, I'd
look carefully at all assumptions about issues of speed.  Some of
these things, maybe even many, are limited by the speed of the
opcode dispatch sequence.

--tom

Re: Recording what we decided *not* to do, and why

[raptor]

hi,
it will be good if all these RFC are put somewhere on the WEB (we can't
follow all those mailing lists if the amout of posts stay the same :") )
also in this way we will get broader picture what is happenning..
=
iVAN
[EMAIL PROTECTED]
=

Re: RFC: Foreign objects in perl

[Ken Fox]

Dan Sugalski wrote:
DS>   TheObj *foo;
DS>   SV *new_sv;
DS>   foo = new TheObj("A parameter");
DS>   sv = perl_new_sv();
DS>   perl_make_sv_object(sv, "Some::Package", foo, &dispatch_routine,
DS> &destroy_routine);
DS>
DS>   perl_return(perl_make_ref(sv));

Ken Fox wrote:
KF> Are you really thinking about keeping "SV *" and two-level refs
KF> and all that? That sounds an awful lot like the current internals.

DS> SV * as it exists in perl 5? No. Some magic cookie that perl's internals
DS> passes to extension code? Yep, you bet. Probably should've used PMC instead
DS> of SV * to distinguish them.

Ok. I just want to make sure you realize that by not returning the (SV *)
on *every* mutating operation that you've eliminated the possibility of
storing values in the (SV *) itself.

Here's an example of something that might be nice.

There are 3 immediate types and 1 extended type -- the bottom 2 bits
of the (SV *) are used to identify them:

  SV *x

  if (x & 3 == 0) x is an extended type -- a pointer to an "object"
  if (x & 3 == 1) x is an integer -- value = x >> 2
  if (x & 3 == 2) x is a string constant -- value = constant[x >> 2]
  if (x & 3 == 3) x is [something else that can fit in 30 bits]

This is fairly traditional value boxing that is used in many lisp
implementations. Yeah, I know bit ops suck and lots of values will
end up being pointer derefs anyways. It seems like something we
should investigate -- the impact on the perl API shouldn't be too
big.

> Your extended way's cool too--RFC it and we can do that as well.

I've been wanting to. It would be nice (at least for me) for you to
start suggesting more RFC "assignments" like this. Community is not
the same as "no ownership" and there are a lot of people that have
a lot more ownership of internals than me.

- Ken