Re: RFC 155 - Remove geometric functions from core

[Russ Allbery]

Andy Dougherty <[EMAIL PROTECTED]> writes:

> I'm sure the glibc folks indeed work very hard at this and are largely
> successful.  I also know, however, that over the past couple of years or
> so, I've had to recompile nearly all of my applications on several
> occasions when I've upgraded glibc.  Other times, glibc upgrades have
> gone without a hitch.  It's probably my fault and probably somewhere
> deep in my personal library I'm incorrectly fiddling with stdio
> internals or something, but I just wanted to offer a counter data point
> that doing this sort of this robustly is, indeed, very hard.

It may not be your fault... my understanding is that glibc 2.0 really
didn't do things right, and that glibc 2.1 did break some binary
compatibility to fix some serious bugs.  It's probably only fair to start
holding glibc to this standard from 2.2 and up.

Perl *should* have a *much* easier task than glibc, given that our
interface is positively tiny compared to the entire C library.

-- 
Russ Allbery ([EMAIL PROTECTED]) 

Re: RFC 155 - Remove geometric functions from core

[Bryan C . Warnock]

On Wed, 30 Aug 2000, Dan Sugalski wrote:
>
> I think we can pull this off if we're careful and draw really strict lines
> about what is and isn't public stuff. It's not easy, and it will mean
> we'll have to have some reference executables in a test suite for
> verification, but I think we can manage that.
> 
> I do want to have a set of C/XS/whatever sources as part of the test suite
> as well--right now perl's test suite only tests the language, and I think
> we should also test the HLL interface we present, as it's just as
> important in some ways.

One of the off-the-cuff suggestions I sent in to one of the myriad
polls for "fixing Perl", is that the bulk of Perl should be written in
the XS replacement.  This would force a two-pass build of perl, of
course - the first pass to build the XSR, and the second to build perl
itself.  But the advantage was that it would really solidify the
internals interface.  And give you your test suite above.  :-)

 -- 
Bryan C. Warnock
([EMAIL PROTECTED])

Re: RFC 155 - Remove geometric functions from core

[Nick Ing-Simmons]

Dan Sugalski <[EMAIL PROTECTED]> writes:
>At 07:32 PM 8/29/00 +, Nick Ing-Simmons wrote:
>>David L . Nicol <[EMAIL PROTECTED]> writes:
>> >
>> >Did I not just describe how a .so or a DLL works currently?
>>
>>And behind the scenes that does something akin to:
>>
>>int fd = open("file_of_posn_indepenant_byte_code",O_RDONLY);
>>struct stat st;
>>fstat(fd,&st);
>>code_t *code = mmap(NULL,st.st_len,PROT_READ,MAP_SHARED,fd,0);
>>close(fd);
>
>Don't forget the fixup work that needs to be done afterwards. Loading the 
>library into memory's only the first part--after that the loader needs to 
>twiddle with transfer vectors and such so the unresolved calls into the 
>routines in the newly loaded library get resolved.

I finessed the "fixup work" by saying "position independant byte code".
The fixups break the shareability of the pages which is why you compile
shared libs -fPIC. So we should strive to have minimal fixups and 
collect them in one place (which vtables do very nicely).


-- 
Nick Ing-Simmons

Re: RFC 155 - Remove geometric functions from core

[Dan Sugalski]

On Wed, 30 Aug 2000, Andy Dougherty wrote:

> On Tue, 29 Aug 2000, Russ Allbery wrote:
> 
> > Not a big deal, and that's certainly doable.  But it's possible to do more
> > than that if you really want to.  The glibc folks have decided to comment
> > to nearly full binary compatibility for essentially forever; the theory is
> > that upgrading libc should never break a working application even if the
> > ABI changes.  I'm not familiar with the exact details of how symbol
> > versioning works, but as I understand it, this is what it lets you do.
> 
> I'm sure the glibc folks indeed work very hard at this and are largely
> successful.  I also know, however, that over the past couple of years or
> so, I've had to recompile nearly all of my applications on several
> occasions when I've upgraded glibc.  Other times, glibc upgrades have gone
> without a hitch.  It's probably my fault and probably somewhere deep in my
> personal library I'm incorrectly fiddling with stdio internals or
> something, but I just wanted to offer a counter data point that doing
> this sort of this robustly is, indeed, very hard.

I think we can pull this off if we're careful and draw really strict lines
about what is and isn't public stuff. It's not easy, and it will mean
we'll have to have some reference executables in a test suite for
verification, but I think we can manage that.

I do want to have a set of C/XS/whatever sources as part of the test suite
as well--right now perl's test suite only tests the language, and I think
we should also test the HLL interface we present, as it's just as
important in some ways.

dan

Re: RFC 155 - Remove geometric functions from core

[Andy Dougherty]

On Tue, 29 Aug 2000, Russ Allbery wrote:

> Not a big deal, and that's certainly doable.  But it's possible to do more
> than that if you really want to.  The glibc folks have decided to comment
> to nearly full binary compatibility for essentially forever; the theory is
> that upgrading libc should never break a working application even if the
> ABI changes.  I'm not familiar with the exact details of how symbol
> versioning works, but as I understand it, this is what it lets you do.

I'm sure the glibc folks indeed work very hard at this and are largely
successful.  I also know, however, that over the past couple of years or
so, I've had to recompile nearly all of my applications on several
occasions when I've upgraded glibc.  Other times, glibc upgrades have gone
without a hitch.  It's probably my fault and probably somewhere deep in my
personal library I'm incorrectly fiddling with stdio internals or
something, but I just wanted to offer a counter data point that doing
this sort of this robustly is, indeed, very hard.

-- 
Andy Dougherty  [EMAIL PROTECTED]
Dept. of Physics
Lafayette College, Easton PA 18042

Re: RFC 155 - Remove geometric functions from core

[Dan Sugalski]

At 07:37 PM 8/29/00 -0700, Russ Allbery wrote:
>Dan Sugalski <[EMAIL PROTECTED]> writes:
> > On 29 Aug 2000, Russ Allbery wrote:
>
> >> I'd love to see Perl aggressively take advantage of new capabilities in
> >> dynamic loaders, though.  Among other things, I'll point out that
> >> symbol versioning is the way that things like libc manage to be
> >> backward compatible while still changing things, and we should probably
> >> seriously consider using symbol versioning in a shared libperl.so as a
> >> means to provide that much desired and extremely difficult to implement
> >> stable API for modules and the XS-equivalent.
>
> > This is where my lack of strange Unix knowledge comes to the fore. Is
> > this really a problem? It seems to me to be a standard sort of thing to
> > be dealing with. (OTOH, my platform of choice has 20-year-old
> > executables as part of its test suite and a strong engineering bent, so
> > I may be coming at things from a different direction than most folks)
>
>Well, it depends on what your goals are, basically.  For most shared
>libraries, people don't take the trouble.

That's OK--we will. :)

>Basically, no matter how well you design the API up front, if it's at all
>complex you'll discover that down the road you really want to *change*
>something, not just add something new (maybe just add a new parameter to a
>function).  At that point, the standard Perl thing up until now to do is
>to just change it in a major release and require people to relink their
>modules against the newer version.  And relink their applications that
>embed Perl.

It's just hit me why VMS' system service interface has managed to handle 
this as well as it has over the years. Unfortunately one of the things that 
helped it's longevity is rather inconvenient for the average C programmer.

I'll write up something more concrete once I've batted it around some in my 
brain, and we can see if I'm off-base or, if not, whether it's worth it.

>Not a big deal, and that's certainly doable.  But it's possible to do more
>than that if you really want to.  The glibc folks have decided to comment
>to nearly full binary compatibility for essentially forever; the theory is
>that upgrading libc should never break a working application even if the
>ABI changes.  I'm not familiar with the exact details of how symbol
>versioning works, but as I understand it, this is what it lets you do.
>Both the old and the new symbol are available, and newly built
>applications use the new one while older applications continue to use the
>previous symbol.  That means that all your older binary modules keep
>working, and if your applications that embed Perl are linked dynamically,
>you can even upgrade Perl underneath them without having to rebuild them.
>
>I'm not sure it's worth the trouble, but it's something to consider.

I'm sure it is. I really, *really* want long-term binary compatibility.

Luckily for us, perl may end up with a reasonably small external API, 
which'll make life easier.

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: RFC 155 - Remove geometric functions from core

[Russ Allbery]

Dan Sugalski <[EMAIL PROTECTED]> writes:
> On 29 Aug 2000, Russ Allbery wrote:
 
>> I'd love to see Perl aggressively take advantage of new capabilities in
>> dynamic loaders, though.  Among other things, I'll point out that
>> symbol versioning is the way that things like libc manage to be
>> backward compatible while still changing things, and we should probably
>> seriously consider using symbol versioning in a shared libperl.so as a
>> means to provide that much desired and extremely difficult to implement
>> stable API for modules and the XS-equivalent.

> This is where my lack of strange Unix knowledge comes to the fore. Is
> this really a problem? It seems to me to be a standard sort of thing to
> be dealing with. (OTOH, my platform of choice has 20-year-old
> executables as part of its test suite and a strong engineering bent, so
> I may be coming at things from a different direction than most folks)

Well, it depends on what your goals are, basically.  For most shared
libraries, people don't take the trouble.

Basically, no matter how well you design the API up front, if it's at all
complex you'll discover that down the road you really want to *change*
something, not just add something new (maybe just add a new parameter to a
function).  At that point, the standard Perl thing up until now to do is
to just change it in a major release and require people to relink their
modules against the newer version.  And relink their applications that
embed Perl.

Not a big deal, and that's certainly doable.  But it's possible to do more
than that if you really want to.  The glibc folks have decided to comment
to nearly full binary compatibility for essentially forever; the theory is
that upgrading libc should never break a working application even if the
ABI changes.  I'm not familiar with the exact details of how symbol
versioning works, but as I understand it, this is what it lets you do.
Both the old and the new symbol are available, and newly built
applications use the new one while older applications continue to use the
previous symbol.  That means that all your older binary modules keep
working, and if your applications that embed Perl are linked dynamically,
you can even upgrade Perl underneath them without having to rebuild them.

I'm not sure it's worth the trouble, but it's something to consider.

-- 
Russ Allbery ([EMAIL PROTECTED]) 

Re: RFC 155 - Remove geometric functions from core

[Dan Sugalski]

On 29 Aug 2000, Russ Allbery wrote:

> I'm not sure I'm completely following what you're arguing for here, but be
> careful not to go too far down the road of duplicating what the dynamic
> loader already knows how to do.  There be dragons; that stuff is seriously
> baroque.  You really don't want to reimplement it.

I'd very much like to not do it. It's bad enough to have to do that sort
of thing with perl code we completely control--having to do it portably
for what's essentially native code is more work than I'd really like to
do. Clever People have already provided the capabilities on individual
platforms and that's just fine by me.
 
> I'd love to see Perl aggressively take advantage of new capabilities in
> dynamic loaders, though.  Among other things, I'll point out that symbol
> versioning is the way that things like libc manage to be backward
> compatible while still changing things, and we should probably seriously
> consider using symbol versioning in a shared libperl.so as a means to
> provide that much desired and extremely difficult to implement stable API
> for modules and the XS-equivalent.

This is where my lack of strange Unix knowledge comes to the fore. Is this
really a problem? It seems to me to be a standard sort of thing to be
dealing with. (OTOH, my platform of choice has 20-year-old executables as
part of its test suite and a strong engineering bent, so I may be coming
at things from a different direction than most folks)

I've been working on a spec for the API that'll hopefully isolate
extensions enough that we can yank the guts around at will without
affecting them. I'm really hoping that an extension built against perl
6.0.1 (presumably the first stable release... :) will work against perl
6.12.4. That's the plan, at least.

Dan

Re: RFC 155 - Remove geometric functions from core

[Russ Allbery]

David L Nicol <[EMAIL PROTECTED]> writes:

> This is what I was talking about when I suggested the language maintain
> a big list of all the addresses of each function, and after the function
> gets loaded or compiled it is added to the big list, and after this
> stage the placeholder in the op can be replaced with a longjump.

> Since the shared segments live at different addresses in different
> processes (or should I have stayed awake through that lecture)

I'm not sure I'm completely following what you're arguing for here, but be
careful not to go too far down the road of duplicating what the dynamic
loader already knows how to do.  There be dragons; that stuff is seriously
baroque.  You really don't want to reimplement it.

I'd love to see Perl aggressively take advantage of new capabilities in
dynamic loaders, though.  Among other things, I'll point out that symbol
versioning is the way that things like libc manage to be backward
compatible while still changing things, and we should probably seriously
consider using symbol versioning in a shared libperl.so as a means to
provide that much desired and extremely difficult to implement stable API
for modules and the XS-equivalent.

-- 
Russ Allbery ([EMAIL PROTECTED]) 

Re: RFC 155 - Remove geometric functions from core

[David L. Nicol]

Dan Sugalski wrote:

> 
> Don't forget the fixup work that needs to be done afterwards. Loading the
> library into memory's only the first part--after that the loader needs to
> twiddle with transfer vectors and such so the unresolved calls into the
> routines in the newly loaded library get resolved.
> 
> Dan


This is what I was talking about when I suggested the language maintain
a big list of all the addresses of each function, and after the function
gets loaded or compiled it is added to the big list, and after this stage
the placeholder in the op can be replaced with a longjump.

Since the shared segments live at different addresses in different
processes (or should I have stayed awake through that lecture)


And there you go, a JIT.


-- 
  David Nicol 816.235.1187 [EMAIL PROTECTED]

Re: RFC 155 - Remove geometric functions from core

[Jarkko Hietaniemi]

On Tue, Aug 29, 2000 at 07:29:00PM -0400, Uri Guttman wrote:
> > "TC" == Tom Christiansen <[EMAIL PROTECTED]> writes:
> 
>   >> Yup.  I think somebody said that mmap() is POSIX.  It isn't.  
>   TC> Are you sure?
> 
>   TC> http://slacvx.slac.stanford.edu/HELP/POSIX/CALLABLE_FUNCTIONS/MMAP
> 
> 
>   TC>The mmap() function maps process addresses to a memory object.
> 
> i think he meant POSIX didn't create the mmap call (which we agree was a
> bsd thing first). jarkko already said that posix has both mmap and their
> own shared memory api.

Valiant attempt to interpret me but I really meant that POSIX doesn't
have mmap, and I was valiantly wrong :-)

-- 
$jhi++; # http://www.iki.fi/jhi/
# There is this special biologist word we use for 'stable'.
# It is 'dead'. -- Jack Cohen

Re: RFC 155 - Remove geometric functions from core

[Uri Guttman]

> "TC" == Tom Christiansen <[EMAIL PROTECTED]> writes:

  >> Yup.  I think somebody said that mmap() is POSIX.  It isn't.  
  TC> Are you sure?

  TC> http://slacvx.slac.stanford.edu/HELP/POSIX/CALLABLE_FUNCTIONS/MMAP


  TC>The mmap() function maps process addresses to a memory object.

i think he meant POSIX didn't create the mmap call (which we agree was a
bsd thing first). jarkko already said that posix has both mmap and their
own shared memory api.

uri

-- 
Uri Guttman  -  [EMAIL PROTECTED]  --  http://www.sysarch.com
SYStems ARCHitecture, Software Engineering, Perl, Internet, UNIX Consulting
The Perl Books Page  ---  http://www.sysarch.com/cgi-bin/perl_books
The Best Search Engine on the Net  --  http://www.northernlight.com

Re: RFC 155 - Remove geometric functions from core

[Jarkko Hietaniemi]

On Tue, Aug 29, 2000 at 05:12:02PM -0600, Tom Christiansen wrote:
> >Yup.  I think somebody said that mmap() is POSIX.  It isn't.  
> 
> Are you sure?

Rats, I was wrong.  I dug up my copy and mmap(), mlock(), etc are all
in the latest edition of 1003.1. 

-- 
$jhi++; # http://www.iki.fi/jhi/
# There is this special biologist word we use for 'stable'.
# It is 'dead'. -- Jack Cohen

Re: RFC 155 - Remove geometric functions from core

[Tom Christiansen]

>Yup.  I think somebody said that mmap() is POSIX.  It isn't.  

Are you sure?

http://slacvx.slac.stanford.edu/HELP/POSIX/CALLABLE_FUNCTIONS/MMAP


   The mmap() function maps process addresses to a memory object.

   IEEE Std 1003.1b-1993, §12.2.1.

   C Format

 #include 

 void *mmap(void *addr, size_t len, int prot, int flags, int
   fildes, off_t off);
 


Googling for "POSIX mmap" comes up with various hits.  But I don't have
the stuff to look up.

--tom

Re: RFC 155 - Remove geometric functions from core

[Jarkko Hietaniemi]

On Tue, Aug 29, 2000 at 06:43:34PM -0400, Uri Guttman wrote:
> > "ST" == Sam Tregar <[EMAIL PROTECTED]> writes:
> 
>   ST> On Tue, 29 Aug 2000, Nick Ing-Simmons wrote:
>   >> David L . Nicol <[EMAIL PROTECTED]> writes:
>   >> >
>   >> >does sysV shm not support the equivalent security as the file system?
>   >> 
>   >> mmap() has the file system.
> 
>   ST> I wasn't aware that mmap() was part of SysV shared memory.  My
>   ST> mistake?  It's not on the SysV IPC man pages on my Linux system.
>   ST> The mmap manpage doesn't mention SysV IPC either.
> 
> mmap came from berkeley. i used it on early versions of sunos which was
> based on BSD. so calling it SysV IPC is wrong.

Yup.  I think somebody said that mmap() is POSIX.  It isn't.  POSIX
realtime extensions (and Single UNIX Spec) have shared memory objects,
which are different from either SysV IPC or mmap().  The smos have a
system-wide flat namespace, not connected to the usual filesystem
namespace (in the definition, that is, nobody of course forbids making
it visible)  mmap() is also in the SUS.

Executive summary: there are three different "shared memory" APIs.

-- 
$jhi++; # http://www.iki.fi/jhi/
# There is this special biologist word we use for 'stable'.
# It is 'dead'. -- Jack Cohen

Re: RFC 155 - Remove geometric functions from core

[Uri Guttman]

> "ST" == Sam Tregar <[EMAIL PROTECTED]> writes:

  ST> On Tue, 29 Aug 2000, Nick Ing-Simmons wrote:
  >> David L . Nicol <[EMAIL PROTECTED]> writes:
  >> >
  >> >does sysV shm not support the equivalent security as the file system?
  >> 
  >> mmap() has the file system.

  ST> I wasn't aware that mmap() was part of SysV shared memory.  My
  ST> mistake?  It's not on the SysV IPC man pages on my Linux system.
  ST> The mmap manpage doesn't mention SysV IPC either.

mmap came from berkeley. i used it on early versions of sunos which was
based on BSD. so calling it SysV IPC is wrong.

uri

-- 
Uri Guttman  -  [EMAIL PROTECTED]  --  http://www.sysarch.com
SYStems ARCHitecture, Software Engineering, Perl, Internet, UNIX Consulting
The Perl Books Page  ---  http://www.sysarch.com/cgi-bin/perl_books
The Best Search Engine on the Net  --  http://www.northernlight.com

Re: RFC 155 - Remove geometric functions from core

[Tom Christiansen]

>>> mmap() has the file system.
>>I wasn't aware that mmap() was part of SysV shared memory. 

>It is NOT. It is another (POSIX) way of getting shared memory bewteen 
>processes. Even without MAP_SHARED OS will share un-modified pages 
>between processes.

...

>SysV IPC is a mess IMHO. 

For a good time, see Camel 3's introductory discussion of SysV IPC. :-)

--tom

Re: RFC 155 - Remove geometric functions from core

[Nick Ing-Simmons]

Sam Tregar <[EMAIL PROTECTED]> writes:
>On Tue, 29 Aug 2000, Nick Ing-Simmons wrote:
>> David L . Nicol <[EMAIL PROTECTED]> writes:
>> >
>> >does sysV shm not support the equivalent security as the file system?
>> 
>> mmap() has the file system.
>
>I wasn't aware that mmap() was part of SysV shared memory. 

It is NOT. It is another (POSIX) way of getting shared memory bewteen 
processes. Even without MAP_SHARED OS will share un-modified pages 
between processes.

It happens to be the way modern UNIX implemements "shared .text".
i.e. the ".text" part of the object file is mmap()'ed  into 
each process.

>My
>mistake?  It's not on the SysV IPC man pages on my Linux system.  The mmap
>manpage doesn't mention SysV IPC either.

SysV IPC is a mess IMHO. 

My point was that if the "file system" is considered
sufficient then mmap()ing file system objects will get you "shared code"
or "shared data" without any tedious reinventing of wheels.

-- 
Nick Ing-Simmons

Re: RFC 155 - Remove geometric functions from core

[Sam Tregar]

On Tue, 29 Aug 2000, Nick Ing-Simmons wrote:
> David L . Nicol <[EMAIL PROTECTED]> writes:
> >
> >does sysV shm not support the equivalent security as the file system?
> 
> mmap() has the file system.

I wasn't aware that mmap() was part of SysV shared memory.  My
mistake?  It's not on the SysV IPC man pages on my Linux system.  The mmap
manpage doesn't mention SysV IPC either.

-sam

Re: RFC 155 - Remove geometric functions from core

[Dan Sugalski]

At 07:32 PM 8/29/00 +, Nick Ing-Simmons wrote:
>David L . Nicol <[EMAIL PROTECTED]> writes:
> >
> >Did I not just describe how a .so or a DLL works currently?
>
>And behind the scenes that does something akin to:
>
>int fd = open("file_of_posn_indepenant_byte_code",O_RDONLY);
>struct stat st;
>fstat(fd,&st);
>code_t *code = mmap(NULL,st.st_len,PROT_READ,MAP_SHARED,fd,0);
>close(fd);

Don't forget the fixup work that needs to be done afterwards. Loading the 
library into memory's only the first part--after that the loader needs to 
twiddle with transfer vectors and such so the unresolved calls into the 
routines in the newly loaded library get resolved.

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: RFC 155 - Remove geometric functions from core

[Nick Ing-Simmons]

David L . Nicol <[EMAIL PROTECTED]> writes:
>
>does sysV shm not support the equivalent security as the file system?

mmap() has the file system.

>
>Did I not just describe how a .so or a DLL works currently?

And behind the scenes that does something akin to:

int fd = open("file_of_posn_indepenant_byte_code",O_RDONLY);
struct stat st;
fstat(fd,&st);
code_t *code = mmap(NULL,st.st_len,PROT_READ,MAP_SHARED,fd,0);
close(fd);

strace (linux) or truss (solaris) will show you what I mean.

And then trusts to OS to honour MAP_SHARED.  (mmap() is POSIX.)

Win32 has "something similar" but I don't remember the function names off
hand.

Or you can embed your bytecode in 

const char script[] = {...};

and link/dlopen() it and then you have classical shared text.



-- 
Nick Ing-Simmons

Re: RFC 155 - Remove geometric functions from core

[Sam Tregar]

On Tue, 29 Aug 2000, David L. Nicol wrote:

> does sysV shm not support the equivalent security as the file system?

Well, no, I don't think it does.  It supports permissions on individual
segments but it doesn't support anything like directory perimssions.  It
might be enough, and it might not be.  A user can run two programs and not
expect one to have an automatic exploit on the other just because they're
both Perl!  Think "nobody".

Yes, you'd provide a paranoid mode for experts to use to avoid the
problems to which most users would be exposed.  Great.

> Did I not just describe how a .so or a DLL works currently?

Certainly not.  You wrote only a few sentences.  I'm no expert but I don't
think that shared libraries are that simple.  I also don't think they're
implemented using SysV IPC shared memory, but you might know differently.

> In the ever-immenent vaporware implementation, this whole thing may
> be represented as a big file into which we can seek() to locate stuff.

Zuh?  What are you talking about?  Is this some kind of Inline.pm-esque
system?

-sam

Re: RFC 155 - Remove geometric functions from core

[Stephen P. Potter]

Lightning flashed, thunder crashed and Tom Christiansen <[EMAIL PROTECTED]
m> whispered:
| Very well, then: I'll save it for an after-the-fact I-TOLD-YOU-SO,
| which, believe it or not, is truly *not* a pleasant thing to be
| able to say.

Tom, we appreciate your constructive comments and your help in making sure
we've considered all issues before we embark on a particular path.  It is
when you start calling us stupid for even suggesting that we look at the
other paths that problems arise.

There are lots of paths out there.  Some of them may have quicksand in the
middle of them, but can be safely navigated if we see them first.  Some of
them may be rabbit paths instead of super highways.  Some of them may even
lead to the pits of despair.  But, if no one ever says "what about this
path?" we'll never go anywhere except around the block.

-spp

Re: RFC 155 - Remove geometric functions from core

[David L. Nicol]

Sam Tregar wrote:
> 
> On Tue, 29 Aug 2000, David L. Nicol wrote:
> 
> > Well then.  It is impossible to rearchitect it to make it shared
> > text?  Perhaps the first instance of perl sets up some vast shared
> > memory segments and a way for the newcomers to link in to it and look
> > at the modules that have been loaded, somewhere on this system, and use
> > the common copy?
> 
> That approach invites big security problems.  Any system that involves one
> program trusting another program to load executable code into their memory
> space is vulnerable to attack.  This kind of thing works for forking
> daemons running identical code since the forked process trusts the parent
> process.  In the general case of a second perl program starting on a
> machine, why would this second program trust the first program to not
> load a poison module?

does sysV shm not support the equivalent security as the file system?

Did I not just describe how a .so or a DLL works currently?

Yes, the later Perls would have to trust the first one to load the modules
into the shared space correctly, and none of them would be allowed to
barf on the couch.  

A paranoid mode would be required in which
you don't use the shared pre-loaded module pool.

In the ever-immenent vaporware implementation, this whole thing may
be represented as a big file into which we can seek() to locate stuff.
 




-- 
  David Nicol 816.235.1187 [EMAIL PROTECTED]
   Yum, sidewalk eggs!

Re: RFC 155 - Remove geometric functions from core

[Andy Dougherty]

On Tue, 29 Aug 2000, Sam Tregar wrote:

> On Tue, 29 Aug 2000, David L. Nicol wrote:
> 
> > Well then.  It is impossible to rearchitect it to make it shared
> > text? 

> I don't believe you can simply "rearchitect it to make it shared text".

That depends on what the meaning of "it" is. :-)

If "it" is primarily XS code, then I can easily imagine building a new
perl binary with the compiled XS code linked in (kind of like the current
static linking of extensions).  If "it" is primarily perl code (like
CGI.pm) then it's harder.  But if the compiler back end works out well
enough, then CGI.o could be linked into the main executable too.  Would
that solve the shared text problem?

Just because we can leave almost everything out and dynamically link it in
later doesn't mean that the default build should be so extreme.  After
all, it's rather silly to dynamically load something you're going to load
almost every time you run.

-- 
Andy Dougherty  [EMAIL PROTECTED]
Dept. of Physics
Lafayette College, Easton PA 18042

Re: RFC 155 - Remove geometric functions from core

[Dan Sugalski]

At 12:02 PM 8/29/00 -0600, Tom Christiansen wrote:
> >Well then.  It is impossible to rearchitect it to make it shared
> >text?  Perhaps the first instance of perl sets up some vast shared
> >memory segments and a way for the newcomers to link in to it and look
> >at the modules that have been loaded, somewhere on this system, and use
> >the common copy?
>
>I'd be astonished to see a general-purpose, cross-platform, and
>maintainable solution to this problem.  I predict that you'd at the
>very best, only address this a few places.  Feel free to astonish me.

It's possible we'll manage this with mmap()ing predigested bytecode, but 
I'm not entirely sure that's feasable--it means the optree (or whatever) 
that perl runs through would have to have no pointers at all in it, and 
that might be a rather big speed hit. (Presumably branches would need to do 
relative lookups and such)

On the other hand, if we go the TIL route (which has its own cross-platform 
headaches) we could precompile segments of code into something that would 
be shareable as it'd be real executable code.

> >This handwringing naysaying is depressing.
>
>Very well, then: I'll save it for an after-the-fact I-TOLD-YOU-SO,
>which, believe it or not, is truly *not* a pleasant thing to be
>able to say.

Personally I'd rather have someone throwing wet blankets now, rather than 
later. If the ideas have merit enough to be worth doing then they'll 
survive the uncomfortable scrutiny. If they don't, better to get the ego 
bruising out of the way now, rather than after spending a month or more of 
wasted time.

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: RFC 155 - Remove geometric functions from core

[Tom Christiansen]

>Well then.  It is impossible to rearchitect it to make it shared
>text?  Perhaps the first instance of perl sets up some vast shared
>memory segments and a way for the newcomers to link in to it and look
>at the modules that have been loaded, somewhere on this system, and use
>the common copy?

I'd be astonished to see a general-purpose, cross-platform, and
maintainable solution to this problem.  I predict that you'd at the
very best, only address this a few places.  Feel free to astonish me.

>This handwringing naysaying is depressing.

Very well, then: I'll save it for an after-the-fact I-TOLD-YOU-SO,
which, believe it or not, is truly *not* a pleasant thing to be
able to say.

--tom

Re: RFC 155 - Remove geometric functions from core

[Sam Tregar]

On Tue, 29 Aug 2000, David L. Nicol wrote:

> Well then.  It is impossible to rearchitect it to make it shared
> text?  Perhaps the first instance of perl sets up some vast shared
> memory segments and a way for the newcomers to link in to it and look
> at the modules that have been loaded, somewhere on this system, and use
> the common copy?

That approach invites big security problems.  Any system that involves one
program trusting another program to load executable code into their memory
space is vulnerable to attack.  This kind of thing works for forking
daemons running identical code since the forked process trusts the parent
process.  In the general case of a second perl program starting on a
machine, why would this second program trust the first program to not
load a poison module?

I don't believe you can simply "rearchitect it to make it shared text".

> This sounds like a problem to be fixed.  Relax, Tom, we'll take it from
> here.

Are you so sure?  From where I'm sitting he's got some pretty tough points
there.  If you've got a solution then I'm quite suprised, which would be
great.  If not then I suggest you avoid writing the proverbial bad check.

-sam

Re: RFC 155 - Remove geometric functions from core

[Nathan Torkington]

David L. Nicol writes:
> This handwringing naysaying is depressing.

Yes, it's depressing to find out there are problems in one's grand
plans.  However, I'm very glad that people (including Tom) are
pointing out problems *before* we commit to a course of action.

Nat

Re: RFC 155 - Remove geometric functions from core

[David L. Nicol]

Well then.  It is impossible to rearchitect it to make it shared
text?  Perhaps the first instance of perl sets up some vast shared
memory segments and a way for the newcomers to link in to it and look
at the modules that have been loaded, somewhere on this system, and use
the common copy?




This handwringing naysaying is depressing.




Tom Christiansen wrote:

> Disastrously, you will then also lose the shared text component,
> which is what makes all this cheap when Perl loads.  Since the
> modules will have to be pasted in the data segment of each process
> that wants them, they aren't going to be in a shared region, except
> perhaps for some of the non-perl parts of them on certain architectures.
> But certainly the Perl parts are *NEVER* shared.

This sounds like a problem to be fixed.  Relax, Tom, we'll take it from
here.


> That's why the
> whole CGI.pm or IO::whatever.pm stuff hurts so badly: you run with
> 10 copies of Perl on your system (as many people do, if not much
> more than that), then you have to load them, from disk, into each
> process that wants them, and eth result of what you've loaded cannot
> be shared, since you loaded and compiled source code into non-shared
> parse trees.  This is completely abysmal.  Loading bytecode is no win:
> it's not shared text.
> 
> --tom

-- 
  David Nicol 816.235.1187 [EMAIL PROTECTED]
Ask me about sidewalk eggs

Re: RFC 155 - Remove geometric functions from core

[Bennett Todd]

Tom Christiansen <[EMAIL PROTECTED]> writes:
> Keywords that *cannot* be overridden are chop, defined, delete, do,
> dump, each , else, elsif, eval, exists, for, foreach, format, glob,
> goto, grep, if, keys, last, local, m, map, my, next, no, package,
> pop, pos, print, printf, prototype, push, q, qq, qw, qx, redo,
> return, s, scalar, shift, sort, splice, split, study, sub, tie,
> tied, tr, undef, unless, unshift, untie, until, use, while, and y.

Thanks! That's a really helpful list!

2000-08-25-05:12:58 Johan Vromans:
> Hmm. Quite a few of these should no longer be special:
> 
>  chop, defined, delete, dump, each, exists, glob, grep, keys, map,
>  pop, pos, print, printf, prototype, push, scalar, shift, sort,
>  splice, split, study, tie, tied, undef?, unshift, untie.

If that were to be done, it'd be a nice clean superset of solving
RFC 70: if (as best I can tell by eyeballing that list) glob, print,
and printf were overridable, then Fatal.pm could be completed
usefully for those of us who would rather have to explicitly catch
errors to prevent them from exiting with an error message, rather
than having to explicitly check for errors to prevent them from
being ignored.

-Bennett

 PGP signature

Re: RFC 155 - Remove geometric functions from core

[Johan Vromans]

Nick Ing-Simmons <[EMAIL PROTECTED]> writes:

> But if perl6 bytecode does not need to be modified to be used 

I'd assume that.

-- Johan

Re: RFC 155 - Remove geometric functions from core

[Stephen P. Potter]

Lightning flashed, thunder crashed and Steve Fink <[EMAIL PROTECTED]>
 whispered:
| Depends on your definition of "module". Many people seem to be assuming
| "module" eq "shared library".

Yes, exactly.  I use module as a generic term for something other than the
main perl binary itself, a black box if you will.  The more modular we can
make perl, I think the easier it is for it to be ported or embedded or
whatever.  It is very easy to simply not install a "module" or write stub
functions, or even to add a new function.  It is not so easy to remove
things from "the core".  Note some of the deprecated features we've lived
with for years (such as EQ and friends).

-spp

Re: RFC 155 - Remove geometric functions from core

[Jarkko Hietaniemi]

On Sun, Aug 27, 2000 at 08:37:38PM +, Nick Ing-Simmons wrote:
> Jarkko Hietaniemi <[EMAIL PROTECTED]> writes:
> > bytes
> >
> >microperl, which has almost nothing os dependent (*) in it   1212416
> >shared libperl 1277952 bytes + perl 32768 bytes  1310720
> >dynamically linked perl  1376256
> >statically linked perl with all the core extensions  2129920
> >
> >  (*) I haven't tried building it in non-UNIX boxes, so I can't be certain
> >  of how fastidiously features have been disabled.
> 
> "bytes" of what? - size of executable, size of .text, ???

Sizes of the executables/libs, code fully optimized -- and symbols stripped.

> If we are taling executable with -g size then  a lot of that is symbol-table
> and is tedious repetition of "sv.h" & co. re-itteerated in each .o file.
> 
> But the basic point is that these things are small.

-- 
$jhi++; # http://www.iki.fi/jhi/
# There is this special biologist word we use for 'stable'.
# It is 'dead'. -- Jack Cohen

Re: RFC 155 - Remove geometric functions from core

[Nick Ing-Simmons]

Jarkko Hietaniemi <[EMAIL PROTECTED]> writes:
>   bytes
>
>microperl, which has almost nothing os dependent (*) in it 1212416
>shared libperl 1277952 bytes + perl 32768 bytes1310720
>dynamically linked perl1376256
>statically linked perl with all the core extensions2129920
>
>  (*) I haven't tried building it in non-UNIX boxes, so I can't be certain
>  of how fastidiously features have been disabled.

"bytes" of what? - size of executable, size of .text, ???
If we are taling executable with -g size then  a lot of that is symbol-table
and is tedious repetition of "sv.h" & co. re-itteerated in each .o file.

But the basic point is that these things are small.

>
>So ripping all this 'cruft' would save us about 100-160 kB, still
>leaving us with well over a 1MB-plus executable.  It's Perl itself
>that's big, not the thin glue to the system functions.

My support for the idea is not to reduce the size of perl in the UNIX
case, but to allow replacement. I would also like to have the mechanism
worked out and "proven" on something that we know gets used so 
that we can have good solid testing of the mechanism. Then something 
less obvious (say Damian's any/all operators) which might be major
extra size and not of universal appeal can use a well tried mechanism,
and we can flip default to re-link sockets or sin/cos/tan into the core.

-- 
Nick Ing-Simmons

Re: RFC 155 - Remove geometric functions from core

[Nick Ing-Simmons]

Tom Christiansen <[EMAIL PROTECTED]> writes:
>
>Disastrously, you will then also lose the shared text component,
>which is what makes all this cheap when Perl loads.  

But we can on modern OSes have shared data too.

>Since the
>modules will have to be pasted in the data segment of each process
>that wants them, they aren't going to be in a shared region, except
>perhaps for some of the non-perl parts of them on certain architectures.
>But certainly the Perl parts are *NEVER* shared.  That's why the
>whole CGI.pm or IO::whatever.pm stuff hurts so badly: you run with
>10 copies of Perl on your system (as many people do, if not much
>more than that), then you have to load them, from disk, into each
>process that wants them, and eth result of what you've loaded cannot
>be shared, since you loaded and compiled source code into non-shared
>parse trees.  This is completely abysmal.  Loading bytecode is no win:
>it's not shared text.

Loading perl5 bytecode is a non-win I agree 110%.

But if perl6 bytecode does not need to be modified to be used
it can be mmap()'ed shared, read-only and hence page-cached and reused.

-- 
Nick Ing-Simmons

Re: RFC 155 - Remove geometric functions from core

[Steve Fink]

Tom Christiansen wrote:
> 
> >So ripping all this 'cruft' would save us about 100-160 kB, still
> >leaving us with well over a 1MB-plus executable.  It's Perl itself
> >that's big, not the thin glue to the system functions.
> 
> Moreover, if you rip out that 100-160 kb, allocating it to modules,
> then I can guarantee you that it will cost significantly more memory
> as a module to pull in that if it here already there.  There's always
> some overhead involved in these things.  Notice how the Byteloader
> produces much bigger executables than the originals.  Some work has
> been done to fix that, but as a minimum, you've got the Byteloader
> module itself.  And it gets worse...

Depends on your definition of "module". Many people seem to be assuming
"module" eq "shared library". I submit that a better definition would be
anything that uses the Perl6 Core API.

The vision I have is that all existing builtins _could_ be linked with
the "perl core". The new builtins (a small list) always will be -- they
require inside knowledge of data structures that should specifically not
be exposed to extensions. (With a "please", not a shotgun.) Any or all
of the "core modules" could be placed into one or separate shared
libraries. They would have no more access to the internals than any 3rd
party module. (That's my real definition of a module -- it's a property
of the architecture, not of the linkage, preprocessing, author, or
whatever else.)

What this means is that we have to define a recommended API for modules
to use (and warn that compatibility is not assured if they dig deeper
than this), and to make sure that sin(), socket() etc. strictly obey
this discipline. If this API is too raw for the convenience of many
extensions, then we can always layer it with a kindler gentler API, but
the bottom-most API should be the important one. At compile time, you
can choose which things you want linked into the perl core,  which ones
to stick in a shared library, and which ones to leave out entirely, with
a warning "what you are about to compile is not Perl" if they choose to
leave out any "core module" entirely. In fact, the binary should then be
called "pht" rather than "perl".

Whether this is implemented via the preprocessor, autogenerated code, or
? is a separate issue. It seems easily possible to define an API that
allows either static or dynamic linking with no performance loss.

Maybe this is what some people are already assuming. I just thought I
should spell it out. It's probably implied by the RFC that says user
subs should be indistinguishable from opcodes (doesn't one that sound
something like that exist? I'm too lazy to go look.)

Re: RFC 155 - Remove geometric functions from core

[Nathan Torkington]

Tom Christiansen writes:
> further abuse as the next bandit decides to chew on you.  As nobody
> else said mum about that scat, I took care of it myself.

90 minutes passed on a Friday night.  That doesn't mean it wasn't
going to be dealt with.

> >(hint: grown-ups would apologise at this point)
> I'm sorry he said that.

Cute, but flippancy doesn't convince me you're capable of maturity.
If you can't behave, I'll treat you like the little children you are
and give you a time out.  If that fails to moderate your behaviour, I
will take the ability to make trouble out of your grasp by preventing
the mailing list software from accepting messages from you.

This goes for everyone, Stephen as well as Tom.  Name-calling and
petulant messages are the kind of things we're trying to leave behind.

This is supposed to be about perl6, not personalities.  It makes me
grumpy that this is distracting from the work we're supposed to be
doing.  Please don't waste any more of our time with this idiocy.

Thank you,

Nat

Re: RFC 155 - Remove geometric functions from core

[Larry Wall]

Tom Christiansen writes:
: >Please act like a grown-up.  Stephen cast the
: >first stone, but that's no excuse for you to reply with a boulder.
: 
: Sure it is: when a hoodlum jumps you with a knife, there's no reason
: to roll over and quietly submit to the death of a thousand cuts.
: No, you pull an Indy by responding with overwhelming firepower to
: dispatch the cretin forthwith before he gets cocky.  Otherwise
: you're a willing victim with an FMH sign on his butt waiting for
: further abuse as the next bandit decides to chew on you.  As nobody
: else said mum about that scat, I took care of it myself.

I despise escalation.  You seem to enjoy it.

Larry

Re: RFC 155 - Remove geometric functions from core

[Tom Christiansen]

>And that's offensive.  

What's offensive to one person is amusing to the next.  

>Please act like a grown-up.  Stephen cast the
>first stone, but that's no excuse for you to reply with a boulder.

Sure it is: when a hoodlum jumps you with a knife, there's no reason
to roll over and quietly submit to the death of a thousand cuts.
No, you pull an Indy by responding with overwhelming firepower to
dispatch the cretin forthwith before he gets cocky.  Otherwise
you're a willing victim with an FMH sign on his butt waiting for
further abuse as the next bandit decides to chew on you.  As nobody
else said mum about that scat, I took care of it myself.

>(hint: grown-ups would apologise at this point)

I'm sorry he said that.

--tom

Re: RFC 155 - Remove geometric functions from core

[Nathan Torkington]

Tom Christiansen writes:
> Your coquettish plot to reveal the desperate yearning of your
> nethermost alimentary canal for multiply redundant new egresses is
> neither charming nor

And that's offensive.  Please act like a grown-up.  Stephen cast the
first stone, but that's no excuse for you to reply with a boulder.

Nat
(hint: grown-ups would apologise at this point)

Re: RFC 155 - Remove geometric functions from core

[Nathan Torkington]

Stephen P. Potter writes:
> =head1 TITLE
> 
> Perl is Tom's private domain.

That's unproductive.

Nat

Re: RFC 155 - Remove geometric functions from core

[Tom Christiansen]

Your coquettish plot to reveal the desperate yearning of your nethermost
alimentary canal for multiply redundant new egresses is neither charming nor
subtle--nor even innovative.  Although it would be no great trouble for me
to don the vestments and sharpen the cutlery to serve as haruspex for so
joyous an event as you have requested, I must confess doubt that these
divinations might augur anything that would prove to our current forum useful.
For the sake of general decorum, I must therefore deny your ardent request
of public evisceration, and instead quietly counsel you to maintain discreetly
hidden from public display the steaming faeculae that your mouth has just now
wetly ejected like some hydrophobic and coprophagous cur subjected to explosive
decompression.

--tom

Re: RFC 155 - Remove geometric functions from core

[Tom Christiansen]

>So ripping all this 'cruft' would save us about 100-160 kB, still
>leaving us with well over a 1MB-plus executable.  It's Perl itself
>that's big, not the thin glue to the system functions.

Moreover, if you rip out that 100-160 kb, allocating it to modules,
then I can guarantee you that it will cost significantly more memory
as a module to pull in that if it here already there.  There's always
some overhead involved in these things.  Notice how the Byteloader
produces much bigger executables than the originals.  Some work has
been done to fix that, but as a minimum, you've got the Byteloader 
module itself.  And it gets worse... 

Disastrously, you will then also lose the shared text component,
which is what makes all this cheap when Perl loads.  Since the
modules will have to be pasted in the data segment of each process
that wants them, they aren't going to be in a shared region, except
perhaps for some of the non-perl parts of them on certain architectures.
But certainly the Perl parts are *NEVER* shared.  That's why the
whole CGI.pm or IO::whatever.pm stuff hurts so badly: you run with
10 copies of Perl on your system (as many people do, if not much
more than that), then you have to load them, from disk, into each
process that wants them, and eth result of what you've loaded cannot
be shared, since you loaded and compiled source code into non-shared
parse trees.  This is completely abysmal.  Loading bytecode is no win:
it's not shared text.

--tom

Re: RFC 155 - Remove geometric functions from core

[Stephen P. Potter]

Lightning flashed, thunder crashed and Tom Christiansen <[EMAIL PROTECTED]
m> whispered:
| More of this nonsense, eh?  I just fail to understand the urge
| to eviscerate.  Why don't we just say that Perl isn't for 
| systems work anymore, and remove everything that diddles $!, 
| or $?, or anything that might call anything from the C library.

Ok, here's my new RFC.  This should handle all of Tom's objections:

=head1 TITLE

Perl is Tom's private domain.

=head1 VERSION

  Maintainer: Stephen P. Potter <[EMAIL PROTECTED]>
  Date: 8/25/2000
  Version: 1
  Mailing List: [EMAIL PROTECTED]

=head1 ABSTRACT

(Dredging up really old jokes) Tom is a highly developed artificial
intelligence, not a person like everyone thinks.  Tom is also completely
perfect.  Therefore, we should take everything that Tom says as gospel.

=head1 DESCRIPTION

It is readily apparent that no one, except Tom, has any good ideas on what
should and should not happen to perl.  Any change that anyone else suggests
must be because either they are too stupid to understand what Perl (or
perl) is, because they want a different language, or is designed strictly
to make Tom's life miserable.  As such, we should all just abandon the
entire idea of perl6, unless Tom suggests a change.  Every RFC which has
been submitted so far (unless Tom submitted it) should be dropped.

=head1 IMPLEMENTATION

There is no implementation.  Tom hasn't said we are allowed to change
anything yet.  Everything is perfect.

=head1 REFERENCES

See any posting of Tom's, especially if it is an answer to someone else's
post.

Re: RFC 155 - Remove geometric functions from core

[Jarkko Hietaniemi]

On Fri, Aug 25, 2000 at 09:20:53AM -0400, Dan Sugalski wrote:
> At 09:12 AM 8/25/00 -0400, Stephen P. Potter wrote:
> >  As you say, 200 lines isn't much.  But combine that with the IPC, the
> >environment, the system, etc it all adds up.
> 
> Not to much, though. We've been down this road for perl 5. You'd be 
> surprised at how little code gets removed if you yank most of the functions 
> under discussion. (They're generally trivial wrappers around library calls, 
> with very little code involved)

Here are some numbers for people who have forgotten the above.
Using the latest perl development sources, using an unnamed UNIX:

bytes

microperl, which has almost nothing os dependent (*) in it  1212416
shared libperl 1277952 bytes + perl 32768 bytes 1310720
dynamically linked perl 1376256
statically linked perl with all the core extensions 2129920

  (*) I haven't tried building it in non-UNIX boxes, so I can't be certain
  of how fastidiously features have been disabled.

So ripping all this 'cruft' would save us about 100-160 kB, still
leaving us with well over a 1MB-plus executable.  It's Perl itself
that's big, not the thin glue to the system functions.

-- 
$jhi++; # http://www.iki.fi/jhi/
# There is this special biologist word we use for 'stable'.
# It is 'dead'. -- Jack Cohen

Re: RFC 155 - Remove geometric functions from core

[Michael G Schwern]

On Fri, Aug 25, 2000 at 09:12:32AM -0400, Stephen P. Potter wrote:
> Lightning flashed, thunder crashed and Michael G Schwern <[EMAIL PROTECTED]> wh
> ispered:
> | PS  The idea of adding acos, asin and tan is good.
> 
> You just answered your own question.  It is very difficult to add new
> functions to the core.  It is very easy to write new modules.  Doesn't it
> make sense that if you have to use Math::Trig to get to acos and friends,
> you might as well make the language definition clean and say all of acos
> friends should be in that module, not some in the core?

Actually I was suggesting that acos, asin and tan be added to the core.

Most likely, splitting this out into a module wouldn't make anything
much simpler.  To get anything like the performance math functions
need, Math::Trig could not be written in plain Perl.  It would have to
be written in C or XS (or whatever perl6 winds up being written in)
probably calling the system's math libraries and still dragging in all
the basic problems of a core patch and configuration.


> That's the basic goal behind my RFCs for moving things to modules.  In
> general, I hope to make the language cleaner, easier to learn and use, and
> easier to extend.  If at the same time the language became better
> performing because of a removal of some of the core, all the better.  As
> you say, 200 lines isn't much.  But combine that with the IPC, the
> environment, the system, etc it all adds up.

I think I basically agree with tchrist here.  This is nickel and dime
stuff.  Cutting out the math functions isn't going to do squat.  IPC
and the system... that might do something, but Dan pointed out that it
doesn't amount to much either.

I think we should back up and reconsider what the intent of this RFC
is.  

If the intent is to make perl more maintainable and easier to patch,
I've already commented on that.  You're just shuffling code around.
And only a small fraction at that.

If you wish to make perl smaller and faster, just tearing things out
isn't going to help.  Its hit-or-miss optimizing.  You could remove
half the core functions and find out you only gained 5%.

Like all other optimizing attempts, the first step is analysis.
People have to sit down and systematically go through and find out
what parts of perl (and Perl) are eating up space and speed.  The
results will be very surprising, I'm sure, but it will give us a
concrete idea of what we can do to really help out perl's performance.

There should probably be an RFC to this effect, and I'm just visiting
here in perl6-language so I dump it on somebody else.

exit('stage' << 1);


-- 

Michael G Schwern  http://www.pobox.com/~schwern/  [EMAIL PROTECTED]
Just Another Stupid Consultant  Perl6 Kwalitee Ashuranse
When faced with desperate circumstances, we must adapt.
- Seven of Nine

Re: RFC 155 - Remove geometric functions from core

[Nathan Torkington]

Larry Wall writes:
> Tom Christiansen writes:
> : More of this nonsense, eh?
> 
> Please don't use fighting words in here.

On the subject of fighting words, I owe everyone an apology for my
language on the subject of Perl being the only thing that can parse
Perl.  I've been banging my head against program filters lately, and
I'm frustrated that it's so hard.  That's no excuse for exposing you
all to my frustration.  Sorry.

So please take the suggestion (that Perl be more easily toked) without
the swear words.  Thanks.

Nat

Re: RFC 155 - Remove geometric functions from core

[Dan Sugalski]

At 11:36 AM 8/25/00 -0600, Tom Christiansen wrote:
> >If the behind-the-scenes shenanigans are invisible and performance is at
> >worst acceptable, does it really matter how things are implemented?
>
>So long as you can still say things like
>
> perl -le 'print scalar getpwuid(1)'
>
>and that script still continue to work unaltered, then I suppose
>not--assuming no seriously grave performance issues.

That's one of the working goals...

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: RFC 155 - Remove geometric functions from core

[Larry Wall]

Tom Christiansen writes:
: More of this nonsense, eh?

Please don't use fighting words in here.

: I just fail to understand the urge to eviscerate.  Why don't we just
: say that Perl isn't for systems work anymore, and remove everything
: that diddles $!, : or $?, or anything that might call anything from
: the C library.

This is confusing Perl and perl again.  This is not terribly useful,
especially in -internals.  The degree to which Perl 6 (the language)
will be useful for systems work has almost nothing to do with the method
by which perl6 (the program) implements the given functionality.  In fact,
we're trying to *decouple* those ideas further so that we can more easily
add such functionality to Perl without forcing people to read miles of
camel entrails before they begin.  If we can make it run faster in
the process, that'll be even better.

We're redesigning everything.  And we'll do the best job we can with
the brains we've got.  We will bear people's fears in mind, but we will
not be controlled by such fears.  Rather, we will try to give fear its
proper weight--which is neither too much, nor too little.

Fasten your seat belts, folks.

Larry

Re: RFC 155 - Remove geometric functions from core

[Hildo Biersma]

Tom Christiansen wrote:
> 
> >If the behind-the-scenes shenanigans are invisible and performance is at
> >worst acceptable, does it really matter how things are implemented?
> 
> So long as you can still say things like
> 
> perl -le 'print scalar getpwuid(1)'
> 
> and that script still continue to work unaltered, then I suppose
> not--assuming no seriously grave performance issues.

One thing to consider for many of these functions - does the performance
matter?  Yes, sin() and cos() need to be fast.  getpid(), getpwuid() etc
need not - a program that uses them will call them once, or twice, and
not in an inner loop.

Hildo

Re: RFC 155 - Remove geometric functions from core

[Tom Christiansen]

>If the behind-the-scenes shenanigans are invisible and performance is at 
>worst acceptable, does it really matter how things are implemented?

So long as you can still say things like

perl -le 'print scalar getpwuid(1)'

and that script still continue to work unaltered, then I suppose
not--assuming no seriously grave performance issues.

--tom

Re: RFC 155 - Remove geometric functions from core

[Dan Sugalski]

At 09:39 AM 8/25/00 -0700, Larry Wall wrote:
>Tom Christiansen writes:
>: >Or, more succinctly, we're not going to screw with perl without a *darned*
>: >good reason.
>:
>: This is the most beautiful thing I've read in days.
>
>Bear in mind there are lots of darned good reasons.  :-)

You'll notice I didn't say otherwise... :-P


Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: RFC 155 - Remove geometric functions from core

[Dan Sugalski]

At 09:38 AM 8/25/00 -0700, Larry Wall wrote:
>Dan Sugalski writes:
>: Or, more succinctly, we're not going to screw with perl without a *darned*
>: good reason.
>
>Er, perl is already screwed--it's Perl we're trying to preserve and grow.

Hey, look! A good reason! :)


Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: RFC 155 - Remove geometric functions from core

[Dan Sugalski]

At 10:52 AM 8/25/00 -0600, Tom Christiansen wrote:
> >But to apply this metric to every fine-grained aspect of perl5 is silly.
> >Perl6 should be a well-designed, consistent, integrated opus.
> >If it goes against the grain of perl6's design to have system-specific
> >system calls built in, then they shouldn't be built in, regardless of
> >how it was in perl5.
>
>That's silly.  *ALL* syscalls are system specific.  This is starting
>to sound like Java, which has its own brand of revisionism.

If the behind-the-scenes shenanigans are invisible and performance is at 
worst acceptable, does it really matter how things are implemented?

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: RFC 155 - Remove geometric functions from core

[John Porter]

Tom Christiansen wrote:
> 
> Why don't we just say that Perl isn't for 
> systems work anymore, and remove everything that diddles $!, 
> or $?, or anything that might call anything from the C library.

As in "remove mountains", yes.

-- 
John Porter

We're building the house of the future together.

Re: RFC 155 - Remove geometric functions from core

[Tom Christiansen]

You don't beneficially make Perl any more anything by ghettoizing
its useful systems functionality.  You certainly don't make it 
more "portable".   I still can't use any of my myriad useful systems
hackery script if I try to run them on CP/M, and you aren't going to
change that one whit.  So what if it's in a ghetto hole?  It still
doesn't work.

--tom

Re: RFC 155 - Remove geometric functions from core

[Tom Christiansen]

>Well, duh.  So what?  Did you really miss my point by so many furlongs?

Whatever.

>> You'll take my fork when you pry it out of my cold, dead hands.

>Perl will ship with fork.  But maybe in a loadable library, eh?

More of this nonsense, eh?  I just fail to understand the urge
to eviscerate.  Why don't we just say that Perl isn't for 
systems work anymore, and remove everything that diddles $!, 
or $?, or anything that might call anything from the C library.

Useless.

--tom

Re: RFC 155 - Remove geometric functions from core

[John Porter]

Tom Christiansen wrote:
> >But to apply this metric to every fine-grained aspect of perl5 is silly.
> >Perl6 should be a well-designed, consistent, integrated opus. 
> >If it goes against the grain of perl6's design to have system-specific
> >system calls built in, then they shouldn't be built in, regardless of
> >how it was in perl5.
> 
> That's silly.  *ALL* syscalls are system specific.  

Well, duh.  So what?  Did you really miss my point by so many furlongs?


> You'll take my fork when you pry it out of my cold, dead hands.

Perl will ship with fork.  But maybe in a loadable library, eh?

-- 
John Porter

We're building the house of the future together.

Re: RFC 155 - Remove geometric functions from core

[Tom Christiansen]

>But to apply this metric to every fine-grained aspect of perl5 is silly.
>Perl6 should be a well-designed, consistent, integrated opus. 
>If it goes against the grain of perl6's design to have system-specific
>system calls built in, then they shouldn't be built in, regardless of
>how it was in perl5.

That's silly.  *ALL* syscalls are system specific.  This is starting
to sound like Java, which has its own brand of revisionism.

You'll take my fork when you pry it out of my cold, dead hands.

--tom

Re: RFC 155 - Remove geometric functions from core

[John Porter]

Larry Wall wrote:
> Dan Sugalski writes:
> : Or, more succinctly, we're not going to screw with perl without a *darned* 
> : good reason.
> 
> Er, perl is already screwed--it's Perl we're trying to preserve and grow.

Man; leave it to Larry to say the right thing, in the best way.  :-)

-- 
John Porter

Re: RFC 155 - Remove geometric functions from core

[Larry Wall]

Tom Christiansen writes:
: >Or, more succinctly, we're not going to screw with perl without a *darned* 
: >good reason.
: 
: This is the most beautiful thing I've read in days.

Bear in mind there are lots of darned good reasons.  :-)

Larry

Re: RFC 155 - Remove geometric functions from core

[John Porter]

Dan Sugalski wrote:
> 
> Or, more succinctly, we're not going to screw with perl without a *darned* 
> good reason.

Well, in the big picture, we *are* screwing with perl, and we know
that we have darned good reason to.

But to apply this metric to every fine-grained aspect of perl5 is silly.
Perl6 should be a well-designed, consistent, integrated opus. 
If it goes against the grain of perl6's design to have system-specific
system calls built in, then they shouldn't be built in, regardless of
how it was in perl5.

-- 
John Porter

We're building the house of the future together.

Re: RFC 155 - Remove geometric functions from core

[Larry Wall]

Dan Sugalski writes:
: Or, more succinctly, we're not going to screw with perl without a *darned* 
: good reason.

Er, perl is already screwed--it's Perl we're trying to preserve and grow.

Larry

Re: RFC 155 - Remove geometric functions from core

[Tom Christiansen]

>Or, more succinctly, we're not going to screw with perl without a *darned* 
>good reason.

This is the most beautiful thing I've read in days.

--tom

Re: RFC 155 - Remove geometric functions from core

[Dan Sugalski]

At 10:53 AM 8/25/00 -0400, [EMAIL PROTECTED] wrote:
>Dan wrote:
> >At 09:12 AM 8/25/00 -0400, Stephen P. Potter wrote:
> >>  As you say, 200 lines isn't much.  But combine that with the IPC, the
> >>environment, the system, etc it all adds up.
> >
> >Not to much, though. We've been down this road for perl 5. You'd be
> >surprised at how little code gets removed if you yank most of the
>functions
> >under discussion. (They're generally trivial wrappers around library
>calls,
> >with very little code involved)
>
>so is all this concern about the size of perl due to the hope
>of making perl "embeddable"? with 128 meg of ram and
>a 40 gig drive, I'll assume its not due to system limitations.

Embeddability is one worry, though not necessarily a primary one. Behind 
the scenes modularity makes things easier to deal with 
conceptually--there's a limit to what'll fit in my brain. It can aid in 
portability and make code cleaner in some ways, since platform-specific 
code can be isolated more easily. (And yes, I know, you can do it now)

System limits are still an issue--loading 2M worth of interpreter still 
takes twice as long, more or less, as loading in 1M of interpreter code, 
and the fewer external symbol fixups you need to do in the general case the 
faster the load time.

Having said that...

Unless Larry decrees otherwise, nothing that's built-in and guaranteed now 
will be missing or need a "use thingie;" in perl 6. Moreover, if things are 
stripped out into separate libraries and automagically loaded in, it won't 
be done unless there's a compelling reason, and "conceptual coolness" just 
doesn't cut it as a reason.

There are good reasons. It may make things easier to upgrade. It may make 
perl faster, either to run or to start. It may make perl smaller, either in 
on-disk or in-memory footprint. It may make the optimizer better (fewer 
things to think about means fewer cases to think about) and thus ultimately 
speed up a program. It may make sense from a modularity standpoint when 
targeting perl at special-purpose gadgets.

Or, more succinctly, we're not going to screw with perl without a *darned* 
good reason.

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: RFC 155 - Remove geometric functions from core

[GregLondon]

Dan wrote:
>At 09:12 AM 8/25/00 -0400, Stephen P. Potter wrote:
>>  As you say, 200 lines isn't much.  But combine that with the IPC, the
>>environment, the system, etc it all adds up.
>
>Not to much, though. We've been down this road for perl 5. You'd be
>surprised at how little code gets removed if you yank most of the
functions
>under discussion. (They're generally trivial wrappers around library
calls,
>with very little code involved)

so is all this concern about the size of perl due to the hope
of making perl "embeddable"? with 128 meg of ram and
a 40 gig drive, I'll assume its not due to system limitations.

Greg London

Re: RFC 155 - Remove geometric functions from core

[Tom Christiansen]

>Not to much, though. We've been down this road for perl 5. You'd be 
>surprised at how little code gets removed if you yank most of the functions 
>under discussion. (They're generally trivial wrappers around library calls, 
>with very little code involved)

Thaniks -- I wish people wouldn't forget this.

And it seems crazy to worry about sacrificing convenient functionality
by saving a few bytes when memory is as incredibly cheap as it is.

--tom

Re: RFC 155 - Remove geometric functions from core

[Tom Christiansen]

>That's the basic goal behind my RFCs for moving things to modules.  In
>general, I hope to make the language cleaner, easier to learn and use, and
>easier to extend.  

"Clean"?  What is "clean"?  Huh?  And since when has Perl ever been 
supposed to be "clean"?  I've got plenty of quotage from Larry to the
contrary.

You must now define objective and quantifiable criteria for what
does and thus what does not constitute:

* Making a language cleaner
* Making a language easier to learn
* Making a language easier to use
* Making a language easier to extend

Provide working frameworks for those in concrete terms that can be
applied to your proposals in a non-feel-good fashion, one that is
discretely measurable by any party, and only then we can see whether
they actually make any sense.  Right now, they seem like random
fantasies without any basis in reality.

Note that it is more important that a language be easier to use
than anything else you've listed.

>If at the same time the language became better
>performing because of a removal of some of the core, all the better.  As
>you say, 200 lines isn't much.  But combine that with the IPC, the
>environment, the system, etc it all adds up.

What the bloody blazes are you going to do with a language that can't
do systems work?  Number crunching, I suppose.  Oh wait--you already
stripped that out, too.  :-(

Remember: big languages make small programs, and small language make
big programs.   Larry in observing this has clearly weighed in on the
side of small programs.  Micro-language people always have forth.

--tom

Re: RFC 155 - Remove geometric functions from core

[Dan Sugalski]

At 09:12 AM 8/25/00 -0400, Stephen P. Potter wrote:
>  As you say, 200 lines isn't much.  But combine that with the IPC, the
>environment, the system, etc it all adds up.

Not to much, though. We've been down this road for perl 5. You'd be 
surprised at how little code gets removed if you yank most of the functions 
under discussion. (They're generally trivial wrappers around library calls, 
with very little code involved)

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: RFC 155 - Remove geometric functions from core

[Stephen P. Potter]

Lightning flashed, thunder crashed and Michael G Schwern <[EMAIL PROTECTED]> wh
ispered:
| However, since those funtions take up about 200 lines in the core, are
| very stable and relatively easy to document, what do we win by
| removing them?
| 
| PS  The idea of adding acos, asin and tan is good.

You just answered your own question.  It is very difficult to add new
functions to the core.  It is very easy to write new modules.  Doesn't it
make sense that if you have to use Math::Trig to get to acos and friends,
you might as well make the language definition clean and say all of acos
friends should be in that module, not some in the core?

That's the basic goal behind my RFCs for moving things to modules.  In
general, I hope to make the language cleaner, easier to learn and use, and
easier to extend.  If at the same time the language became better
performing because of a removal of some of the core, all the better.  As
you say, 200 lines isn't much.  But combine that with the IPC, the
environment, the system, etc it all adds up.

-spp

Re: RFC 155 - Remove geometric functions from core

[Johan Vromans]

Tom Christiansen <[EMAIL PROTECTED]> writes:

> Keywords that *cannot* be overridden are chop, defined, delete, do,
> dump, each , else, elsif, eval, exists, for, foreach, format, glob,
> goto, grep, if, keys, last, local, m, map, my, next, no, package,
> pop, pos, print, printf, prototype, push, q, qq, qw, qx, redo,
> return, s, scalar, shift, sort, splice, split, study, sub, tie,
> tied, tr, undef, unless, unshift, untie, until, use, while, and y.

Hmm. Quite a few of these should no longer be special:

 chop, defined, delete, dump, each, exists, glob, grep, keys, map,
 pop, pos, print, printf, prototype, push, scalar, shift, sort,
 splice, split, study, tie, tied, undef?, unshift, untie.

-- Johan

Re: RFC 155 - Remove geometric functions from core

[Tom Christiansen]

>A friend pointed out, technically most are trigonometric functions,
>not geometric.  atan2, cos, sin, acos, asin and tan are all trig.
>exp, log, sqrt are... just math I guess.

>So I suppose the proposed module would be Math::Trig or some such.  Or
>maybe, as the source code suggests, Math::High:Falutin.

>However, since those funtions take up about 200 lines in the core, are
>very stable and relatively easy to document, what do we win by
>removing them?

>PS  The idea of adding acos, asin and tan is good.

It's sounding like folks want to delete this, delete that.  Let's
just cut to the chase and propose that anything that can be redefined
be deleted.  After all, if it's so mutable and boring that people
can diddle its definition, then who cares?  Yes, that means that
anything that's not a reserved word should be banished from Perl,
relegated to some third-class back-of-the-bus module.

Which are those?

Those that return a negative number in the C-language keyword()
function in the toke.c file in your Perl source kit may be overridden.
Keywords that *cannot* be overridden are chop, defined, delete, do,
dump, each , else, elsif, eval, exists, for, foreach, format, glob,
goto, grep, if, keys, last, local, m, map, my, next, no, package,
pop, pos, print, printf, prototype, push, q, qq, qw, qx, redo,
return, s, scalar, shift, sort, splice, split, study, sub, tie,
tied, tr, undef, unless, unshift, untie, until, use, while, and y.
All the rest can--and so, it seems, should go to the back of the
module bus.  If it had been all that important, it wouldn't have
been mutable.

--tom

Re: RFC 155 - Remove geometric functions from core

[Michael G Schwern]

A friend pointed out, technically most are trigonometric functions,
not geometric.  atan2, cos, sin, acos, asin and tan are all trig.
exp, log, sqrt are... just math I guess.

So I suppose the proposed module would be Math::Trig or some such.  Or
maybe, as the source code suggests, Math::High:Falutin.


However, since those funtions take up about 200 lines in the core, are
very stable and relatively easy to document, what do we win by
removing them?


PS  The idea of adding acos, asin and tan is good.


-- 

Michael G Schwern  http://www.pobox.com/~schwern/  [EMAIL PROTECTED]
Just Another Stupid Consultant  Perl6 Kwalitee Ashuranse
Our business in life is not to succeed but to continue to fail in high
spirits.
-- Robert Louis Stevenson