Re: Removing stuff to shareable thingies

[Dan Sugalski]

On Wed, 30 Aug 2000, Bradley M. Kuhn wrote:

> Dan Sugalski wrote:
> > At 04:25 AM 8/30/00 -0400, Bradley M. Kuhn wrote:
> > > > 2) Having a mechanism to automagically load in chunks of executable code
> > > > only when needed would be nice
> > >
> > >I would take this one a bit further:
> > >
> > >  2a) It should be possible, at compile-time, to detect what chunks will be
> > >  needed and perhaps warn the user that they have gone beyond a subset of
> > >  Perl they sought to stay within.
> > >
> > >This is primarily for compiling Perl to be run on embedded devices.
>  
> > I was thinking of generalizing this so that the parser would pass on to
> > the bytecompiler (and then to the execution engine, whatever that may be)
> > that an add-on piece was used.
> 
> That sounds like it fits with my idea.  The question then becomes what is
> considered an "add-on" piece.  I'd like to get very fine-grained with; even
> being able to turn of things as fundamental as tied hashes at compile time.

Well, the bytecode compiler will get all the bytecodes, so it can
presumably do whatever analysis it wants and if it sees bytecode for a
fundamental language feature that isn't supported (string eval, say, or
do/require) then it can yell.
 
On the other hand, it really wouldn't hurt to figure this out and pass on
the info to the compiler unit--if it doesn't get used, that's OK,
something else might later.

> I doubt anyone would have a need to do this except for the embedded device
> developers, but it would be best to be able to tell the embedded device
> developer at compile time that per code won't run because the embedded
> device is so small that it doesn't have the tied-hash support.  ;)

Yup, I agree. Though I can see tied hashes making FurbiePerl easier... :-)

Dan

Re: Removing stuff to shareable thingies

[Bradley M. Kuhn]

Dan Sugalski wrote:
> At 04:25 AM 8/30/00 -0400, Bradley M. Kuhn wrote:
> > > 2) Having a mechanism to automagically load in chunks of executable code
> > > only when needed would be nice
> >
> >I would take this one a bit further:
> >
> >  2a) It should be possible, at compile-time, to detect what chunks will be
> >  needed and perhaps warn the user that they have gone beyond a subset of
> >  Perl they sought to stay within.
> >
> >This is primarily for compiling Perl to be run on embedded devices.
 
> I was thinking of generalizing this so that the parser would pass on to
> the bytecompiler (and then to the execution engine, whatever that may be)
> that an add-on piece was used.

That sounds like it fits with my idea.  The question then becomes what is
considered an "add-on" piece.  I'd like to get very fine-grained with; even
being able to turn of things as fundamental as tied hashes at compile time.

I doubt anyone would have a need to do this except for the embedded device
developers, but it would be best to be able to tell the embedded device
developer at compile time that per code won't run because the embedded
device is so small that it doesn't have the tied-hash support.  ;)

-- 
Bradley M. Kuhn  -  http://www.ebb.org/bkuhn

 PGP signature

Re: Removing stuff to shareable thingies

[Dan Sugalski]

At 04:25 AM 8/30/00 -0400, Bradley M. Kuhn wrote:
>Dan Sugalski wrote:
>
> > 2) Having a mechanism to automagically load in chunks of executable code
> > only when needed would be nice
>
>I would take this one a bit further:
>
>  2a) It should be possible, at compile-time, to detect what chunks will be
>  needed and perhaps warn the user that they have gone beyond a subset of
>  Perl they sought to stay within.
>
>This is primarily for compiling Perl to be run on embedded devices.

I was thinking of generalizing this so that the parser would pass on to the 
bytecompiler (and then to the execution engine, whatever that may be) that 
an add-on piece was used.

If the execution engine's really a perl->Java translator, the info could be 
used to load in some sort of extra code or library. If the execution 
engine's a targeted bytecode saver, it could warn that the extras aren't 
available. If the execution engine's just the execution engine then, well, 
presumably it'd load in the code. :)

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: Removing stuff to shareable thingies

[Grant M.]

 Dan Sugalski <[EMAIL PROTECTED]> writes:

>What I'm
>thinking of specifically could be used to yank the functions out of the
>base perl binary (which is sort of where it started) but doesn't actually
>have to be used that way on any particular platform. (Or, rather, probably
>will, but just reference a chunk of code already loaded in)

Is this thing working? I've been making suggestions regarding this for about
a month now. I actually started an RFC on it, but never heard a word back
about it.

It really can have a 'dual' functionality. Static Build or DSO. Potentially,
there can even been a combination version (see my email of 8/28/00), with
some libraries static, and others DSO. This would allow a core functionality
required by the installation, while still allow the extended functionality
needed to run applications not originally expected in the installation. In
my estimation, it could possibly allow for better system tuning, and
potentially allow for higher security in sensitive areas (i.e.,
'uncertified' libraries could be left off the list based upon a
'' directive in the configuration?).

If the designers of Perl wish to make it a successful 'platform independent'
language, then it really needs to be as flexible as possible. This means the
ability to allow the implementor to decide what's the right way in their
particular instance, and although that may not be the 'recommended' way,
then who are we to tell them that they are wrong. I started as an embedded
systems developer for the 8051, and I know full well what sorts of
limitations a system like that has. And don't kid yourselves, there are
still alot of those systems still chugging away.

Grant M.

Re: Removing stuff to shareable thingies

[Grant M.]

RANDOM THOUGHT (please don't hurt me):
What about allow a 'daemon-mode'? This might be really not an option, but it
could be beneficial in a sense that there would be one core, and any other
instances would be spawned child processes?
Does that make any sense? Mind you, it's still early.
Grant M.

Re: Removing stuff to shareable thingies

[Bradley M. Kuhn]

Dan Sugalski wrote:

> 2) Having a mechanism to automagically load in chunks of executable code
> only when needed would be nice

I would take this one a bit further:

 2a) It should be possible, at compile-time, to detect what chunks will be
 needed and perhaps warn the user that they have gone beyond a subset of
 Perl they sought to stay within.

This is primarily for compiling Perl to be run on embedded devices.
 

-- 
Bradley M. Kuhn  -  http://www.ebb.org/bkuhn

 PGP signature

Re: Removing stuff to shareable thingies

[Dan Sugalski]

On 29 Aug 2000, Russ Allbery wrote:

> Dan Sugalski <[EMAIL PROTECTED]> writes:
> 
> > It's not unreasonable to expect this sort of feature to possibly be used
> > for more esoteric extensions to the perl core or commonly and heavily
> > used extensions. I wouldn't, for example, want to always load in
> > DBD::Oracle or a full complex math library (complete with FFT and
> > gaussian filters) every time I was ripping through a text file.
> 
> > If the feature exists as part of the design from the start, it puts
> > certain requirements for the lexer/parser and core interpreter that will
> > make modularizing things a neccessity and thus functional for those
> > situations where it is reasonable to do it.
> 
> Yes, this part I agree with... it's pretty close to our current dynamic
> module system, though, isn't it?  Or is it the on-demand part specifically
> that would be new?

It's the on-demand and 'use-less' part that's new, really. What I'm
thinking of specifically could be used to yank the functions out of the
base perl binary (which is sort of where it started) but doesn't actually
have to be used that way on any particular platform. (Or, rather, probably
will, but just reference a chunk of code already loaded in)

You also get the side-effect of being able to upgrade pieces of core perl
without having to upgrade the whole thing and, while I don't know that
that particular capability would be used much, it is a nice extra.

Dan

Re: Removing stuff to shareable thingies

[Russ Allbery]

Dan Sugalski <[EMAIL PROTECTED]> writes:

> It's not unreasonable to expect this sort of feature to possibly be used
> for more esoteric extensions to the perl core or commonly and heavily
> used extensions. I wouldn't, for example, want to always load in
> DBD::Oracle or a full complex math library (complete with FFT and
> gaussian filters) every time I was ripping through a text file.

> If the feature exists as part of the design from the start, it puts
> certain requirements for the lexer/parser and core interpreter that will
> make modularizing things a neccessity and thus functional for those
> situations where it is reasonable to do it.

Yes, this part I agree with... it's pretty close to our current dynamic
module system, though, isn't it?  Or is it the on-demand part specifically
that would be new?

Shared libraries opened at run-time make sense to me for things that make
a large and noticeable thud (such as DBD::Oracle with all the accompanying
Oracle shared libraries and whatnot) or things that are distributed
independently (which is where we pick up a whole bunch of stuff probably
not large enough to warrant a shared library by itself except that it just
makes things infinitely more convenient).

-- 
Russ Allbery ([EMAIL PROTECTED]) 

Re: Removing stuff to shareable thingies

[Dan Sugalski]

On 29 Aug 2000, Russ Allbery wrote:

> Dan Sugalski <[EMAIL PROTECTED]> writes:
> 
> > 2) Having a mechanism to automagically load in chunks of executable code
> > only when needed would be nice
> 
> It's not clear to me how useful this really is from an internals speed
> standpoint on modern systems.  It's no longer always true that increasing
> the size of an executable will make it start slower or consume more
> memory, and I expect that to become less true over time.  And loading
> dynamic libraries is actually fairly slow; static code loads faster
> because it doesn't have to do the relocations and the additional disk
> accesses.

I do understand that a fully static link does tend to be faster, since all
the relocation fixups are done at link time, even on those systems with
efficient link-time resolution. (And the corresponding lack of flexibility
that goes with it)

As I said, this may very well not be a win in the general case for many
systems. Odds are most recent versions of Unices won't win anything from
separating out things from the core, and neither will Windows or VMS.

However, this point was vague on purpose. It's *not* referring only to
pieces deemed "really, really core perl" (like, say, POSIX) nor just at
the latest and greatest in OS technology. (SunOS 4.x anyone? Or the Palm
or embedded systems that might end up referencing ROM-based (and usually
slower) overlays?)

It's not unreasonable to expect this sort of feature to possibly be used
for more esoteric extensions to the perl core or commonly and heavily used
extensions. I wouldn't, for example, want to always load in DBD::Oracle or
a full complex math library (complete with FFT and gaussian filters) every
time I was ripping through a text file.

If the feature exists as part of the design from the start, it puts
certain requirements for the lexer/parser and core interpreter that will
make modularizing things a neccessity and thus functional for those
situations where it is reasonable to do it.

Dan

Re: Removing stuff to shareable thingies

[Russ Allbery]

Dan Sugalski <[EMAIL PROTECTED]> writes:

> 2) Having a mechanism to automagically load in chunks of executable code
> only when needed would be nice

It's not clear to me how useful this really is from an internals speed
standpoint on modern systems.  It's no longer always true that increasing
the size of an executable will make it start slower or consume more
memory, and I expect that to become less true over time.  And loading
dynamic libraries is actually fairly slow; static code loads faster
because it doesn't have to do the relocations and the additional disk
accesses.

-- 
Russ Allbery ([EMAIL PROTECTED]) 

Removing stuff to shareable thingies

[Dan Sugalski]

Folks,

I think at this point we've beaten this one to death. The obvious 
conclusions seem to be:

1) Removing all functions that do X (whatever X is) may or may not buy us 
anything
2) Having a mechanism to automagically load in chunks of executable code 
only when needed would be nice
3) It'd be nice to statically link in code that would otherwise be 
dynamically loaded

I've got part of an RFC done--I'll finish it and send it off.

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk