On 10/3/2012 9:53 AM, Paul Homer wrote:
"people will clean things up so long as they are sufficiently painful,
but once this is achieved, people no longer care."
The idea I've been itching to try is to go backwards. Right now we use
programmers to assemble larger and larger pieces of software, but as
time goes on they get inconsistent and the inconsistencies propagate
upwards. The result is that each new enhancement adds something, but
also degenerates the overall stability. Eventually it all hits a ceiling.
If instead, programmers just built little pieces, and it was the
computer itself that was responsible for assembling it all together
into mega-systems, then we could reach scales that are unimaginable
today. To do this of course, the pieces would have to be tightly
organized. Contributors wouldn't have the freedom they do now, but
that's a necessity to move from what is essentially a competitive
environment to a cooperative one. Some very fixed rules become necessary.
One question that arises from this type of idea is whether or not it
is even possible for a computer to assemble a massive working system
from say, a billion little code fragments. Would it run straight into
NP? In the general case I don't think the door could be closed on
this, but realistically the assembling doesn't have to happen in
real-time, most of it can be cached and reused. Also, the final system
would be composed of many major pieces, each with their own breakdown
into sub-pieces and each of those being also decomposable. Thus
getting to a super-system could take a tremendous amount of time, but
that could be distributed over a wide and mostly independent set of
work. As it is, we have 1M+ programmers right now, most of whom are
essentially writing the same things (just slightly different). With
that type of man-power directed, some pretty cool things could be created.
as to whether or not this can be applied "generally".
but, some of this I think has to do, not so much with code, but with
"data" and "metadata".
code is much better about executing a particular action, representing a
particular algorithm, ...
but, otherwise, code isn't all that inherently flexible. you can't
really "reinterpret" code.
data then, isn't so much about doing something, but describing something
in some particular format.
the code then can partly operate by "interpreting" the data, and
completing a particular action with it.
some flexibility and scalability can be gained then by taking some
things which would be otherwise hard-coded, and making them be part of
the data.
however, trying to make data into code, or code into data, looses much
of its advantages.
hard-coding things that would otherwise be data, often makes things
brittle and makes maintenance and expansion harder.
trying to make code into data tends to impair its ability to "actually
do stuff", and may often end up being bulkier and/or less clear than
actual code would have been, and unlike "true" data, greatly diminishes
its ability to be reinterpreted (often, "less is more" when it comes to
data representations).
a few semi-effective compromises seems to be:
A, scripting, where code is still code (or, at least, dynamically loaded
scripts), but which may be embedded in or invoked by data (a major
practical example of this is the whole HTML+JavaScript thing);
B, "plug-board" representation of logic within data, where the data will
indicate which actions invoke which piece of code (without describing
the code itself).
there is no clear way to scale this up to an "everything" level though.
(you wouldn't want the whole world to be HTML+JavaScript, just, no...).
metadata is, OTOH, data about code.
so, it is not code, since you can't really "run" metadata;
but, it isn't really "data" either, given what it describes is typically
tightly integrated with the code itself (most often, it is stuff that
the compiler knows while compiling the code, but is routinely discarded
once the compiler finishes its work).
so, I guess it can be noted that there are several major types of
metadata in current use:
symbolic debugging information, whose main purpose is to relate the
compiled output back to the original source code;
reflective metadata, whose main purpose is generally to allow code to
work on data-structures and objects;
specialized structures, such as class-definition and exception-unwinding
tables, which are generally used as a part of the ABI.
on various systems, these may be more-or-less integrated, for example,
on current Linux systems, all 3 cases are (sort-of) handled by DWARF,
but on Windows using the MS tool-chain, these parts are more-or-less
independent.
C and C++ typically lack reflective metadata, but this is commonly used
in languages like Java and C#. in dynamically-typed languages, the
object is often itself its own metadata.
meanwhile, in my case, I build and use reflective metadata for C.
the big part of the "power" of my VM project is not that I have such big
fancy code, or fancy code generation, but rather in that my C code has
reflection facilities. even for plain old C code, reflection can be
pretty useful sometimes (allowing doing things that would otherwise be
impossible, or at least, rather impractical).
so, in a way, reflection metadata is what makes my fancy C FFI possible.
this part was made to work fairly well, even if, admittedly, this is
something of a fairly limited scope.
much larger "big concept" things though would likely require "big
concept" metadata, and this is where the pain would begin.
with FFI gluing, the task is simpler, like:
"on one side, I have a 'string' or 'char[]' type, and on the other, a
'char *' type, what do I do?...".
usually, the types are paired in a reasonably straightforward way, and
the number of arguments match, ... so the code can either succeed (and
generate the needed interface glue), or fail at doing so.
but, admittedly, figuring out something like "how do I make these two
unrelated things interact?", where there is not a clear 1:1 mapping, is
not such an easy task.
this is then where we get into APIs, ABIs, protocols, ... where each
side defines a particular (and, usually narrow) set of defined
interactions, and interacts in a particular way.
and this is, itself, ultimately limited.
for example, a person can plug all manner of filesystems into the Linux
VFS subsystem, but ultimately there is a restriction here: it has to be
able to present itself as a hierarchical filesystem.
more so, given the way it is implemented in Linux, it has to be possible
to enumerate the files, so sad as it is, you can't really just implement
"the internet" as a Linux VFS driver (say,
"/mnt/http/www.google.com/#hl=en&..."), albeit some other VFS-style
systems allow this.
so, in this sense, it still requires "intelligence" to put the pieces
together, and design the various ways in which they may interoperate...
I really don't know if this helps, or is just me going off on a tangent.
Paul.
------------------------------------------------------------------------
*From:* BGB <cr88...@gmail.com>
*To:* fonc@vpri.org
*Sent:* Tuesday, October 2, 2012 5:48:14 PM
*Subject:* Re: [fonc] How it is
On 10/2/2012 12:19 PM, Paul Homer wrote:
It always seems to be that each new generation of programmers
goes straight for the low-hanging fruit, ignoring that most of it
has already been solved many times over. Meanwhile the real
problems remain. There has been progress, but over the couple of
decades I've been working, I've always felt that it was '2 steps
forward, 1.999999 steps back".
it depends probably on how one measures things, but I don't think
it is quite that bad.
more like, I suspect, a lot has to do with pain-threshold:
people will clean things up so long as they are sufficiently
painful, but once this is achieved, people no longer care.
the rest is people mostly recreating the past, often poorly,
usually under the idea "this time we will do it right!", often
without looking into what the past technologies did or did not do
well engineering-wise.
or, they end up trying for "something different", but usually this
turns out to be recreating something which already exists and
turns out to typically be a dead-end (IOW: where many have gone
before, and failed). often the people will think "why has no one
done it before this way?" but, usually they have, and usually it
didn't turn out well.
so, a blind "rebuild starting from nothing" probably wont achieve
much.
like, it requires taking account of history to improve on it
(classifying various options and design choices, ...).
it is like trying to convince other language/VM
designers/implementers that expecting the end programmer to have
to write piles of boilerplate to interface with C is a problem
which should be addressed, but people just go and use terrible
APIs usually based on "registering" the C callbacks with the VM
(or they devise something like JNI or JNA and congratulate
themselves, rather than being like "this still kind of sucks").
though in a way it sort of makes sense:
many language designers end up thinking like "this language will
replace C anyways, why bother to have a half-decent FFI?...".
whereas it is probably a minority position to design a language
and VM with the attitude "C and C++ aren't going away anytime soon".
but, at least I am aware that most of my stuff is poor imitations
of other stuff, and doesn't really do much of anything actually
original, or necessarily even all that well, but at least I can
try to improve on things (like, rip-off and refine).
even, yes, as misguided and wasteful as it all may seem sometimes...
in a way it can be distressing though when one has created
something that is lame and ugly, but at the same time is aware of
the various design tradeoffs that has caused them to design it
that way (like, a cleaner and more elegant design could have been
created, but might have suffered in another way).
in a way, it is a slightly different experience I suspect...
Paul.
------------------------------------------------------------------------
*From:* John Pratt <jpra...@gmail.com> <mailto:jpra...@gmail.com>
*To:* fonc@vpri.org <mailto:fonc@vpri.org>
*Sent:* Tuesday, October 2, 2012 11:21:59 AM
*Subject:* [fonc] How it is
Basically, Alan Kay is too polite to say what
we all know to be the case, which is that things
are far inferior to where they could have been
if people had listened to what he was saying in the 1970's.
Inefficient chip architectures, bloated frameworks,
and people don't know at all.
It needs a reboot from the core, all of it, it's just that
people are too afraid to admit it. New programming languages,
not aging things tied to the keyboard from the 1960's.
It took me 6 months to figure out how to write a drawing program
in cocoa, but a 16-year-old figured it out in the 1970's easily
with Smalltalk.
_______________________________________________
fonc mailing list
fonc@vpri.org <mailto:fonc@vpri.org>
http://vpri.org/mailman/listinfo/fonc
_______________________________________________
fonc mailing list
fonc@vpri.org <mailto:fonc@vpri.org>
http://vpri.org/mailman/listinfo/fonc
_______________________________________________
fonc mailing list
fonc@vpri.org <mailto:fonc@vpri.org>
http://vpri.org/mailman/listinfo/fonc
_______________________________________________
fonc mailing list
fonc@vpri.org
http://vpri.org/mailman/listinfo/fonc
_______________________________________________
fonc mailing list
fonc@vpri.org
http://vpri.org/mailman/listinfo/fonc
