On 12/31/12 6:36 PM, Marcus G. Daniels wrote:
2. The programmer has a belief or preference that the code is easier to
work with if it isn't abstracted.
It's all right in front of them in the context they want it. Perhaps
they are copying the code from foreign modules they don't maintain and
don't want to. They don't think in terms of global redundancy and the
cost for the project but just their personal ease of maintenance after
they assimilate the code. If they have to study any indirect
mechanisms, they become agitated or lose their momentum.
A lot of programmers don't seem to get abstraction (I say this having
taught programming at the college level), and that is probably a major
reason code has more duplication and lack of conciseness than it should.
Nonetheless, a lot of programmers who don't really get abstraction
(including ideas like recursion) can still get a lot of useful-to-them
stuff done by writing programs within their own comfort zone. My guess
is that 80% to 90% or more of people who actually make their living
"programming" these days fall into this category of not really being
able to deal well with abstraction to any great degree much beyond being
able to write a class to represent some problem domain object. So, such
developers would not be very comfortable thinking about writing the code
to compile those classes etc. -- even if many of them might understand a
problem domain very well. These are the people who love Visual Basic.
:-) And probably SQL. :-) And they love both of them precisely for all
the reason that probably many people on this list may feel very
constrained and unhappy when using either of those two -- stuff like
abstraction or recursion in the case of SQL is hard to do in them. So,
code in such languages in practice tends to stay within the abstraction
limits of what such programmers can understand without a lot of effort.
Still, those who do get abstraction eventually realize that every layer
of abstraction imposes its own cost -- both on the original developer
and any subsequent maintainer. One can hope that new computer languages
(whatever?) and new programming tools (like for refactoring) and new
cultural habits (unit tests) could reduce those costs, but it does not
seem like we are quite there yet -- although there are some signs of
progress. Often invention is about tradeoffs, you can have some of this
(reduced redundancy) if you give up some of that (the support code being
straightforward). True breakthroughs come when someone figures out how
to have lots of both (reduced redundancy and the supporting code being
straightforward.
Related:
http://en.wikipedia.org/wiki/Indirection
"A famous aphorism of David Wheeler goes: "All problems in computer
science can be solved by another level of indirection";[1] this is often
deliberately mis-quoted with "abstraction layer" substituted for "level
of indirection". Kevlin Henney's corollary to this is, "...except for
the problem of too many layers of indirection.""
My programming these days is a lot less abstract (clever?) than it used
to be, and I don't think that is mostly because I am less sharp having
programmed for about thirty years (although it is true that I am indeed
somewhat less sharp than I used to be). I'm willing to tolerate somewhat
more duplication than I used to because I know the price that needs to
be paid for introducing abstraction. Abstractions need to be maintained.
They may be incomplete. They may leak. They are often harder to debug.
They can be implemented badly because a couple examples of something are
often not enough to generalize well from. And so on (much of this known
from personal experience). So, these days, I tend more towards the
programming language equivalent of the "Principle of least privilege",
where I would like a computer language where, somewhat like DrScheme, I
could specify a "language level" for every section of code to reduce the
amount of cleverness allowed in that code. :-) That way, the "Visual
Basic" like parts might be 90% of the code, and when I got into sections
that did a lot of recursion or implemented parsers or whatever, it would
be clear I was in code working at a higher level of abstraction.
Considering how you can write VMs in Fortran, maybe that idea would not
work in practice, but it is at least a thing to think about. Related:
http://en.wikipedia.org/wiki/Principle_of_least_privilege
http://c2.com/cgi/wiki?PrincipleOfLeastPrivilege
And on top of that, it is quite likely that future maintainers who have
more problems with abstractions. So, the code in practice is likely to
be less easily maintained by a random programmer. So, I now try to apply
cleverness in other directions. For example, I might make tools that may
stand outside the system and help with maintaining or understanding it
(where even if the tool was not maintained, it would have served its
purpose and given a return on time invested). Or I might think harder
about achieving clarity about the essence of an issue or problem domain,
but still working within the constraints of trying to program things
straightforwardly without a new underlying interpreter layer. Smalltalk
shined so bright because it had a consistent application of an
approachable abstraction (message passing and classes) which made it
possible to create great tools to work with it. Lisp, on the other hand,
makes creating arbitrary abstractions much easier (including "mini
languages") and thus you really never be sure what any small piece of
Lisp code really does until you understand the entire system. That can
be true in Smalltalk too, but it is much rarer. So, in practice, that
moves Lisp closer in practice to the camp of write-only languages like
APL and Perl in my opinion (even though people don't have to write Lisp
that way, but I'd expect many people whole love Lisp love it precisely
because it makes that increased conciseness via macros so easy to do).
While I liked Alan Kay's 1997 OOPSLA talk, I'm not sure that focusing on
the book "The art of the meta-object protocol" as he did at the end is
really the best way forward, because it bumps into this abstraction
problem. There may be something there as far as talking about having
good abstractions, and supporting the ability to make good abstractions,
but there still seems a huge gulf somewhere between theory and practice.
That seems especially true as relates to LISPy languages tending towards
write-only when the level of abstraction increases and the numbers of
macros proliferates and so on. Related:
http://en.wikipedia.org/wiki/The_Art_of_the_Metaobject_Protocol
As an analogy, Smalltalk-80 succeeded where Smalltalk-72 failed in part
because Smalltalk-80 standardized the syntax and calling patterns of how
objects talked to each other, rather than leaving things more
"flexible". It's a fine line between flexible and flabby sometimes.
Also, programmers, even the very best, have limited attention. If they
put it into parsing arbitrary mini-languages when they read code, then
they are not putting that attention into other areas (like thinking
about the problem domain). Manuel De Landa makes an excellent related
point here:
http://www.t0.or.at/delanda/meshwork.htm
"Indeed, one must resist the temptation to make hierarchies into
villains and meshworks into heroes, not only because, as I said, they
are constantly turning into one another, but because in real life we
find only mixtures and hybrids, and the properties of these cannot be
established through theory alone but demand concrete experimentation.
Certain standardizations, say, of electric outlet designs or of
data-structures traveling through the Internet, may actually turn out to
promote heterogenization at another level, in terms of the appliances
that may be designed around the standard outlet, or of the services that
a common data-structure may make possible."
So, when Smalltalk-80 standardized to the keyword etc. syntax, it freed
up programmer mental capacity for handling more variety at another
level. Lisp macros, C macros, Forth "build" commands, and similar things
all seem like a great idea for supporting abstraction and
domain-specific languages, and can indeed be used by those skilled to
make concise notations, but they do that at a cost of maintenance and
programmer distraction (as you alluded to when you wrote about loosing
momentum). It becomes a judgement call based on expected needs whether
the cost is worth paying up-front -- a judgment call that includes the
number of future problems to solve in the problem domain and
expectations about the type of programmers who will be around to solve
them. I feel Smalltalk-80 found a good sweet spot for many problem
domains in that sense. Of course, Smalltalk-80 failed to conquer the
world (or even most of the tri-state area :-) due to the corporate
history of Smalltalk licensing and ParcPlace, PP-Digitalk, etc. sadly
whereas languages with communities more with roots in either free
licensing like Python or free runtimes like Java stormed ahead.
Here is a related essay I wrote on the follies of domain specific
languages (not sure how long that site is going to stay up though, it is
no longer in the year of free Amazon EC2 micro instance service and I
was just using it mainly to play around with Drupal):
http://twirlip.com/main/?q=on-the-follies-of-crafting-domain-specific-languages
"At a meeting I was at recently, as it was ending a person made a
comment on domain specific languages in open source code I had written
(originally about fifteen years ago), where my short reply as everyone
was leaving was "That was a long time ago...". Here is a longer response
to explain what I meant by that, involving four points -- plain text,
leveraging interpreted computer languages, using web standards, and the
issue that "end users are not programmers". But just as a disclaimer, I
think many programs are essentially "domain specific languages" in terms
of the set of functions programmers create to solve problems in various
domains. So, none of this is to be against doing that; the issue is how
we do that effectively. ..."
I had a discussion with someone at OOPSLA 97 about one of the values of
Smalltalk, including the keyword syntax, that Smalltalk supported
creating mini-languages natively in a sense. So, it is fairly easy to
create classes that let you define a datastructure for some domain by
using plain easy to read Smalltalk, where you could then easily debug
the construction process and you did not have to write any special
parser. You can do that to some extent in any language, but the keywork
syntax lends itself to more expressiveness and better documentation.
Smalltalk does it well enough that even though you could write an even
more concise mini-language, it is less often worth it in Smalltalk than
in other languages that are less clear. So in that sense, that is a key
aspect of Smalltalk as a breakthrough for the redundancy vs. support
complexity tradeoff mentioned above -- something that is that widely
appreciated.
Some ideas that have recurred in different languages though -- of which
JSON is an emergent example looking a lot like what one could do in
VisualWorks years ago to define nested data structures at compile time.
I guess once something like that becomes a common place abstraction for
defining some data, and then that data can be easily parsed, then it
reduces the general cognitive burden across the programming community
when people use that. Ideally we'd have a widely shared set of standards
like that, and perhaps we are seeing them emerge. JavaScript, HTML, and
CSS being potentially another set, where now more people get the notion
of modifying behavior at runtime, and so that reduces the cost to the
community of using those ideas. RDF is another example that is
popularizing the notion of working with triples. In that sense, common
abstractions used in web programming may be part of new directions for
the memetic center of gravity of the programming community as it relates
to abstractions that can be used effectively in code that other
programmers are going to be likely to have to maintain. It's sad though
that it was not something more like Smalltalk the language instead
though. Still, bright people are ever at work on that; example, like
with a Smalltalk on top of Javascript and HTML:
http://amber-lang.net/
But if one was going to generalize from this to issues about new
computing, having better languages and tools for working with multiple
levels of abstractions is important. But how that translates into
practice is the hard part. I posted some metaphorical comments on that
posted to the FONC list a couple years ago, using an analogy to creating
the microscope and telescope for computing abstractions:
"On inventing the computing microscope/telescope for the dynamic
semantic web"
http://www.mail-archive.com/fonc@vpri.org/msg01445.html
Those reprised some of my comments on the original FONC proposal (made
in 2007) although were a bit more narrowly focused:
http://mail.python.org/pipermail/edu-sig/2007-March/007822.html
In any case, a manager confronted with this situation can choose to
reward individuals who slow this type of proliferation. An objective
justification being an increase in the percentage of lines touched by
coverage analysis profiles across test suites, unit tests, etc.
Good points. Although, as above, there are still judgement calls
involved. I've found it easier (read, more enjoyable) to maintain code
that has multiple cut-and-paste copies of a class with minor variations
than code that has poorly thought out and badly named abstractions. I
have not yet seen any code analysis tools that tell me whether most
variables, functions, and classes/prototypes are well-named for the
problem domain. I'd much rather maintain code with three or four or
event ten slightly different copies of classes but with well chosen
names and well-thought out algorithms than code that has only one of
everything but each one has cryptic variables and problematical
algorithms.It would be a great to have some tool that would be able to
rate code based on that -- well, assuming that code would not also
replace all programming jobs because it would probably be an advanced
AI. :-) (Although event that might be OK if we had a "basic income" from
taxes on the robots and AIs like Marshall Brain talked about in "Manna".)
So, I guess there is some limits defining the shape of these tools of
new computing and where they are going if humans are going to continue
to be part of the programming loop. And those limits may be imposed at
this point more by the relatively-fixed-in-capacity human psychological
side than the exponentially increasing computing side.
--Paul Fernhout
http://www.pdfernhout.net/
====
The biggest challenge of the 21st century is the irony of technologies
of abundance in the hands of those thinking in terms of scarcity.
_______________________________________________
fonc mailing list
fonc@vpri.org
http://vpri.org/mailman/listinfo/fonc
