Re: Knuth's literate programming tangle function in Clojure

2011-01-08 Thread Robert McIntyre
You may find 
http://blog.higher-order.net/2009/02/01/understanding-clojures-persistentvector-implementation/
useful for a clear explanation of PersistentVectors.  Maybe even get
in touch with the guy for an addition to the book?
Thanks for your work on a literate clojure.

sincerely,
--Robert McIntyre

On Fri, Jan 7, 2011 at 12:28 AM, Ken Wesson kwess...@gmail.com wrote:
 On Fri, Jan 7, 2011 at 12:07 AM, Tim Daly d...@axiom-developer.org wrote:
 Hmmm. I may have misunderstood your point. I thought you were suggesting
 writing code that is not part of the distribution in order to get a
 minimal running system and then working from that. If that is not what
 you're suggesting then I'm confused.

 No, I was just suggesting that the order of the material put the stuff
 in the distribution that's necessary to bootstrap a minimally
 functional repl first, culminating in the eval function and the
 command-line repl class, then flesh out the rest of Clojure's
 feature-set with the rest of the stuff in the distribution. No new
 code.

 The pamphlet sources are in a git repository so they are immutable.

 Wikis are fine for a lot of things but not for linearizing the
 ideas into a readable literate form. Books fulfill that role.

 I suggested *maybe* letting the wiki users try to decide,
 collectively, on a linearization; maybe that would prove workable and
 maybe not. If not, you'd have to linearize it yourself to make the
 book version. But if you're looking for section submissons and user
 proof-reading a wiki can at least organize that activity, and can
 provide seeds by having unwritten sections in there with just the
 source code that is to be explained. And without potential
 contributors maybe being put off by having to learn a whole extra set
 of tools (namely, github and whatever client software) and get a login
 at some site (github). Some might not have used git. A few might not
 have used any code repository system. A wiki on the other hand can be
 edited by anyone who can type stuff into a web form and can be
 configured not to require a login (ala Wikipedia itself).

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your 
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-08 Thread Tim Daly

 Sweet! Thanks for the pointer.
It is a challenge to reverse-engineer someone else's code.
It is even more challenging to understand how it relates
to the algorithm and the idea. This will help a lot.

Tim Daly

On 1/8/2011 6:22 AM, Robert McIntyre wrote:

You may find 
http://blog.higher-order.net/2009/02/01/understanding-clojures-persistentvector-implementation/
useful for a clear explanation of PersistentVectors.  Maybe even get
in touch with the guy for an addition to the book?
Thanks for your work on a literate clojure.

sincerely,
--Robert McIntyre

On Fri, Jan 7, 2011 at 12:28 AM, Ken Wessonkwess...@gmail.com  wrote:

On Fri, Jan 7, 2011 at 12:07 AM, Tim Dalyd...@axiom-developer.org  wrote:

Hmmm. I may have misunderstood your point. I thought you were suggesting
writing code that is not part of the distribution in order to get a
minimal running system and then working from that. If that is not what
you're suggesting then I'm confused.

No, I was just suggesting that the order of the material put the stuff
in the distribution that's necessary to bootstrap a minimally
functional repl first, culminating in the eval function and the
command-line repl class, then flesh out the rest of Clojure's
feature-set with the rest of the stuff in the distribution. No new
code.


The pamphlet sources are in a git repository so they are immutable.

Wikis are fine for a lot of things but not for linearizing the
ideas into a readable literate form. Books fulfill that role.

I suggested *maybe* letting the wiki users try to decide,
collectively, on a linearization; maybe that would prove workable and
maybe not. If not, you'd have to linearize it yourself to make the
book version. But if you're looking for section submissons and user
proof-reading a wiki can at least organize that activity, and can
provide seeds by having unwritten sections in there with just the
source code that is to be explained. And without potential
contributors maybe being put off by having to learn a whole extra set
of tools (namely, github and whatever client software) and get a login
at some site (github). Some might not have used git. A few might not
have used any code repository system. A wiki on the other hand can be
edited by anyone who can type stuff into a web form and can be
configured not to require a login (ala Wikipedia itself).

--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-06 Thread Eric Schulte
Tim Daly d...@axiom-developer.org writes:

 On 1/6/2011 12:03 AM, Eric Schulte wrote:
 Can you post examples of these? I'd love to see some other examples.
 Sure thing, check out this old version of a file which tangles out into
 the directory layout expected by lein.
 http://gitweb.adaptive.cs.unm.edu/?p=asm.git;a=blob;f=asm.org;h=f043a8c8b0a917f58b62bdeac4c0dca441b8e2cb;hb=HEAD
 I see that this file is using chunk markup but I don't see
 the ideas-to-implementation (I2I) explanation. That is, I
 don't see text that is written so someone like me can read the
 text, understand the ideas and how they translate to code.


True, unfortunately I don't have any good examples where I write in an
I2I style.  I think the discipline needed to sit down and document ones
initial ideas may be one of the largest hurdles to a wider adoption of
LP practices.

However there are more examples of org-mode usage available online at
http://orgmode.org/worg/org-contrib/babel/uses.html


 Also, this project has an org-mode front page with code examples, the
 html woven from this front page is shown at
 http://repo.or.cz/w/neural-net.git
 and the raw org file is available here
 http://repo.or.cz/w/neural-net.git/blob/HEAD:/neural-net.org
 I love the graphics in this example! But, alas, this also seems
 to be disconnected from the I2I property. But this is VERY
 encouraging since it shows that you can get beautifully formatted
 documents from the source.

Thanks, you may also notice that the graphics are actually produced from
inline LaTeX (using tikz) avoiding the need for external binary files
and allowing everything to be handled nicely by git.


 I'll have to check  out clojure.pamphlet, it sounds like an elegant
 alternative.  It's always interesting to see other solutions in this
 space.  For example I think scribble is a nice tool from the scheme
 world.  http://lambda-the-ultimate.org/node/4017
 Take a look at http://daly.axiom-developer.org/clojure.pdf


Very nice, I think this has become my new reference for code in the
Clojure core.  Is the .clj or .tex file available as well?


 The key difference, at least from my perspective, is that the CISP
 is (well, is slowly being) organized around ideas. The goal is to
 introduce the ideas and explain them in enough detail to *motivate*
 the code that follows the explanation.

 If you handed your printed document (or pdf) to someone will they
 come away with enough understanding of neural networks to be able
 to explain your code? Can they read the pdf and know what to change
 and where to change it? Do they have literature references to follow
 for deeper information? Is there a usable index with proper cross
 references so they can hyperlink to the explanation and its associated
 code block? These things seem, in my opinion, necessary (but not
 sufficient) for the I2I property to hold.

 I think that the tool you have is very, very nice. It shows me that
 it would make a proper substitute for my beloved Latex as a viable
 alternative for literate programming.


Thanks, and I agree the tool should support LP/I2I style composition,
the main hurdle is getting users (including myself) to write code in
that style (regardless of the tool used).

Best Regards -- Eric


 Tim Daly

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-06 Thread Michael Wood
Hi

On 6 January 2011 07:33, Tim Daly d...@axiom-developer.org wrote:
[...]
 Take a look at http://daly.axiom-developer.org/clojure.pdf

I like it :)

Some simple corrections:

You have a typo on the front page:

Based on Version 1.3.0-alphs4 (alphs4 instead of alpha4).

In the foreword, This is a literate fork of Clojure, invented by Rich
Hickey. sounds a little like Rich Hickey invented this fork of
Clojure rather than Clojure itself.

I have no idea what that ((iHy)) means after the date at the end of
the foreword.

Page 2:

Last sentence of 1.3.2 says We will look now look

In 1.3.3 the methods for updating the right member are identical to
the ones for the left member.  I suspect left should be replace with
right in all three lines?

Page 5:

Fourth paragraph of 1.3.5 contains a sentence with a missing a:
Since it is copy operation

Also in 1.3.5, the list of Black node types duplicates the information
in 1.3.3.  (Same with the list of Red nodes in 1.3.11.)

In the introductions to the Black, BlackVal, BlackBranch and
BlackBranchVal descriptions you say This is constructed by
PersistenTreeMap's black method.  Are all of them constructed with
the black method?  If so, perhaps you should say that somewhere
instead of saying it separately for each one?  Perhaps 1.3.10 is
enough.  Just when I was reading through the introductions to Black,
BlackVal etc., I kept wondering if the next one was going to say it
was constructed by some method other than black, since you were
explicitly mentioning it every time.

Page 9:

There seems to be a spurious new in the first sentence: If either
of the children exist but new have no value for this node...

The second sentence of 1.3.11 is confusing to me.  It seems to me that
the colour of the node returned has nothing to do with whether or not
subtrees are balanced, so it doesn't make sense to me to contrast
these two things.  (Of course I know nothing about Red Black trees
other than what I've read so far in this document.)

The next paragraph is a little confusing to me too.  Why would Red
nodes have to be returned only because the default colour is Black?
Surely even if the default colour were Red, Red nodes would need to be
returned?  Perhaps I'm missing something fundamental about how Red
Black trees work.

Page 13:

1.3.16 seems to be more or less a copy of 1.3.10 with Black replaced
with Red, so it also has the extraneous new.

Page 14:

The second sentence of 1.3.17 says ... all four cases all
balance  It would be better without the second all.

The next paragraph says Each cases shows the ML code  This
should of course be Each case  Also, what is ML code?

Page 16:

There's a missing 's in the second sentence: ... a black node (p5)
removeRight method...

Anyway, that's as far as I've got so far.

-- 
Michael Wood esiot...@gmail.com

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-06 Thread Tim Daly



On 1/6/2011 11:16 AM, Eric Schulte wrote:

Tim Dalyd...@axiom-developer.org  writes:


On 1/6/2011 12:03 AM, Eric Schulte wrote:

Can you post examples of these? I'd love to see some other examples.

Sure thing, check out this old version of a file which tangles out into
the directory layout expected by lein.
http://gitweb.adaptive.cs.unm.edu/?p=asm.git;a=blob;f=asm.org;h=f043a8c8b0a917f58b62bdeac4c0dca441b8e2cb;hb=HEAD

I see that this file is using chunk markup but I don't see
the ideas-to-implementation (I2I) explanation. That is, I
don't see text that is written so someone like me can read the
text, understand the ideas and how they translate to code.


True, unfortunately I don't have any good examples where I write in an
I2I style.  I think the discipline needed to sit down and document ones
initial ideas may be one of the largest hurdles to a wider adoption of
LP practices.

However there are more examples of org-mode usage available online at
http://orgmode.org/worg/org-contrib/babel/uses.html


Also, this project has an org-mode front page with code examples, the
html woven from this front page is shown at
http://repo.or.cz/w/neural-net.git
and the raw org file is available here
http://repo.or.cz/w/neural-net.git/blob/HEAD:/neural-net.org

I love the graphics in this example! But, alas, this also seems
to be disconnected from the I2I property. But this is VERY
encouraging since it shows that you can get beautifully formatted
documents from the source.

Thanks, you may also notice that the graphics are actually produced from
inline LaTeX (using tikz) avoiding the need for external binary files
and allowing everything to be handled nicely by git.


I'll have to check  out clojure.pamphlet, it sounds like an elegant
alternative.  It's always interesting to see other solutions in this
space.  For example I think scribble is a nice tool from the scheme
world.  http://lambda-the-ultimate.org/node/4017

Take a look at http://daly.axiom-developer.org/clojure.pdf


Very nice, I think this has become my new reference for code in the
Clojure core.  Is the .clj or .tex file available as well?

The source documents are
http://daly.axiom-developer.org/clojure.pamphlet
http://daly.axiom-developer.org/clojure.sty

The instructions for running tangle are in the preface.
The tangle.c function is in the document but it is standalone here:

http://daly.axiom-developer.org/tangle.c

A pamphlet file IS a tex file. I just chose the name to distinguish
it as a literate document from other tex files I have.

You might find it interesting to change the pamphlet to use org-mode
(or change org-mode to read pamphlets :-) )

--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-06 Thread Tim Daly



On 1/6/2011 12:07 PM, Michael Wood wrote:

Hi

On 6 January 2011 07:33, Tim Dalyd...@axiom-developer.org  wrote:
[...]

Take a look at http://daly.axiom-developer.org/clojure.pdf

I like it :)

Some simple corrections:

You have a typo on the front page:

Based on Version 1.3.0-alphs4 (alphs4 instead of alpha4).

In the foreword, This is a literate fork of Clojure, invented by Rich
Hickey. sounds a little like Rich Hickey invented this fork of
Clojure rather than Clojure itself.

You're right. Rich did not invent the fork. The spoon, perhaps.
This is why projects need an Editor-in-Chief.

I have no idea what that ((iHy)) means after the date at the end of
the foreword.

It is an lisp joke between myself and certain others. Ignore it.

Page 2:

Last sentence of 1.3.2 says We will look now look

In 1.3.3 the methods for updating the right member are identical to
the ones for the left member.  I suspect left should be replace with
right in all three lines?

Page 5:

Fourth paragraph of 1.3.5 contains a sentence with a missing a:
Since it is copy operation

Also in 1.3.5, the list of Black node types duplicates the information
in 1.3.3.  (Same with the list of Red nodes in 1.3.11.)

In the introductions to the Black, BlackVal, BlackBranch and
BlackBranchVal descriptions you say This is constructed by
PersistenTreeMap's black method.  Are all of them constructed with
the black method?  If so, perhaps you should say that somewhere
instead of saying it separately for each one?  Perhaps 1.3.10 is
enough.  Just when I was reading through the introductions to Black,
BlackVal etc., I kept wondering if the next one was going to say it
was constructed by some method other than black, since you were
explicitly mentioning it every time.

Page 9:

There seems to be a spurious new in the first sentence: If either
of the children exist but new have no value for this node...

The second sentence of 1.3.11 is confusing to me.  It seems to me that
the colour of the node returned has nothing to do with whether or not
subtrees are balanced, so it doesn't make sense to me to contrast
these two things.  (Of course I know nothing about Red Black trees
other than what I've read so far in this document.)

The next paragraph is a little confusing to me too.  Why would Red
nodes have to be returned only because the default colour is Black?
Surely even if the default colour were Red, Red nodes would need to be
returned?  Perhaps I'm missing something fundamental about how Red
Black trees work.

One of the struggles about these paragraphs is that I am trying
to reverse engineer Rich's code. There are no comments to guide the
connection between ideas and code. I have to stare at it and guess
his intentions and mindset. Ideally code would be born literate
but that is not the case here.

The posted version

Page 13:

1.3.16 seems to be more or less a copy of 1.3.10 with Black replaced
with Red, so it also has the extraneous new.

I did a self-study of errors once and found that 50 percent of all
the errors I ever make are caused by copy-pasting. You think I would
learn but apparently I don't.

Page 14:

The second sentence of 1.3.17 says ... all four cases all
balance  It would be better without the second all.

The next paragraph says Each cases shows the ML code  This
should of course be Each case  Also, what is ML code?

You're right. I should define all my abbreviations like ML before
they are used. I added a line to define it along with two literature
references.


Page 16:

There's a missing 's in the second sentence: ... a black node (p5)
removeRight method...

Anyway, that's as far as I've got so far.

Very useful feedback. Thanks.

Tim Daly

--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-06 Thread Ken Wesson
I've now had a quick look at this too.

Aside from the grammatical and spelling nitty-gritty, there are some
larger scale concerns, ones that unfortunately go to the heart of the
entire literate programming concept.

Namely, at the start it jumps very quickly into a red-black tree
implementation without really motivating it.

Now, it mentions wanting a binary search tree, and that a red-black
tree maintains certain invariants that limit how unbalanced it can
get, but the explicit motivation for this (avoiding the linear
worst-case performance of a fully unbalanced tree in favor of a tree
that gives guaranteed logarithmic bounds on its operations, other than
the obviously-linear-best-case full traversal of course).

More significantly, there's no explanation for introducing a binary
search tree in the first place -- and mention of Java but no direct
motivation for not using java.util.TreeMap or similar instead of
rolling our own. There's a brief mention of immutability and
persistence, without the connecting thoughts -- namely we want clojure
to have an immutable sorted-map type, and while we could use a factory
function that spits out (Collections/unmodifiableMap (doto (TreeMap.)
#(doseq [[k v] kvs] (.put % k v, every assoc would require copying
the entire map, in linear time and space, etc. etc. etc., vs. using a
persistent data structure specifically designed for structure
sharing with modified versions.

And then of course the overarching purpose of Clojure itself isn't
even given, let alone why it has sorted-map, and why it's an immutable
sorted-map ...

I know, I know, tall order, right? I hope you're at least thinking
about this stuff for the eventual final version, though. (And why
start with the red-black tree, anyway? After introducing Clojure I'd
probably start with getting a minimalist implementation that is
runnable first -- so the compiler and evaluator and repl and
s-expressions, lists and any other infrastructure these need under the
hood, and whatever else is needed for this -- some of the numerics
stuff, symbols and symbol interning, etc.; then with already a basic
Lisp environment in play, adding the other functions and data
structures in core that weren't needed just to get things up and
running with a basically functional repl. Maybe sorted-map (or a
persistent immutable binary search tree implemented in Java) is one of
the things that is needed to get the compiler working, but if so, I'd
think a truly literate version is going to explain what Clojure is,
and then how we're gonna implement it, and then details of the planned
compiler implementation, and why this needs a persistent immutable
binary search tree, and how that will also be useful later for making
the user-visible sorted-map and sorted-set features, and THEN get down
into the weeds of how we make a persistent immutable binary search
tree, how we can avoid the linear worst-time behavior of unbalanced
trees by ensuring the tree never gets too far out of balance (and even
what balanced means in this context), and that red-black trees are
among several kinds that have such properties, and then why red-black
trees were chosen over, say, two-three trees, etc.

Of course, you might need to ask Rich some questions to get some of
this information. It's pretty clear how a persistent immutable
balanced search tree is needed to make sorted-map and sorted-set work
well and in a pure-functional way; it may or may not be obvious on
close examination that a red-black tree is a superior choice over the
alternatives; maybe it isn't and Rich's choice was more or less
arbitrary, or based on what he already knew how to implement or had a
reference handy for; or Rich tried several alternatives and found that
he could make a Java implementation of a persistent, immutable
red-black tree faster than any of the others, or better at
structure-sharing; to really be sure, unless it really does become
obvious on looking into the code itself why, you'd have to ask Rich
why he chose red-black trees for this job. And there will probably be
many other similar points in the code where there's an implementation
choice that may not be obvious without Rich's input. And all of them
will need some explanation, if the program is to be truly literate --
even if there turn out to be cases where that explanation will be
something like tests showed that X and Y were equally good
alternatives for implementing this, so Rich just flipped a coin and it
came up tails so he used Y. :)

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-06 Thread Tim Daly



On 1/6/2011 9:15 PM, Ken Wesson wrote:

I've now had a quick look at this too.

Aside from the grammatical and spelling nitty-gritty, there are some
larger scale concerns, ones that unfortunately go to the heart of the
entire literate programming concept.

Namely, at the start it jumps very quickly into a red-black tree
implementation without really motivating it.

Actually the document was released because it contained a complete
working version of Clojure so it was the initial starting point.

I started on the Red Black tree section so I could show an example
of how to reorganize the code presentation so that it fit the
presentation in text rather than fitting the requirements of Java.

Red Black trees are mentioned in one of Rich's lectures as an
implementation detail so I decided to try and reverse engineer
their use.

Clearly there needs to be some discussion about Persistence and
Okasaki and Rich's research references motivating why they were
chosen.

I did this with the section on Bit-partitioned hash tries.
That version has not been published yet.



Now, it mentions wanting a binary search tree, and that a red-black
tree maintains certain invariants that limit how unbalanced it can
get, but the explicit motivation for this (avoiding the linear
worst-case performance of a fully unbalanced tree in favor of a tree
that gives guaranteed logarithmic bounds on its operations, other than
the obviously-linear-best-case full traversal of course).

More significantly, there's no explanation for introducing a binary
search tree in the first place -- and mention of Java but no direct
motivation for not using java.util.TreeMap or similar instead of
rolling our own. There's a brief mention of immutability and
persistence, without the connecting thoughts -- namely we want clojure
to have an immutable sorted-map type, and while we could use a factory
function that spits out (Collections/unmodifiableMap (doto (TreeMap.)
#(doseq [[k v] kvs] (.put % k v, every assoc would require copying
the entire map, in linear time and space, etc. etc. etc., vs. using a
persistent data structure specifically designed for structure
sharing with modified versions.

And then of course the overarching purpose of Clojure itself isn't
even given, let alone why it has sorted-map, and why it's an immutable
sorted-map ...

There is very little that needs to be said about the overarching
purpose of Clojure for the expected target audience. Since this
includes detailed source code in two languages it is reasonable to
assume that it is being read by a programmer and, more specifically,
it is being read by a programmer who is interested in the details
of Clojure's implementation.

I know, I know, tall order, right? I hope you're at least thinking
about this stuff for the eventual final version, though. (And why
start with the red-black tree, anyway? After introducing Clojure I'd
probably start with getting a minimalist implementation that is
runnable first

The included implementation is runnable. I was not advocating
literate program at the introduction of Clojure 1.0 so the first
version is 1.3 which is not minimalist in any sense.


  -- so the compiler and evaluator and repl and
s-expressions, lists and any other infrastructure these need under the
hood, and whatever else is needed for this -- some of the numerics
stuff, symbols and symbol interning, etc.; then with already a basic
Lisp environment in play, adding the other functions and data
structures in core that weren't needed just to get things up and
running with a basically functional repl.

I wish Clojure was factored in terms of the implementation
to suit the I2I (ideas-to-implementation) approach but it isn't.
The literate goal here isn't to rewrite or refactor it but to
explain how the ideas are realized in code. A different goal
would be to factor Clojure in layers but that's someone else's book.

  Maybe sorted-map (or a
persistent immutable binary search tree implemented in Java) is one of
the things that is needed to get the compiler working, but if so, I'd
think a truly literate version is going to explain what Clojure is,
and then how we're gonna implement it, and then details of the planned
compiler implementation, and why this needs a persistent immutable
binary search tree, and how that will also be useful later for making
the user-visible sorted-map and sorted-set features, and THEN get down
into the weeds of how we make a persistent immutable binary search
tree, how we can avoid the linear worst-time behavior of unbalanced
trees by ensuring the tree never gets too far out of balance (and even
what balanced means in this context), and that red-black trees are
among several kinds that have such properties, and then why red-black
trees were chosen over, say, two-three trees, etc.

Of course, you might need to ask Rich some questions to get some of
this information.

Rich obviously knows the answers to these questions but he is
a valuable and highly constrained 

Re: Knuth's literate programming tangle function in Clojure

2011-01-06 Thread Ken Wesson
On Thu, Jan 6, 2011 at 11:05 PM, Tim Daly d...@axiom-developer.org wrote:
 On 1/6/2011 9:15 PM, Ken Wesson wrote:
 And then of course the overarching purpose of Clojure itself isn't
 even given, let alone why it has sorted-map, and why it's an immutable
 sorted-map ...

 There is very little that needs to be said about the overarching
 purpose of Clojure for the expected target audience. Since this
 includes detailed source code in two languages it is reasonable to
 assume that it is being read by a programmer and, more specifically,
 it is being read by a programmer who is interested in the details
 of Clojure's implementation.

Perhaps, but I think it might be worth considering broadening the
potential target audience to include people with some Java knowledge
that are curious about Clojure, but don't necessarily know much about
Clojure.

 I know, I know, tall order, right? I hope you're at least thinking
 about this stuff for the eventual final version, though. (And why
 start with the red-black tree, anyway? After introducing Clojure I'd
 probably start with getting a minimalist implementation that is
 runnable first

 The included implementation is runnable. I was not advocating
 literate program at the introduction of Clojure 1.0 so the first
 version is 1.3 which is not minimalist in any sense.

Oh, I wasn't suggesting to leave anything out entirely; just to
organize the text to put the bootstrap-necessary material first, until
the code up to that point can get you a working repl with some
functionality, and then after that fleshing out the rest of the
feature-set and standard (clojure.core; contrib would be biting off
more than anyone could probably chew) library.

  -- so the compiler and evaluator and repl and
 s-expressions, lists and any other infrastructure these need under the
 hood, and whatever else is needed for this -- some of the numerics
 stuff, symbols and symbol interning, etc.; then with already a basic
 Lisp environment in play, adding the other functions and data
 structures in core that weren't needed just to get things up and
 running with a basically functional repl.

 I wish Clojure was factored in terms of the implementation
 to suit the I2I (ideas-to-implementation) approach but it isn't.

I thought you had been saying all kinds of stuff about how your
literate-programming tools allow you to organize the material for the
benefit of human readers, rather than the requirements of the
compiling tools? This seems somewhat at odds with that.

 Of course, you might need to ask Rich some questions to get some of
 this information.

 Rich obviously knows the answers to these questions but he is
 a valuable and highly constrained resource. It would be nice if
 he provided some motivating material but I think that we, as programmers,
 can figure out how bit-partitioning works and how the code implements
 it. Every new developer needs to do that to make changes. Literate
 source code would decrease the time it takes to get up to speed.
 So while it would be nice if Rich contributed I think he has better
 things to do.
...
 Rich has provided a lot of discussion about his motivations
 in his videos.

Then those videos should prove a valuable resource for you.

 There was a discussion a while back about providing
 this same information in textual form and as a result of that
 I ended up advocating literate programming which led to this.

I seem to recall being involved in that discussion. :)

 I am hoping that showing an example of literate Clojure will
 motivate others to pick a section (e.g. agents) and work through
 the details. Once a section is written it can benefit from the
 many eyes of open source to improve the style and accuracy
 of presentation. In that way we all end up developing a document
 that quickly brings up the average quality material for developers.

That seems to me to suggest creating a wiki or similar for this
project. Why stop with half-measures? Let anyone contribute. Sections
can be marked as code-only, code and partial text, and code with
completed text; encourage people to flesh out the first two and
proofread and correct errors in the latter. Sections with completed
text would also have an uneditable canonical version that could be
added to the .pamphlet file, though still updated from time to time
with typo fixes and other corrections. (And the whole thing would
retain an uneditable history, like wikis commonly do, as a defense
against vandalism. Think of it as persistent, immutable text editing.
:))

There's also the need though for motivating sections that may not have
any code themselves but would preface some series of code-sections
related to implementing a particular larger element of the
architecture, such as the compiler, maps, or lazy sequences.

Any such wiki should probably allow such sections to be created, and
provide a discussion forum of some sort for debating which such
sections are needed, creations/deletions, any other wiki

Re: Knuth's literate programming tangle function in Clojure

2011-01-06 Thread Tim Daly



On 1/6/2011 11:42 PM, Ken Wesson wrote:

On Thu, Jan 6, 2011 at 11:05 PM, Tim Dalyd...@axiom-developer.org  wrote:

On 1/6/2011 9:15 PM, Ken Wesson wrote:

And then of course the overarching purpose of Clojure itself isn't
even given, let alone why it has sorted-map, and why it's an immutable
sorted-map ...

There is very little that needs to be said about the overarching
purpose of Clojure for the expected target audience. Since this
includes detailed source code in two languages it is reasonable to
assume that it is being read by a programmer and, more specifically,
it is being read by a programmer who is interested in the details
of Clojure's implementation.

Perhaps, but I think it might be worth considering broadening the
potential target audience to include people with some Java knowledge
that are curious about Clojure, but don't necessarily know much about
Clojure.

I'm open to including such a section at the beginning of the book.
I'm not a Java programmer though, I come from the Lisp side.
If you're willing to write it, we can add it (Advocacy... :-) )

I know, I know, tall order, right? I hope you're at least thinking
about this stuff for the eventual final version, though. (And why
start with the red-black tree, anyway? After introducing Clojure I'd
probably start with getting a minimalist implementation that is
runnable first

The included implementation is runnable. I was not advocating
literate program at the introduction of Clojure 1.0 so the first
version is 1.3 which is not minimalist in any sense.

Oh, I wasn't suggesting to leave anything out entirely; just to
organize the text to put the bootstrap-necessary material first, until
the code up to that point can get you a working repl with some
functionality, and then after that fleshing out the rest of the
feature-set and standard (clojure.core; contrib would be biting off
more than anyone could probably chew) library.


  -- so the compiler and evaluator and repl and
s-expressions, lists and any other infrastructure these need under the
hood, and whatever else is needed for this -- some of the numerics
stuff, symbols and symbol interning, etc.; then with already a basic
Lisp environment in play, adding the other functions and data
structures in core that weren't needed just to get things up and
running with a basically functional repl.

I wish Clojure was factored in terms of the implementation
to suit the I2I (ideas-to-implementation) approach but it isn't.

I thought you had been saying all kinds of stuff about how your
literate-programming tools allow you to organize the material for the
benefit of human readers, rather than the requirements of the
compiling tools? This seems somewhat at odds with that.

Hmmm. I may have misunderstood your point. I thought you were suggesting
writing code that is not part of the distribution in order to get a
minimal running system and then working from that. If that is not what
you're suggesting then I'm confused.

Of course, you might need to ask Rich some questions to get some of
this information.

Rich obviously knows the answers to these questions but he is
a valuable and highly constrained resource. It would be nice if
he provided some motivating material but I think that we, as programmers,
can figure out how bit-partitioning works and how the code implements
it. Every new developer needs to do that to make changes. Literate
source code would decrease the time it takes to get up to speed.
So while it would be nice if Rich contributed I think he has better
things to do.

...

Rich has provided a lot of discussion about his motivations
in his videos.

Then those videos should prove a valuable resource for you.

Yes, I am reviewing the video and slides so the explanation
is as close to Rich's message as possible. These are Rich's
ideas. The goal is to write them down.

There was a discussion a while back about providing
this same information in textual form and as a result of that
I ended up advocating literate programming which led to this.

I seem to recall being involved in that discussion. :)


I am hoping that showing an example of literate Clojure will
motivate others to pick a section (e.g. agents) and work through
the details. Once a section is written it can benefit from the
many eyes of open source to improve the style and accuracy
of presentation. In that way we all end up developing a document
that quickly brings up the average quality material for developers.

That seems to me to suggest creating a wiki or similar for this
project. Why stop with half-measures? Let anyone contribute. Sections
can be marked as code-only, code and partial text, and code with
completed text; encourage people to flesh out the first two and
proofread and correct errors in the latter. Sections with completed
text would also have an uneditable canonical version that could be
added to the .pamphlet file, though still updated from time to time
with typo fixes and other corrections. (And the whole thing 

Re: Knuth's literate programming tangle function in Clojure

2011-01-06 Thread Ken Wesson
On Fri, Jan 7, 2011 at 12:07 AM, Tim Daly d...@axiom-developer.org wrote:
 Hmmm. I may have misunderstood your point. I thought you were suggesting
 writing code that is not part of the distribution in order to get a
 minimal running system and then working from that. If that is not what
 you're suggesting then I'm confused.

No, I was just suggesting that the order of the material put the stuff
in the distribution that's necessary to bootstrap a minimally
functional repl first, culminating in the eval function and the
command-line repl class, then flesh out the rest of Clojure's
feature-set with the rest of the stuff in the distribution. No new
code.

 The pamphlet sources are in a git repository so they are immutable.

 Wikis are fine for a lot of things but not for linearizing the
 ideas into a readable literate form. Books fulfill that role.

I suggested *maybe* letting the wiki users try to decide,
collectively, on a linearization; maybe that would prove workable and
maybe not. If not, you'd have to linearize it yourself to make the
book version. But if you're looking for section submissons and user
proof-reading a wiki can at least organize that activity, and can
provide seeds by having unwritten sections in there with just the
source code that is to be explained. And without potential
contributors maybe being put off by having to learn a whole extra set
of tools (namely, github and whatever client software) and get a login
at some site (github). Some might not have used git. A few might not
have used any code repository system. A wiki on the other hand can be
edited by anyone who can type stuff into a web form and can be
configured not to require a login (ala Wikipedia itself).

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-05 Thread Seth
The literate programming is actually a contrib to org-mode.
http://orgmode.org/worg/org-contrib/babel/

Ive actually used it to create my emacs.el, by having code in
emacs.org and have init.el tangle out the emacs code. Of course i
never documented
anything and did it for the novelty of being able to organize all that
code in one file, instead of expanding it to other files :)

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-05 Thread Seth
Just discovered org-mode myself --- does anyone know of guide to using
it with clojure for a total newbie?

I havent actually used it for clojure per se. I was just imagining how
it could be used. You have the ability to embed arbitrary code (from
many different languages). You can edit the code in its own emacs
major mode and then it will automatically be saved back once done. You
can then document it using org-modes awesome abilities.
However, this is sort of clumsy.

I would rather be able to have all of my code in all of its 'little
files' arranged in directories. And when im editing the clojure files,
i would like to be like 'oh, i want to document this better/introduce
the motivation etc! And then automatically have the code, or parts of
the code, copied to the org file and then i could document it. And
then jump back to the code to continue developing. And have changes in
the clojure file automatically reflected in the org file. I was
thinking that 'chunk' labels could be embedded in the source code
(like in marginalia in github: just comments like ;;##Block Name) so
that we wouldn't have to have all code in one file in one chunk, but
could split it up.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-05 Thread Tim Daly



On 1/5/2011 9:27 AM, Seth wrote:

Just discovered org-mode myself --- does anyone know of guide to using
it with clojure for a total newbie?

I havent actually used it for clojure per se. I was just imagining how
it could be used. You have the ability to embed arbitrary code (from
many different languages). You can edit the code in its own emacs
major mode and then it will automatically be saved back once done. You
can then document it using org-modes awesome abilities.
However, this is sort of clumsy.

I would rather be able to have all of my code in all of its 'little
files' arranged in directories.

Just out of curiosity, what is the advantage of maintaining code
in 'little files'? The main reason people use an IDE is that they
get a whole-project view. The IDE lets you move around and find
things as though your project was effectively one big file. It really
is a kind of patch on the little-file organization. What is it about
'little file format' that is actually useful? Except for habit, is
there a real advantage?

Conceptually separate efforts, such as the Clojure-contrib effort
could be done in separate volumes. But you would expect this kind of
natural organization, just as you might expect a multi-volume story
like the Asimov Foundation series. Wouldn't a well organized contrib
literate document be useful? When you program and reach for your
books, don't you find a good index and cross-reference the best way
to find the idea you need?


And when im editing the clojure files,
i would like to be like 'oh, i want to document this better/introduce
the motivation etc! And then automatically have the code, or parts of
the code, copied to the org file and then i could document it. And
then jump back to the code to continue developing.

Viewing the step of 'documenting the code' as just another step in
development is one of the reasons that documentation is rarely done
and even more rarely done well. The target audience is usually another
developer so tools like Javadoc exist to make it easy to look up class
details. What tools exist to support whole-project, idea-to-implementation
documentation?

Viewing documentation as a phase of development is conceptually and
actually very different from viewing the project as a literate effort.
Development targets the machine whereas a literate effort targets people.
The difference in target audience makes a qualitative difference in what
you write, when you write it, and why you write it. There is nothing
about the Javadoc organization (to pick on one tool) that encourages or
even supports the 'idea-to-implementation' flow. Are there open source
examples of documentation from the ideas to the implementation?

Literate programming allows you to reorganize your code by ideas. For
instance, in the clojure example, the PersistentTreeMap class is split
into its subclasses like Node. Okasaki's work starts with the idea of
Nodes so we highlight and explain the Node structure of PersistentTreeMap
before we get into the top level class details. In this way you motivate
the need for the Node class and 'bring the reader along' so that when
they get to PersistentTreeMap they already understand Nodes.

Because of the way Java forces you to organize your code you have to
introduce the PersistentTreeMap class before you introduce the Node class.
This is the late 90s and we ought to be able to organize our code any
way we want rather than be forced to organize it for the compilers,
linkers, and loaders. Why would we want to organize our code for the
convenience of our tools?



And have changes in
the clojure file automatically reflected in the org file. I was
thinking that 'chunk' labels could be embedded in the source code
(like in marginalia in github: just comments like ;;##Block Name) so
that we wouldn't have to have all code in one file in one chunk, but
could split it up.


Having literate code in more than one file is certainly possible because
Latex supports an 'include' command. You could include the code chunks or
you could include the chapters and keep them in 'litte file format'.

I'm not sure what advantage this confers. Working in a single file or
working in multiple files is pretty transparent. Emacs lets me split
buffers in the same file as easily as having two files in split windows.
Finding things is SO much easier (hey, its certainly in THIS file :-) )
The hardest initial part is using apt-get texlive.

The difference is not the physical organization but the mental organization.
A single file format gives the impression of a book and with that comes the
skills of organizing the material for presenting the ideas in a logical
fashion. It is this change of mental viewpoint that is the critical part
of literate programming rather than the tools like emacs-org-mode or latex.

The mental transition to this style of programming is at least as hard as
the mental transition from Object Oriented programming to Functional 
programming.

In my experience, the gain is 

Re: Knuth's literate programming tangle function in Clojure

2011-01-05 Thread Eric Schulte
Seth wbu...@gmail.com writes:

 The literate programming is actually a contrib to org-mode.
 http://orgmode.org/worg/org-contrib/babel/


This has been moved out of contrib and into the Org-mode core, so with
recent versions of Org-mode the code block Literate Programming and
Reproducible Research support is built in.

In fact when Emacs 24 is released this will be part of the Emacs core.


 Ive actually used it to create my emacs.el, by having code in
 emacs.org and have init.el tangle out the emacs code. Of course i
 never documented
 anything and did it for the novelty of being able to organize all that
 code in one file, instead of expanding it to other files :)

I do this myself and find it very convenient.  In fact I maintain a
Literate fork of Phil Hagelberg's emacs-starter-kit which does exactly
this, allowing you to keep you emacs customizations in either .org files
or .el files.

The git repo for this is here
https://github.com/eschulte/emacs-starter-kit

and the documentation (exported from the literate .org files) is here
http://eschulte.github.com/emacs-starter-kit/

Cheers -- Eric

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-05 Thread Eric Schulte
Hi,

Seth wbu...@gmail.com writes:

Just discovered org-mode myself --- does anyone know of guide to using
it with clojure for a total newbie?

 I havent actually used it for clojure per se. I was just imagining how
 it could be used. You have the ability to embed arbitrary code (from
 many different languages). You can edit the code in its own emacs
 major mode and then it will automatically be saved back once done. You
 can then document it using org-modes awesome abilities.
 However, this is sort of clumsy.


There are a variety of options here

- you can write *all* of your code in a single large Org-mode file, and
  tangle out .clj files for compilation.

- you can write *all* of your code in .clj files, and simply link to the
  code from your .org files

- you can write some code in external .clj files, and some embedded in
  .org files

- with current versions of Org-mode it is even possible to propagate
  changes from a tangled .clj file back into the code blocks in a .org
  file if e.g. you are working on a project with non-org users who would
  rather edit the .clj files directly.  See the `org-babel-detangle'
  function.


 I would rather be able to have all of my code in all of its 'little
 files' arranged in directories. And when im editing the clojure files,
 i would like to be like 'oh, i want to document this better/introduce
 the motivation etc! And then automatically have the code, or parts of
 the code, copied to the org file and then i could document it. And
 then jump back to the code to continue developing. And have changes in
 the clojure file automatically reflected in the org file. I was
 thinking that 'chunk' labels could be embedded in the source code
 (like in marginalia in github: just comments like ;;##Block Name) so
 that we wouldn't have to have all code in one file in one chunk, but
 could split it up.

I am a grad student and spend much of my time writing code and running
experiments in Clojure.  I do all of this in an environment of mixed
.org and .clj files.  I find I prefer to write larger libraries directly
in .clj files, but then I often embed the snippets of code required for
running experiments, generating tables/graphs and analyzing experimental
results in code blocks embedded in Org-mode files.  From these code
blocks I can either tangle the clojure code out into executable scripts,
or execute it /in situ/ in the .org file with the results dumped
directly into my org-mode buffer.

I find this to be a *very* comfortable research and development
environment, although as one of the main developers of the code block
support for Org-mode I'm certainly biased.

Cheers -- Eric

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-05 Thread Eric Schulte
Hi Tim,

I'm confused as to what parts of LP practice are not supported by
Org-mode.  Are you aware that Org-mode files can be exported to formats
more suitable for publication and human consumption (e.g. woven).  See
http://orgmode.org/manual/Exporting.html

Tim Daly d...@axiom-developer.org writes:

  I looked at org-mode.

 Note that 'literate programming' involves writing literature
 for other people to read. The executable code is included as
 a 'reduction to practice' but the emphasis is on describing
 the ideas. Rich has some powerful ideas that he has reduced
 to running code. What we need to do is start with a description
 of the ideas and bridge the gap to the actual implementation.

 Ideally you can read a literate program like a novel, from
 beginning to end, and find that every line of code has a
 'motivation' for being introduced. The side-effect is that
 there is a reason why the idea is implemented in a particular
 way rather than 'just because it worked'. Literate programming
 tends to improve code quality because you have to explain it.

 Emacs org-mode, on the other hand, is a useful development
 technology but it really isn't literate programming.


I would be interested to hear your thoughts as to why Org-mode is not a
literate programming tool.

Thanks -- Eric


 Tim Daly

 On 1/4/2011 9:34 AM, Seth wrote:
 have you guys checked out org-mode + babel for emacs? This would be an
 excellent place to start  to do literate programming. Interesting
 ideas ... maybe i will try this in my own code ...


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-05 Thread Eric Schulte
For the most up-to-date and comprehensive documentation of using
Org-mode to work with code blocks (e.g. Literate Programming or
Reproducible Research) the online manual is also very useful.

http://orgmode.org/manual/Working-With-Source-Code.html

also, for a good review of Org-mode's support for the practices of
Literate Programming and Reproducible Research, see this draft
manuscript (currently in submission).

http://cs.unm.edu/~eschulte/org-paper/
both the .org source and the exported .pdf are available

Cheers -- Eric

Hubert Iwaniuk neo...@kungfoo.pl writes:

 I would say start here:
 http://orgmode.org/worg/org-contrib/babel/languages/ob-doc-clojure.html

 Cheers,
 Hubert



 On Tue, Jan 4, 2011 at 4:12 PM, Robert McIntyre r...@mit.edu wrote:
 Just discovered org-mode myself --- does anyone know of guide to using
 it with clojure for a total newbie?

 sincerely,
 --Robert McIntyre

 On Tue, Jan 4, 2011 at 9:57 AM, Hubert Iwaniuk neo...@kungfoo.pl wrote:
 Hi Seth,

 Yes I did play with org-mode + babel for clojure.
 It works great :-)
 Just make sure you are using latest and greatest of org-mode.

 Cheers,
 Hubert.



 On Tue, Jan 4, 2011 at 3:34 PM, Seth wbu...@gmail.com wrote:
 have you guys checked out org-mode + babel for emacs? This would be an
 excellent place to start  to do literate programming. Interesting
 ideas ... maybe i will try this in my own code ...

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with 
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en


 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with 
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your 
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-05 Thread Tim Daly



On 1/5/2011 7:37 PM, Eric Schulte wrote:

Hi Tim,

I'm confused as to what parts of LP practice are not supported by
Org-mode.  Are you aware that Org-mode files can be exported to formats
more suitable for publication and human consumption (e.g. woven).  See
http://orgmode.org/manual/Exporting.html

I am truly impressed with the number of formats org-mode can
support. Of course, I expect nothing less as a heavy emacs user.

Tim Dalyd...@axiom-developer.org  writes:


  I looked at org-mode.

Note that 'literate programming' involves writing literature
for other people to read. The executable code is included as
a 'reduction to practice' but the emphasis is on describing
the ideas. Rich has some powerful ideas that he has reduced
to running code. What we need to do is start with a description
of the ideas and bridge the gap to the actual implementation.

Ideally you can read a literate program like a novel, from
beginning to end, and find that every line of code has a
'motivation' for being introduced. The side-effect is that
there is a reason why the idea is implemented in a particular
way rather than 'just because it worked'. Literate programming
tends to improve code quality because you have to explain it.

Emacs org-mode, on the other hand, is a useful development
technology but it really isn't literate programming.


I would be interested to hear your thoughts as to why Org-mode is not a
literate programming tool.

I never said org-mode wasn't a 'literate programming tool'. It is clearly an
outstanding version of a literate programming tool. What I said was that
org-mode really isn't literate programming.

I am trying to distinguish between tool and task.

Literate programming, as a tool, can be done with notepad.

Literate programming, as a task, is a radical change of mindset.
It is at least as difficult as going from Object Oriented programming
to Functional programming.

The point of the clojure.pamphlet file isn't to highlight how it
was created (emacs, fundamental mode). The point is to begin to
think about documentation as an ideas to implementation, speaking
from human to human, way of looking at the problem.

I made the machinery as simple as possible so people could experiment
with a new way of creating software. It is hardly new, and it isn't
my idea (see Knuth). I just have come to understand that it is a very
efficient and effective way to develop software that can live.

Clojure breaks with the past in many ways. I am advocating breaking
with the past in terms of the 'little files' idea, 'javadoc', and
other ways of documenting. And, since Advocacy is Volunteering, I
pretty much put myself into a position where I had to demonstrate
what I was advocating. Thus, the Clojure in Small Pieces book.



Thanks -- Eric


Tim Daly

On 1/4/2011 9:34 AM, Seth wrote:

have you guys checked out org-mode + babel for emacs? This would be an
excellent place to start  to do literate programming. Interesting
ideas ... maybe i will try this in my own code ...



--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-05 Thread Mark Engelberg
On Wed, Jan 5, 2011 at 4:44 PM, Eric Schulte schulte.e...@gmail.com wrote:
 For the most up-to-date and comprehensive documentation of using
 Org-mode to work with code blocks (e.g. Literate Programming or
 Reproducible Research) the online manual is also very useful.

In literate programming org-mode, will Clojure code be properly
highlighted and indented?
Is there a keystroke (like Ctrl-c,Ctrl-k) that will evaluate the
Clojure code in the entire file and send it to the swank REPL?
Will stacktraces point at the correct line number?

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-05 Thread Tim Daly



On 1/5/2011 8:27 PM, Mark Engelberg wrote:

On Wed, Jan 5, 2011 at 4:44 PM, Eric Schulteschulte.e...@gmail.com  wrote:

For the most up-to-date and comprehensive documentation of using
Org-mode to work with code blocks (e.g. Literate Programming or
Reproducible Research) the online manual is also very useful.

In literate programming org-mode, will Clojure code be properly
highlighted and indented?
Is there a keystroke (like Ctrl-c,Ctrl-k) that will evaluate the
Clojure code in the entire file and send it to the swank REPL?
Will stacktraces point at the correct line number?


It seems that your real question is whether Clojure knows about
a literate document. It does not. But it would be possible to
modify the reader behavior when given a pamphlet file. The REPL
uses a line numbering reader. Anything between the last
\end{chunk} and the next \begin{chunk} could be considered
as comments to be ignored but the line numbers for the function
would be correct and therefore the stack traces would be correct.

I suppose it would be reasonably easy (there is no such thing
as a simple job) to write a literate reader for the REPL. All
it would need to know is where to turn-on and turn-off the normal
read semantics.

When I get to documenting the REPL I will look at how reading
is done and think about writing a LiterateReader class.

Tim

--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-05 Thread Ken Wesson
On Wed, Jan 5, 2011 at 8:39 PM, Tim Daly d...@axiom-developer.org wrote:
 On 1/5/2011 8:27 PM, Mark Engelberg wrote:
 On Wed, Jan 5, 2011 at 4:44 PM, Eric Schulteschulte.e...@gmail.com
  wrote:

 For the most up-to-date and comprehensive documentation of using
 Org-mode to work with code blocks (e.g. Literate Programming or
 Reproducible Research) the online manual is also very useful.

 In literate programming org-mode, will Clojure code be properly
 highlighted and indented?
 Is there a keystroke (like Ctrl-c,Ctrl-k) that will evaluate the
 Clojure code in the entire file and send it to the swank REPL?
 Will stacktraces point at the correct line number?

 It seems that your real question is whether Clojure knows about
 a literate document. It does not. But it would be possible to
 modify the reader behavior when given a pamphlet file. The REPL
 uses a line numbering reader. Anything between the last
 \end{chunk} and the next \begin{chunk} could be considered
 as comments to be ignored but the line numbers for the function
 would be correct and therefore the stack traces would be correct.

 I suppose it would be reasonably easy (there is no such thing
 as a simple job) to write a literate reader for the REPL. All
 it would need to know is where to turn-on and turn-off the normal
 read semantics.

If we had custom reader macros in Clojure we wouldn't even be having
this discussion; you would probably have already implemented it by
now. :)

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-05 Thread Tim Daly



On 1/5/2011 10:20 PM, Ken Wesson wrote:

On Wed, Jan 5, 2011 at 8:39 PM, Tim Dalyd...@axiom-developer.org  wrote:

On 1/5/2011 8:27 PM, Mark Engelberg wrote:

On Wed, Jan 5, 2011 at 4:44 PM, Eric Schulteschulte.e...@gmail.com
  wrote:

For the most up-to-date and comprehensive documentation of using
Org-mode to work with code blocks (e.g. Literate Programming or
Reproducible Research) the online manual is also very useful.

In literate programming org-mode, will Clojure code be properly
highlighted and indented?
Is there a keystroke (like Ctrl-c,Ctrl-k) that will evaluate the
Clojure code in the entire file and send it to the swank REPL?
Will stacktraces point at the correct line number?


It seems that your real question is whether Clojure knows about
a literate document. It does not. But it would be possible to
modify the reader behavior when given a pamphlet file. The REPL
uses a line numbering reader. Anything between the last
\end{chunk} and the next \begin{chunk} could be considered
as comments to be ignored but the line numbers for the function
would be correct and therefore the stack traces would be correct.

I suppose it would be reasonably easy (there is no such thing
as a simple job) to write a literate reader for the REPL. All
it would need to know is where to turn-on and turn-off the normal
read semantics.

If we had custom reader macros in Clojure we wouldn't even be having
this discussion; you would probably have already implemented it by
now. :)


LispReader is a class that appears to have a read function that does
Clojure s-expression parsing. Wrapping that around a LiterateReader
stream would seem to do the job. The LiterateReader stream only has
to change any non-chunk line into a Clojure comment by prepending
a semicolon. The code to scan a line that begins with a \begin{chunk}
or \end{chunk} does not seem all that challenging.

In fact, a slightly smarter LiterateReader stream could be given the
particular line as an argument and only call LispReader on that
s-expression. So a LiterateReader with an optional line number
argument would allow an editor to specify where to start .read.

A reader macro would require special syntax. This may be reasonable
but it seems that a simple (load-literate filename N) would be
all that is needed, requiring no special syntax.

Alternatively the reader could use the file extension so that
pamphlet files would invoke the LiterateReader stream automatically.
Or if the parse of the first line begins with \ then use LiterateReader.

I'm documenting Bit-partitioned hash tries at the moment. I'll
see if I can document the reader next and get an idea of exactly
how it works and what it would take to change it.

Tim Daly

--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-05 Thread Seth
Now that i think of it, it is mostly a fear of having decreased
productivity in writing code that affected my statement that i liked
the little files. Im used to, i suppose, developing code for a
specific function in a file, being able to compile, goto line numbers
where there are errors,
send code to slime, etc. Looking over your example made things much
clearer. Its like your guiding your reader to specific parts of the
'little files', describing the theory behind them, moving on, etc. And
each code fragment has a chunk name associated with it, and all of
them are combined into the final .clj file using the code fragment
names (in a separate chunk).

At first, i thought this would be less productive than simply putting
all of the code in one clj file, but now that i think about it i think
it would, with the appropriate tools. And it wouldn't even be too
difficult, with org-mode (prefer it over latex any day!)

Im going to start transferring a subsection of my program to literate
programming, using org-mode. See how it goes...

Oh, and Tim, you might want to take a closer look at org-mode. Instead
of having to tangle out the code that builds everything, you could
create an executable shell script block in org-mode - the makefile
script could be tangled into a string using noweb syntax, and then
everything could go from there. You can execute the block by hitting C-
c c-c in the org file (or something like that). Pretty cool, in my
opinion!

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-05 Thread Eric Schulte
Mark Engelberg mark.engelb...@gmail.com writes:

 On Wed, Jan 5, 2011 at 4:44 PM, Eric Schulte schulte.e...@gmail.com wrote:
 For the most up-to-date and comprehensive documentation of using
 Org-mode to work with code blocks (e.g. Literate Programming or
 Reproducible Research) the online manual is also very useful.

 In literate programming org-mode, will Clojure code be properly
 highlighted and indented?

yes, Clojure code is displayed using the Emacs major mode for Clojure
code, so the appearance is as you would expect.  See this screenshot
from the file I am working on at the moment, the upper frame is a .clj
file and the lower is a .org file.  http://i.imgur.com/kdbDp.png

 
 Is there a keystroke (like Ctrl-c,Ctrl-k) that will evaluate the
 Clojure code in the entire file and send it to the swank REPL?

Yes, C-c C-c evaluates the code block under the point, and many other
keystrokes bind to various other functions [1], specifically C-c C-v b
executes the entire buffer.

 Will stacktraces point at the correct line number?

No, this is one of the reasons that I currently tend to do large-scale
development in .clj files and reserve embedded code for shorter chunks
of code.  That said I have successfully completed large clojure projects
in which the entirety of the code was tangled from a single literate
.org file.

Cheers -- Eric

Footnotes: 
[1]  M-x org-babel-describe-bindings

Major Mode Bindings Starting With C-c C-v:
key binding
--- ---

C-c C-v a   org-babel-sha1-hash
C-c C-v b   org-babel-execute-buffer
C-c C-v d   org-babel-demarcate-block
C-c C-v e   org-babel-execute-maybe
C-c C-v f   org-babel-tangle-file
C-c C-v g   org-babel-goto-named-src-block
C-c C-v h   org-babel-describe-bindings
C-c C-v i   org-babel-lob-ingest
C-c C-v l   org-babel-load-in-session
C-c C-v n   org-babel-next-src-block
C-c C-v o   org-babel-open-src-block-result
C-c C-v p   org-babel-previous-src-block
C-c C-v r   org-babel-goto-named-result
C-c C-v s   org-babel-execute-subtree
C-c C-v t   org-babel-tangle
C-c C-v u   org-babel-goto-src-block-head
C-c C-v v   org-babel-expand-src-block
C-c C-v x   org-babel-do-key-sequence-in-edit-buffer
C-c C-v z   org-babel-switch-to-session-with-code

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-05 Thread Eric Schulte

 Emacs org-mode, on the other hand, is a useful development
 technology but it really isn't literate programming.

 I would be interested to hear your thoughts as to why Org-mode is not a
 literate programming tool.
 I never said org-mode wasn't a 'literate programming tool'. It is clearly an
 outstanding version of a literate programming tool. What I said was that
 org-mode really isn't literate programming.


Ok, then it sounds like we're in agreement.  I just was confused by the
use of org-mode as a verb to describe one particular task when it
supports many disparate tasks, including LP as you have defined it.

Thanks for the explanation.

Cheers -- Eric

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-05 Thread Tim Daly



On 1/5/2011 10:58 PM, Seth wrote:

Now that i think of it, it is mostly a fear of having decreased
productivity in writing code that affected my statement that i liked
the little files. Im used to, i suppose, developing code for a
specific function in a file, being able to compile, goto line numbers
where there are errors,

Try inserting a syntax error anywhere in the code. Then type
'make' to the shell. You'll get a traceback that shows you the exact
line in the file that failed since the literate document is really
feeding Java files to the compiler. This is forced by Java since
there is a (bogus) connection between filename and contents.

In any case, you still get the same traceback you always got.


send code to slime, etc. Looking over your example made things much
clearer. Its like your guiding your reader to specific parts of the
'little files', describing the theory behind them, moving on, etc. And
each code fragment has a chunk name associated with it, and all of
them are combined into the final .clj file using the code fragment
names (in a separate chunk).

The REPL makes it much easier to develop lisp code in a literate
style because you can kill/yank an s-expression into a shell buffer
(or use slime).

My usual method of working is to build and test a function in the
REPL. Once it works I have the source already in the file so I can
just save it, build the whole system, run the tests, and make sure
I didn't break anything (it takes less than a minute).

At first, i thought this would be less productive than simply putting
all of the code in one clj file, but now that i think about it i think
it would, with the appropriate tools. And it wouldn't even be too
difficult, with org-mode (prefer it over latex any day!)

You see all of the source as one file. The compiler sees all of the
source in little files. The beauty of literate programming is that you
no longer have to pay attention to the compiler's failings.

Im going to start transferring a subsection of my program to literate
programming, using org-mode. See how it goes...

Oh, and Tim, you might want to take a closer look at org-mode. Instead
of having to tangle out the code that builds everything, you could
create an executable shell script block in org-mode - the makefile
script could be tangled into a string using noweb syntax, and then
everything could go from there. You can execute the block by hitting C-
c c-c in the org file (or something like that). Pretty cool, in my
opinion!

I used noweb (ref: Norman Ramsey) for years in Axiom. It is a useful
tool. However I finally understood that I can get rid of the 'weave'
function with a couple latex macros I wrote (see a prior post) and
I could get rid of the 'tangle' function by modifying the reader.
I patched lisp and build tangle directly into the image.
Thus, with some simple changes I no longer need any special tools.
That makes life simple and I like simple.

Org-mode sounds great and from what I've seen from the docs it does
everything but cook rice. I would highly recommend it as a tool if you
like that sort of thing. It would integrate well in a slime environment
if you like that sort of thing.

ANYTHING that helps you write literate programs is a win in my book.

I'm afraid that I have two personal failing that make org-mode unlikely.
I don't like modes (My emacs mode table is smashed to be fundamental mode
for everything). Editors should not change my source code.
Thus, org-mode is right-out, to quote Monty Python.

Second, I'm addicted to Latex. Latex is an amazing program, simply
stunning. I cannot imagine trying to write Clojure in Small Pieces
without it. It is just a markup language like HTML and thus not hard
to learn but it is also a turing complete language that has a huge
ecosystem of tools and techniques. I am creating the graphics for
CISP at the moment in Latex. I could do it using some other tool such
as gimp but life would not be as simple and the clojure.pamphlet file
would now need image files (more 'little files' cruft).

But whatever works for you is great. Please post an example of
a literate document using org-mode. We can then compare and contrast,
as my English teacher used to say. It would be interesting to see another
example of a literate document for Clojure. Slime and org-mode may
be the proper way to go.

Tim Daly

--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-05 Thread Tim Daly



On 1/5/2011 11:18 PM, Eric Schulte wrote:

Mark Engelbergmark.engelb...@gmail.com  writes:


On Wed, Jan 5, 2011 at 4:44 PM, Eric Schulteschulte.e...@gmail.com  wrote:

For the most up-to-date and comprehensive documentation of using
Org-mode to work with code blocks (e.g. Literate Programming or
Reproducible Research) the online manual is also very useful.

In literate programming org-mode, will Clojure code be properly
highlighted and indented?

yes, Clojure code is displayed using the Emacs major mode for Clojure
code, so the appearance is as you would expect.  See this screenshot
from the file I am working on at the moment, the upper frame is a .clj
file and the lower is a .org file.  http://i.imgur.com/kdbDp.png


Is there a keystroke (like Ctrl-c,Ctrl-k) that will evaluate the
Clojure code in the entire file and send it to the swank REPL?

Yes, C-c C-c evaluates the code block under the point, and many other
keystrokes bind to various other functions [1], specifically C-c C-v b
executes the entire buffer.


Will stacktraces point at the correct line number?

No, this is one of the reasons that I currently tend to do large-scale
development in .clj files and reserve embedded code for shorter chunks
of code.  That said I have successfully completed large clojure projects
in which the entirety of the code was tangled from a single literate
.org file.

Can you post examples of these? I'd love to see some other examples.

Cheers -- Eric

Footnotes:
[1]  M-x org-babel-describe-bindings

Major Mode Bindings Starting With C-c C-v:
key binding
--- ---

C-c C-v a   org-babel-sha1-hash
C-c C-v b   org-babel-execute-buffer
C-c C-v d   org-babel-demarcate-block
C-c C-v e   org-babel-execute-maybe
C-c C-v f   org-babel-tangle-file
C-c C-v g   org-babel-goto-named-src-block
C-c C-v h   org-babel-describe-bindings
C-c C-v i   org-babel-lob-ingest
C-c C-v l   org-babel-load-in-session
C-c C-v n   org-babel-next-src-block
C-c C-v o   org-babel-open-src-block-result
C-c C-v p   org-babel-previous-src-block
C-c C-v r   org-babel-goto-named-result
C-c C-v s   org-babel-execute-subtree
C-c C-v t   org-babel-tangle
C-c C-v u   org-babel-goto-src-block-head
C-c C-v v   org-babel-expand-src-block
C-c C-v x   org-babel-do-key-sequence-in-edit-buffer
C-c C-v z   org-babel-switch-to-session-with-code



--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-05 Thread Tim Daly



On 1/5/2011 11:19 PM, Eric Schulte wrote:

Emacs org-mode, on the other hand, is a useful development
technology but it really isn't literate programming.


I would be interested to hear your thoughts as to why Org-mode is not a
literate programming tool.

I never said org-mode wasn't a 'literate programming tool'. It is clearly an
outstanding version of a literate programming tool. What I said was that
org-mode really isn't literate programming.


Ok, then it sounds like we're in agreement.  I just was confused by the
use of org-mode as a verb to describe one particular task when it
supports many disparate tasks, including LP as you have defined it.

So my comment was unclear. I so badly need an Editor-in-Chief to
scan my emails for clarity, punctuation, and other things literate
people ought have mastered by now. Sorry about that.

Thanks for the explanation.

Cheers -- Eric



--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-05 Thread Eric Schulte
 Can you post examples of these? I'd love to see some other examples.

Sure thing, check out this old version of a file which tangles out into
the directory layout expected by lein.
http://gitweb.adaptive.cs.unm.edu/?p=asm.git;a=blob;f=asm.org;h=f043a8c8b0a917f58b62bdeac4c0dca441b8e2cb;hb=HEAD

Also, this project has an org-mode front page with code examples, the
html woven from this front page is shown at
http://repo.or.cz/w/neural-net.git
and the raw org file is available here
http://repo.or.cz/w/neural-net.git/blob/HEAD:/neural-net.org

I'll have to check out clojure.pamphlet, it sounds like an elegant
alternative.  It's always interesting to see other solutions in this
space.  For example I think scribble is a nice tool from the scheme
world.  http://lambda-the-ultimate.org/node/4017

Best -- Eric

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-05 Thread Tim Daly



On 1/6/2011 12:03 AM, Eric Schulte wrote:

Can you post examples of these? I'd love to see some other examples.

Sure thing, check out this old version of a file which tangles out into
the directory layout expected by lein.
http://gitweb.adaptive.cs.unm.edu/?p=asm.git;a=blob;f=asm.org;h=f043a8c8b0a917f58b62bdeac4c0dca441b8e2cb;hb=HEAD

I see that this file is using chunk markup but I don't see
the ideas-to-implementation (I2I) explanation. That is, I
don't see text that is written so someone like me can read the
text, understand the ideas and how they translate to code.


Also, this project has an org-mode front page with code examples, the
html woven from this front page is shown at
http://repo.or.cz/w/neural-net.git
and the raw org file is available here
http://repo.or.cz/w/neural-net.git/blob/HEAD:/neural-net.org

I love the graphics in this example! But, alas, this also seems
to be disconnected from the I2I property. But this is VERY
encouraging since it shows that you can get beautifully formatted
documents from the source.

I'll have to check  out clojure.pamphlet, it sounds like an elegant
alternative.  It's always interesting to see other solutions in this
space.  For example I think scribble is a nice tool from the scheme
world.  http://lambda-the-ultimate.org/node/4017

Take a look at http://daly.axiom-developer.org/clojure.pdf

The key difference, at least from my perspective, is that the CISP
is (well, is slowly being) organized around ideas. The goal is to
introduce the ideas and explain them in enough detail to *motivate*
the code that follows the explanation.

If you handed your printed document (or pdf) to someone will they
come away with enough understanding of neural networks to be able
to explain your code? Can they read the pdf and know what to change
and where to change it? Do they have literature references to follow
for deeper information? Is there a usable index with proper cross
references so they can hyperlink to the explanation and its associated
code block? These things seem, in my opinion, necessary (but not
sufficient) for the I2I property to hold.

I think that the tool you have is very, very nice. It shows me that
it would make a proper substitute for my beloved Latex as a viable
alternative for literate programming.

Tim Daly


--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-04 Thread Seth
have you guys checked out org-mode + babel for emacs? This would be an
excellent place to start  to do literate programming. Interesting
ideas ... maybe i will try this in my own code ...

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-04 Thread Hubert Iwaniuk
Hi Seth,

Yes I did play with org-mode + babel for clojure.
It works great :-)
Just make sure you are using latest and greatest of org-mode.

Cheers,
Hubert.



On Tue, Jan 4, 2011 at 3:34 PM, Seth wbu...@gmail.com wrote:
 have you guys checked out org-mode + babel for emacs? This would be an
 excellent place to start  to do literate programming. Interesting
 ideas ... maybe i will try this in my own code ...

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your 
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-04 Thread Robert McIntyre
Just discovered org-mode myself --- does anyone know of guide to using
it with clojure for a total newbie?

sincerely,
--Robert McIntyre

On Tue, Jan 4, 2011 at 9:57 AM, Hubert Iwaniuk neo...@kungfoo.pl wrote:
 Hi Seth,

 Yes I did play with org-mode + babel for clojure.
 It works great :-)
 Just make sure you are using latest and greatest of org-mode.

 Cheers,
 Hubert.



 On Tue, Jan 4, 2011 at 3:34 PM, Seth wbu...@gmail.com wrote:
 have you guys checked out org-mode + babel for emacs? This would be an
 excellent place to start  to do literate programming. Interesting
 ideas ... maybe i will try this in my own code ...

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your 
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en


 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your 
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-04 Thread Hubert Iwaniuk
I would say start here:
http://orgmode.org/worg/org-contrib/babel/languages/ob-doc-clojure.html

Cheers,
Hubert



On Tue, Jan 4, 2011 at 4:12 PM, Robert McIntyre r...@mit.edu wrote:
 Just discovered org-mode myself --- does anyone know of guide to using
 it with clojure for a total newbie?

 sincerely,
 --Robert McIntyre

 On Tue, Jan 4, 2011 at 9:57 AM, Hubert Iwaniuk neo...@kungfoo.pl wrote:
 Hi Seth,

 Yes I did play with org-mode + babel for clojure.
 It works great :-)
 Just make sure you are using latest and greatest of org-mode.

 Cheers,
 Hubert.



 On Tue, Jan 4, 2011 at 3:34 PM, Seth wbu...@gmail.com wrote:
 have you guys checked out org-mode + babel for emacs? This would be an
 excellent place to start  to do literate programming. Interesting
 ideas ... maybe i will try this in my own code ...

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with 
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en


 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your 
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your 
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-04 Thread Tim Daly

 The latest version of clojure.pamphlet can build Clojure
directly from the book. It dynamically builds the source
tree from the book, runs tests, creates the pdf, and
starts the REPL.

At least in theory. I am stuck with running a couple
tests. The only real change I've made to the sources
is to make it fit a printable page which involves
changing a line to make it shorter.

I've run into a syntax for strings that I don't understand.
The string #some string is used in the test files. The
documentation on the reader does not list this as a possible
input case. What does it mean?

Once I cross this hurdle everything else works and I can
post a new version for your experiments.

Tim

On 1/4/2011 10:49 AM, Hubert Iwaniuk wrote:

I would say start here:
http://orgmode.org/worg/org-contrib/babel/languages/ob-doc-clojure.html

Cheers,
Hubert



On Tue, Jan 4, 2011 at 4:12 PM, Robert McIntyrer...@mit.edu  wrote:

Just discovered org-mode myself --- does anyone know of guide to using
it with clojure for a total newbie?

sincerely,
--Robert McIntyre

On Tue, Jan 4, 2011 at 9:57 AM, Hubert Iwaniukneo...@kungfoo.pl  wrote:

Hi Seth,

Yes I did play with org-mode + babel for clojure.
It works great :-)
Just make sure you are using latest and greatest of org-mode.

Cheers,
Hubert.



On Tue, Jan 4, 2011 at 3:34 PM, Sethwbu...@gmail.com  wrote:

have you guys checked out org-mode + babel for emacs? This would be an
excellent place to start  to do literate programming. Interesting
ideas ... maybe i will try this in my own code ...

--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en



--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-04 Thread Jeff Valk
On Tuesday, January 04, 2011 at 10:18 am, Tim Daly wrote:
 I've run into a syntax for strings that I don't understand.
 The string #some string is used in the test files. The
 documentation on the reader does not list this as a possible
 input case. What does it mean?

It's reader syntax for a regular expression.

user= (type #some string)
java.util.regex.Pattern

It and its reader macro friends can be found here:
http://clojure.org/reader

 Once I cross this hurdle everything else works and I can
 post a new version for your experiments.

Thanks for your efforts on this. I'm quite interested in its potential!

- Jeff

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-04 Thread Robert McIntyre
the # is a reader macro for regexes.

hope that helps,
--Robert McIntyre

On Tue, Jan 4, 2011 at 11:18 AM, Tim Daly d...@axiom-developer.org wrote:
  The latest version of clojure.pamphlet can build Clojure
 directly from the book. It dynamically builds the source
 tree from the book, runs tests, creates the pdf, and
 starts the REPL.

 At least in theory. I am stuck with running a couple
 tests. The only real change I've made to the sources
 is to make it fit a printable page which involves
 changing a line to make it shorter.

 I've run into a syntax for strings that I don't understand.
 The string #some string is used in the test files. The
 documentation on the reader does not list this as a possible
 input case. What does it mean?

 Once I cross this hurdle everything else works and I can
 post a new version for your experiments.

 Tim

 On 1/4/2011 10:49 AM, Hubert Iwaniuk wrote:

 I would say start here:
 http://orgmode.org/worg/org-contrib/babel/languages/ob-doc-clojure.html

 Cheers,
 Hubert



 On Tue, Jan 4, 2011 at 4:12 PM, Robert McIntyrer...@mit.edu  wrote:

 Just discovered org-mode myself --- does anyone know of guide to using
 it with clojure for a total newbie?

 sincerely,
 --Robert McIntyre

 On Tue, Jan 4, 2011 at 9:57 AM, Hubert Iwaniukneo...@kungfoo.pl  wrote:

 Hi Seth,

 Yes I did play with org-mode + babel for clojure.
 It works great :-)
 Just make sure you are using latest and greatest of org-mode.

 Cheers,
 Hubert.



 On Tue, Jan 4, 2011 at 3:34 PM, Sethwbu...@gmail.com  wrote:

 have you guys checked out org-mode + babel for emacs? This would be an
 excellent place to start  to do literate programming. Interesting
 ideas ... maybe i will try this in my own code ...

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en


 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-04 Thread Tim Daly

 I thought it was also but it appears to be used in place
of string, which I thought was odd. I'll look again.
Thanks for the answers.

On 1/4/2011 11:38 AM, Robert McIntyre wrote:

the # is a reader macro for regexes.

hope that helps,
--Robert McIntyre

On Tue, Jan 4, 2011 at 11:18 AM, Tim Dalyd...@axiom-developer.org  wrote:

  The latest version of clojure.pamphlet can build Clojure
directly from the book. It dynamically builds the source
tree from the book, runs tests, creates the pdf, and
starts the REPL.

At least in theory. I am stuck with running a couple
tests. The only real change I've made to the sources
is to make it fit a printable page which involves
changing a line to make it shorter.

I've run into a syntax for strings that I don't understand.
The string #some string is used in the test files. The
documentation on the reader does not list this as a possible
input case. What does it mean?

Once I cross this hurdle everything else works and I can
post a new version for your experiments.

Tim

On 1/4/2011 10:49 AM, Hubert Iwaniuk wrote:

I would say start here:
http://orgmode.org/worg/org-contrib/babel/languages/ob-doc-clojure.html

Cheers,
Hubert



On Tue, Jan 4, 2011 at 4:12 PM, Robert McIntyrer...@mit.eduwrote:

Just discovered org-mode myself --- does anyone know of guide to using
it with clojure for a total newbie?

sincerely,
--Robert McIntyre

On Tue, Jan 4, 2011 at 9:57 AM, Hubert Iwaniukneo...@kungfoo.plwrote:

Hi Seth,

Yes I did play with org-mode + babel for clojure.
It works great :-)
Just make sure you are using latest and greatest of org-mode.

Cheers,
Hubert.



On Tue, Jan 4, 2011 at 3:34 PM, Sethwbu...@gmail.comwrote:

have you guys checked out org-mode + babel for emacs? This would be an
excellent place to start  to do literate programming. Interesting
ideas ... maybe i will try this in my own code ...

--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with
your first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with
your first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with
your first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-04 Thread Tim Daly

 I looked at org-mode.

Note that 'literate programming' involves writing literature
for other people to read. The executable code is included as
a 'reduction to practice' but the emphasis is on describing
the ideas. Rich has some powerful ideas that he has reduced
to running code. What we need to do is start with a description
of the ideas and bridge the gap to the actual implementation.

Ideally you can read a literate program like a novel, from
beginning to end, and find that every line of code has a
'motivation' for being introduced. The side-effect is that
there is a reason why the idea is implemented in a particular
way rather than 'just because it worked'. Literate programming
tends to improve code quality because you have to explain it.

Emacs org-mode, on the other hand, is a useful development
technology but it really isn't literate programming.

Tim Daly

On 1/4/2011 9:34 AM, Seth wrote:

have you guys checked out org-mode + babel for emacs? This would be an
excellent place to start  to do literate programming. Interesting
ideas ... maybe i will try this in my own code ...



--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2011-01-04 Thread Tim Daly

 The new version of Clojure in Small Pieces is up at:

http://daly.axiom-developer.org/clojure.pdf
http://daly.axiom-developer.org/clojure.pamphlet
http://daly.axiom-developer.org/clojure.sty

This version of the literate document contains a
complete, working system. The steps for building
it are in the preface.

Essentially you compile the tangle function from
the document (or use the same source code here:
http://daly.axiom-developer.org/tangle.c )

Then you run tangle to extract the Makefile.
Then you type make.

Or, for the programmers:

1) edit the file, clip out and save tangle.c
2) gcc -o tangle tangle.c
3) tangle clojure.pamphlet Makefile Makefile
4) make

It should extract the sources, build Clojure,
test it, build the pdf, and leave you at a
REPL prompt.

The source tree lives under the 'tpd' directory.
You can put it anywhere with an argument to make, e.g.

4) make WHERE=myplace

This means that you only need the latex document
to develop (resist the urge to edit the other
files).

Now the problem is to write the ideas and connect
them to the code. I started doing this for the
Red Black tree idea and PersistentTreeMap. Feel
free to pick an idea (or suggest one) and work
out the details.

I urge you to try the edit/build cycle using
literate tools as a possible different way to
work. My usual command line after every change is:

rm -rf tpd  tangle clojure.pamphlet Makefile Makefile  make

A complete rebuild from scratch takes less than a
minute on a fast machine.

Tim

--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2010-12-26 Thread Robert McIntyre
That's really cool. I was _just reading_ your comments from 2006 at
http://www.mail-archive.com/gardeners@lispniks.com/msg01006.html and
wondering about how hard something like this would be to write.  If
possible, could you expand on how one might use this in a development
work-flow with emacs or texmacs? Is there some sort of lisp-pamphlet
mode that could be extended to clojure? I'm assuming you've already
solved all these problems and more while working with Axiom. I'd
appreciate any pointers or best practices you might have found.

Thanks so much for your work!
--Robert McIntyre


On Sun, Dec 26, 2010 at 1:01 AM, Tim Daly d...@axiom-developer.org wrote:
  ;  0 AUTHOR and LICENSE
 ;  1 ABSTRACT and USE CASES
 ;  2 THE LATEX SUPPORT CODE
 ;  3 IMPORTS
 ;  4 THE TANGLE COMMAND
 ;  5 SAY
 ;  6 READ-FILE
 ;  7 ISCHUNK
 ;  8 HASHCHUNKS
 ;  9 EXPAND
 ; 10 TANGLE


 
 ;;; 0 AUTHOR and LICENSE

 ;;; Timothy Daly (d...@axiom-developer.org)
 ;;; License: Public Domain

 
 ;;; 1 ABSTRACT and USE CASES

 ;;; Don Knuth has defined literate programming as a combination of
 ;;; documentation and source code in a single file. The TeX language
 ;;; is documented this way in books. Knuth defined two functions
 ;;;    tangle - extract the source code from a literate file
 ;;;    weave  - extract the latex from a literate file

 ;;; This seems unnecessarily complex. Latex is a full programming
 ;;; language and is capable of defining environments that can
 ;;; handle code directly in Latex. Here we define the correct environment
 ;;; macros. Thus, the weave function is not needed.

 ;;; If this tangle function were added to Clojure then Clojure could
 ;;; read literate files in Latex format and extract the code. We create
 ;;; the necessary tangle function here.



 ;;; This program will extract the source code from a literate file.

 ;;; A literate lisp file contains a mixture of latex and lisp sources code.
 ;;; The file is intended to be in standard latex format. In order to
 ;;; delimit code chunks we define a latex chunk environment.

 ;;; Latex format files defines a newenvironment so that code chunks
 ;;; can be delimited by \begin{chunk}{name}  \end{chunk} blocks
 ;;; This is supported by the following latex code.

 ;;; So a trivial example of a literate latex file might look like
 ;;; (ignore the prefix semicolons. that's for lisp)

 ; this is a file that is in a literate
 ; form it has a chunk called
 ; \begin{chunk}{first chunk}
 ; THIS IS THE FIRST CHUNK
 ; \end{chunk}
 ; and this is a second chunk
 ; \begin{chunk}{second chunk}
 ; THIS IS THE SECOND CHUNK
 ; \end{chunk}
 ; and this is more in the first chunk
 ; \begin{chunk}{first chunk}
 ; \getchunk{second chunk}
 ; THIS IS MORE IN THE FIRST CHUNK
 ; \end{chunk}
 ; \begin{chunk}{all}
 ; \getchunk{first chunk}
 ; \getchunk{second chunk}
 ; \end{chunk}
 ; and that's it

 ;;; From a file called testcase that contains the above text
 ;;; we want to extract the chunk names second chunk. We do this with:

 ; (tangle testcase second chunk)

 ; which yields:

 ; THIS IS THE SECOND CHUNK

 ;;; From the same file we might extract the chunk named first chunk.
 ;;; Notice that this has the second chunk embedded recursively inside.
 ;;; So we execute:

 ; (tangle testcase first chunk)

 ; which yields:

 ; THIS IS THE FIRST CHUNK
 ; THIS IS THE SECOND CHUNK
 ; THIS IS MORE IN THE FIRST CHUNK

 ;;; There is a third chunk called all which will extract both chunks:

 ; (tangle testcase all)

 ; which yields

 ; THIS IS THE FIRST CHUNK
 ; THIS IS THE SECOND CHUNK
 ; THIS IS MORE IN THE FIRST CHUNK
 ; THIS IS THE SECOND CHUNK

 ;;; The tangle function takes a third argument which is the name of
 ;;; an output file. Thus, you can write the same results to a file with:

 ; (tangle testcase all outputfile)

 ;;; It is also worth noting that all chunks with the same name will be
 ;;; merged into one chunk so it is possible to split chunks in mulitple
 ;;; parts and have them extracted as one. That is,

 ; \begin{chunk}{a partial chunk}
 ; part 1 of the partial chunk
 ; \end{chunk}
 ; not part of the chunk
 ; \begin{chunk}{a partial chunk}
 ; part 2 of the partial chunk
 ; \end{chunk}

 ;;; These will be combined on output as a single chunk. Thus

 ; (tangle testmerge a partial chunk)

 ; will yield

 ; part 1 of the partial chunk
 ; part 2 of the partial chunk


 
 ;;; 2 THE LATEX SUPPORT CODE

 ;;; The verbatim package quotes everything within its grasp and is used to
 ;;; hide and quote the source code during latex formatting. The verbatim
 ;;; environment is built in but the package form lets us use it in our
 ;;; chunk environment and it lets us change the font.
 ;;;
 ;;; \usepackage{verbatim}
 ;;;
 ;;; Make the verbatim font smaller
 ;;; Note that we have to 

Re: Knuth's literate programming tangle function in Clojure

2010-12-26 Thread Praki Prakash
Tim,

This approach is very interesting. My choice of mode for LP has always been
noweb-mode but it doesn't seem to work with my version of emacs anymore. My
current approach is to embed prose and clojure code in a latex document and
generate a .tex file with formatted clojure code and .clj containing only
clojure code. Needless to say, it is a hack and I would like to see if I can
adopt your approach.

However, I have a question on mapping of line numbers in clojure stacktrace
to its source. AFAIK, there is no support in clojure compiler for #LINE
directive. In my case, a code is always in one location and I just replace
latex lines with empty lines. How do you address this issue?

Thanks,
Praki
On Sat, Dec 25, 2010 at 10:01 PM, Tim Daly d...@axiom-developer.org wrote:

  ;  0 AUTHOR and LICENSE
 ;  1 ABSTRACT and USE CASES
 ;  2 THE LATEX SUPPORT CODE
 ;  3 IMPORTS
 ;  4 THE TANGLE COMMAND
 ;  5 SAY
 ;  6 READ-FILE
 ;  7 ISCHUNK
 ;  8 HASHCHUNKS
 ;  9 EXPAND
 ; 10 TANGLE




-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Re: Knuth's literate programming tangle function in Clojure

2010-12-26 Thread Tim Daly



On 12/26/2010 9:56 PM, Praki Prakash wrote:

Tim,

This approach is very interesting. My choice of mode for LP has always 
been noweb-mode but it doesn't seem to work with my version of emacs 
anymore. My current approach is to embed prose and clojure code in a 
latex document and generate a .tex file with formatted clojure code 
and .clj containing only clojure code. Needless to say, it is a hack 
and I would like to see if I can adopt your approach.

Well, I'm something of a primitivist so I don't use modes in Emacs.
I know there is a noweb mode and a clojure mode and a latex mode but
I've never used any of them so I can't comment.

In the Knuth web approach (ala noweb) the chunk markup is not valid
latex. Therefore you need to run weave to extract valid latex.
I found this pointless so I wrote the latex environment macros to
wrap the code chunks. This means that the document you are writing
is always pure latex. So you might find it useful to use the chunk
environment and skip the generate .tex file step.

All that is left is to extract the clojure code. While writing the
code I have a REPL open so I can kill/yank changes directly. The
full build/test cycle uses the attached code to extract from the
literate document (which I call a pamphlet).



However, I have a question on mapping of line numbers in clojure 
stacktrace to its source. AFAIK, there is no support in clojure 
compiler for #LINE directive. In my case, a code is always in one 
location and I just replace latex lines with empty lines. How do you 
address this issue?

I don't use the #LINE directive. I usually use a split screen
with lisp or clojure in a shell buffer (not a slime setup) and
my code in the other buffer. I've hand-checked the code in the
REPL before adding to or changing the document. After each change
I do a complete system rebuild/test cycle.

Thus, if it fails I know exactly what I broke and where I broke it.
It is broken at the last change, which is likely still in a buffer.
When it works, the code is already documented in latex and pdf (since
the build/test automatically regenerates the pdf).

I would like to see Clojure move toward literate programming and
have direct support for reading and compiling latex documents.
It is surprisingly efficient and effective. In my last 3 year
project I implemented over 60k lines of lisp and 6k pages of
documentation (with the embedded lisp code in literate form, of
course). When the program was done, the documentation was done.
In fact, that was true from the first day of the project and was
an invariant throughout. I highly recommend it.

Tim

--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en


Re: Knuth's literate programming tangle function in Clojure

2010-12-26 Thread Tim Daly



On 12/26/2010 8:33 PM, Robert McIntyre wrote:

That's really cool. I was _just reading_ your comments from 2006 at
http://www.mail-archive.com/gardeners@lispniks.com/msg01006.html and
wondering about how hard something like this would be to write.  If
possible, could you expand on how one might use this in a development
work-flow with emacs or texmacs? Is there some sort of lisp-pamphlet
mode that could be extended to clojure? I'm assuming you've already
solved all these problems and more while working with Axiom. I'd
appreciate any pointers or best practices you might have found.

Wow. That's an old reference. Most of those links are dead. Try
http://en.wikipedia.org/wiki/Axiom_computer_algebra_system
or http://axiom-developer.org

As I mentioned in my other email on this subject I don't use Emacs
modes. I don't want the computer editing my text. I just want it to
do exactly what I told it to do. But I'm a curmudgeon :-)

My style of programming is to change one thing and then do a complete
system rebuild and test, which takes about 1 hour on my fastest machine,
5 on my slowest. I work a pipeline of changes so the build/test time
is overlapped with changes. I can have up to 5 machines doing build/test
while I'm working on the 6th change. (Git helps a LOT here. I strongly
recommend git also.)

I usually use a split screen with the literate latex document (I call
it a pamphlet) in one buffer (not a slime setup) and the REPL in the
other buffer. I use the REPL to hand-check the code before modifying
the pamphlet.

Thus, if (when :-)) it fails I know what I broke and where I broke it.
When it works, the code is already documented in latex and pdf.
The build/test automatically regenerates the pdf.

The goal is to make the smallest change possible and concentrate on getting
it right. It might require writing or updating test cases which are also
in the same literate document under a test chunk name. The automated build
extracts and runs all test chunks.

So the essential loop is:
  edit the pamphlet in a split buffer
  define the functions in the REPL buffer
  change the function
  hand-test the change in the REPL
  update the pamphlet with the changed function
  write/update the test code
  save the pamphlet
  do a full system build and test
  check that the test pass
  check the pdf documentation

The second step would be a lot easier if Clojure knew how to load
a latex file. I'm thinking of writing a load-pamphlet function to
do this. It could specify a chunk name to extract as in:
   (defn
 (load-pamphlet [file] (load-pamphlet file *))
 (load-pamphlet [file chunk] (tangle file chunk)))
Thus, to do step 2 above I just need to (load-pamphlet...) the file.
I could run the tests with (load-pamphlet file test)

\begin{rant}{
\tl;dr{Let's move programming out of the dark ages.}

We of the Clojure community are trying hard to move Lisp out of
the 60s programming model. We can also move programming out of
the 60s code-then-maybe-document model. The Knuth technology is
30+ years old.

When I started 40 years ago (http://www.dilbert.com/fast/2010-12-23/)
we had limits like 8k of memory so no file was bigger than 4k. Thus,
C programs were tiny piles of sand with include/overlay linkers/etc.
Few of you have ever worked on a fully loaded PDP 11/40. Why do you
program like you only have 4k?

Now we can have a machine with 128Gig of memory. Its about time for
programmers to lose the one function-one file idea based on keeping
the machine happy. We need to think about one-program-one-document to
keep the humans happy. For a perfect example, see the book:
Lisp In Small Pieces by Christian Queinnec [1]

Queinnec's book is my ultimate example. He MOTIVATES every piece of
code that gets introduced and he talks about everything. The code
contains a complete lisp system including the interpreter, compiler,
and garbage collector. If you want to know lisp, read this book.
If you want to learn literate programming, study this book.

I want Clojure In Small Pieces, a literate form of Clojure that
I can execute from the book. Then I can open to the chapter on
PersistentHashMap and read all about log32 tries, why they matter,
and how Clojure implements them, with literature references and a
good index. Oh, yeah, and the ACTUAL code that gets executed.
Forget the jar file. Send me the pamphlet containing Clojure.

Programming will come out of the dark ages when we employ English
majors as project leads with the title Editor in Chief. If you
can explain it to an English major in text, you probably understand
the problem :-)

Let's move programming out of the dark ages.

\end{rant}

Tim Daly

[1] http://www.amazon.com/Lisp-Small-Pieces-Christian-Queinnec/dp/0521545668


--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe 

Knuth's literate programming tangle function in Clojure

2010-12-25 Thread Tim Daly

 ;  0 AUTHOR and LICENSE
;  1 ABSTRACT and USE CASES
;  2 THE LATEX SUPPORT CODE
;  3 IMPORTS
;  4 THE TANGLE COMMAND
;  5 SAY
;  6 READ-FILE
;  7 ISCHUNK
;  8 HASHCHUNKS
;  9 EXPAND
; 10 TANGLE



;;; 0 AUTHOR and LICENSE

;;; Timothy Daly (d...@axiom-developer.org)
;;; License: Public Domain


;;; 1 ABSTRACT and USE CASES

;;; Don Knuth has defined literate programming as a combination of
;;; documentation and source code in a single file. The TeX language
;;; is documented this way in books. Knuth defined two functions
;;;tangle - extract the source code from a literate file
;;;weave  - extract the latex from a literate file

;;; This seems unnecessarily complex. Latex is a full programming
;;; language and is capable of defining environments that can
;;; handle code directly in Latex. Here we define the correct environment
;;; macros. Thus, the weave function is not needed.

;;; If this tangle function were added to Clojure then Clojure could
;;; read literate files in Latex format and extract the code. We create
;;; the necessary tangle function here.



;;; This program will extract the source code from a literate file.

;;; A literate lisp file contains a mixture of latex and lisp sources code.
;;; The file is intended to be in standard latex format. In order to
;;; delimit code chunks we define a latex chunk environment.

;;; Latex format files defines a newenvironment so that code chunks
;;; can be delimited by \begin{chunk}{name}  \end{chunk} blocks
;;; This is supported by the following latex code.

;;; So a trivial example of a literate latex file might look like
;;; (ignore the prefix semicolons. that's for lisp)

; this is a file that is in a literate
; form it has a chunk called
; \begin{chunk}{first chunk}
; THIS IS THE FIRST CHUNK
; \end{chunk}
; and this is a second chunk
; \begin{chunk}{second chunk}
; THIS IS THE SECOND CHUNK
; \end{chunk}
; and this is more in the first chunk
; \begin{chunk}{first chunk}
; \getchunk{second chunk}
; THIS IS MORE IN THE FIRST CHUNK
; \end{chunk}
; \begin{chunk}{all}
; \getchunk{first chunk}
; \getchunk{second chunk}
; \end{chunk}
; and that's it

;;; From a file called testcase that contains the above text
;;; we want to extract the chunk names second chunk. We do this with:

; (tangle testcase second chunk)

; which yields:

; THIS IS THE SECOND CHUNK

;;; From the same file we might extract the chunk named first chunk.
;;; Notice that this has the second chunk embedded recursively inside.
;;; So we execute:

; (tangle testcase first chunk)

; which yields:

; THIS IS THE FIRST CHUNK
; THIS IS THE SECOND CHUNK
; THIS IS MORE IN THE FIRST CHUNK

;;; There is a third chunk called all which will extract both chunks:

; (tangle testcase all)

; which yields

; THIS IS THE FIRST CHUNK
; THIS IS THE SECOND CHUNK
; THIS IS MORE IN THE FIRST CHUNK
; THIS IS THE SECOND CHUNK

;;; The tangle function takes a third argument which is the name of
;;; an output file. Thus, you can write the same results to a file with:

; (tangle testcase all outputfile)

;;; It is also worth noting that all chunks with the same name will be
;;; merged into one chunk so it is possible to split chunks in mulitple
;;; parts and have them extracted as one. That is,

; \begin{chunk}{a partial chunk}
; part 1 of the partial chunk
; \end{chunk}
; not part of the chunk
; \begin{chunk}{a partial chunk}
; part 2 of the partial chunk
; \end{chunk}

;;; These will be combined on output as a single chunk. Thus

; (tangle testmerge a partial chunk)

; will yield

; part 1 of the partial chunk
; part 2 of the partial chunk



;;; 2 THE LATEX SUPPORT CODE

;;; The verbatim package quotes everything within its grasp and is used to
;;; hide and quote the source code during latex formatting. The verbatim
;;; environment is built in but the package form lets us use it in our
;;; chunk environment and it lets us change the font.
;;;
;;; \usepackage{verbatim}
;;;
;;; Make the verbatim font smaller
;;; Note that we have to temporarily change the '@' to be just a character
;;; because the \verba...@font name uses it as a character
;;;
;;; \chardef\atcode=\catcode`\@
;;; \catcod...@=11
;;; \renewcommand{\verba...@font}{\ttfamily\small}
;;; \catcod...@=\atcode

;;; This declares a new environment named ``chunk'' which has one
;;; argument that is the name of the chunk. All code needs to live
;;; between the \begin{chunk}{name} and the \end{chunk}
;;; The ``name'' is used to define the chunk.
;;; Reuse of the same chunk name later concatenates the chunks

;;; For those of you who can't read latex this says:
;;; Make a new environment named chunk with one argument
;;; The first block is the code for the \begin{chunk}{name}
;;; The second block is the code for the \end{chunk}
;;; The %