Just started using Clojure, found myself asking a similar question. Was
writing programs operating as part of unix command shell pipes, so I wrote
a macro that did something like the perl diamond operator. It iterates over
a series of files and standard input, opening each in turn and reading line
by line. This enables code like:
(do-read-input-source-lines [x cmdline-args]
(println x))
where cmdline-args is a sequence of files. Useful for simple scripts. The
above expands roughly to:
(doseq [input-source# cmdline-args]
(with-open [rdr# (clojure.java.io/reader input-source#)]
(doseq [x (line-seq rdr#)]
(println x))))
Basically, iterate over the files, opening each and iterating over the
lines. There's a bit more to it to handle standard input correctly, but
that's essentially it.
This was fine until needing to aggregate across line and file boundaries.
For example, consider reading files containing weather data (time,
temperature, etc.), aggregating data, and outputting a summary line for
every hour interval (eg. average temp). Assume it is undesirable to read
entire files into memory, and the data used in an hour summary can cross
file boundaries. I've been trying to understand what the options are for
structuring code of this form. Approaches that seem obvious:
a) The doseq/with-open/doseq pattern as in the macro I wrote. This appears
to require mutable data structures for the aggregation and state
transitions. Straightforward, but not in keeping with Clojure's preferred
style.
b) loop-recur - Structurally similar to doseq solution. Would be something
like:
(loop [...]
(with-open [...]
(loop [...]
(recur ...)))
(recur ...))
Haven't tried coding this yet, but looks like it should work. Code could be
a little messy, but workable.
c) Lazy sequence - Something along the lines of read-lines described in
this thread. Would be quite nice, as it would fit smoothly with all the
sequence processing constructs available. The downside is that it's not
clear how to retain the nice resource management properties one gets from
the with-open style clauses. In particular, the guarantee that any open
files are closed in the event of exceptions.
My guess is that this type of interaction between sequences and resource
management has come up any of times before. Are there preferred ways to
handle it? Language facilities I haven't discovered yet?
--Jon
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
