On Feb 2, 10:06 am, Keith Bennett <keithrbenn...@gmail.com> wrote:
> I'm curious to know how to minimize memory consumption in Clojure for
> large lists. I did the following test in repl:
>
> (def d ())
> (dotimes [_ 100000] (def d (cons "x" d)))
Let me translate this into pseudocode for you:
Make d into a global reference wrapped with a mutex
Set d to point to an empty list
for i = 1 to 100000:
If the symbol d has not yet been defined previously:
Create it
Within a critical section protecting d:
Replace the value of d with (cons "x" d)
Note in particular that
1. You're probably doing it wrong if you call "def" more than twice
for the same symbol. "def" is NOT a replacement for SETF (in Common
Lisp) or "set!" (in Scheme). The point of Clojure is not to use
destructive updates whenever possible.
2. If you keep the head of a sequence, as in "(def a (map ...))", you
have to store all elements of the sequence in memory.
> Are there alternate strategies for building a large sequence that
> would consume less memory per element?
If you _build_ the sequence, you'll have to store the whole sequence.
If you construct the sequence lazily, only use one element of it at a
time, and don't keep that element anywhere, then you'll never need to
store the whole sequence. But if you store the head of the sequence
(e.g., with a def), then you'll have to store all elements of the
sequence that you visit. If you print the whole sequence (as you
would at the REPL), then you visit all the elements of the sequence.
Therefore, in keeping the head of the sequence and printing it, you're
storing the whole sequence in memory. That's where all the list
objects (a.k.a. conses) come from. The GC doesn't clean them up
because you still have references to all of them; they are still
reachable. You could do GC a million times and they would all stay.
That's how lazy seqs work. It has nothing to do with Clojure and
everything to do with the language construct. If you don't like it,
then check out "streams," which Rich is currently building. Streams
are ephemeral as long as you don't put their elements into a seq. Of
course that means you can only iterate over a stream once. Again,
that has nothing to do with Clojure and everything to do with the
language construct.
mfh
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To post to this group, send email to clojure@googlegroups.com
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~----------~----~----~----~------~----~------~--~---
