The trick with these listish things is to not calculate the tail until you need it, and to throw away the head when you're done with it.
On Mon, Feb 2, 2009 at 7:06 PM, Keith Bennett <keithrbenn...@gmail.com> wrote: > > All - > > I'm curious to know how to minimize memory consumption in Clojure for > large lists. I did the following test in repl: > > (def d ()) > (dotimes [_ 100000] (def d (cons "x" d))) Here, you are keeping the head of the list in the "d" and do nothing but append to the "d" list and redefine "d" as the new, longer list. Instead, try defining "d" like this: (def d (take 100000 (repeat "x"))) And take a gander at how much memory that consumes. However, the ideal thing here is to not define "d" as a list at all. Make instead a function that produces the value of "d" on demand - you don't actually need "d", you need it's value. All "d" does is to reference the head of your list and thereby preventing it from being garbage collected. > > Then, I used VisualVM, an awesome free profiling tool (https:// > visualvm.dev.java.net/) to examine the results. It indicated that the > number of clojure.lang.PersistentList instances increased by 100,000. > Each instance appeared to consume 48 bytes (not including the actual > value, but only its reference, I presume). I don't think any were > eligible for garbage collection, because I initiated gc several times, > and they were not removed. (I know that gc() is not deterministic, > but am pretty sure that they would have been removed. Feel free to > correct me.) > > Thinking that maybe the special functions like map did some magic to > save memory, I tried this: > > (def a (map #(* 2 %) (range 100000))) > > The result was 100,000 clojure.lang.LazyCons objects, each of which > consuming 52 bytes. But not until the list was realized. However, your "a" reference will prevent this list from being garbage collected as well. > > Are there alternate strategies for building a large sequence that > would consume less memory per element? Don't keep the head around on any list (well, seq really) unless you really mean/need it. > > Thanks, > Keith > > > > -- Venlig hilsen / Kind regards, Christian Vest Hansen. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~----------~----~----~----~------~----~------~--~---
