Hello, In line 9, you retain the head in a var.
Transform it into a function: (defn xml [] (-> large-file java.io/input-stream clojure.data.xml/source-seq)) HTH, Laurent 2014/1/9 Peter Ullah <peterul...@gmail.com> > > Hi Everyone, > > I am trying to write some Clojure to process a large (310M) XML file, in > doing so, the code seems to consume the heap. I wrote a smaller script to > simplify the problem but I have now reduced it to a point where I am just > confused with why the heap is being used up; I'm consuming a lazy sequence > and not (that I am aware) maintaining a reference to the head. I have also > removed Leiningen from the equation just in case the REPL was causing any > references to be kept. > > I have managed to process the file as desired by increasing -Xmx and -Xms > passed to the VM but it doesn't address the underlying problem and my > confusion. > > Here is the code: https://gist.github.com/anonymous/8332220 > > Here are some details: > > - XML file has 6,321,720 lines. > - Using clojure.data.xml/source-seq to create a lazy sequence, which > translates into ~15,000,000 elements in the sequence > - Keeping a counter to determine how far through the processing I have > got before failure... (i.e. GC out of memory errors) > - Increased available and initial heap size to such a point that the > processing completes. > - Used VisualVM to profile memory characteristics - see attachment > > > My problem is in understanding why the used heap is increasing and not > constant. > > Any help would be great. thanks. > > -- > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to the Google Groups > "Clojure" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to clojure+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.