On Mon, Mar 16, 2009 at 4:56 PM, Brian Doyle <brianpdo...@gmail.com> wrote: > I've been using Clojure for about 6 months now and really like it. I am > somewhat new to multi-threading > and using any of the parallel features in Clojure though. I have a > situation where I need to convert > 7 files from CSV to XML. Each one of these files is about 180MB apiece in > size. I have dual core machine > with 2GB of RAM and would like some advice on the best strategy for > processing these files in a way that > really utilizes both cores and my memory to really speed up the > processing. I'm sure this isn't the best > way, but I've only come up with starting up two threads at first, having > each thread open up a file, > call line-seq on that file, write out the XML for each line and then go to > the next file when it's complete. Any > advice would be great. Thanks.
Most libraries that write XML build a data structure to represent the XML and then write it out. That doesn't work for writing out large XML documents because you'll run out of memory before you finish building the data structure. I don't know if there is a Clojure library that writes XML as you specify it. If there isn't then you should consider WAX, a Java library I wrote. You can learn more about it at http://ociweb.com/wax/. You could use it from Clojure. Check out the tutorial. It's a really simple library, it's really fast and it's very memory efficient! -- R. Mark Volkmann Object Computing, Inc. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~----------~----~----~----~------~----~------~--~---
