The java overhead for Strings is incredible. Even moderate-sized input files will consume all memory. Are there good existing solutions?
I found iota, which looks like a good solution for the read portion of the problem. However I also need to process the data in the file. If I start with an iota/vec and need to sort it, something like (sort (iota/vec "foo")) will end up building Strings, again. The memory usage is absurd. It's also possible to use iota's random-access support instead of converting the whole collection to Strings, like (def f (iota/vec "foo")) (sort (fn [i j] (compare (nth f i) (nth f j))) (into [] (range (count f))))) to return a list of row indexes in sorted order. But this is painfully slow. iota doesn't appear to provide a solution for efficient storage of transformed collections of strings. Are there other libraries, or standard java techniques for dealing with this? -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
