Re: Big Excel (xlsx) file parsing (docjure, incanter...)
Have you tried increasing the heap size of your JVM using the -Xmx and -Xms options? We're loading some spreadsheets that are 80M in size and have to run with a ~1.5G heap to handle this... On Saturday, 7 September 2013 02:22:27 UTC+1, Stanislav Sobolev wrote: Hello guys. I have excel file with huge columns in there. When i used some plugin(docjure, or anything else) it shows me CompilerException java.lang.OutOfMemoryError: Java heap space, compiling How can i handle that big file without converting it to csv? Primary problem in xlsx structure, it doesnt have lines or something, unlike csv. -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Big Excel (xlsx) file parsing (docjure, incanter...)
orthogonal, perhaps helpful: I wrote a clojure (wrapper) library that streams data in-and-out of Excel quite easily (and other applications, mostly finance, that use DDE). especially useful if you need to monitor or updates changes to cells. https://github.com/tuddman/clj-dde feedback welcome. On Saturday, September 7, 2013 10:28:57 AM UTC-4, Stanislav Sobolev wrote: Allright, but if use SXSSF how can i stream excel file to that? https://github.com/ktsujister/clj-tsv2xls/blob/master/src/tsv2xls/core.clj Like in this example can anybody provide me code example of using that streaming please? with-open [out-stream (io/output-stream outfile) and further? суббота, 7 сентября 2013 г., 12:30:43 UTC+6 пользователь Vijay Kiran написал: Hi, I don't think docjure supports streaming options yet - but you can try using Apache POI streaming API - http://poi.apache.org/spreadsheet/index.html - that should help in reading large files. HTH, @vijaykiran On Saturday, September 7, 2013 3:22:27 AM UTC+2, Stanislav Sobolev wrote: Hello guys. I have excel file with huge columns in there. When i used some plugin(docjure, or anything else) it shows me CompilerException java.lang.OutOfMemoryError: Java heap space, compiling How can i handle that big file without converting it to csv? Primary problem in xlsx structure, it doesnt have lines or something, unlike csv. -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Big Excel (xlsx) file parsing (docjure, incanter...)
Hi, I don't think docjure supports streaming options yet - but you can try using Apache POI streaming API - http://poi.apache.org/spreadsheet/index.html - that should help in reading large files. HTH, @vijaykiran On Saturday, September 7, 2013 3:22:27 AM UTC+2, Stanislav Sobolev wrote: Hello guys. I have excel file with huge columns in there. When i used some plugin(docjure, or anything else) it shows me CompilerException java.lang.OutOfMemoryError: Java heap space, compiling How can i handle that big file without converting it to csv? Primary problem in xlsx structure, it doesnt have lines or something, unlike csv. -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Big Excel (xlsx) file parsing (docjure, incanter...)
Allright, but if use SXSSF how can i stream excel file to that? https://github.com/ktsujister/clj-tsv2xls/blob/master/src/tsv2xls/core.clj Like in this example can anybody provide me code example of using that streaming please? with-open [out-stream (io/output-stream outfile) and further? суббота, 7 сентября 2013 г., 12:30:43 UTC+6 пользователь Vijay Kiran написал: Hi, I don't think docjure supports streaming options yet - but you can try using Apache POI streaming API - http://poi.apache.org/spreadsheet/index.html - that should help in reading large files. HTH, @vijaykiran On Saturday, September 7, 2013 3:22:27 AM UTC+2, Stanislav Sobolev wrote: Hello guys. I have excel file with huge columns in there. When i used some plugin(docjure, or anything else) it shows me CompilerException java.lang.OutOfMemoryError: Java heap space, compiling How can i handle that big file without converting it to csv? Primary problem in xlsx structure, it doesnt have lines or something, unlike csv. -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
