[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

2018-12-04 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 krzysztof.rzymkow...@gmail.com changed: What|Removed |Added Status|NEW |RESOLVED Resol

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

2018-09-06 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #26 from PJ Fanning --- POI 4.0.0 and poi-shared-strings 1.0.0 are now released. -- You are receiving this mail because: You are the assignee for the bug. - To un

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

2018-09-05 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #25 from PJ Fanning --- https://github.com/pjfanning/poi-shared-strings is an extension to poi that may help - will be released as soon as POI 4.0.0 is released (it relies on changes in POI 4.0.0). -- You are receiving this mail b

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

2018-09-05 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #24 from Thamodharan --- @Fanning, In below link, https://poi.apache.org/changes.html under History of Changes Version 4.0.0-SNAPSHOT (2018-08-??) The change 61832 / Github-85 is not involved. As you replied to marion, can we e

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

2018-05-15 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #23 from PJ Fanning --- There have been no changes made to POI for this. 1. If you use inline strings, the issue appears to be that Excel itself baulks at the large dataset (eg Comment 7 above). 2. Comment 16 above highlights a work

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

2018-05-14 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #22 from Marion --- This is still an issue. Can anyone from the Apache team confirm this issue will be resolved with the new release? -- You are receiving this mail because: You are the assignee for the bug. --

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

2018-01-16 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #21 from Thamodharan --- Hi Fanning, I went through the pull request in github, really great. Is there a chance to expect complete code in upcoming version of apache poi ? -- You are receiving this mail because: You are the assig

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

2018-01-11 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #20 from PJ Fanning --- There is a Pull Request which uses temp files for constructing the shared strings table. https://github.com/apache/poi/pull/85 -- You are receiving this mail because: You are the assignee for the bug. -

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

2018-01-11 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #19 from Thamodharan --- My machine specs, Jboss server RAM 32 GB Heap size allocated to jboss 28 GB Harddisk 1 TB I am good to go with generating excel file upto a limit of 0.3 million X 200 rows with enabling shared string. Ahea

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

2018-01-11 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #18 from Thamodharan --- Hi mewalig, Kindly help us by explaining the calculation which you stated on your last reply. -- You are receiving this mail because: You are the assignee for the bug.

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

2017-12-27 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #17 from Thamodharan --- Hi mewalig, Does 1mm what you mentioned refer to 0.1 million or 1 million rows ? Can you please explain this calculation [(256+32)/8*150*1mm = 5.4GB of memory] in detail, so that it would be easy for me to

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

2017-12-18 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #16 from PJ Fanning --- I have the initial work done on a prototype version of SharedStringTable that can be used in SXSSF to reduce the memory footprint. The Pull Request is for discussion as opposed to suggesting this work is anyw

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

2017-12-18 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #15 from mewa...@gmail.com --- I do not agree with this at all: "Some tasks just take a large amount of RAM, which is cheap these days, including the ability to "rent" time and space on any number of cloud provider platforms [...]".

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

2017-12-18 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #14 from Thamodharan --- Though I adopt 64GB system RAM and 30GB to JBOSS heap space, I cant even produce two parallel workbook of such large size at a time. In case of adopting to write a patch, speed is concerned as you said. Is

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

2017-12-12 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #13 from Greg Woolsey --- At some point, you just have to allocate more heap, no way around it. Sounds like you might be there, needing to allow for a very large shared strings table. Some tasks just take a large amount of RAM, whi

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

2017-12-12 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #12 from Thamodharan --- (In reply to Yegor Kozlov from comment #11) > try to construct SXSSFWorkbook in a different way: > > wb = new SXSSFWorkbook( > null, /* template workbook. pass null to start with a blank workbook */ >

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

2017-12-08 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #11 from Yegor Kozlov --- try to construct SXSSFWorkbook in a different way: wb = new SXSSFWorkbook( null, /* template workbook. pass null to start with a blank workbook */ 1000, /* size of the sliding window*/ true,

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

2017-12-07 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #10 from Thamodharan --- Hi Yegor Kozlov, Hope, you mean to set this limit 10K as mentioned below. SXSSFWorkbook wb = new SXSSFWorkbook(1); Though I set this, I face content issue while opening it. I tried extending it to 30K

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

2017-12-06 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #9 from Yegor Kozlov --- In theory SXSSF can use a fixed-size cache of shared strings, e.g. we can allow SXSSF to put 10K distinct strings in the SST. If the cache is full then the string goes inline. The 10K limit can be configurab

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

2017-12-06 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #8 from Thamodharan --- Hi Greg Woolsey, SXSSF by default takes inline string, it can be changed to shared string for which some patch is available. Since Shared string takes the heap space memory, its difficult to opt for it, due

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

2017-12-04 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #7 from Greg Woolsey --- I ran the attached code, and examined the generated workbook. All XML validates successfully with xmllint. The worksheet file, however, expands to > 5GB due to all the inlineStr attributes for all the cell

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

2017-11-30 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #6 from Thamodharan --- (In reply to Yegor Kozlov from comment #5) > Can you please do one more test: generate a 500K rows x 150 columns > spreadsheet containing only numbers and see if it opens? Due to its > streaming nature SXSSF

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

2017-11-30 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #5 from Yegor Kozlov --- Can you please do one more test: generate a 500K rows x 150 columns spreadsheet containing only numbers and see if it opens? Due to its streaming nature SXSSF stores strings differently than Excel does: Exce

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

2017-11-30 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #4 from Thamodharan --- Created attachment 35571 --> https://bz.apache.org/bugzilla/attachment.cgi?id=35571&action=edit ExcelErrorAfterRepair -- You are receiving this mail because: You are the assignee for the bug.

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

2017-11-30 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #3 from Thamodharan --- (In reply to Yegor Kozlov from comment #2) > (In reply to Thamodharan from comment #1) > > Created attachment 35566 [details] > > Excel 2013 - Alert Message - While opening such large file > > Is it a 32 or

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

2017-11-30 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #2 from Yegor Kozlov --- (In reply to Thamodharan from comment #1) > Created attachment 35566 [details] > Excel 2013 - Alert Message - While opening such large file Is it a 32 or 64-bit Excel ? I wonder if it matters. What happens

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

2017-11-29 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #1 from Thamodharan --- Created attachment 35566 --> https://bz.apache.org/bugzilla/attachment.cgi?id=35566&action=edit Excel 2013 - Alert Message - While opening such large file -- You are receiving this mail because: You are t