[Libreoffice-bugs] [Bug 84246] Calc crashes on loading / saving large (~500MB) CSV files

[bugzilla-daemon]

https://bugs.documentfoundation.org/show_bug.cgi?id=84246

--- Comment #17 from John Nagle  ---
Update: saving as .ods took about 1.5 hours, but finished.
Opening file thus saved took about 5 minutes to open.

-- 
You are receiving this mail because:
You are the assignee for the bug.___
Libreoffice-bugs mailing list
Libreoffice-bugs@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/libreoffice-bugs

[Libreoffice-bugs] [Bug 84246] Calc crashes on loading / saving large (~500MB) CSV files

[bugzilla-daemon]

https://bugs.documentfoundation.org/show_bug.cgi?id=84246

--- Comment #16 from John Nagle  ---
File load time for 233MB file is only 30 seconds. Scrolling around in the file
works fine.  It's only "Save" that's slow.

Save as .ods is also extremely slow. It's not just .csv. So it's not the CSV
exporter.  About 5 minutes into that now. As before, 100% of 1 CPU in use, very
little I/O, no thrashing, not out of memory.

-- 
You are receiving this mail because:
You are the assignee for the bug.___
Libreoffice-bugs mailing list
Libreoffice-bugs@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/libreoffice-bugs

[Libreoffice-bugs] [Bug 84246] Calc crashes on loading / saving large (~500MB) CSV files

[bugzilla-daemon]

https://bugs.documentfoundation.org/show_bug.cgi?id=84246

--- Comment #15 from John Nagle  ---
Update: after over 2 hours, the writing of the CSV file finished. The output
file is valid. Calc is back to its normal operating state.

Strongly suggest profiling and looking for something that's O(N^2) in file
exporting.

-- 
You are receiving this mail because:
You are the assignee for the bug.___
Libreoffice-bugs mailing list
Libreoffice-bugs@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/libreoffice-bugs

[Libreoffice-bugs] [Bug 84246] Calc crashes on loading / saving large (~500MB) CSV files

[bugzilla-daemon]

https://bugs.documentfoundation.org/show_bug.cgi?id=84246

--- Comment #14 from John Nagle  ---
I'm seeing a similar bug.  Read a 233MB CSV file into Calc without problems.
Attempted to write it out as CSV with slightly different options ("quote
everything"), and it's half done after an hour.  About 2/3 done after an hour
and a half. Progress continues. Calc is in the "greyed out" state, not
processing mouse clicks. 

Since this is on Linux, it doesn't affect other processes much, and I'm letting
this run to see if it finishes. It's not hung; the progress bar advances
slowly. 

CPU utilization 100% of 1 CPU. Very little disk I/O. Memory not full. Not
thrashing. Very little I/O, maybe one disk write every 10 seconds.

This looks like there's something in writing out a CSV file that's worse than
O(1), so it chokes on large files.  Somebody with a debug build should profile
this while writing out a large CSV file and see where it's spending all the CPU
time.

Ubuntu 16.04LTS x64 8GB.  
Calc version: libreoffice-calc 1:5.1.6-rc2.0ubuntu1-xenial2

-- 
You are receiving this mail because:
You are the assignee for the bug.___
Libreoffice-bugs mailing list
Libreoffice-bugs@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/libreoffice-bugs

[Libreoffice-bugs] [Bug 84246] Calc crashes on loading / saving large (~500MB) CSV files

[bugzilla-daemon]

https://bugs.documentfoundation.org/show_bug.cgi?id=84246

Telesto  changed:

   What|Removed |Added

   See Also||https://bugs.documentfounda
   ||tion.org/show_bug.cgi?id=95
   ||346

-- 
You are receiving this mail because:
You are the assignee for the bug.___
Libreoffice-bugs mailing list
Libreoffice-bugs@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/libreoffice-bugs

[Libreoffice-bugs] [Bug 84246] Calc crashes on loading / saving large (~500MB) CSV files

[bugzilla-daemon]

https://bugs.documentfoundation.org/show_bug.cgi?id=84246

--- Comment #13 from Jean-Baptiste Faure  ---
I tried to export the attached csv file to .ods with LO 5.4.1.0.0+ built at
home under Ubuntu 16.04 x86-64 but it failed with error I/O message. No problem
to load the csv file, it was long but it worked.
The PC has 8 Go of RAM and a SSD 512 Go. System monitor showed that process
soffice.bin used 1,1 Go of RAM.

Best regards. JBF

-- 
You are receiving this mail because:
You are the assignee for the bug.___
Libreoffice-bugs mailing list
Libreoffice-bugs@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/libreoffice-bugs

[Libreoffice-bugs] [Bug 84246] Calc crashes on loading / saving large (~500MB) CSV files

[bugzilla-daemon]

https://bugs.documentfoundation.org/show_bug.cgi?id=84246

Telesto  changed:

   What|Removed |Added

   See Also||https://bugs.documentfounda
   ||tion.org/show_bug.cgi?id=10
   ||8631

-- 
You are receiving this mail because:
You are the assignee for the bug.___
Libreoffice-bugs mailing list
Libreoffice-bugs@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/libreoffice-bugs

[Libreoffice-bugs] [Bug 84246] Calc crashes on loading / saving large (~500MB) CSV files

[bugzilla-daemon]

https://bugs.documentfoundation.org/show_bug.cgi?id=84246

Bartosz  changed:

   What|Removed |Added

 Blocks|88175   |


Referenced Bugs:

https://bugs.documentfoundation.org/show_bug.cgi?id=88175
[Bug 88175] [Meta] Tracking bug for limitations of XLSX (OOXML) format
-- 
You are receiving this mail because:
You are the assignee for the bug.___
Libreoffice-bugs mailing list
Libreoffice-bugs@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/libreoffice-bugs

[Libreoffice-bugs] [Bug 84246] Calc crashes on loading / saving large (~500MB) CSV files

[bugzilla-daemon]

https://bugs.documentfoundation.org/show_bug.cgi?id=84246

Telesto  changed:

   What|Removed |Added

   Keywords||filter:xlsx
 Blocks||88175


Referenced Bugs:

https://bugs.documentfoundation.org/show_bug.cgi?id=88175
[Bug 88175] [Meta] Tracking bug for limitations of XLSX (OOXML) format
-- 
You are receiving this mail because:
You are the assignee for the bug.___
Libreoffice-bugs mailing list
Libreoffice-bugs@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/libreoffice-bugs

[Libreoffice-bugs] [Bug 84246] Calc crashes on loading / saving large (~500MB) CSV files

[bugzilla-daemon]

https://bugs.documentfoundation.org/show_bug.cgi?id=84246

Bartosz  changed:

   What|Removed |Added

 CC||gan...@poczta.onet.pl

--- Comment #12 from Bartosz  ---
Created attachment 128996
  --> https://bugs.documentfoundation.org/attachment.cgi?id=128996=edit
Random value CSV file ~237MB (unpacked)

I have tested attached CSV export to .ODS and .XLSX.
In both format there are needed about 25GB of memory to save to desired
formats.

The difference is that during exporting to .ODS, the hard disc is used (you
could noticed that free disc space is decreasing). The RAM usage for
LibreOffice during exporting to .ods is constant (about 1200MB) Generally
exporting into .ods is successful.

Unforunately during exporting into .xlsx, only RAM is used.  There is no disc
usage, during exporting into .xlsx  It means that during exporting to .xlsx,
the amount of free RAM memory is dramatically decreasing until it crash.

In my opinion desired solution for exporting to .xlsx, is saving data on disc,
as it is already done for .ods.

-- 
You are receiving this mail because:
You are the assignee for the bug.___
Libreoffice-bugs mailing list
Libreoffice-bugs@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/libreoffice-bugs

[Libreoffice-bugs] [Bug 84246] Calc crashes on loading / saving large (~500MB) CSV files

[bugzilla-daemon]

https://bugs.documentfoundation.org/show_bug.cgi?id=84246

Bartosz  changed:

   What|Removed |Added

 Status|REOPENED|NEW

-- 
You are receiving this mail because:
You are the assignee for the bug.___
Libreoffice-bugs mailing list
Libreoffice-bugs@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/libreoffice-bugs

[Libreoffice-bugs] [Bug 84246] Calc crashes on loading / saving large (~500MB) CSV files

[bugzilla-daemon]

https://bugs.documentfoundation.org/show_bug.cgi?id=84246

--- Comment #11 from Holli Bleßmann  ---
This IS still a Bug. Have same issue with a 237MB large CSV file on xubuntu
16.04LTS 64bit running on an intel core i7 with 8GB RAM. Trying to save that
file as a M$ Excel 2010 .xlsx ended up in getting out of memory (full use of
RAM AND 5GB SWAP PARTITION!) and -after 30 minutes- a crashed LibreOffice Calc
(v5.1.2.x).

Btw: xubuntu with loaded LibreOffice Calc with an empty sheet needs aprox.
250MB of RAM and no use of the swap partition.

As Markus Mohrhard mentioned above, I expected a normal RAM consumption of
(temporary) 2x document size + needed additional spreadsheet data. In my case
something about 500MB, NOT 12.5GB!

So, this bug still exists.

-- 
You are receiving this mail because:
You are the assignee for the bug.___
Libreoffice-bugs mailing list
Libreoffice-bugs@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/libreoffice-bugs

[Libreoffice-bugs] [Bug 84246] Calc crashes on loading / saving large (~500MB) CSV files

[bugzilla-daemon]

https://bugs.documentfoundation.org/show_bug.cgi?id=84246

Holli Bleßmann  changed:

   What|Removed |Added

 Status|RESOLVED|REOPENED
 Resolution|NOTABUG |---

--- Comment #10 from Holli Bleßmann  ---
This IS still a Bug. Have same issue with a 237MB large CSV file on xubuntu
16.04LTS 64bit running on an intel core i7 with 8GB RAM. Trying to save that
file as a M$ Excel 2010 .xlsx ended up in getting out of memory (full use of
RAM AND 5GB SWAP PARTITION!) and -after 30 minutes- a crashed LibreOffice Calc
(v5.1.2.x).

Btw: xubuntu with loaded LibreOffice Calc with an empty sheet needs aprox.
250MB of RAM and no use of the swap partition.

As Markus Mohrhard mentioned above, I expected a normal RAM consumption of
(temporary) 2x document size + needed additional spreadsheet data. In my case
something about 500MB, NOT 12.5GB!

So, this bug still exists.

-- 
You are receiving this mail because:
You are the assignee for the bug.___
Libreoffice-bugs mailing list
Libreoffice-bugs@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/libreoffice-bugs

[Libreoffice-bugs] [Bug 84246] Calc crashes on loading / saving large (~500MB) CSV files

[bugzilla-daemon]

https://bugs.documentfoundation.org/show_bug.cgi?id=84246

Markus Mohrhard  changed:

   What|Removed |Added

 Status|ASSIGNED|RESOLVED
 Resolution|--- |NOTABUG

--- Comment #9 from Markus Mohrhard  ---
I'm not sure what people expect. Just storing the 500 MB string would require
us to use 500 MB of memory. There is not yet anything additional that we have
allocated then for this data like cell information, ...

If you need to process 500 MB csv files in a spreadsheet you need quite some
memory. There is no way around it.

-- 
You are receiving this mail because:
You are the assignee for the bug.___
Libreoffice-bugs mailing list
Libreoffice-bugs@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/libreoffice-bugs

[Libreoffice-bugs] [Bug 84246] Calc crashes on loading / saving large (~500MB) CSV files

[bugzilla-daemon]

https://bugs.documentfoundation.org/show_bug.cgi?id=84246

Bartosz  changed:

   What|Removed |Added

 Status|NEW |ASSIGNED

-- 
You are receiving this mail because:
You are the assignee for the bug.___
Libreoffice-bugs mailing list
Libreoffice-bugs@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/libreoffice-bugs

[Libreoffice-bugs] [Bug 84246] Calc crashes on loading / saving large (~500MB) CSV files

[bugzilla-daemon]

https://bugs.documentfoundation.org/show_bug.cgi?id=84246

Holli Bleßmann  changed:

   What|Removed |Added

 CC||holli.blessm...@gmx.de

--- Comment #8 from Holli Bleßmann  ---
*** Bug 103267 has been marked as a duplicate of this bug. ***

-- 
You are receiving this mail because:
You are the assignee for the bug.___
Libreoffice-bugs mailing list
Libreoffice-bugs@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/libreoffice-bugs

[Libreoffice-bugs] [Bug 84246] Calc crashes on loading / saving large (~500MB) CSV files

[bugzilla-daemon]

https://bugs.documentfoundation.org/show_bug.cgi?id=84246

--- Comment #7 from QA Administrators  ---
** Please read this message in its entirety before responding **

To make sure we're focusing on the bugs that affect our users today,
LibreOffice QA is asking bug reporters and confirmers to retest open, confirmed
bugs which have not been touched for over a year.

There have been thousands of bug fixes and commits since anyone checked on this
bug report. During that time, it's possible that the bug has been fixed, or the
details of the problem have changed. We'd really appreciate your help in
getting confirmation that the bug is still present.

If you have time, please do the following:

   Test to see if the bug is still present on a currently supported version of
LibreOffice (5.0.1 or preferably 5.0.2.2 or later)
   https://www.libreoffice.org/download/

   If the bug is present, please leave a comment that includes the version of
LibreOffice and your operating system, and any changes you see in the bug
behavior

 If the bug is NOT present, please set the bug's Status field to
RESOLVED-WORKSFORME and leave a short comment that includes your version of
LibreOffice and Operating System

Please DO NOT

   Update the version field
   Reply via email (please reply directly on the bug tracker)
   Set the bug's Status field to RESOLVED - FIXED (this status has a particular
meaning that is not appropriate in this case)


If you want to do more to help you can test to see if your issue is a
REGRESSION. To do so: 

1. Download and install oldest version of LibreOffice (usually 3.3 unless your
bug pertains to a feature added after 3.3)

http://downloadarchive.documentfoundation.org/libreoffice/old/

2. Test your bug 
3. Leave a comment with your results. 
4a. If the bug was present with 3.3 - set version to "inherited from OOo"; 
4b. If the bug was not present in 3.3 - add "regression" to keyword


Feel free to come ask questions or to say hello in our QA chat:
http://webchat.freenode.net/?channels=libreoffice-qa

Thank you for your help!

-- The LibreOffice QA Team This NEW Message was generated on: 2015-10-14

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
Libreoffice-bugs mailing list
Libreoffice-bugs@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/libreoffice-bugs

[Libreoffice-bugs] [Bug 84246] Calc crashes on loading / saving large (~500MB) CSV files

[bugzilla-daemon]

https://bugs.freedesktop.org/show_bug.cgi?id=84246

--- Comment #5 from Marcelo soaresi...@gmail.com ---
I have 4GB RAM, processor Intel Core i5-3230M 2.60GHz, running elementary OS
0.2.1 64-bit

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
Libreoffice-bugs mailing list
Libreoffice-bugs@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/libreoffice-bugs

[Libreoffice-bugs] [Bug 84246] Calc crashes on loading / saving large (~500MB) CSV files

[bugzilla-daemon]

https://bugs.freedesktop.org/show_bug.cgi?id=84246

Owen Genat owen.ge...@gmail.com changed:

   What|Removed |Added

 Status|UNCONFIRMED |NEW
  Component|Libreoffice |Spreadsheet
Summary|Calc hanging|Calc crashes on loading /
   ||saving large (~500MB) CSV
   ||files
 Ever confirmed|0   |1

--- Comment #3 from Owen Genat owen.ge...@gmail.com ---
I am not sure whether there is anything the developers can do about handling
CSV files of this magnitude, but for now confirmed that there is an issue.
Status set to NEW. Component set to Spreadsheet. Summary amended for clarity.

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
Libreoffice-bugs mailing list
Libreoffice-bugs@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/libreoffice-bugs

[Libreoffice-bugs] [Bug 84246] Calc crashes on loading / saving large (~500MB) CSV files

[bugzilla-daemon]

https://bugs.freedesktop.org/show_bug.cgi?id=84246

Jean-Baptiste Faure jbfa...@libreoffice.org changed:

   What|Removed |Added

 CC||jbfa...@libreoffice.org

--- Comment #4 from Jean-Baptiste Faure jbfa...@libreoffice.org ---
@Marcello: 

How much RAM do you have on your machine?

Best regards. JBF

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
Libreoffice-bugs mailing list
Libreoffice-bugs@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/libreoffice-bugs