Hi, after some "git bisecting" this problem, it turns out that likely change https://svn.apache.org/viewvc?view=revision&revision=1825277 is somehow increasing memory usage when writing the document. It is not caused by XMLBeans per se, but how it is used at this point and how XMLBeans handles arrays.
We need to look if we can do this differently, naturally this change tried to fix another bug, which we don't want to re-introduce again here as well. Regards... Dominik. On Tue, Jul 16, 2019 at 1:36 PM Kirill R <[email protected]> wrote: > Dear POI Users Group, > > > I created a smaller example regarding this issue excel-writer.zip with > readme.md inside how to run it. Additionally I made a heap memory dump > after OutOfMemoryError has been thrown. I assume that the root cause of the > issue is in the dependency xmlbeans-3.1.0.jar which was updated as well and > according to the heap dump the most memory is used by > org.apache.xmlbeans.impl.store.Xobj$AttrXobj. Here is the stack trace of > the error: > > Exception in thread "main" java.lang.OutOfMemoryError: Java heap space > at java.base/java.util.Arrays.copyOf(Arrays.java:3745) > at > java.base/java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:120) > at > java.base/java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:95) > at > java.base/java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:156) > at > org.apache.poi.openxml4j.opc.internal.MemoryPackagePartOutputStream.write(MemoryPackagePartOutputStream.java:88) > at org.apache.xmlbeans.impl.store.Cursor._save(Cursor.java:590) > at org.apache.xmlbeans.impl.store.Cursor.save(Cursor.java:2544) > at > org.apache.xmlbeans.impl.values.XmlObjectBase.save(XmlObjectBase.java:223) > at > org.apache.poi.xssf.model.SharedStringsTable.writeTo(SharedStringsTable.java:279) > at > org.apache.poi.xssf.model.SharedStringsTable.commit(SharedStringsTable.java:286) > at > org.apache.poi.ooxml.POIXMLDocumentPart.onSave(POIXMLDocumentPart.java:463) > at > org.apache.poi.ooxml.POIXMLDocumentPart.onSave(POIXMLDocumentPart.java:468) > at org.apache.poi.ooxml.POIXMLDocument.write(POIXMLDocument.java:236) > at > org.apache.poi.xssf.streaming.SXSSFWorkbook.write(SXSSFWorkbook.java:943) > at com.excel.writer.ExcelWriter.write(ExcelWriter.java:48) > at com.excel.writer.ExcelWriter.main(ExcelWriter.java:74) > > > For the poi-ooxml with version 3.17 and its dependency xmlbeans-2.6.0.jar, > memory usage was twice better and it is enough to set 512MB to run this > example, whereas using the latest 4.1.0 version, it requires more than > twice more memory (1.1GB ) to be executed without > > OutOfMemoryError. It is not critical for small files like in this new > example, but it becomes critical if the file is much bigger and required > 10GB or even much more memory to be written. > > Best regards, > Kirill > > > -----Original Message----- > From: Mark Murphy <[email protected]> > Sent: Friday, 12 July, 2019 17:12 > To: POI Users List <[email protected]> > Subject: Re: POI memory usage degradation > > > > Maybe you could take a look and see if you can narrow down what the issues > are. Then maybe you could contribute a patch? > > > > On Fri, Jul 12, 2019 at 11:10 AM Mark Murphy <[email protected]> > wrote: > > > > > I see no issue in the Bugzilla. Without an issue there, the chances of > > > it being worked on or even considered are poor at best. > > > > > > On Fri, Jul 12, 2019 at 9:29 AM Kirill wrote: > > > > > >> Dear POI Users Group, > > >> > > >> Unfortunately I did not subscribe to POI Users List and did not get > > >> any information about the raised issue. > > >> Now I have a subscription. > > >> Could you please update me regarding the issue? > > >> > > >> Best regards, > > >> Kirill > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected]
