[jira] [Commented] (PDFBOX-2512) OutOfMemory while signing large documents
[
https://issues.apache.org/jira/browse/PDFBOX-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17263931#comment-17263931
]
Ralf Hauser commented on PDFBOX-2512:
-
and even more continuation in PDFBOX-5068
> OutOfMemory while signing large documents
> -
>
> Key: PDFBOX-2512
> URL: https://issues.apache.org/jira/browse/PDFBOX-2512
> Project: PDFBox
> Issue Type: Bug
> Components: Parsing, Signing
>Affects Versions: 1.8.7
>Reporter: Thomas Chojecki
>Assignee: Thomas Chojecki
>Priority: Major
> Fix For: 1.8.8
>
> Attachments: keystore.p12
>
>
> While working with large documents, we found some memory issues.
> 1. The method close() in the COSDocument, clones the objectpool and does not
> clean it properly. The cloning in getObjects() cause a OutOfMemory exception.
> 2.The COSWriter copy the whole pdf into the memory for signing and does not
> use BufferedInputStream for the FileInputStream which also has a big
> performance impact. (PDFBOX-1798)
> 3. The cloning of COSStreams cause a OutOfMemory exception
> I used the CreateSignature example with a about 150 MB big document from here:
> https://cdn-reichelt.de/bilder/downloads/reichelt_01-2015_DE_B_HQ.pdf
> Additionaly I add a RandomAccessFile to the PDDocument.load in the
> CreateSignature class.
> PDDocument doc = PDDocument.load(document,new RandomAccessFile(new
> File("d:\\temp.bin"), "rw")); (this prevent the OOM for the third case)
> The use of a BuffedInputStream in case two, will increase the signing speed
> from more than 5 minutes to less than 1 minute.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@pdfbox.apache.org
For additional commands, e-mail: dev-h...@pdfbox.apache.org
[jira] [Commented] (PDFBOX-2512) OutOfMemory while signing large documents
[
https://issues.apache.org/jira/browse/PDFBOX-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17257721#comment-17257721
]
Ralf Hauser commented on PDFBOX-2512:
-
ok, let's address this in PDFBOX-5067
> OutOfMemory while signing large documents
> -
>
> Key: PDFBOX-2512
> URL: https://issues.apache.org/jira/browse/PDFBOX-2512
> Project: PDFBox
> Issue Type: Bug
> Components: Parsing, Signing
>Affects Versions: 1.8.7
>Reporter: Thomas Chojecki
>Assignee: Thomas Chojecki
>Priority: Major
> Fix For: 1.8.8
>
> Attachments: keystore.p12
>
>
> While working with large documents, we found some memory issues.
> 1. The method close() in the COSDocument, clones the objectpool and does not
> clean it properly. The cloning in getObjects() cause a OutOfMemory exception.
> 2.The COSWriter copy the whole pdf into the memory for signing and does not
> use BufferedInputStream for the FileInputStream which also has a big
> performance impact. (PDFBOX-1798)
> 3. The cloning of COSStreams cause a OutOfMemory exception
> I used the CreateSignature example with a about 150 MB big document from here:
> https://cdn-reichelt.de/bilder/downloads/reichelt_01-2015_DE_B_HQ.pdf
> Additionaly I add a RandomAccessFile to the PDDocument.load in the
> CreateSignature class.
> PDDocument doc = PDDocument.load(document,new RandomAccessFile(new
> File("d:\\temp.bin"), "rw")); (this prevent the OOM for the third case)
> The use of a BuffedInputStream in case two, will increase the signing speed
> from more than 5 minutes to less than 1 minute.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@pdfbox.apache.org
For additional commands, e-mail: dev-h...@pdfbox.apache.org
[jira] [Commented] (PDFBOX-2512) OutOfMemory while signing large documents
[
https://issues.apache.org/jira/browse/PDFBOX-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17257718#comment-17257718
]
Tilman Hausherr commented on PDFBOX-2512:
-
This is a closed issue, please don't write there.
> OutOfMemory while signing large documents
> -
>
> Key: PDFBOX-2512
> URL: https://issues.apache.org/jira/browse/PDFBOX-2512
> Project: PDFBox
> Issue Type: Bug
> Components: Parsing, Signing
>Affects Versions: 1.8.7
>Reporter: Thomas Chojecki
>Assignee: Thomas Chojecki
>Priority: Major
> Fix For: 1.8.8
>
> Attachments: keystore.p12
>
>
> While working with large documents, we found some memory issues.
> 1. The method close() in the COSDocument, clones the objectpool and does not
> clean it properly. The cloning in getObjects() cause a OutOfMemory exception.
> 2.The COSWriter copy the whole pdf into the memory for signing and does not
> use BufferedInputStream for the FileInputStream which also has a big
> performance impact. (PDFBOX-1798)
> 3. The cloning of COSStreams cause a OutOfMemory exception
> I used the CreateSignature example with a about 150 MB big document from here:
> https://cdn-reichelt.de/bilder/downloads/reichelt_01-2015_DE_B_HQ.pdf
> Additionaly I add a RandomAccessFile to the PDDocument.load in the
> CreateSignature class.
> PDDocument doc = PDDocument.load(document,new RandomAccessFile(new
> File("d:\\temp.bin"), "rw")); (this prevent the OOM for the third case)
> The use of a BuffedInputStream in case two, will increase the signing speed
> from more than 5 minutes to less than 1 minute.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@pdfbox.apache.org
For additional commands, e-mail: dev-h...@pdfbox.apache.org
[jira] [Commented] (PDFBOX-2512) OutOfMemory while signing large documents
[
https://issues.apache.org/jira/browse/PDFBOX-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17257716#comment-17257716
]
Ralf Hauser commented on PDFBOX-2512:
-
Did a quick test with [^programWinter2015_20210103_091853-sig_LTV.pdf] 35MB
when doing -Xmx70m , the signature works
with -Xmx50m
java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.lang.StringBuilder.toString(StringBuilder.java:407)
at org.apache.pdfbox.pdfparser.BaseParser.readLong(BaseParser.java:1281)
at
org.apache.pdfbox.pdfparser.BaseParser.readObjectNumber(BaseParser.java:1212)
at
org.apache.pdfbox.pdfparser.PDFObjectStreamParser.privateReadObjectNumbers(PDFObjectStreamParser.java:104)
at
org.apache.pdfbox.pdfparser.PDFObjectStreamParser.parseObject(PDFObjectStreamParser.java:77)
at
org.apache.pdfbox.pdfparser.COSParser.parseObjectStreamObject(COSParser.java:779)
at
org.apache.pdfbox.pdfparser.COSParser.parseObjectDynamically(COSParser.java:637)
at
org.apache.pdfbox.pdfparser.COSParser.dereferenceCOSObject(COSParser.java:586)
at org.apache.pdfbox.cos.COSObject.getObject(COSObject.java:115)
at org.apache.pdfbox.pdfwriter.COSWriter.prepareIncrement(COSWriter.java:327)
at org.apache.pdfbox.pdfwriter.COSWriter.write(COSWriter.java:1425)
at org.apache.pdfbox.pdmodel.PDDocument.saveIncremental(PDDocument.java:997)
...
with -Xmx30m
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:3236)
at java.io.ByteArrayOutputStream.toByteArray(ByteArrayOutputStream.java:191)
at org.apache.pdfbox.cos.COSStream.createView(COSStream.java:218)
at
org.apache.pdfbox.pdfparser.PDFObjectStreamParser.(PDFObjectStreamParser.java:48)
at
org.apache.pdfbox.pdfparser.COSParser.parseObjectStreamObject(COSParser.java:778)
at
org.apache.pdfbox.pdfparser.COSParser.parseObjectDynamically(COSParser.java:637)
at
org.apache.pdfbox.pdfparser.COSParser.dereferenceCOSObject(COSParser.java:586)
at org.apache.pdfbox.cos.COSObject.getObject(COSObject.java:115)
at org.apache.pdfbox.pdfwriter.COSWriter.prepareIncrement(COSWriter.java:327)
at org.apache.pdfbox.pdfwriter.COSWriter.write(COSWriter.java:1425)
at org.apache.pdfbox.pdmodel.PDDocument.saveIncremental(PDDocument.java:997)
at
org.apache.pdfbox.examples.signature.CreateVisibleSignature.signPDF(CreateVisibleSignature.java:...
...
> OutOfMemory while signing large documents
> -
>
> Key: PDFBOX-2512
> URL: https://issues.apache.org/jira/browse/PDFBOX-2512
> Project: PDFBox
> Issue Type: Bug
> Components: Parsing, Signing
>Affects Versions: 1.8.7
>Reporter: Thomas Chojecki
>Assignee: Thomas Chojecki
>Priority: Major
> Fix For: 1.8.8
>
> Attachments: keystore.p12
>
>
> While working with large documents, we found some memory issues.
> 1. The method close() in the COSDocument, clones the objectpool and does not
> clean it properly. The cloning in getObjects() cause a OutOfMemory exception.
> 2.The COSWriter copy the whole pdf into the memory for signing and does not
> use BufferedInputStream for the FileInputStream which also has a big
> performance impact. (PDFBOX-1798)
> 3. The cloning of COSStreams cause a OutOfMemory exception
> I used the CreateSignature example with a about 150 MB big document from here:
> https://cdn-reichelt.de/bilder/downloads/reichelt_01-2015_DE_B_HQ.pdf
> Additionaly I add a RandomAccessFile to the PDDocument.load in the
> CreateSignature class.
> PDDocument doc = PDDocument.load(document,new RandomAccessFile(new
> File("d:\\temp.bin"), "rw")); (this prevent the OOM for the third case)
> The use of a BuffedInputStream in case two, will increase the signing speed
> from more than 5 minutes to less than 1 minute.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@pdfbox.apache.org
For additional commands, e-mail: dev-h...@pdfbox.apache.org
[jira] [Commented] (PDFBOX-2512) OutOfMemory while signing large documents
[ https://issues.apache.org/jira/browse/PDFBOX-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245909#comment-14245909 ] Thomas Chojecki commented on PDFBOX-2512: - I will try to apply the fix next year to the trunk and take a look at PDFBOX-2515. The last test with the nonSeq parser and the large document linked in the issue description workes more or less and created a valid signature out of the box. But the size was more than twice of the original document OutOfMemory while signing large documents - Key: PDFBOX-2512 URL: https://issues.apache.org/jira/browse/PDFBOX-2512 Project: PDFBox Issue Type: Bug Components: Parsing, Signing Affects Versions: 1.8.7 Reporter: Thomas Chojecki Assignee: Thomas Chojecki Fix For: 1.8.8 Attachments: keystore.p12 While working with large documents, we found some memory issues. 1. The method close() in the COSDocument, clones the objectpool and does not clean it properly. The cloning in getObjects() cause a OutOfMemory exception. 2.The COSWriter copy the whole pdf into the memory for signing and does not use BufferedInputStream for the FileInputStream which also has a big performance impact. (PDFBOX-1798) 3. The cloning of COSStreams cause a OutOfMemory exception I used the CreateSignature example with a about 150 MB big document from here: https://cdn-reichelt.de/bilder/downloads/reichelt_01-2015_DE_B_HQ.pdf Additionaly I add a RandomAccessFile to the PDDocument.load in the CreateSignature class. PDDocument doc = PDDocument.load(document,new RandomAccessFile(new File(d:\\temp.bin), rw)); (this prevent the OOM for the third case) The use of a BuffedInputStream in case two, will increase the signing speed from more than 5 minutes to less than 1 minute. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PDFBOX-2512) OutOfMemory while signing large documents
[ https://issues.apache.org/jira/browse/PDFBOX-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245321#comment-14245321 ] Andreas Lehmkühler commented on PDFBOX-2512: I've just realized that the code in the trunk is different. We should consider some similar solution when working PDFBOX-2515. OutOfMemory while signing large documents - Key: PDFBOX-2512 URL: https://issues.apache.org/jira/browse/PDFBOX-2512 Project: PDFBox Issue Type: Bug Components: Parsing, Signing Affects Versions: 1.8.7 Reporter: Thomas Chojecki Assignee: Thomas Chojecki Fix For: 1.8.8 Attachments: keystore.p12 While working with large documents, we found some memory issues. 1. The method close() in the COSDocument, clones the objectpool and does not clean it properly. The cloning in getObjects() cause a OutOfMemory exception. 2.The COSWriter copy the whole pdf into the memory for signing and does not use BufferedInputStream for the FileInputStream which also has a big performance impact. (PDFBOX-1798) 3. The cloning of COSStreams cause a OutOfMemory exception I used the CreateSignature example with a about 150 MB big document from here: https://cdn-reichelt.de/bilder/downloads/reichelt_01-2015_DE_B_HQ.pdf Additionaly I add a RandomAccessFile to the PDDocument.load in the CreateSignature class. PDDocument doc = PDDocument.load(document,new RandomAccessFile(new File(d:\\temp.bin), rw)); (this prevent the OOM for the third case) The use of a BuffedInputStream in case two, will increase the signing speed from more than 5 minutes to less than 1 minute. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: [jira] [Commented] (PDFBOX-2512) OutOfMemory while signing large documents
how can i unsubscribe from this email list? On Dec 13, 2014, at 6:07 PM, Andreas Lehmkühler (JIRA) wrote: [ https://issues.apache.org/jira/browse/PDFBOX-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245321#comment-14245321 ] Andreas Lehmkühler commented on PDFBOX-2512: I've just realized that the code in the trunk is different. We should consider some similar solution when working PDFBOX-2515. OutOfMemory while signing large documents - Key: PDFBOX-2512 URL: https://issues.apache.org/jira/browse/PDFBOX-2512 Project: PDFBox Issue Type: Bug Components: Parsing, Signing Affects Versions: 1.8.7 Reporter: Thomas Chojecki Assignee: Thomas Chojecki Fix For: 1.8.8 Attachments: keystore.p12 While working with large documents, we found some memory issues. 1. The method close() in the COSDocument, clones the objectpool and does not clean it properly. The cloning in getObjects() cause a OutOfMemory exception. 2.The COSWriter copy the whole pdf into the memory for signing and does not use BufferedInputStream for the FileInputStream which also has a big performance impact. (PDFBOX-1798) 3. The cloning of COSStreams cause a OutOfMemory exception I used the CreateSignature example with a about 150 MB big document from here: https://cdn-reichelt.de/bilder/downloads/reichelt_01-2015_DE_B_HQ.pdf Additionaly I add a RandomAccessFile to the PDDocument.load in the CreateSignature class. PDDocument doc = PDDocument.load(document,new RandomAccessFile(new File(d:\\temp.bin), rw)); (this prevent the OOM for the third case) The use of a BuffedInputStream in case two, will increase the signing speed from more than 5 minutes to less than 1 minute. -- This message was sent by Atlassian JIRA (v6.3.4#6332) Sherif A. Bishr Chairman AKNAN Company for Data Management Services T +20 2 3347 5258 l F +20 2 3346 0537 M +20 122 215 4059 Website: www.aknan-company.com
Re: [jira] [Commented] (PDFBOX-2512) OutOfMemory while signing large documents
Hi, same way than subscribing. Send a mail to dev-unsubscr...@pdfbox.apache.org See http://pdfbox.apache.org/mailinglists.html for further details BR Andreas Lehmkühler Am 13.12.2014 um 16:30 schrieb Sherif Bishr: how can i unsubscribe from this email list? On Dec 13, 2014, at 6:07 PM, Andreas Lehmkühler (JIRA) wrote: [ https://issues.apache.org/jira/browse/PDFBOX-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245321#comment-14245321 ] Andreas Lehmkühler commented on PDFBOX-2512: I've just realized that the code in the trunk is different. We should consider some similar solution when working PDFBOX-2515. OutOfMemory while signing large documents - Key: PDFBOX-2512 URL: https://issues.apache.org/jira/browse/PDFBOX-2512 Project: PDFBox Issue Type: Bug Components: Parsing, Signing Affects Versions: 1.8.7 Reporter: Thomas Chojecki Assignee: Thomas Chojecki Fix For: 1.8.8 Attachments: keystore.p12 While working with large documents, we found some memory issues. 1. The method close() in the COSDocument, clones the objectpool and does not clean it properly. The cloning in getObjects() cause a OutOfMemory exception. 2.The COSWriter copy the whole pdf into the memory for signing and does not use BufferedInputStream for the FileInputStream which also has a big performance impact. (PDFBOX-1798) 3. The cloning of COSStreams cause a OutOfMemory exception I used the CreateSignature example with a about 150 MB big document from here: https://cdn-reichelt.de/bilder/downloads/reichelt_01-2015_DE_B_HQ.pdf Additionaly I add a RandomAccessFile to the PDDocument.load in the CreateSignature class. PDDocument doc = PDDocument.load(document,new RandomAccessFile(new File(d:\\temp.bin), rw)); (this prevent the OOM for the third case) The use of a BuffedInputStream in case two, will increase the signing speed from more than 5 minutes to less than 1 minute. -- This message was sent by Atlassian JIRA (v6.3.4#6332) Sherif A. Bishr Chairman AKNAN Company for Data Management Services T +20 2 3347 5258 l F +20 2 3346 0537 M +20 122 215 4059 Website: www.aknan-company.com
[jira] [Commented] (PDFBOX-2512) OutOfMemory while signing large documents
[ https://issues.apache.org/jira/browse/PDFBOX-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14237747#comment-14237747 ] Andreas Lehmkühler commented on PDFBOX-2512: Are these changes limited to the 1.8-branch or should we add them to the trunk as well? OutOfMemory while signing large documents - Key: PDFBOX-2512 URL: https://issues.apache.org/jira/browse/PDFBOX-2512 Project: PDFBox Issue Type: Bug Components: Parsing, Signing Affects Versions: 1.8.7 Reporter: Thomas Chojecki Assignee: Thomas Chojecki Fix For: 1.8.8 Attachments: keystore.p12 While working with large documents, we found some memory issues. 1. The method close() in the COSDocument, clones the objectpool and does not clean it properly. The cloning in getObjects() cause a OutOfMemory exception. 2.The COSWriter copy the whole pdf into the memory for signing and does not use BufferedInputStream for the FileInputStream which also has a big performance impact. (PDFBOX-1798) 3. The cloning of COSStreams cause a OutOfMemory exception I used the CreateSignature example with a about 150 MB big document from here: https://cdn-reichelt.de/bilder/downloads/reichelt_01-2015_DE_B_HQ.pdf Additionaly I add a RandomAccessFile to the PDDocument.load in the CreateSignature class. PDDocument doc = PDDocument.load(document,new RandomAccessFile(new File(d:\\temp.bin), rw)); (this prevent the OOM for the third case) The use of a BuffedInputStream in case two, will increase the signing speed from more than 5 minutes to less than 1 minute. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PDFBOX-2512) OutOfMemory while signing large documents
[ https://issues.apache.org/jira/browse/PDFBOX-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14237806#comment-14237806 ] Thomas Chojecki commented on PDFBOX-2512: - If we can port it, we should do it. There are only small changes, that improve the performance and solve the OOM problematic. OutOfMemory while signing large documents - Key: PDFBOX-2512 URL: https://issues.apache.org/jira/browse/PDFBOX-2512 Project: PDFBox Issue Type: Bug Components: Parsing, Signing Affects Versions: 1.8.7 Reporter: Thomas Chojecki Assignee: Thomas Chojecki Fix For: 1.8.8 Attachments: keystore.p12 While working with large documents, we found some memory issues. 1. The method close() in the COSDocument, clones the objectpool and does not clean it properly. The cloning in getObjects() cause a OutOfMemory exception. 2.The COSWriter copy the whole pdf into the memory for signing and does not use BufferedInputStream for the FileInputStream which also has a big performance impact. (PDFBOX-1798) 3. The cloning of COSStreams cause a OutOfMemory exception I used the CreateSignature example with a about 150 MB big document from here: https://cdn-reichelt.de/bilder/downloads/reichelt_01-2015_DE_B_HQ.pdf Additionaly I add a RandomAccessFile to the PDDocument.load in the CreateSignature class. PDDocument doc = PDDocument.load(document,new RandomAccessFile(new File(d:\\temp.bin), rw)); (this prevent the OOM for the third case) The use of a BuffedInputStream in case two, will increase the signing speed from more than 5 minutes to less than 1 minute. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PDFBOX-2512) OutOfMemory while signing large documents
[ https://issues.apache.org/jira/browse/PDFBOX-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14219650#comment-14219650 ] ASF subversion and git services commented on PDFBOX-2512: - Commit 1640774 from [~tchojecki] in branch 'pdfbox/branches/1.8' [ https://svn.apache.org/r1640774 ] PDFBOX-2512: OutOfMemory while signing large documents - 1. clean the objectpool while closing the document - 2. pass a BufferedInputStream to the SignatureInteface / remove some code - pass a scratchfile to the load method in the example 'CreateSignature' OutOfMemory while signing large documents - Key: PDFBOX-2512 URL: https://issues.apache.org/jira/browse/PDFBOX-2512 Project: PDFBox Issue Type: Bug Components: Parsing, Signing Affects Versions: 1.8.7 Reporter: Thomas Chojecki Assignee: Thomas Chojecki While working with large documents, we found some memory issues. 1. The method close() in the COSDocument, clones the objectpool and does not clean it properly. The cloning in getObjects() cause a OutOfMemory exception. 2.The COSWriter copy the whole pdf into the memory for signing and does not use BufferedInputStream for the FileInputStream which also has a big performance impact. (PDFBOX-1798) 3. The cloning of COSStreams cause a OutOfMemory exception I used the CreateSignature example with a about 150 MB big document from here: https://cdn-reichelt.de/bilder/downloads/reichelt_01-2015_DE_B_HQ.pdf Additionaly I add a RandomAccessFile to the PDDocument.load in the CreateSignature class. PDDocument doc = PDDocument.load(document,new RandomAccessFile(new File(d:\\temp.bin), rw)); (this prevent the OOM for the third case) The use of a BuffedInputStream in case two, will increase the signing speed from more than 5 minutes to less than 1 minute. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
