[jira] [Commented] (TIKA-3795) General upgrades for 2.4.2
[ https://issues.apache.org/jira/browse/TIKA-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17576143#comment-17576143 ] Hudson commented on TIKA-3795: -- FAILURE: Integrated in Jenkins build Tika » tika-main-jdk8 #732 (See [https://ci-builds.apache.org/job/Tika/job/tika-main-jdk8/732/]) TIKA-3795: update azure-core-http-netty (tilman: [https://github.com/apache/tika/commit/e1d90bed5dd5dffb1305335575bb62de6d999dc8]) * (edit) tika-pipes/pom.xml > General upgrades for 2.4.2 > -- > > Key: TIKA-3795 > URL: https://issues.apache.org/jira/browse/TIKA-3795 > Project: Tika > Issue Type: Improvement > Components: build >Reporter: Tilman Hausherr >Priority: Minor > Fix For: 2.4.2 > > Attachments: image-2022-07-12-06-46-00-215.png > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (TIKA-3831) Allow for retries in S3Fetcher
[ https://issues.apache.org/jira/browse/TIKA-3831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17576078#comment-17576078 ] Hudson commented on TIKA-3831: -- SUCCESS: Integrated in Jenkins build Tika » tika-main-jdk8 #731 (See [https://ci-builds.apache.org/job/Tika/job/tika-main-jdk8/731/]) TIKA-3831 -- allow for retries in S3Fetcher (tallison: [https://github.com/apache/tika/commit/f50d514a47e751fca17c18fa13457f2051123b23]) * (edit) tika-pipes/tika-fetchers/tika-fetcher-s3/src/main/java/org/apache/tika/pipes/fetcher/s3/S3Fetcher.java > Allow for retries in S3Fetcher > -- > > Key: TIKA-3831 > URL: https://issues.apache.org/jira/browse/TIKA-3831 > Project: Tika > Issue Type: Task >Reporter: Tim Allison >Priority: Trivial > > We should allow for retries. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (TIKA-3827) Word Document extracted mpga file extension instead of bitmap
[ https://issues.apache.org/jira/browse/TIKA-3827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17575945#comment-17575945 ] Tika User edited comment on TIKA-3827 at 8/5/22 11:20 PM: -- Okay was (Author: vamsi452): When this fix will be available? Next version? > Word Document extracted mpga file extension instead of bitmap > -- > > Key: TIKA-3827 > URL: https://issues.apache.org/jira/browse/TIKA-3827 > Project: Tika > Issue Type: Bug > Components: parser >Reporter: Tika User >Priority: Major > Attachments: Screenshot from 2022-08-04 06-05-09.png, example.DOC, > example.zip, file_1.bmp, file_2.bmp, image-2022-08-04-10-52-44-800.png, > image-2022-08-04-10-53-48-894.png, image-2022-08-04-15-44-48-396.png, > image-2022-08-04-15-45-10-892.png > > > When tried to parser the .doc document it is extracted two mpga files which > can't be open to play. We are suspecting they should be bitmap image files. > The Tika version we are using is 2.4.1. > [^example.DOC] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (TIKA-3831) Allow for retries in S3Fetcher
[ https://issues.apache.org/jira/browse/TIKA-3831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Allison updated TIKA-3831: -- Summary: Allow for retries in S3Fetcher (was: Small improvements to S3Fetcher) > Allow for retries in S3Fetcher > -- > > Key: TIKA-3831 > URL: https://issues.apache.org/jira/browse/TIKA-3831 > Project: Tika > Issue Type: Task >Reporter: Tim Allison >Priority: Trivial > > We should allow for retries. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (TIKA-3831) Small improvements to S3Fetcher
[ https://issues.apache.org/jira/browse/TIKA-3831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Allison updated TIKA-3831: -- Description: We should allow for retries. (was: When using the s3fetcher with aws public datasets, no credentials are required. We currently support instance and profile, and we require one of those. Let's drop this requirement. There are a couple of other improvements too that I'd like to make, like allowing retries and not requiring region.) > Small improvements to S3Fetcher > --- > > Key: TIKA-3831 > URL: https://issues.apache.org/jira/browse/TIKA-3831 > Project: Tika > Issue Type: Task >Reporter: Tim Allison >Priority: Trivial > > We should allow for retries. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (TIKA-3831) Small improvements to S3Fetcher
[ https://issues.apache.org/jira/browse/TIKA-3831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Allison updated TIKA-3831: -- Summary: Small improvements to S3Fetcher (was: S3Fetcher does not need to require credentials) > Small improvements to S3Fetcher > --- > > Key: TIKA-3831 > URL: https://issues.apache.org/jira/browse/TIKA-3831 > Project: Tika > Issue Type: Task >Reporter: Tim Allison >Priority: Trivial > > When using the s3fetcher with aws public datasets, no credentials are > required. We currently support instance and profile, and we require one of > those. Let's drop this requirement. > There are a couple of other improvements too that I'd like to make, like > allowing retries and not requiring region. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (TIKA-3832) Required array length is too large (OOM) error when reading a PDF file
[
https://issues.apache.org/jira/browse/TIKA-3832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17575951#comment-17575951
]
Lakatos Gyula commented on TIKA-3832:
-
[~tallison] Thanks a lot for fixing the problem! Tika is awesome. :)
> Required array length is too large (OOM) error when reading a PDF file
> --
>
> Key: TIKA-3832
> URL: https://issues.apache.org/jira/browse/TIKA-3832
> Project: Tika
> Issue Type: Bug
> Components: parser
>Affects Versions: 2.4.1
>Reporter: Lakatos Gyula
>Priority: Major
> Fix For: 1.28.5, 2.4.2
>
> Attachments: 7581cfbf-8c1e-4154-bfbb-4e633d858d5f.pdf
>
>
> I'm working on a web crawler and it got obliterated with an OutOfMemory error
> by a random PDF from the internet.
> {code:java}
> Exception in thread "main" java.lang.OutOfMemoryError: Required array length
> 2147483638 + 14 is too large
> at
> java.base/jdk.internal.util.ArraysSupport.hugeLength(ArraysSupport.java:649)
> at
> java.base/jdk.internal.util.ArraysSupport.newLength(ArraysSupport.java:642)
> at
> java.base/java.lang.AbstractStringBuilder.newCapacity(AbstractStringBuilder.java:257)
> at
> java.base/java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:229)
> at
> java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:740)
> at java.base/java.lang.StringBuffer.append(StringBuffer.java:410)
> at java.base/java.io.StringWriter.write(StringWriter.java:99)
> at
> org.apache.tika.sax.ToTextContentHandler.characters(ToTextContentHandler.java:108)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.WriteOutContentHandler.characters(WriteOutContentHandler.java:160)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.xpath.MatchingContentHandler.characters(MatchingContentHandler.java:81)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.SafeContentHandler.access$201(SafeContentHandler.java:47)
> at
> org.apache.tika.sax.SafeContentHandler.lambda$new$0(SafeContentHandler.java:57)
> at
> org.apache.tika.sax.SafeContentHandler.filter(SafeContentHandler.java:106)
> at
> org.apache.tika.sax.SafeContentHandler.characters(SafeContentHandler.java:250)
> at
> org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:270)
> at
> org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:295)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.extractBookmarkText(AbstractPDF2XHTML.java:977)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.extractBookmarkText(AbstractPDF2XHTML.java:981)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.extractBookmarkText(AbstractPDF2XHTML.java:959)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.endDocument(AbstractPDF2XHTML.java:907)
> at
> org.apache.pdfbox.text.PDFTextStripper.writeText(PDFTextStripper.java:239)
> at org.apache.tika.parser.pdf.PDF2XHTML.process(PDF2XHTML.java:108)
> at org.apache.tika.parser.pdf.PDFParser.parse(PDFParser.java:196)
> at com.example.TikaOOMExample.main(TikaOOMExample.java:31)
> {code}
> I reproduced the error in this repository:
> [https://github.com/laxika/apache-tika-oom-reproduction|http://example.com/]
> Uploaded the PDF into the attachments as well. It can be opened and read by
> the PDF readers I tried (Edge, Adobe, Chrome).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (TIKA-3827) Word Document extracted mpga file extension instead of bitmap
[ https://issues.apache.org/jira/browse/TIKA-3827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17575945#comment-17575945 ] Tika User commented on TIKA-3827: - When this fix will be available? Next version? > Word Document extracted mpga file extension instead of bitmap > -- > > Key: TIKA-3827 > URL: https://issues.apache.org/jira/browse/TIKA-3827 > Project: Tika > Issue Type: Bug > Components: parser >Reporter: Tika User >Priority: Major > Attachments: Screenshot from 2022-08-04 06-05-09.png, example.DOC, > example.zip, file_1.bmp, file_2.bmp, image-2022-08-04-10-52-44-800.png, > image-2022-08-04-10-53-48-894.png, image-2022-08-04-15-44-48-396.png, > image-2022-08-04-15-45-10-892.png > > > When tried to parser the .doc document it is extracted two mpga files which > can't be open to play. We are suspecting they should be bitmap image files. > The Tika version we are using is 2.4.1. > [^example.DOC] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (TIKA-3832) Required array length is too large (OOM) error when reading a PDF file
[
https://issues.apache.org/jira/browse/TIKA-3832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17575936#comment-17575936
]
Hudson commented on TIKA-3832:
--
SUCCESS: Integrated in Jenkins build Tika » tika-main-jdk8 #730 (See
[https://ci-builds.apache.org/job/Tika/job/tika-main-jdk8/730/])
TIKA-3832 -- defend against an infinite loop in handling bookmarks in PDFs.
(tallison:
[https://github.com/apache/tika/commit/dcea49b41ae8dad79497d645c72b4d1b297f983b])
* (edit)
tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/AbstractPDF2XHTML.java
TIKA-3832 -- move item count check into inner loop as added protection
(tallison:
[https://github.com/apache/tika/commit/a3072aeb7cf01f91cdb5d4e94408937cdade1086])
* (edit)
tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/AbstractPDF2XHTML.java
> Required array length is too large (OOM) error when reading a PDF file
> --
>
> Key: TIKA-3832
> URL: https://issues.apache.org/jira/browse/TIKA-3832
> Project: Tika
> Issue Type: Bug
> Components: parser
>Affects Versions: 2.4.1
>Reporter: Lakatos Gyula
>Priority: Major
> Fix For: 1.28.5, 2.4.2
>
> Attachments: 7581cfbf-8c1e-4154-bfbb-4e633d858d5f.pdf
>
>
> I'm working on a web crawler and it got obliterated with an OutOfMemory error
> by a random PDF from the internet.
> {code:java}
> Exception in thread "main" java.lang.OutOfMemoryError: Required array length
> 2147483638 + 14 is too large
> at
> java.base/jdk.internal.util.ArraysSupport.hugeLength(ArraysSupport.java:649)
> at
> java.base/jdk.internal.util.ArraysSupport.newLength(ArraysSupport.java:642)
> at
> java.base/java.lang.AbstractStringBuilder.newCapacity(AbstractStringBuilder.java:257)
> at
> java.base/java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:229)
> at
> java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:740)
> at java.base/java.lang.StringBuffer.append(StringBuffer.java:410)
> at java.base/java.io.StringWriter.write(StringWriter.java:99)
> at
> org.apache.tika.sax.ToTextContentHandler.characters(ToTextContentHandler.java:108)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.WriteOutContentHandler.characters(WriteOutContentHandler.java:160)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.xpath.MatchingContentHandler.characters(MatchingContentHandler.java:81)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.SafeContentHandler.access$201(SafeContentHandler.java:47)
> at
> org.apache.tika.sax.SafeContentHandler.lambda$new$0(SafeContentHandler.java:57)
> at
> org.apache.tika.sax.SafeContentHandler.filter(SafeContentHandler.java:106)
> at
> org.apache.tika.sax.SafeContentHandler.characters(SafeContentHandler.java:250)
> at
> org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:270)
> at
> org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:295)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.extractBookmarkText(AbstractPDF2XHTML.java:977)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.extractBookmarkText(AbstractPDF2XHTML.java:981)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.extractBookmarkText(AbstractPDF2XHTML.java:959)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.endDocument(AbstractPDF2XHTML.java:907)
> at
> org.apache.pdfbox.text.PDFTextStripper.writeText(PDFTextStripper.java:239)
> at org.apache.tika.parser.pdf.PDF2XHTML.process(PDF2XHTML.java:108)
> at org.apache.tika.parser.pdf.PDFParser.parse(PDFParser.java:196)
> at com.example.TikaOOMExample.main(TikaOOMExample.java:31)
> {code}
> I reproduced the error in this repository:
> [https://github.com/laxika/apache-tika-oom-reproduction|http://example.com/]
> Uploaded the PDF into the attachments as well. It can be opened and read by
> the PDF readers I tried (Edge, Adobe, Chrome).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (TIKA-3827) Word Document extracted mpga file extension instead of bitmap
[ https://issues.apache.org/jira/browse/TIKA-3827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17575935#comment-17575935 ] Hudson commented on TIKA-3827: -- SUCCESS: Integrated in Jenkins build Tika » tika-main-jdk8 #730 (See [https://ci-builds.apache.org/job/Tika/job/tika-main-jdk8/730/]) TIKA-3827 -- override image mime if raw bitmap in RTF (tallison: [https://github.com/apache/tika/commit/99533c971d5db7d7f3c501bc6cf67082a8d7f0cc]) * (edit) tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-microsoft-module/src/main/java/org/apache/tika/parser/microsoft/rtf/RTFEmbObjHandler.java * (edit) tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-microsoft-module/src/main/java/org/apache/tika/parser/microsoft/rtf/TextExtractor.java > Word Document extracted mpga file extension instead of bitmap > -- > > Key: TIKA-3827 > URL: https://issues.apache.org/jira/browse/TIKA-3827 > Project: Tika > Issue Type: Bug > Components: parser >Reporter: Tika User >Priority: Major > Attachments: Screenshot from 2022-08-04 06-05-09.png, example.DOC, > example.zip, file_1.bmp, file_2.bmp, image-2022-08-04-10-52-44-800.png, > image-2022-08-04-10-53-48-894.png, image-2022-08-04-15-44-48-396.png, > image-2022-08-04-15-45-10-892.png > > > When tried to parser the .doc document it is extracted two mpga files which > can't be open to play. We are suspecting they should be bitmap image files. > The Tika version we are using is 2.4.1. > [^example.DOC] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (TIKA-3832) Required array length is too large (OOM) error when reading a PDF file
[
https://issues.apache.org/jira/browse/TIKA-3832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17575915#comment-17575915
]
Hudson commented on TIKA-3832:
--
SUCCESS: Integrated in Jenkins build Tika » tika-branch1x-jdk8 #244 (See
[https://ci-builds.apache.org/job/Tika/job/tika-branch1x-jdk8/244/])
TIKA-3832 -- prevent infinite cycle in bookmark extraction (tallison:
[https://github.com/apache/tika/commit/b77e8ba6c9cb3effd0c5fe785ac54b25a5fa9118])
* (edit)
tika-parsers/src/main/java/org/apache/tika/parser/pdf/AbstractPDF2XHTML.java
> Required array length is too large (OOM) error when reading a PDF file
> --
>
> Key: TIKA-3832
> URL: https://issues.apache.org/jira/browse/TIKA-3832
> Project: Tika
> Issue Type: Bug
> Components: parser
>Affects Versions: 2.4.1
>Reporter: Lakatos Gyula
>Priority: Major
> Fix For: 1.28.5, 2.4.2
>
> Attachments: 7581cfbf-8c1e-4154-bfbb-4e633d858d5f.pdf
>
>
> I'm working on a web crawler and it got obliterated with an OutOfMemory error
> by a random PDF from the internet.
> {code:java}
> Exception in thread "main" java.lang.OutOfMemoryError: Required array length
> 2147483638 + 14 is too large
> at
> java.base/jdk.internal.util.ArraysSupport.hugeLength(ArraysSupport.java:649)
> at
> java.base/jdk.internal.util.ArraysSupport.newLength(ArraysSupport.java:642)
> at
> java.base/java.lang.AbstractStringBuilder.newCapacity(AbstractStringBuilder.java:257)
> at
> java.base/java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:229)
> at
> java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:740)
> at java.base/java.lang.StringBuffer.append(StringBuffer.java:410)
> at java.base/java.io.StringWriter.write(StringWriter.java:99)
> at
> org.apache.tika.sax.ToTextContentHandler.characters(ToTextContentHandler.java:108)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.WriteOutContentHandler.characters(WriteOutContentHandler.java:160)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.xpath.MatchingContentHandler.characters(MatchingContentHandler.java:81)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.SafeContentHandler.access$201(SafeContentHandler.java:47)
> at
> org.apache.tika.sax.SafeContentHandler.lambda$new$0(SafeContentHandler.java:57)
> at
> org.apache.tika.sax.SafeContentHandler.filter(SafeContentHandler.java:106)
> at
> org.apache.tika.sax.SafeContentHandler.characters(SafeContentHandler.java:250)
> at
> org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:270)
> at
> org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:295)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.extractBookmarkText(AbstractPDF2XHTML.java:977)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.extractBookmarkText(AbstractPDF2XHTML.java:981)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.extractBookmarkText(AbstractPDF2XHTML.java:959)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.endDocument(AbstractPDF2XHTML.java:907)
> at
> org.apache.pdfbox.text.PDFTextStripper.writeText(PDFTextStripper.java:239)
> at org.apache.tika.parser.pdf.PDF2XHTML.process(PDF2XHTML.java:108)
> at org.apache.tika.parser.pdf.PDFParser.parse(PDFParser.java:196)
> at com.example.TikaOOMExample.main(TikaOOMExample.java:31)
> {code}
> I reproduced the error in this repository:
> [https://github.com/laxika/apache-tika-oom-reproduction|http://example.com/]
> Uploaded the PDF into the attachments as well. It can be opened and read by
> the PDF readers I tried (Edge, Adobe, Chrome).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (TIKA-3827) Word Document extracted mpga file extension instead of bitmap
[
https://issues.apache.org/jira/browse/TIKA-3827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17575891#comment-17575891
]
Tim Allison commented on TIKA-3827:
---
For now, I've added a mediatype hint that the bytes are of type
{{image/x-rtf-raw-bitmap}}. This prevents parsers from being applied.
The correct solution would be to figure out the algorithm to manipulate the
bytes to convert them to an actual image file, but that is beyond my reach atm.
> Word Document extracted mpga file extension instead of bitmap
> --
>
> Key: TIKA-3827
> URL: https://issues.apache.org/jira/browse/TIKA-3827
> Project: Tika
> Issue Type: Bug
> Components: parser
>Reporter: Tika User
>Priority: Major
> Attachments: Screenshot from 2022-08-04 06-05-09.png, example.DOC,
> example.zip, file_1.bmp, file_2.bmp, image-2022-08-04-10-52-44-800.png,
> image-2022-08-04-10-53-48-894.png, image-2022-08-04-15-44-48-396.png,
> image-2022-08-04-15-45-10-892.png
>
>
> When tried to parser the .doc document it is extracted two mpga files which
> can't be open to play. We are suspecting they should be bitmap image files.
> The Tika version we are using is 2.4.1.
> [^example.DOC]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (TIKA-3827) Word Document extracted mpga file extension instead of bitmap
[ https://issues.apache.org/jira/browse/TIKA-3827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17575890#comment-17575890 ] Tim Allison commented on TIKA-3827: --- That's the client code, but we don't know what "getImageData()" is doing under the hood to transform the raw bytes. > Word Document extracted mpga file extension instead of bitmap > -- > > Key: TIKA-3827 > URL: https://issues.apache.org/jira/browse/TIKA-3827 > Project: Tika > Issue Type: Bug > Components: parser >Reporter: Tika User >Priority: Major > Attachments: Screenshot from 2022-08-04 06-05-09.png, example.DOC, > example.zip, file_1.bmp, file_2.bmp, image-2022-08-04-10-52-44-800.png, > image-2022-08-04-10-53-48-894.png, image-2022-08-04-15-44-48-396.png, > image-2022-08-04-15-45-10-892.png > > > When tried to parser the .doc document it is extracted two mpga files which > can't be open to play. We are suspecting they should be bitmap image files. > The Tika version we are using is 2.4.1. > [^example.DOC] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (TIKA-3829) java.lang.IllegalArgumentException: The document is really a XLS file exception while parsing doc file
[ https://issues.apache.org/jira/browse/TIKA-3829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17575883#comment-17575883 ] Tim Allison edited comment on TIKA-3829 at 8/5/22 2:47 PM: --- You can exclude parsers and exclude specific mime types from parsers via tika-config. See: https://tika.apache.org/2.4.1/configuring.html I'm not sure how that would help you. You can also turn off this logging via configuration of log4j2.xml. was (Author: talli...@mitre.org): You can exclude parsers and exclude specific mime types from parsers via tika-config. See: https://tika.apache.org/2.4.1/configuring.html > java.lang.IllegalArgumentException: The document is really a XLS file > exception while parsing doc file > -- > > Key: TIKA-3829 > URL: https://issues.apache.org/jira/browse/TIKA-3829 > Project: Tika > Issue Type: Bug > Components: parser >Affects Versions: 1.23 >Reporter: John >Priority: Major > > Getting following exception while parsing doc file: > WARN Ignoring unexpected exception while parsing summary entry > DocumentSummaryInformation > java.lang.IllegalArgumentException: The document is really a XLS file > at > org.apache.poi.poifs.filesystem.DirectoryNode.getEntry(DirectoryNode.java:322) > at > org.apache.tika.parser.microsoft.SummaryExtractor.parseSummaryEntryIfExists(SummaryExtractor.java:82) > at > org.apache.tika.parser.microsoft.SummaryExtractor.parseSummaries(SummaryExtractor.java:74) > at > org.apache.tika.parser.microsoft.OfficeParser.parse(OfficeParser.java:155) > at > org.apache.tika.parser.microsoft.OfficeParser.parse(OfficeParser.java:131) > at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280) > at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280) > at > org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:143) > > What is the meaning of this exception? when it will be thrown? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (TIKA-3827) Word Document extracted mpga file extension instead of bitmap
[
https://issues.apache.org/jira/browse/TIKA-3827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17575881#comment-17575881
]
Tika User commented on TIKA-3827:
-
Below is the code:
You can easily extract text from the document using code like this:
{{Document doc = new Document("C:\Temp\in.doc");
doc.save("C:\Temp\out.txt");}}
Or of you need to extract text into a String, you can use code like this:
{{Document doc = new Document("C:\Temp\in.doc");
String docText = doc.toString(SaveFormat.TEXT);}}
The following code can be used for image extraction:
{{Document doc = new Document("C:\Temp\in.doc");
Iterable shapes = doc.getChildNodes(NodeType.SHAPE, true);
int counter = 0;
for (Shape s : shapes)
\{
if (s.hasImage())
{
s.getImageData().save("C:\Temp\img_" + counter +
FileFormatUtil.imageTypeToExtension(s.getImageData().getImageType()));
counter++;
}
}}}
> Word Document extracted mpga file extension instead of bitmap
> --
>
> Key: TIKA-3827
> URL: https://issues.apache.org/jira/browse/TIKA-3827
> Project: Tika
> Issue Type: Bug
> Components: parser
>Reporter: Tika User
>Priority: Major
> Attachments: Screenshot from 2022-08-04 06-05-09.png, example.DOC,
> example.zip, file_1.bmp, file_2.bmp, image-2022-08-04-10-52-44-800.png,
> image-2022-08-04-10-53-48-894.png, image-2022-08-04-15-44-48-396.png,
> image-2022-08-04-15-45-10-892.png
>
>
> When tried to parser the .doc document it is extracted two mpga files which
> can't be open to play. We are suspecting they should be bitmap image files.
> The Tika version we are using is 2.4.1.
> [^example.DOC]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (TIKA-3829) java.lang.IllegalArgumentException: The document is really a XLS file exception while parsing doc file
[ https://issues.apache.org/jira/browse/TIKA-3829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17575883#comment-17575883 ] Tim Allison commented on TIKA-3829: --- You can exclude parsers and exclude specific mime types from parsers via tika-config. See: https://tika.apache.org/2.4.1/configuring.html > java.lang.IllegalArgumentException: The document is really a XLS file > exception while parsing doc file > -- > > Key: TIKA-3829 > URL: https://issues.apache.org/jira/browse/TIKA-3829 > Project: Tika > Issue Type: Bug > Components: parser >Affects Versions: 1.23 >Reporter: John >Priority: Major > > Getting following exception while parsing doc file: > WARN Ignoring unexpected exception while parsing summary entry > DocumentSummaryInformation > java.lang.IllegalArgumentException: The document is really a XLS file > at > org.apache.poi.poifs.filesystem.DirectoryNode.getEntry(DirectoryNode.java:322) > at > org.apache.tika.parser.microsoft.SummaryExtractor.parseSummaryEntryIfExists(SummaryExtractor.java:82) > at > org.apache.tika.parser.microsoft.SummaryExtractor.parseSummaries(SummaryExtractor.java:74) > at > org.apache.tika.parser.microsoft.OfficeParser.parse(OfficeParser.java:155) > at > org.apache.tika.parser.microsoft.OfficeParser.parse(OfficeParser.java:131) > at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280) > at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280) > at > org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:143) > > What is the meaning of this exception? when it will be thrown? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (TIKA-3832) Required array length is too large (OOM) error when reading a PDF file
[
https://issues.apache.org/jira/browse/TIKA-3832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Allison resolved TIKA-3832.
---
Fix Version/s: 1.28.5
2.4.2
Resolution: Fixed
Thank you [~Laxika] for opening this and sharing the file that triggers the
problem!
> Required array length is too large (OOM) error when reading a PDF file
> --
>
> Key: TIKA-3832
> URL: https://issues.apache.org/jira/browse/TIKA-3832
> Project: Tika
> Issue Type: Bug
> Components: parser
>Affects Versions: 2.4.1
>Reporter: Lakatos Gyula
>Priority: Major
> Fix For: 1.28.5, 2.4.2
>
> Attachments: 7581cfbf-8c1e-4154-bfbb-4e633d858d5f.pdf
>
>
> I'm working on a web crawler and it got obliterated with an OutOfMemory error
> by a random PDF from the internet.
> {code:java}
> Exception in thread "main" java.lang.OutOfMemoryError: Required array length
> 2147483638 + 14 is too large
> at
> java.base/jdk.internal.util.ArraysSupport.hugeLength(ArraysSupport.java:649)
> at
> java.base/jdk.internal.util.ArraysSupport.newLength(ArraysSupport.java:642)
> at
> java.base/java.lang.AbstractStringBuilder.newCapacity(AbstractStringBuilder.java:257)
> at
> java.base/java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:229)
> at
> java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:740)
> at java.base/java.lang.StringBuffer.append(StringBuffer.java:410)
> at java.base/java.io.StringWriter.write(StringWriter.java:99)
> at
> org.apache.tika.sax.ToTextContentHandler.characters(ToTextContentHandler.java:108)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.WriteOutContentHandler.characters(WriteOutContentHandler.java:160)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.xpath.MatchingContentHandler.characters(MatchingContentHandler.java:81)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.SafeContentHandler.access$201(SafeContentHandler.java:47)
> at
> org.apache.tika.sax.SafeContentHandler.lambda$new$0(SafeContentHandler.java:57)
> at
> org.apache.tika.sax.SafeContentHandler.filter(SafeContentHandler.java:106)
> at
> org.apache.tika.sax.SafeContentHandler.characters(SafeContentHandler.java:250)
> at
> org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:270)
> at
> org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:295)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.extractBookmarkText(AbstractPDF2XHTML.java:977)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.extractBookmarkText(AbstractPDF2XHTML.java:981)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.extractBookmarkText(AbstractPDF2XHTML.java:959)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.endDocument(AbstractPDF2XHTML.java:907)
> at
> org.apache.pdfbox.text.PDFTextStripper.writeText(PDFTextStripper.java:239)
> at org.apache.tika.parser.pdf.PDF2XHTML.process(PDF2XHTML.java:108)
> at org.apache.tika.parser.pdf.PDFParser.parse(PDFParser.java:196)
> at com.example.TikaOOMExample.main(TikaOOMExample.java:31)
> {code}
> I reproduced the error in this repository:
> [https://github.com/laxika/apache-tika-oom-reproduction|http://example.com/]
> Uploaded the PDF into the attachments as well. It can be opened and read by
> the PDF readers I tried (Edge, Adobe, Chrome).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (TIKA-3832) Required array length is too large (OOM) error when reading a PDF file
[
https://issues.apache.org/jira/browse/TIKA-3832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17575856#comment-17575856
]
Tim Allison commented on TIKA-3832:
---
We have to defend against cycles in BookMarks... Facepalm, we do in the outer
loop, effectively, but not the inner loop.
> Required array length is too large (OOM) error when reading a PDF file
> --
>
> Key: TIKA-3832
> URL: https://issues.apache.org/jira/browse/TIKA-3832
> Project: Tika
> Issue Type: Bug
> Components: parser
>Affects Versions: 2.4.1
>Reporter: Lakatos Gyula
>Priority: Major
> Attachments: 7581cfbf-8c1e-4154-bfbb-4e633d858d5f.pdf
>
>
> I'm working on a web crawler and it got obliterated with an OutOfMemory error
> by a random PDF from the internet.
> {code:java}
> Exception in thread "main" java.lang.OutOfMemoryError: Required array length
> 2147483638 + 14 is too large
> at
> java.base/jdk.internal.util.ArraysSupport.hugeLength(ArraysSupport.java:649)
> at
> java.base/jdk.internal.util.ArraysSupport.newLength(ArraysSupport.java:642)
> at
> java.base/java.lang.AbstractStringBuilder.newCapacity(AbstractStringBuilder.java:257)
> at
> java.base/java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:229)
> at
> java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:740)
> at java.base/java.lang.StringBuffer.append(StringBuffer.java:410)
> at java.base/java.io.StringWriter.write(StringWriter.java:99)
> at
> org.apache.tika.sax.ToTextContentHandler.characters(ToTextContentHandler.java:108)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.WriteOutContentHandler.characters(WriteOutContentHandler.java:160)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.xpath.MatchingContentHandler.characters(MatchingContentHandler.java:81)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.SafeContentHandler.access$201(SafeContentHandler.java:47)
> at
> org.apache.tika.sax.SafeContentHandler.lambda$new$0(SafeContentHandler.java:57)
> at
> org.apache.tika.sax.SafeContentHandler.filter(SafeContentHandler.java:106)
> at
> org.apache.tika.sax.SafeContentHandler.characters(SafeContentHandler.java:250)
> at
> org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:270)
> at
> org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:295)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.extractBookmarkText(AbstractPDF2XHTML.java:977)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.extractBookmarkText(AbstractPDF2XHTML.java:981)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.extractBookmarkText(AbstractPDF2XHTML.java:959)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.endDocument(AbstractPDF2XHTML.java:907)
> at
> org.apache.pdfbox.text.PDFTextStripper.writeText(PDFTextStripper.java:239)
> at org.apache.tika.parser.pdf.PDF2XHTML.process(PDF2XHTML.java:108)
> at org.apache.tika.parser.pdf.PDFParser.parse(PDFParser.java:196)
> at com.example.TikaOOMExample.main(TikaOOMExample.java:31)
> {code}
> I reproduced the error in this repository:
> [https://github.com/laxika/apache-tika-oom-reproduction|http://example.com/]
> Uploaded the PDF into the attachments as well. It can be opened and read by
> the PDF readers I tried (Edge, Adobe, Chrome).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (TIKA-3832) Required array length is too large (OOM) error when reading a PDF file
[
https://issues.apache.org/jira/browse/TIKA-3832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17575847#comment-17575847
]
Tim Allison commented on TIKA-3832:
---
Thank you for sharing the file. PDFBox's ExtractText has no problem with this.
Tika is entering an infinite loop here:
{noformat}
Abweichender Beschluss
Bekanntmachung 18.5.2021 Gemeinderat
1 Beschlussvorlage 43/2021 - Verkehrsplanung;
Sanierung der Zornedinger Straße in Harthausen - Variantenuntersuchung
Zwischenbericht Vorplanung
Lageplan (Teil Süd)
--Pläne und Ansichten
140102_Unterlage 5 utm-Blatt 1
140102_Unterlage 5 utm-Blatt 1
{noformat}
> Required array length is too large (OOM) error when reading a PDF file
> --
>
> Key: TIKA-3832
> URL: https://issues.apache.org/jira/browse/TIKA-3832
> Project: Tika
> Issue Type: Bug
> Components: parser
>Affects Versions: 2.4.1
>Reporter: Lakatos Gyula
>Priority: Major
> Attachments: 7581cfbf-8c1e-4154-bfbb-4e633d858d5f.pdf
>
>
> I'm working on a web crawler and it got obliterated with an OutOfMemory error
> by a random PDF from the internet.
> {code:java}
> Exception in thread "main" java.lang.OutOfMemoryError: Required array length
> 2147483638 + 14 is too large
> at
> java.base/jdk.internal.util.ArraysSupport.hugeLength(ArraysSupport.java:649)
> at
> java.base/jdk.internal.util.ArraysSupport.newLength(ArraysSupport.java:642)
> at
> java.base/java.lang.AbstractStringBuilder.newCapacity(AbstractStringBuilder.java:257)
> at
> java.base/java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:229)
> at
> java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:740)
> at java.base/java.lang.StringBuffer.append(StringBuffer.java:410)
> at java.base/java.io.StringWriter.write(StringWriter.java:99)
> at
> org.apache.tika.sax.ToTextContentHandler.characters(ToTextContentHandler.java:108)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.WriteOutContentHandler.characters(WriteOutContentHandler.java:160)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.xpath.MatchingContentHandler.characters(MatchingContentHandler.java:81)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.SafeContentHandler.access$201(SafeContentHandler.java:47)
> at
> org.apache.tika.sax.SafeContentHandler.lambda$new$0(SafeContentHandler.java:57)
> at
> org.apache.tika.sax.SafeContentHandler.filter(SafeContentHandler.java:106)
> at
> org.apache.tika.sax.SafeContentHandler.characters(SafeContentHandler.java:250)
> at
> org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:270)
> at
> org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:295)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.extractBookmarkText(AbstractPDF2XHTML.java:977)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.extractBookmarkText(AbstractPDF2XHTML.java:981)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.extractBookmarkText(AbstractPDF2XHTML.java:959)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.endDocument(AbstractPDF2XHTML.java:907)
> at
> org.apache.pdfbox.text.PDFTextStripper.writeText(PDFTextStripper.java:239)
> at org.apache.tika.parser.pdf.PDF2XHTML.process(PDF2XHTML.java:108)
> at org.apache.tika.parser.pdf.PDFParser.parse(PDFParser.java:196)
> at com.example.TikaOOMExample.main(TikaOOMExample.java:31)
> {code}
> I reproduced the error in this repository:
> [https://github.com/laxika/apache-tika-oom-reproduction|http://example.com/]
> Uploaded the PDF into the attachments as well. It can be opened and read by
> the PDF readers I tried (Edge, Adobe, Chrome).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (TIKA-3832) Required array length is too large (OOM) error when reading a PDF file
[
https://issues.apache.org/jira/browse/TIKA-3832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17575814#comment-17575814
]
Nick Burch commented on TIKA-3832:
--
Any chance you could try with Apache PDFBox directly? They've got a handy
command line tool you can use:
[https://cwiki.apache.org/confluence/display/TIKA/Troubleshooting+Tika#TroubleshootingTika-PDFTextProblems]
That will help us narrow down if it's a Tika bug, or one in the underlying
PDFBox library
> Required array length is too large (OOM) error when reading a PDF file
> --
>
> Key: TIKA-3832
> URL: https://issues.apache.org/jira/browse/TIKA-3832
> Project: Tika
> Issue Type: Bug
> Components: parser
>Affects Versions: 2.4.1
>Reporter: Lakatos Gyula
>Priority: Major
> Attachments: 7581cfbf-8c1e-4154-bfbb-4e633d858d5f.pdf
>
>
> I'm working on a web crawler and it got obliterated with an OutOfMemory error
> by a random PDF from the internet.
> {code:java}
> Exception in thread "main" java.lang.OutOfMemoryError: Required array length
> 2147483638 + 14 is too large
> at
> java.base/jdk.internal.util.ArraysSupport.hugeLength(ArraysSupport.java:649)
> at
> java.base/jdk.internal.util.ArraysSupport.newLength(ArraysSupport.java:642)
> at
> java.base/java.lang.AbstractStringBuilder.newCapacity(AbstractStringBuilder.java:257)
> at
> java.base/java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:229)
> at
> java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:740)
> at java.base/java.lang.StringBuffer.append(StringBuffer.java:410)
> at java.base/java.io.StringWriter.write(StringWriter.java:99)
> at
> org.apache.tika.sax.ToTextContentHandler.characters(ToTextContentHandler.java:108)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.WriteOutContentHandler.characters(WriteOutContentHandler.java:160)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.xpath.MatchingContentHandler.characters(MatchingContentHandler.java:81)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.SafeContentHandler.access$201(SafeContentHandler.java:47)
> at
> org.apache.tika.sax.SafeContentHandler.lambda$new$0(SafeContentHandler.java:57)
> at
> org.apache.tika.sax.SafeContentHandler.filter(SafeContentHandler.java:106)
> at
> org.apache.tika.sax.SafeContentHandler.characters(SafeContentHandler.java:250)
> at
> org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:270)
> at
> org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:295)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.extractBookmarkText(AbstractPDF2XHTML.java:977)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.extractBookmarkText(AbstractPDF2XHTML.java:981)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.extractBookmarkText(AbstractPDF2XHTML.java:959)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.endDocument(AbstractPDF2XHTML.java:907)
> at
> org.apache.pdfbox.text.PDFTextStripper.writeText(PDFTextStripper.java:239)
> at org.apache.tika.parser.pdf.PDF2XHTML.process(PDF2XHTML.java:108)
> at org.apache.tika.parser.pdf.PDFParser.parse(PDFParser.java:196)
> at com.example.TikaOOMExample.main(TikaOOMExample.java:31)
> {code}
> I reproduced the error in this repository:
> [https://github.com/laxika/apache-tika-oom-reproduction|http://example.com/]
> Uploaded the PDF into the attachments as well. It can be opened and read by
> the PDF readers I tried (Edge, Adobe, Chrome).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (TIKA-3832) Required array length is too large when reading a PDF file
Lakatos Gyula created TIKA-3832:
---
Summary: Required array length is too large when reading a PDF file
Key: TIKA-3832
URL: https://issues.apache.org/jira/browse/TIKA-3832
Project: Tika
Issue Type: Bug
Components: parser
Affects Versions: 2.4.1
Reporter: Lakatos Gyula
Attachments: 7581cfbf-8c1e-4154-bfbb-4e633d858d5f.pdf
I'm working on a web crawler and it got obliterated with an OutOfMemory error
by a random PDF from the internet.
{code:java}
Exception in thread "main" java.lang.OutOfMemoryError: Required array length
2147483638 + 14 is too large
at
java.base/jdk.internal.util.ArraysSupport.hugeLength(ArraysSupport.java:649)
at
java.base/jdk.internal.util.ArraysSupport.newLength(ArraysSupport.java:642)
at
java.base/java.lang.AbstractStringBuilder.newCapacity(AbstractStringBuilder.java:257)
at
java.base/java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:229)
at
java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:740)
at java.base/java.lang.StringBuffer.append(StringBuffer.java:410)
at java.base/java.io.StringWriter.write(StringWriter.java:99)
at
org.apache.tika.sax.ToTextContentHandler.characters(ToTextContentHandler.java:108)
at
org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
at
org.apache.tika.sax.WriteOutContentHandler.characters(WriteOutContentHandler.java:160)
at
org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
at
org.apache.tika.sax.xpath.MatchingContentHandler.characters(MatchingContentHandler.java:81)
at
org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
at
org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
at
org.apache.tika.sax.SafeContentHandler.access$201(SafeContentHandler.java:47)
at
org.apache.tika.sax.SafeContentHandler.lambda$new$0(SafeContentHandler.java:57)
at
org.apache.tika.sax.SafeContentHandler.filter(SafeContentHandler.java:106)
at
org.apache.tika.sax.SafeContentHandler.characters(SafeContentHandler.java:250)
at
org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:270)
at
org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:295)
at
org.apache.tika.parser.pdf.AbstractPDF2XHTML.extractBookmarkText(AbstractPDF2XHTML.java:977)
at
org.apache.tika.parser.pdf.AbstractPDF2XHTML.extractBookmarkText(AbstractPDF2XHTML.java:981)
at
org.apache.tika.parser.pdf.AbstractPDF2XHTML.extractBookmarkText(AbstractPDF2XHTML.java:959)
at
org.apache.tika.parser.pdf.AbstractPDF2XHTML.endDocument(AbstractPDF2XHTML.java:907)
at
org.apache.pdfbox.text.PDFTextStripper.writeText(PDFTextStripper.java:239)
at org.apache.tika.parser.pdf.PDF2XHTML.process(PDF2XHTML.java:108)
at org.apache.tika.parser.pdf.PDFParser.parse(PDFParser.java:196)
at com.example.TikaOOMExample.main(TikaOOMExample.java:31)
{code}
I reproduced the error in this repository:
[https://github.com/laxika/apache-tika-oom-reproduction|http://example.com/]
Uploaded the PDF into the attachments as well. It can be opened and read by the
PDF readers I tried (Edge, Adobe, Chrome).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (TIKA-3832) Required array length is too large (OOM) error when reading a PDF file
[
https://issues.apache.org/jira/browse/TIKA-3832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lakatos Gyula updated TIKA-3832:
Summary: Required array length is too large (OOM) error when reading a PDF
file (was: Required array length is too large when reading a PDF file)
> Required array length is too large (OOM) error when reading a PDF file
> --
>
> Key: TIKA-3832
> URL: https://issues.apache.org/jira/browse/TIKA-3832
> Project: Tika
> Issue Type: Bug
> Components: parser
>Affects Versions: 2.4.1
>Reporter: Lakatos Gyula
>Priority: Major
> Attachments: 7581cfbf-8c1e-4154-bfbb-4e633d858d5f.pdf
>
>
> I'm working on a web crawler and it got obliterated with an OutOfMemory error
> by a random PDF from the internet.
> {code:java}
> Exception in thread "main" java.lang.OutOfMemoryError: Required array length
> 2147483638 + 14 is too large
> at
> java.base/jdk.internal.util.ArraysSupport.hugeLength(ArraysSupport.java:649)
> at
> java.base/jdk.internal.util.ArraysSupport.newLength(ArraysSupport.java:642)
> at
> java.base/java.lang.AbstractStringBuilder.newCapacity(AbstractStringBuilder.java:257)
> at
> java.base/java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:229)
> at
> java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:740)
> at java.base/java.lang.StringBuffer.append(StringBuffer.java:410)
> at java.base/java.io.StringWriter.write(StringWriter.java:99)
> at
> org.apache.tika.sax.ToTextContentHandler.characters(ToTextContentHandler.java:108)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.WriteOutContentHandler.characters(WriteOutContentHandler.java:160)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.xpath.MatchingContentHandler.characters(MatchingContentHandler.java:81)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:141)
> at
> org.apache.tika.sax.SafeContentHandler.access$201(SafeContentHandler.java:47)
> at
> org.apache.tika.sax.SafeContentHandler.lambda$new$0(SafeContentHandler.java:57)
> at
> org.apache.tika.sax.SafeContentHandler.filter(SafeContentHandler.java:106)
> at
> org.apache.tika.sax.SafeContentHandler.characters(SafeContentHandler.java:250)
> at
> org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:270)
> at
> org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:295)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.extractBookmarkText(AbstractPDF2XHTML.java:977)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.extractBookmarkText(AbstractPDF2XHTML.java:981)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.extractBookmarkText(AbstractPDF2XHTML.java:959)
> at
> org.apache.tika.parser.pdf.AbstractPDF2XHTML.endDocument(AbstractPDF2XHTML.java:907)
> at
> org.apache.pdfbox.text.PDFTextStripper.writeText(PDFTextStripper.java:239)
> at org.apache.tika.parser.pdf.PDF2XHTML.process(PDF2XHTML.java:108)
> at org.apache.tika.parser.pdf.PDFParser.parse(PDFParser.java:196)
> at com.example.TikaOOMExample.main(TikaOOMExample.java:31)
> {code}
> I reproduced the error in this repository:
> [https://github.com/laxika/apache-tika-oom-reproduction|http://example.com/]
> Uploaded the PDF into the attachments as well. It can be opened and read by
> the PDF readers I tried (Edge, Adobe, Chrome).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (TIKA-3829) java.lang.IllegalArgumentException: The document is really a XLS file exception while parsing doc file
[ https://issues.apache.org/jira/browse/TIKA-3829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17575607#comment-17575607 ] John edited comment on TIKA-3829 at 8/5/22 7:01 AM: Ok. Will check and get you back if we faced this problem again. There is any way in tika to exclude some file types from extracting content? It also should be excluded even if files are available inside embedded files. was (Author: JIRAUSER292452): Ok. Will check and get you back if we faced this problem again. There is any way in tika to exclude some file types from scanning? It also should be excluded even if files are available inside embedded files. > java.lang.IllegalArgumentException: The document is really a XLS file > exception while parsing doc file > -- > > Key: TIKA-3829 > URL: https://issues.apache.org/jira/browse/TIKA-3829 > Project: Tika > Issue Type: Bug > Components: parser >Affects Versions: 1.23 >Reporter: John >Priority: Major > > Getting following exception while parsing doc file: > WARN Ignoring unexpected exception while parsing summary entry > DocumentSummaryInformation > java.lang.IllegalArgumentException: The document is really a XLS file > at > org.apache.poi.poifs.filesystem.DirectoryNode.getEntry(DirectoryNode.java:322) > at > org.apache.tika.parser.microsoft.SummaryExtractor.parseSummaryEntryIfExists(SummaryExtractor.java:82) > at > org.apache.tika.parser.microsoft.SummaryExtractor.parseSummaries(SummaryExtractor.java:74) > at > org.apache.tika.parser.microsoft.OfficeParser.parse(OfficeParser.java:155) > at > org.apache.tika.parser.microsoft.OfficeParser.parse(OfficeParser.java:131) > at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280) > at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280) > at > org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:143) > > What is the meaning of this exception? when it will be thrown? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (TIKA-3829) java.lang.IllegalArgumentException: The document is really a XLS file exception while parsing doc file
[ https://issues.apache.org/jira/browse/TIKA-3829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17575607#comment-17575607 ] John commented on TIKA-3829: Ok. Will check and get you back if we faced this problem again. There is any way in tika to exclude some file types from scanning? It also should be excluded even if files are available inside embedded files. > java.lang.IllegalArgumentException: The document is really a XLS file > exception while parsing doc file > -- > > Key: TIKA-3829 > URL: https://issues.apache.org/jira/browse/TIKA-3829 > Project: Tika > Issue Type: Bug > Components: parser >Affects Versions: 1.23 >Reporter: John >Priority: Major > > Getting following exception while parsing doc file: > WARN Ignoring unexpected exception while parsing summary entry > DocumentSummaryInformation > java.lang.IllegalArgumentException: The document is really a XLS file > at > org.apache.poi.poifs.filesystem.DirectoryNode.getEntry(DirectoryNode.java:322) > at > org.apache.tika.parser.microsoft.SummaryExtractor.parseSummaryEntryIfExists(SummaryExtractor.java:82) > at > org.apache.tika.parser.microsoft.SummaryExtractor.parseSummaries(SummaryExtractor.java:74) > at > org.apache.tika.parser.microsoft.OfficeParser.parse(OfficeParser.java:155) > at > org.apache.tika.parser.microsoft.OfficeParser.parse(OfficeParser.java:131) > at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280) > at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280) > at > org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:143) > > What is the meaning of this exception? when it will be thrown? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [tika] THausherr merged pull request #641: Bump aws.version from 1.12.275 to 1.12.276
THausherr merged PR #641: URL: https://github.com/apache/tika/pull/641 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@tika.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tika] THausherr merged pull request #640: Bump google-cloud-storage from 2.11.0 to 2.11.2
THausherr merged PR #640: URL: https://github.com/apache/tika/pull/640 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@tika.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tika] THausherr merged pull request #642: Bump maven-site-plugin from 3.12.0 to 3.12.1
THausherr merged PR #642: URL: https://github.com/apache/tika/pull/642 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@tika.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
