[
https://issues.apache.org/jira/browse/TIKA-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17562484#comment-17562484
]
Nick Burch commented on TIKA-3809:
----------------------------------
If the uncompressed XML is 250mb, then you're going to need a heap a lot lot
bigger than 750mb = 3x the uncompressed size, if you want to use the DOM-based
parsers. I'd try with about 3gb (so a bit over 10x) and be prepared to go up to
about 20x uncompressed size for your heap
> OutOfMemoryError occurs while reading doc file
> ----------------------------------------------
>
> Key: TIKA-3809
> URL: https://issues.apache.org/jira/browse/TIKA-3809
> Project: Tika
> Issue Type: Bug
> Components: app
> Affects Versions: 1.23
> Reporter: earl
> Priority: Blocker
>
> OutOfMemoryError occurs while parsing a docx file of size 8 MB (uncompressed
> size 250 MB). while analyzing the heapdump(.hprof), the thread that parses
> the file consumes about 750 MB heap size. while looking into a
> dominator_tree,
> {code:java}
> org.apache.xmlbeans.impl.store.Xobj$ElementXobj
> {code}
> This object has been created many times!
> I've also attached the stacktrace,
> {code:java}
> at
> org.apache.xmlbeans.impl.store.Cur.createElementXobj(Lorg/apache/xmlbeans/impl/store/Locale;Ljavax/xml/namespace/QName;Ljavax/xml/namespace/QName;)Lorg/apache/xmlbeans/impl/store/Xobj;
> (Cur.java:260)
> at
> org.apache.xmlbeans.impl.store.Cur$CurLoadContext.startElement(Ljavax/xml/namespace/QName;)V
> (Cur.java:2997)
> at
> org.apache.xmlbeans.impl.store.Locale$SaxHandler.startElement(Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;Lorg/xml/sax/Attributes;)V
> (Locale.java:3164)
> at
> org.apache.xerces.parsers.AbstractSAXParser.startElement(Lorg/apache/xerces/xni/QName;Lorg/apache/xerces/xni/XMLAttributes;Lorg/apache/xerces/xni/Augmentations;)V
> (Unknown Source)
> at
> org.apache.xerces.parsers.AbstractXMLDocumentParser.emptyElement(Lorg/apache/xerces/xni/QName;Lorg/apache/xerces/xni/XMLAttributes;Lorg/apache/xerces/xni/Augmentations;)V
> (Unknown Source)
> at org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanStartElement()Z
> (Unknown Source)
> at
> org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Z)Z
> (Unknown Source)
> at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Z)Z
> (Unknown Source)
> at org.apache.xerces.parsers.XML11Configuration.parse(Z)Z (Unknown Source)
> at
> org.apache.xerces.parsers.XML11Configuration.parse(Lorg/apache/xerces/xni/parser/XMLInputSource;)V
> (Unknown Source)
> at
> org.apache.xerces.parsers.XMLParser.parse(Lorg/apache/xerces/xni/parser/XMLInputSource;)V
> (Unknown Source)
> at
> org.apache.xerces.parsers.AbstractSAXParser.parse(Lorg/xml/sax/InputSource;)V
> (Unknown Source)
> at
> org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Lorg/xml/sax/InputSource;)V
> (Unknown Source)
> at
> org.apache.xmlbeans.impl.store.Locale$SaxLoader.load(Lorg/apache/xmlbeans/impl/store/Locale;Lorg/xml/sax/InputSource;Lorg/apache/xmlbeans/XmlOptions;)Lorg/apache/xmlbeans/impl/store/Cur;
> (Locale.java:3422)
> at
> org.apache.xmlbeans.impl.store.Locale.parseToXmlObject(Ljava/io/InputStream;Lorg/apache/xmlbeans/SchemaType;Lorg/apache/xmlbeans/XmlOptions;)Lorg/apache/xmlbeans/XmlObject;
> (Locale.java:1272)
> at
> org.apache.xmlbeans.impl.store.Locale.parseToXmlObject(Lorg/apache/xmlbeans/SchemaTypeLoader;Ljava/io/InputStream;Lorg/apache/xmlbeans/SchemaType;Lorg/apache/xmlbeans/XmlOptions;)Lorg/apache/xmlbeans/XmlObject;
> (Locale.java:1259)
> at
> org.apache.xmlbeans.impl.schema.SchemaTypeLoaderBase.parse(Ljava/io/InputStream;Lorg/apache/xmlbeans/SchemaType;Lorg/apache/xmlbeans/XmlOptions;)Lorg/apache/xmlbeans/XmlObject;
> (SchemaTypeLoaderBase.java:345)
> at
> org.openxmlformats.schemas.wordprocessingml.x2006.main.DocumentDocument$Factory.parse(Ljava/io/InputStream;Lorg/apache/xmlbeans/XmlOptions;)Lorg/openxmlformats/schemas/wordprocessingml/x2006/main/DocumentDocument;
> (Unknown Source)
> at org.apache.poi.xwpf.usermodel.XWPFDocument.onDocumentRead()V
> (XWPFDocument.java:178)
> at
> org.apache.poi.ooxml.POIXMLDocument.load(Lorg/apache/poi/ooxml/POIXMLFactory;)V
> (POIXMLDocument.java:184)
> at
> org.apache.poi.xwpf.usermodel.XWPFDocument.<init>(Lorg/apache/poi/openxml4j/opc/OPCPackage;)V
> (XWPFDocument.java:138)
> at
> org.apache.poi.xwpf.extractor.XWPFWordExtractor.<init>(Lorg/apache/poi/openxml4j/opc/OPCPackage;)V
> (XWPFWordExtractor.java:60)
> at
> org.apache.poi.ooxml.extractor.ExtractorFactory.createExtractor(Lorg/apache/poi/openxml4j/opc/OPCPackage;)Lorg/apache/poi/extractor/POITextExtractor;
> (ExtractorFactory.java:224)
> at
> org.apache.tika.parser.microsoft.ooxml.OOXMLExtractorFactory.parse(Ljava/io/InputStream;Lorg/xml/sax/ContentHandler;Lorg/apache/tika/metadata/Metadata;Lorg/apache/tika/parser/ParseContext;)V
> (OOXMLExtractorFactory.java:170)
> at
> org.apache.tika.parser.microsoft.ooxml.OOXMLParser.parse(Ljava/io/InputStream;Lorg/xml/sax/ContentHandler;Lorg/apache/tika/metadata/Metadata;Lorg/apache/tika/parser/ParseContext;)V
> (OOXMLParser.java:113)
> at
> org.apache.tika.parser.CompositeParser.parse(Ljava/io/InputStream;Lorg/xml/sax/ContentHandler;Lorg/apache/tika/metadata/Metadata;Lorg/apache/tika/parser/ParseContext;)V
> (CompositeParser.java:280)
> at
> org.apache.tika.parser.CompositeParser.parse(Ljava/io/InputStream;Lorg/xml/sax/ContentHandler;Lorg/apache/tika/metadata/Metadata;Lorg/apache/tika/parser/ParseContext;)V
> (CompositeParser.java:280)
> at
> org.apache.tika.parser.AutoDetectParser.parse(Ljava/io/InputStream;Lorg/xml/sax/ContentHandler;Lorg/apache/tika/metadata/Metadata;Lorg/apache/tika/parser/ParseContext;)V
> (AutoDetectParser.java:143) {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
