[ https://issues.apache.org/jira/browse/TIKA-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17562484#comment-17562484 ]
Nick Burch commented on TIKA-3809: ---------------------------------- If the uncompressed XML is 250mb, then you're going to need a heap a lot lot bigger than 750mb = 3x the uncompressed size, if you want to use the DOM-based parsers. I'd try with about 3gb (so a bit over 10x) and be prepared to go up to about 20x uncompressed size for your heap > OutOfMemoryError occurs while reading doc file > ---------------------------------------------- > > Key: TIKA-3809 > URL: https://issues.apache.org/jira/browse/TIKA-3809 > Project: Tika > Issue Type: Bug > Components: app > Affects Versions: 1.23 > Reporter: earl > Priority: Blocker > > OutOfMemoryError occurs while parsing a docx file of size 8 MB (uncompressed > size 250 MB). while analyzing the heapdump(.hprof), the thread that parses > the file consumes about 750 MB heap size. while looking into a > dominator_tree, > {code:java} > org.apache.xmlbeans.impl.store.Xobj$ElementXobj > {code} > This object has been created many times! > I've also attached the stacktrace, > {code:java} > at > org.apache.xmlbeans.impl.store.Cur.createElementXobj(Lorg/apache/xmlbeans/impl/store/Locale;Ljavax/xml/namespace/QName;Ljavax/xml/namespace/QName;)Lorg/apache/xmlbeans/impl/store/Xobj; > (Cur.java:260) > at > org.apache.xmlbeans.impl.store.Cur$CurLoadContext.startElement(Ljavax/xml/namespace/QName;)V > (Cur.java:2997) > at > org.apache.xmlbeans.impl.store.Locale$SaxHandler.startElement(Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;Lorg/xml/sax/Attributes;)V > (Locale.java:3164) > at > org.apache.xerces.parsers.AbstractSAXParser.startElement(Lorg/apache/xerces/xni/QName;Lorg/apache/xerces/xni/XMLAttributes;Lorg/apache/xerces/xni/Augmentations;)V > (Unknown Source) > at > org.apache.xerces.parsers.AbstractXMLDocumentParser.emptyElement(Lorg/apache/xerces/xni/QName;Lorg/apache/xerces/xni/XMLAttributes;Lorg/apache/xerces/xni/Augmentations;)V > (Unknown Source) > at org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanStartElement()Z > (Unknown Source) > at > org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Z)Z > (Unknown Source) > at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Z)Z > (Unknown Source) > at org.apache.xerces.parsers.XML11Configuration.parse(Z)Z (Unknown Source) > at > org.apache.xerces.parsers.XML11Configuration.parse(Lorg/apache/xerces/xni/parser/XMLInputSource;)V > (Unknown Source) > at > org.apache.xerces.parsers.XMLParser.parse(Lorg/apache/xerces/xni/parser/XMLInputSource;)V > (Unknown Source) > at > org.apache.xerces.parsers.AbstractSAXParser.parse(Lorg/xml/sax/InputSource;)V > (Unknown Source) > at > org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Lorg/xml/sax/InputSource;)V > (Unknown Source) > at > org.apache.xmlbeans.impl.store.Locale$SaxLoader.load(Lorg/apache/xmlbeans/impl/store/Locale;Lorg/xml/sax/InputSource;Lorg/apache/xmlbeans/XmlOptions;)Lorg/apache/xmlbeans/impl/store/Cur; > (Locale.java:3422) > at > org.apache.xmlbeans.impl.store.Locale.parseToXmlObject(Ljava/io/InputStream;Lorg/apache/xmlbeans/SchemaType;Lorg/apache/xmlbeans/XmlOptions;)Lorg/apache/xmlbeans/XmlObject; > (Locale.java:1272) > at > org.apache.xmlbeans.impl.store.Locale.parseToXmlObject(Lorg/apache/xmlbeans/SchemaTypeLoader;Ljava/io/InputStream;Lorg/apache/xmlbeans/SchemaType;Lorg/apache/xmlbeans/XmlOptions;)Lorg/apache/xmlbeans/XmlObject; > (Locale.java:1259) > at > org.apache.xmlbeans.impl.schema.SchemaTypeLoaderBase.parse(Ljava/io/InputStream;Lorg/apache/xmlbeans/SchemaType;Lorg/apache/xmlbeans/XmlOptions;)Lorg/apache/xmlbeans/XmlObject; > (SchemaTypeLoaderBase.java:345) > at > org.openxmlformats.schemas.wordprocessingml.x2006.main.DocumentDocument$Factory.parse(Ljava/io/InputStream;Lorg/apache/xmlbeans/XmlOptions;)Lorg/openxmlformats/schemas/wordprocessingml/x2006/main/DocumentDocument; > (Unknown Source) > at org.apache.poi.xwpf.usermodel.XWPFDocument.onDocumentRead()V > (XWPFDocument.java:178) > at > org.apache.poi.ooxml.POIXMLDocument.load(Lorg/apache/poi/ooxml/POIXMLFactory;)V > (POIXMLDocument.java:184) > at > org.apache.poi.xwpf.usermodel.XWPFDocument.<init>(Lorg/apache/poi/openxml4j/opc/OPCPackage;)V > (XWPFDocument.java:138) > at > org.apache.poi.xwpf.extractor.XWPFWordExtractor.<init>(Lorg/apache/poi/openxml4j/opc/OPCPackage;)V > (XWPFWordExtractor.java:60) > at > org.apache.poi.ooxml.extractor.ExtractorFactory.createExtractor(Lorg/apache/poi/openxml4j/opc/OPCPackage;)Lorg/apache/poi/extractor/POITextExtractor; > (ExtractorFactory.java:224) > at > org.apache.tika.parser.microsoft.ooxml.OOXMLExtractorFactory.parse(Ljava/io/InputStream;Lorg/xml/sax/ContentHandler;Lorg/apache/tika/metadata/Metadata;Lorg/apache/tika/parser/ParseContext;)V > (OOXMLExtractorFactory.java:170) > at > org.apache.tika.parser.microsoft.ooxml.OOXMLParser.parse(Ljava/io/InputStream;Lorg/xml/sax/ContentHandler;Lorg/apache/tika/metadata/Metadata;Lorg/apache/tika/parser/ParseContext;)V > (OOXMLParser.java:113) > at > org.apache.tika.parser.CompositeParser.parse(Ljava/io/InputStream;Lorg/xml/sax/ContentHandler;Lorg/apache/tika/metadata/Metadata;Lorg/apache/tika/parser/ParseContext;)V > (CompositeParser.java:280) > at > org.apache.tika.parser.CompositeParser.parse(Ljava/io/InputStream;Lorg/xml/sax/ContentHandler;Lorg/apache/tika/metadata/Metadata;Lorg/apache/tika/parser/ParseContext;)V > (CompositeParser.java:280) > at > org.apache.tika.parser.AutoDetectParser.parse(Ljava/io/InputStream;Lorg/xml/sax/ContentHandler;Lorg/apache/tika/metadata/Metadata;Lorg/apache/tika/parser/ParseContext;)V > (AutoDetectParser.java:143) {code} > -- This message was sent by Atlassian Jira (v8.20.10#820010)