Cristian Vat created TIKA-2837: ---------------------------------- Summary: Performance/Stability problem in ToHTMLContentHandler Key: TIKA-2837 URL: https://issues.apache.org/jira/browse/TIKA-2837 Project: Tika Issue Type: Bug Reporter: Cristian Vat
I got a StackOverflowError while parsing a large PDF file using ToHTMLContentHandler. Trace: {noformat} java.lang.StackOverflowError: null at java.base/java.util.HashMap.hash(HashMap.java:339) ~[na:na] at java.base/java.util.HashMap.get(HashMap.java:552) ~[na:na] at org.apache.tika.sax.ToXMLContentHandler$ElementInfo.getPrefix(ToXMLContentHandler.java:54) ~[tika-core-1.20.jar:1.20] at org.apache.tika.sax.ToXMLContentHandler$ElementInfo.getPrefix(ToXMLContentHandler.java:58) ~[tika-core-1.20.jar:1.20] ....about 1000 recursive calls... at org.apache.tika.sax.ToXMLContentHandler$ElementInfo.getPrefix(ToXMLContentHandler.java:58) ~[tika-core-1.20.jar:1.20] {noformat} Error was received in a Spring Boot command-line app also doing other processing. I couldn't duplicate it with a standalone example, possibly standalone it doesn't completely fill up the stack. Also no error in standalone tika app running with GUI or as command-line. PDF File: "10.1007-s00268-016-3727-3.pdf" can be downloaded from [https://www.researchgate.net/publication/309385633_Safety_of_Nonsteroidal_Anti-inflammatory_Drugs_in_Major_Gastrointestinal_Surgery_A_Prospective_Multicenter_Cohort_Study] Generated output has 4681 <meta> tags Maximum tag depth of generated (X)HTML is 6 I then timed parsing with ToHTMLContentHandler versus directly with ToXMLContentHandler. After a warmup of a few hundred parse calls times were: - ToHTMLContentHandler: avg 500 ms - ToXMLContentHandler: avg 80-90 ms Profiling with YourKit showed a hotspot and very deep stack in recursive calls on ToXMLContentHandler$ElementInfo.getPrefix(String) in ToXMLContentHandler.java:58, same as was in the StackOverflowError Checking the code I found ToXMLContentHandler.endElement has a mention and a fix of old similar issue TIKA-1070: {code:java} // Reset the position in the tree, to avoid endless stack overflow // chains (see TIKA-1070) currentElement = currentElement.parent; {code} But ToHTMLContentHandler.endElement doesn't call super.endElement in case of empty elements including the <meta> tag. Thus the currentElement parents keep growing in this case? I created my own version of ToHTMLContentHandler where I called super.endElement inside the EMPTY_ELEMENTS if and: - no more StackOverflowError in the spring boot app - parse times reduced to XML version one, so 5x speed improvement at least - output is identical except additional "</meta>" closing tag. Questions: - should anybody be using ToHTMLContentHandler instead of ToXMLContentHandler ? Not sure on the exact use-case since information seems to be the same and there exist unaffected XML and XHTML content handlers -- any way that ToHTMLContentHandler could be improved but without emitting extra "</meta>" closing tag? -- This message was sent by Atlassian JIRA (v7.6.3#76005)