Luís Filipe Nassif created TIKA-3237:
----------------------------------------
Summary: Great optimization in ForkParser
Key: TIKA-3237
URL: https://issues.apache.org/jira/browse/TIKA-3237
Project: Tika
Issue Type: Improvement
Components: core
Affects Versions: 1.24.1
Environment: Windows 10, Oracle JDK 1.8.0_261 x64, 24 cores
Reporter: Luís Filipe Nassif
Assignee: Luís Filipe Nassif
There is a huge overhead in ForkParser ContentHandlerProxy and
ContentHandlerResource read/write char[]/string methods. A simple change to not
loop reading/writing each char of strings can result in speed ups of several
orders of magnitude.
Tests with ~10k small RFC822 files (22MB), 24 parsing threads, have shown parse
times below:
ForkParser off: 4s
ForkParser on: 30min
ForkParser patched: 17s
Also, there is a bug when running at least with Oracle JDK8 that, if a String
greater than 65535 is written, a UTFDataFormatException is thrown from
DataOutputStream.writeUTF(String) method.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
