Mark Payne created NIFI-2851:
--------------------------------
Summary: Improve performance of SplitText
Key: NIFI-2851
URL: https://issues.apache.org/jira/browse/NIFI-2851
Project: Apache NiFi
Issue Type: Improvement
Components: Core Framework
Reporter: Mark Payne
Assignee: Mark Payne
Fix For: 1.1.0
SplitText is fairly CPU-intensive and quite slow. A simple flow that splits a
1.4 million line text file into 5k line chunks and then splits those 5k line
chunks into 1 line chunks is only capable of pushing through about 10k lines
per second. This equates to about 10 MB/sec. JVisualVM shows that the majority
of the time is spent in the locateSplitPoint() method. Isolating this code and
inspecting how it works, and using some micro-benchmarking, it appears that if
we refactor the calls to InputStream.read() to instead read into a byte array,
we can improve performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
