[ https://issues.apache.org/jira/browse/NIFI-2851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Oleg Zhurakousky updated NIFI-2851: ----------------------------------- Status: Patch Available (was: Open) > Improve performance of SplitText > -------------------------------- > > Key: NIFI-2851 > URL: https://issues.apache.org/jira/browse/NIFI-2851 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework > Reporter: Mark Payne > Assignee: Oleg Zhurakousky > Fix For: 1.1.0 > > > SplitText is fairly CPU-intensive and quite slow. A simple flow that splits a > 1.4 million line text file into 5k line chunks and then splits those 5k line > chunks into 1 line chunks is only capable of pushing through about 10k lines > per second. This equates to about 10 MB/sec. JVisualVM shows that the > majority of the time is spent in the locateSplitPoint() method. Isolating > this code and inspecting how it works, and using some micro-benchmarking, it > appears that if we refactor the calls to InputStream.read() to instead read > into a byte array, we can improve performance. -- This message was sent by Atlassian JIRA (v6.3.4#6332)