[ https://issues.apache.org/jira/browse/NIFI-7002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Handermann resolved NIFI-7002. ------------------------------------ Fix Version/s: 1.13.0 Resolution: Fixed > Handle large files internally in EC toolkit > ------------------------------------------- > > Key: NIFI-7002 > URL: https://issues.apache.org/jira/browse/NIFI-7002 > Project: Apache NiFi > Issue Type: Sub-task > Components: Tools and Build > Affects Versions: 1.10.0 > Reporter: Andy LoPresto > Priority: Major > Labels: encryption, heap, security, streaming, toolkit > Fix For: 1.13.0 > > > The EC toolkit cannot handle large flow definition files (>100 MB > uncompressed) due to its use of {{String}} to hold the XML content internally > through frequent manipulations. As Strings are immutable in Java, this causes > a lot of heap usage and duplication. The read/write to the file system also > operates on the String directly rather than perform a streaming read/write. > We should change the internal representation of the file content (for all > files) to be a {{BufferedInputStream}} or other similar stream representation > rather than a String. We should test with very large files (1 GB) and clearly > provide instructions and warnings if the file size exceeds that. > The logic of value encryption/replacement will have to change to handle this > approach. -- This message was sent by Atlassian Jira (v8.3.4#803005)