[ https://issues.apache.org/jira/browse/NIFI-7964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17259684#comment-17259684 ]
ASF subversion and git services commented on NIFI-7964: ------------------------------------------------------- Commit 1e227ca6436899b8109a3443aa4487bd43146fca in nifi's branch refs/heads/main from Joey Frazee [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=1e227ca ] NIFI-7964 Force PutAzureBlobStorage to stream writes NIFI-7964 Remove unused import NIFI-7964 Reverted to using BufferedInputStream along with unmarkable NIFI-7964 Made UnmarkableInputStream static and added factory method NIFI-7964 Remove unnecessary factory method from UnmarkableInputStream This closes #4632. Signed-off-by: Peter Turcsanyi <turcsa...@apache.org> > PutAzureBlobStorage OutOfMemory Exception > ----------------------------------------- > > Key: NIFI-7964 > URL: https://issues.apache.org/jira/browse/NIFI-7964 > Project: Apache NiFi > Issue Type: Bug > Affects Versions: 1.12.1 > Reporter: Eric Secules > Assignee: Joey Frazee > Priority: Blocker > Fix For: 1.13.0 > > Time Spent: 10m > Remaining Estimate: 0h > > As part of my flow I upload files to azure blob storage. They can be several > hundred MB in size. I putting a 300 MB file into my flow and it choked on the > PutAzureBlobStorage processor with the following log message. > {code:java} > 2020-10-28 19:34:10,717 ERROR [Timer-Driven Process Thread-6] > o.a.n.p.a.storage.PutAzureBlobStorage > PutAzureBlobStorage[id=74b80a47-016d-3430-fd74-ece7653158d5] > PutAzureBlobStorage[id=74b80a47-016d-3430-fd74-ece7653158d5] failed to > process session due to java.lang.OutOfMemoryError: Java heap space; Processor > Administratively Yielded for 1 sec: java.lang.OutOfMemoryError: Java heap > space > java.lang.OutOfMemoryError: Java heap space > 2020-10-28 19:34:10,717 WARN [Timer-Driven Process Thread-6] > o.a.n.controller.tasks.ConnectableTask Administratively Yielding > PutAzureBlobStorage[id=74b80a47-016d-3430-fd74-ece7653158d5] due to uncaught > Exception: java.lang.OutOfMemoryError: Java heap space > java.lang.OutOfMemoryError: Java heap space > {code} > I did not expect this to happen because I think the PutAzureBlob processor > should be streaming the flowfile from disk directly to blob. But this > behaviour suggests to me that it's getting read into memory in its entirety. > My JVM heap size is set to 512 MB, which shouldn't be a problem if streaming > was used to upload to blob storage in chunks. -- This message was sent by Atlassian Jira (v8.3.4#803005)