Thanks Corey, that configuration change has solved the issue so far If anyone comes across this in the future, the configuration options were:
nifi.components.status.repository.buffer.size=280 nifi.components.status.snapshot.frequency=5 min Many thanks, Michael From: Corey Flowers [mailto:[email protected]] Sent: 26 July 2016 11:51 To: [email protected] Subject: Re: 0.6.1 Master running out of heap space This is probably because of your stats. All the stats across your cluster are kept within the NCMs heap space. Within Apache nifi it stores 1440 stats per processor, per minute. If you look in the conf file you will see the settings toward the bottom of the file. Sorry, I don't have a running instance in front of me to get the exact lines. You still may have issues with heap at that size but if you change the settings to 280 every 5 minutes it should help. Good luck! Sent from my iPhone On Jul 26, 2016, at 4:06 AM, "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> wrote: Hi all, I have a cluster with a relatively large amount of processors (150ish) in a flow. I’m finding that the master frequently runs out of heap space (set to a maximum of 1GB) after a few hours of being online. Is this a known issue, or is the 1GB of ‘allocated’ RAM not enough to hold that many processors? As the master doesn’t do any processing, I’m unsure why I’m seeing the master run out of heap space and not the slaves themselves. “2016-07-26 09:05:08,115 WARN [Process NCM Request-17] org.apache.nifi.io.socket.SocketListener Dispatching socket request encountered exception due to: java.lang.OutOfMemoryError: Java heap space” From my bootstrap file: # JVM memory settings java.arg.2=-Xms1024m java.arg.3=-Xmx1024m Many thanks, Michael Michael Griffiths System Developer BAE Systems Applied Intelligence ___________________________________________________________ E: [email protected]<mailto:[email protected]> BAE Systems Applied Intelligence, Surrey Research Park, Guildford, Surrey, GU2 7RQ. www.baesystems.com/ai<http://www.baesystems.com/ai> Please consider the environment before printing this email. This message should be regarded as confidential. If you have received this email in error please notify the sender and destroy it immediately. Statements of intent shall only become binding when confirmed in hard copy by an authorised signatory. The contents of this email may relate to dealings with other companies under the control of BAE Systems Applied Intelligence Limited, details of which can be found at http://www.baesystems.com/Businesses/index.htm. Please consider the environment before printing this email. This message should be regarded as confidential. If you have received this email in error please notify the sender and destroy it immediately. Statements of intent shall only become binding when confirmed in hard copy by an authorised signatory. The contents of this email may relate to dealings with other companies under the control of BAE Systems Applied Intelligence Limited, details of which can be found at http://www.baesystems.com/Businesses/index.htm.
