Re: How I put the cluster down.

2016-10-28 Thread Andrew Grande
Hi, I'd suggest couple things. Have you configured backpressure controls on connections? NiFi 1.0.0 adds 1evt/1GB by default IIRC. This can help avoid overwhelming components in a flow. Next, the 2 core CPU is really inadequate for high throughput system, see if you can get something better.

Re: How I put the cluster down.

2016-10-28 Thread Alessio Palma
Hello Witt, before anything else thanks for your help. Fortunatly I put down only the NIFI cluster, otherwise I was already in vacation :) After I posted this problem I kept to torture staging NIFI and discovered that when CPU LOAD gets very high, nodes loose connection and anything starts going

Re: How I put the cluster down.

2016-10-28 Thread Joe Witt
Alessio You have two clusters here potentially. The NiFi cluster and the Hadoop cluster. Which one went down? If NiFi went down I'd suspect memory exhaustion issues because other resource exhaustion issues like full file system, exhausted file handles, pegged CPU, etc.. tend not to cause it to

How I put the cluster down.

2016-10-28 Thread Alessio Palma
Hello all, yesterday, for a mistake, basically I executed " ls -R / " using the ListHDFS processor and the whole cluster gone down ( not just a node ). Something like this also happened when I was playing with some DO WHILE / WHILE DO patterns. I have only the nifi logs and they show the