Hi there,
The issue I have isn’t my processor, it’s the fact I can’t get Nifi to run
more threads. I’ve noticed that if I bump the max thread counts in the
controller settings to total over 256, then nifi as a whole locks up and I
have to clear the state and restart it.
Thanks,
Phil
On Mon, 21
Phil,
It sounds like you’re dealing with a large amount of lock contention. Either
using explicit java.util.concurrency.Lock or synchronized blocks. Would review
the code for lock contention or run a profiler against nifi and see what that
shows.
Sent from my iPhone
> On Sep 20, 2020, at
Various numbers - max of 96 so far.
On Sat, 19 Sep 2020 at 21:44, Pierre Villard
wrote:
> How many concurrent tasks did you set on the two custom processors?
>
>
>
> Le sam. 19 sept. 2020 à 02:16, Phil H a écrit :
>
>
>
> > Basically I have two custom processors that are slow (data intensive)
Google is your friend.
https://docs.cloudera.com/HDPDocuments/HDF3/HDF-3.5.1/nifi-configuration-best-practices/content/basic-cluster-setup.html
https://bryanbende.com/development/2018/10/23/apache-nifi-secure-cluster-setup
https://mintopsblog.com/2017/11/12/apache-nifi-cluster-configuration/
