How many concurrent tasks did you set on the two custom processors? Le sam. 19 sept. 2020 à 02:16, Phil H <[email protected]> a écrit :
> Basically I have two custom processors that are slow (data intensive) that > create backlogs. All other aspects of the flow work well - I am running SSD > disks so that sort of I/O is not biting me. The process itself is highly > ‘parallelisable’ but I can’t get NiFi to use more than about 10% of the CPU > load. This is the only purpose this machine has - I want it to work much > harder! > > On Fri, 18 Sep 2020 at 20:56, Pierre Villard <[email protected]> > wrote: > > > Hi, > > > > > > > > Do you have backpressure with flow files accumulating somewhere in your > > > > flow? If not, then you probably don't need more threads. > > > > If you do find bottlenecks in your flow design, then you can look at > > > > increasing the concurrent tasks on specific processors to have > > > > multi-threading. Note that it's not always the solution. > > > > Changing the pool size does not mean NiFi will use more resources. > > > > Actually, I usually recommend changing the pool size only when you see > that > > > > all the threads available are used. > > > > Overall we'd need more details about your flow and observations. > > > > > > > > Thanks, > > > > Pierre > > > > > > > > Le ven. 18 sept. 2020 à 03:49, Phil H <[email protected]> a écrit : > > > > > > > > > Hi there, > > > > > > > > > > Regardless of what I set in the maximum thread count settings, I never > > see > > > > > more than 128 active threads. The CPU load of the system is only 5-8% > > when > > > > > NiFi is running at this rate. It’s a Solaris system with 64 cores. > > > > > > > > > > Any help getting more performance would be appreciated! > > > > > > > > > > Regards, > > > > > Phil > > > > > > > > > >
