I have not test throughput of InvokeHTTP so am not sure what the maximum is, but can give some general guidance.
1kb isn't bad. For bottleneck I'd use top, iotop, etc. to figure out system resources usage while your flow is running. You should be able to increase both your nifi count by more and adjust your processors until you are limited by your system resources. Have you verified the endpoint you are sending to is not the bottle neck? Chad On Tue, Mar 17, 2020 at 3:47 PM Midhun Mohan <midhun.mo...@esginc.us> wrote: > Size of file around kb not more than that, > How can I find the bottle neck, > > Yes I adjust the count to 11 then all other processors stopped but > processing improved. > Totally at present 6 is showing not more than that. > > I just need a way to post more records to endpoint to make it realtime > > Hope i was able to give more details > > On Wed, 18 Mar 2020, 1:05 am Chad Zobrisky, <czobri...@gmail.com> wrote: > > > Midhun, > > A little more information would help. > > > > What size files are you sending? > > Have you looked at resource usage to see what the bottleneck is? > > Did you adjust your nifi system thread count from the hamburger menu? > > How many threads are running total for nifi? It's the number in the top > > left of the screen. > > > > Thanks, > > Chad > > > > > > > > On Tue, Mar 17, 2020 at 3:25 PM Midhun Mohan <midhun.mo...@esginc.us> > > wrote: > > > > > Do anyone have better idea on this > > > > > > On Tue, 17 Mar 2020, 6:32 pm Midhun Mohan, <midhun.mo...@esginc.us> > > wrote: > > > > > > > Hi all, > > > > > > > > I have a queue which will be filling up with realtime records of > around > > > > 20k records. Which is posting to an endpoint using Invokehttp > > processor. > > > > > > > > currently the average throughput is for 10k records it takes around > 20 > > > > minutes to complete the invokehttp processor posting. > > > > I Increased the concurrent thread to larger number which is > increasing > > > > the throughput, but it is causing other processor to terminate. I > think > > > > that can be solved by running on larger instances. > > > > > > > > What all things I need to take care to make sure I get a real time > > > > posting on endpoint. > > > > > > > > I need 20k records to be processed as they receive it > > > > > > > > > > > > thanks in advance > > > > > > > > -- > > > > > > > > > > > > Regards, > > > > Midhun Mohan > > > > > > > > > >
