Joe, The problem is that NiFi is not able to load balance the data. What errors do you see in the logs around load balancing?
Thanks -Mark On Nov 22, 2022, at 9:15 AM, Joe Obernberger <joseph.obernber...@gmail.com<mailto:joseph.obernber...@gmail.com>> wrote: Hi Joe - this is happening only in queues that do round robin load balancing: <jtJG4ulk4w5IKidr.png> When I list queue on that queue it shows no flow files. If I stop the processors, and change the queue to no longer load balance, then I can not only list the queue, but the 22k messages are then processed. I did the dump as suggested - and it nifi-bootstrap.log file is attached. -Joe On 11/9/2022 5:09 PM, Joe Witt wrote: Please get back to the bad state and then do a thread dump. Please share ./bin/nifi.sh dump Only if LB appears stuck. Thanks On Wed, Nov 9, 2022 at 3:05 PM Joe Obernberger <joseph.obernber...@gmail.com<mailto:joseph.obernber...@gmail.com>> wrote: If I stop the RouteOnContent and InvokeHTTP processors, and then change the queue to no longer load balance, then the messages appear and the processor start to run. I can also then list the messages in the queue. -Joe On 11/9/2022 12:04 PM, Joe Obernberger wrote: The processor consuming from the queue runs on all nodes and is a InokeHTTP processor. That processor is currently idle. I can stop/start InvokeHTTP, but the queue size remains. I can't list what is in the queue. This seems like a bug? What I'm confused about is - do I have data to process or not? I can empty the queue, and something is removed, but I can't see what...so I don't know if I lost messages or not. <6aueDc0O2l8gSHvp.png> -Joe On 11/9/2022 11:50 AM, Joe Witt wrote: Hello This likely means the processor consuming from this queue has the flowfiles held being processed. MergeContent is a common processor that would do this but others certainly can. What processor do you have there? If you stop the target processor then delete it should always work though you generally should not need to do so but normally deleting queue content is a debug thing so you can. Thanks On Wed, Nov 9, 2022 at 9:47 AM Joe Obernberger <joseph.obernber...@gmail.com<mailto:joseph.obernber...@gmail.com>> wrote: Hi - I'm using NiFi 1.18.0 in a three node cluster using internal zookeeper. Occasionally, I will see a queue showing queued messages, but when I list the queue, the UI says that there are no flow files. Also the consumer of this queue seems to be idle. I can then empty the queue and it reports that n messages were deleted. Any idea what is happening here? <QGIOIT37jHpHzA5e.png> <ZDDoxeQhEFQdJezH.png> -Joe [https://s-install.avcdn.net/ipm/preview/icons/icon-envelope-tick-green-avg-v1.png]<http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> Virus-free.www.avg.com<http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> <x-msg://32/#m_-5532430073619279110_m_5948377795112736779_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> <nifi-bootstrap.log>
