Yeah that description of behavior means there was a bug in the processor (for sure). I have to believe in the last 5 years that it has been made better. You should never have to stop/start the processor to get it to behave right.
Thanks On Tue, Sep 26, 2023 at 10:39 AM Phillip Lord <phillord0...@gmail.com> wrote: > Honestly if it were me… I’d chalk it up to a glitch in the > matrix. Troubleshooting step 1 for me would be to restart that processor, > hopefully it works and you can move on with your day. > If it’s still occurring then maybe try adding some debug logging for that > processor? > > As mentioned though, I’d also definitely look into upgrading to a newer > version… I haven’t dug into any issues re that component. But a lot has > changed since 1.8. > On Sep 26, 2023 at 1:02 PM -0400, Ivan Dolinin > <idoli...@cleverdevices.com.invalid>, wrote: > > Understood. > > I stopped the processor and started it back. The processor successfully > read all 7+k messages which were in the queue. Was there something I could > do to see why the processor believed that the queue was empty? > > Thanks, > > Ivan Dolinin / (416) 583-5833 x2012 / idoli...@cleverdevices.com<mailto: > idoli...@cleverdevices.com> > > From: Joe Witt <joew...@apache.org> > Sent: Tuesday, September 26, 2023 12:54 PM > To: Ivan Dolinin <idoli...@cleverdevices.com>; dev@nifi.apache.org > Subject: Re: Processor stuck in yielding > > Unfortunately that does not tell us why yield was called. But it likely > means the processor believes the queue it is talking with contains no data. > The code shows that as the most likely culprit. http > External (joew...@apache.org<mailto:joew...@apache.org>) > Report This Email< > https://protection.inkyphishfence.com/report?id=c2tvdXQtY2xldmVyLWRldmljZXMvaWRvbGluaW5AY2xldmVyZGV2aWNlcy5jb20vNTRmZTg2ZmQ0YTFjOWIxMTUyYzBmZjk4NjI2ZDE0YzkvMTY5NTc0NzI3Ny40OQ==#key=a8d6f5821ef32eee25a2b7063049fd6d> > FAQ<https://getskout.com/emailprotectionfaq/> Skout Email Protection< > https://getskout.com/emailprotection/> > > Unfortunately that does not tell us why yield was called. But it likely > means the processor believes the queue it is talking with contains no data. > The code shows that as the most likely culprit. > > > https://github.com/apache/nifi/blob/98aabf2c50f857efc72fd6f2bfdd9965b97fa195/nifi-nar-bundles/nifi-amqp-bundle/nifi-amqp-processors/src/main/java/org/apache/nifi/amqp/processors/ConsumeAMQP.java#L130-L133 > < > https://shared.outlook.inky.com/link?domain=github.com&t=h.eJxVj71ygzAQhF_FQ9rAIRkh5MqZtMlM8gj6NbJBIpIgRSbvHmRckOaK73Znd3-KOQzF6VD0KU3xBHCxqZ9FJf0IfOKy1-CssSAGL4B1nAuDJalNR6g2kmKjWoOFUYqxlghGDUeM3C2l46EUs1ODjhvg49f0IDswBS91jD5EiEHCyK2DK184-HD5VyGrYad-9S7Oo355__yosuHpDR3rcj3H4vlQ3PKoePNzKuWgFx1KpRe7esEqP1hn3XnjD3xfTBqju9aohiPJBEIEy9oY1rW4VaiRDFDLCG0oprRqWI7ROebq9bdN6by1rdbi-aXya4d-_wB0XHiT.MEQCIHfldm6PVe5TryWRj4uzK6s48lz95tME0s_ZGSVejUcAAiBSfrnPqI6Kio7IplZOa3uez4MZxuT8NejbMeTHaD6c3g > > > if (lastReceived == null) { > // If no messages received, then yield. > context.yield(); > } > > Thanks > > On Tue, Sep 26, 2023 at 9:48 AM Ivan Dolinin <idoli...@cleverdevices.com > <mailto:idoli...@cleverdevices.com>> wrote: > Adding log in text. Looks like the image didn’t go through. > > 2023-09-26 08:16:58,207 DEBUG [Timer-Driven Process Thread-7] > o.a.nifi.amqp.processors.ConsumeAMQP > ConsumeAMQP[id=78f68d05-f59d-3b9b-746e-5d480be723df] has chosen to yield > its resources; will not be scheduled to run again for 1000 milliseconds > 2023-09-26 08:16:58,207 DEBUG [Timer-Driven Process Thread-9] > o.a.nifi.amqp.processors.ConsumeAMQP > ConsumeAMQP[id=85c40dd5-3f41-3895-25e9-ee3569331185] has chosen to yield > its resources; will not be scheduled to run again for 1000 milliseconds > > > Ivan Dolinin / (416) 583-5833 x2012 / idoli...@cleverdevices.com<mailto: > idoli...@cleverdevices.com> > > From: Ivan Dolinin <idoli...@cleverdevices.com.INVALID<mailto: > idoli...@cleverdevices.com.INVALID>> > Sent: Tuesday, September 26, 2023 12:20 PM > To: Joe Witt <joew...@apache.org<mailto:joew...@apache.org>>; > dev@nifi.apache.org<mailto:dev@nifi.apache.org> > Subject: RE: Processor stuck in yielding > > > Joe, > > thank you for a very quick response. > > Here is the screenshot of the log. There is nothing else recorded for the > UUID 78f68d05-f59d-3b9b-746e-5d480be723df. I could not trace the beginning > of this state. We have 3 ConsumeAMQP processors reading from RabbitMQ, only > this one is yielding, other 2 reading messages as expected. > > > > > Ivan Dolinin / (416) 583-5833 x2012 / idoli...@cleverdevices.com<mailto: > idoli...@cleverdevices.com> > > From: Joe Witt <joew...@apache.org<mailto:joew...@apache.org>> > Sent: Tuesday, September 26, 2023 11:49 AM > To: dev@nifi.apache.org<mailto:dev@nifi.apache.org> > Subject: Re: Processor stuck in yielding > > > Ivan, > > Please show what the logs state specifically if you can. > > That processor could yield for the following reasons from a quick look: > 1. The connection between it and the next processor in the flow is a full > queue not being processed fast enough. > 2. The data connection to AMQP is working but it is told there is no data > currently to pull. > 3. ... > > That is about it actually. I am looking at the current code though. NiFi > 1.8.0 is 5 years old (today ironically) and that processor and the > associated libraries and even framework have changed a lot since then. You > will definitely need to try your flow on the latest 1.23.x line if the > issue being seen is not a simple config problem. > > Thanks > > > On Tue, Sep 26, 2023 at 8:40 AM Ivan Dolinin > <idoli...@cleverdevices.com.invalid<mailto: > idoli...@cleverdevices.com.invalid>> wrote: > Hello, > > In one of our production instances running NiFi 1.8.0, the ConsumeAMQP > processor is stuck in yielding. All the log is saying that it has chosen to > yield. No errors are happening in the downstream processors. There are > thousands of messages in the queue it is reading from. The processor is > connecting to the queue. What could be the reason? > > > Thanks, > > Ivan Dolinin > >
