Understood. Thanks everyone for quick help.
Ivan Dolinin / (416) 583-5833 x2012 / [email protected]<mailto:[email protected]> From: Joe Witt <[email protected]> Sent: Tuesday, September 26, 2023 1:44 PM To: Phillip Lord <[email protected]> Cc: Joe Witt <[email protected]>; [email protected] Subject: Re: Processor stuck in yielding Yeah that description of behavior means there was a bug in the processor (for sure). I have to believe in the last 5 years that it has been made better. You should never have to stop/start the process External ([email protected]<mailto:[email protected]>) Report This Email<https://protection.inkyphishfence.com/report?id=c2tvdXQtY2xldmVyLWRldmljZXMvaWRvbGluaW5AY2xldmVyZGV2aWNlcy5jb20vNzE3NThlZjEzZWQxOGIxZWEzYjVjZjY5ODZiODY1OGMvMTY5NTc1MDI5NS4zMw==#key=9e2d0d7f406523cb0988a68f7971c83f> FAQ<https://getskout.com/emailprotectionfaq/> Skout Email Protection<https://getskout.com/emailprotection/> Yeah that description of behavior means there was a bug in the processor (for sure). I have to believe in the last 5 years that it has been made better. You should never have to stop/start the processor to get it to behave right. Thanks On Tue, Sep 26, 2023 at 10:39 AM Phillip Lord <[email protected]<mailto:[email protected]>> wrote: > Honestly if it were me… I’d chalk it up to a glitch in the > matrix. Troubleshooting step 1 for me would be to restart that processor, > hopefully it works and you can move on with your day. > If it’s still occurring then maybe try adding some debug logging for that > processor? > > As mentioned though, I’d also definitely look into upgrading to a newer > version… I haven’t dug into any issues re that component. But a lot has > changed since 1.8. > On Sep 26, 2023 at 1:02 PM -0400, Ivan Dolinin > <[email protected]<mailto:[email protected]>>, > wrote: > > Understood. > > I stopped the processor and started it back. The processor successfully > read all 7+k messages which were in the queue. Was there something I could > do to see why the processor believed that the queue was empty? > > Thanks, > > Ivan Dolinin / (416) 583-5833 x2012 / > [email protected]<mailto<mailto:[email protected]%3cmailto>: > [email protected]<mailto:[email protected]>> > > From: Joe Witt <[email protected]<mailto:[email protected]>> > Sent: Tuesday, September 26, 2023 12:54 PM > To: Ivan Dolinin > <[email protected]<mailto:[email protected]>>; > [email protected]<mailto:[email protected]> > Subject: Re: Processor stuck in yielding > > Unfortunately that does not tell us why yield was called. But it likely > means the processor believes the queue it is talking with contains no data. > The code shows that as the most likely culprit. http > External > ([email protected]<mailto:[email protected]<mailto:[email protected]%3cmailto:[email protected]>>) > Report This Email< > https://protection.inkyphishfence.com/report?id=c2tvdXQtY2xldmVyLWRldmljZXMvaWRvbGluaW5AY2xldmVyZGV2aWNlcy5jb20vNTRmZTg2ZmQ0YTFjOWIxMTUyYzBmZjk4NjI2ZDE0YzkvMTY5NTc0NzI3Ny40OQ==#key=a8d6f5821ef32eee25a2b7063049fd6d<https://shared.outlook.inky.com/link?domain=protection.inkyphishfence.com&t=h.eJxNj11vgjAYRv-KYbcTSrEIJmRu2WZMJkbDRLgr7YuUrxKoTFz23zf2kezqSc65OHnetXNbaouJlinVdAvDaFqpgCkha13UxdBkostSqBnoTFZGC41s1Z3gHsOq58edivCl5NVheAn3X1vm8XHT03DfJ6vyTENy_-fj1QHT0C_ZQPIEo94P9lUcnHBc7VAUPOfbcH3ZBK9DdH2o4ryY-fkax49PKLoW_SaIiB8w5F_Xlj_M0HbneTcFDB51uJ0SB5uQWhgAMKE4mSPbQjM35TbXbidaMb7rCnlWU1ZCD-2UQy8YdIbgshS1qJc__Bd__5ybc-JAalrATScxgVoJYantOnbi2MRhhmm7ZE4QdoluWWMGxkwu4U0otaQNZRnosj2Nio_qH_r4BK_Ceo4.MEQCIAVSAxcHHbrsyMZvXOe1lr-77nTemhvQcuA-KtGfrYSvAiBLsN_Ok5Cvz0tEfRgnXt2vi29Bh6oRM1p0dkHwt4Thsg>> > FAQ<https://getskout.com/emailprotectionfaq/<https://shared.outlook.inky.com/link?domain=getskout.com&t=h.eJxNjc0SgiAURl_FYZ0SMSC68lUQrkqiEF5t0fTuDdai7Tnfz4vsyZO2IBNi3FpKR8BtDjtWJiwUFu18TAHBoAvroB-UXAoy58KZKo2HA1Jp4XAGNups8G51a_flP3xu1awWCgbGwTLVM9C8F2aQjZK9kkIZymQjanG9NaLiPN9AvrkHeDrETkdtJqhCGrOyWf2h9weptkDY.MEUCIQDxHlKLavGFxX6VOHkVqR_m3pipH2QYMOYWSYdYgLEOTQIgayhvjV0Xeeo_j4pXSsJw17uJv4TEs2dZowOBTeIDav0>> > Skout Email Protection< > https://getskout.com/emailprotection/<https://shared.outlook.inky.com/link?domain=getskout.com&t=h.eJxNjcEOgyAQBX-l4VyllIDoyV9BWHUrisHVHpr-e4PtodeZlzcvtqfAmgsbidat4XwA2qa4U-nizGG2GNYUCRxhXDi7XtiU1-ekcAEOSIWHAx1sHH0MuODSfvkPn0eVqJSBXkjwwnQCrOyU63VtdGe0Mo4LXatK3e61KqXMGciZR4QnErV2tW6EMqYhK5_VH3p_ALvRP6A.MEUCIQDTtwJBI94l-Mnwbd5-UanKGJgWvG-UrjljkJnP-teOIwIgR1KziqfimPEfXF6QDKPeHonh1QoxU1wZPeL1OLYfjDQ>> > > Unfortunately that does not tell us why yield was called. But it likely > means the processor believes the queue it is talking with contains no data. > The code shows that as the most likely culprit. > > > https://github.com/apache/nifi/blob/98aabf2c50f857efc72fd6f2bfdd9965b97fa195/nifi-nar-bundles/nifi-amqp-bundle/nifi-amqp-processors/src/main/java/org/apache/nifi/amqp/processors/ConsumeAMQP.java#L130-L133<https://shared.outlook.inky.com/link?domain=github.com&t=h.eJxVj81ugzAQhF8lotfCYpD_ckrVayu1j-CfdXACNrWBHqq-eyHkkF728O2MZuanmFNfHA9FN01jPgKc_dTNujJxADUq0yEE7zzoPmqQQintGkNrJyhHZ3jjLHONdtZKyaiW3Cki6c1SBpVKPQfbY96BGr7GO3kAY4oGc44pQ04GBuUDXNSiIKbzvwqbGh7UrzHkecCX98-PajM8vZG2LtfTFs-H4rqNytc4T6XpccFUWlz86gVvY--DD6ed3_FtMSecCnSkRUuEJqhaTY1jUjAtGBUGCJOU07qRtNpjcIu5RPz203Ta21Zr8e1lt9cD-v0DcAx4kA.MEQCIHHAZJD2wkrjfWzJiXqzx2UWaMUqmOWnPjmne-njb0O_AiAp5i2-PxpC7bK3bVFdfVT65X67otIzxgydJgo3GR3UUg> > < > https://shared.outlook.inky.com/link?domain=github.com&t=h.eJxVj71ygzAQhF_FQ9rAIRkh5MqZtMlM8gj6NbJBIpIgRSbvHmRckOaK73Znd3-KOQzF6VD0KU3xBHCxqZ9FJf0IfOKy1-CssSAGL4B1nAuDJalNR6g2kmKjWoOFUYqxlghGDUeM3C2l46EUs1ODjhvg49f0IDswBS91jD5EiEHCyK2DK184-HD5VyGrYad-9S7Oo355__yosuHpDR3rcj3H4vlQ3PKoePNzKuWgFx1KpRe7esEqP1hn3XnjD3xfTBqju9aohiPJBEIEy9oY1rW4VaiRDFDLCG0oprRqWI7ROebq9bdN6by1rdbi-aXya4d-_wB0XHiT.MEQCIHfldm6PVe5TryWRj4uzK6s48lz95tME0s_ZGSVejUcAAiBSfrnPqI6Kio7IplZOa3uez4MZxuT8NejbMeTHaD6c3g<https://shared.outlook.inky.com/link?domain=github.com&t=h.eJxVj81ugzAQhF8lotfCYpD_ckrVayu1j-CfdXACNrWBHqq-eyHkkF728O2MZuanmFNfHA9FN01jPgKc_dTNujJxADUq0yEE7zzoPmqQQintGkNrJyhHZ3jjLHONdtZKyaiW3Cki6c1SBpVKPQfbY96BGr7GO3kAY4oGc44pQ04GBuUDXNSiIKbzvwqbGh7UrzHkecCX98-PajM8vZG2LtfTFs-H4rqNytc4T6XpccFUWlz86gVvY--DD6ed3_FtMSecCnSkRUuEJqhaTY1jUjAtGBUGCJOU07qRtNpjcIu5RPz203Ta21Zr8e1lt9cD-v0DcAx4kA.MEQCIBtVSUUZb6mXCYqSjRzWF8-HfZLqBUNuzlrpnV79gm1CAiAO74abPPnUwcBxNQTm4oOBI9lflgcAX8pWN4O9S-24nA> > > > if (lastReceived == null) { > // If no messages received, then yield. > context.yield(); > } > > Thanks > > On Tue, Sep 26, 2023 at 9:48 AM Ivan Dolinin <[email protected] > <mailto:[email protected]>> wrote: > Adding log in text. Looks like the image didn’t go through. > > 2023-09-26 08:16:58,207 DEBUG [Timer-Driven Process Thread-7] > o.a.nifi.amqp.processors.ConsumeAMQP > ConsumeAMQP[id=78f68d05-f59d-3b9b-746e-5d480be723df] has chosen to yield > its resources; will not be scheduled to run again for 1000 milliseconds > 2023-09-26 08:16:58,207 DEBUG [Timer-Driven Process Thread-9] > o.a.nifi.amqp.processors.ConsumeAMQP > ConsumeAMQP[id=85c40dd5-3f41-3895-25e9-ee3569331185] has chosen to yield > its resources; will not be scheduled to run again for 1000 milliseconds > > > Ivan Dolinin / (416) 583-5833 x2012 / > [email protected]<mailto<mailto:[email protected]%3cmailto>: > [email protected]<mailto:[email protected]>> > > From: Ivan Dolinin <[email protected]<mailto: > [email protected]<mailto:[email protected]>>> > Sent: Tuesday, September 26, 2023 12:20 PM > To: Joe Witt > <[email protected]<mailto:[email protected]<mailto:[email protected]%3cmailto:[email protected]>>>; > [email protected]<mailto:[email protected]<mailto:[email protected]%3cmailto:[email protected]>> > Subject: RE: Processor stuck in yielding > > > Joe, > > thank you for a very quick response. > > Here is the screenshot of the log. There is nothing else recorded for the > UUID 78f68d05-f59d-3b9b-746e-5d480be723df. I could not trace the beginning > of this state. We have 3 ConsumeAMQP processors reading from RabbitMQ, only > this one is yielding, other 2 reading messages as expected. > > > > > Ivan Dolinin / (416) 583-5833 x2012 / > [email protected]<mailto<mailto:[email protected]%3cmailto>: > [email protected]<mailto:[email protected]>> > > From: Joe Witt > <[email protected]<mailto:[email protected]<mailto:[email protected]%3cmailto:[email protected]>>> > Sent: Tuesday, September 26, 2023 11:49 AM > To: > [email protected]<mailto:[email protected]<mailto:[email protected]%3cmailto:[email protected]>> > Subject: Re: Processor stuck in yielding > > > Ivan, > > Please show what the logs state specifically if you can. > > That processor could yield for the following reasons from a quick look: > 1. The connection between it and the next processor in the flow is a full > queue not being processed fast enough. > 2. The data connection to AMQP is working but it is told there is no data > currently to pull. > 3. ... > > That is about it actually. I am looking at the current code though. NiFi > 1.8.0 is 5 years old (today ironically) and that processor and the > associated libraries and even framework have changed a lot since then. You > will definitely need to try your flow on the latest 1.23.x line if the > issue being seen is not a simple config problem. > > Thanks > > > On Tue, Sep 26, 2023 at 8:40 AM Ivan Dolinin > <[email protected]<mailto: > [email protected]<mailto:[email protected]>>> > wrote: > Hello, > > In one of our production instances running NiFi 1.8.0, the ConsumeAMQP > processor is stuck in yielding. All the log is saying that it has chosen to > yield. No errors are happening in the downstream processors. There are > thousands of messages in the queue it is reading from. The processor is > connecting to the queue. What could be the reason? > > > Thanks, > > Ivan Dolinin > >
