I don’t know which version of nifi you are using. But in the newer versions (1.13->) there are the possibility to only allow one incoming flow file to a process group. You could create a PG for unzipping where you can have a loop back if there are zip files inside the parent zip. As long there are files in the PG no new files will enter.
Regards Jens M. Kofoed > Den 20. aug. 2021 kl. 09.49 skrev Aurélien Mazoyer <aurel...@aepsilon.com>: > > Hi Shawn, > > Thanks for your answer: that sounds good. However I did not manage to remove > the backpressure on queues in Nifi... Is it possible to do it or should I put > a super high limit so it would be as if there would be no limit? > > Thanks! > > Aurelien > >> Le jeu. 19 août 2021 à 18:10, Shawn Weeks <swe...@weeksconsulting.us> a >> écrit : >> I’ve had to deal with this exact problem. You must ensure that inside the >> loop is set to no back pressure. You can set back pressure on the entrance >> and final exit from the loop to control how much is in the loop but not in >> the middle. >> >> >> >> Thanks >> >> Shawn >> >> >> >> From: Aurélien Mazoyer <aurel...@aepsilon.com> >> Sent: Thursday, August 19, 2021 10:55 AM >> To: users@nifi.apache.org >> Subject: Re: Deadlock in loop >> >> >> >> Hello, >> >> >> >> Let me give a bit more info. Here is a screen of my blocked flow : >> >> <image001.png> >> >> The queues in the loop are all reaching their back pressure limit (set to >> 0.2 on my test cluster to reproduce more easily the behavior of my prod >> cluster). As a consequence, it seems that all processors that are part of >> the loop are stopped. >> >> >> Any thoughts about this problem? >> >> >> Thank you >> >> >> >> Aurelien >> >> >> >> >> >> Le mer. 18 août 2021 à 16:18, Aurélien Mazoyer <aurel...@aepsilon.com> a >> écrit : >> >> Hello Joe, >> >> >> >> Thank you for your email. Sure, please find attached a template that >> contains the loop itself. >> >> >> >> Best, >> >> >> >> Aurelien >> >> >> >> Le mer. 18 août 2021 à 14:31, Joe Witt <joe.w...@gmail.com> a écrit : >> >> Hello >> >> >> >> This case should work very well. Please share the details of the flow >> configuration. Can you download a flow template and share that? >> >> >> >> thanks >> >> >> >> On Wed, Aug 18, 2021 at 8:20 AM Aurélien Mazoyer <aurel...@aepsilon.com> >> wrote: >> >> Hi, >> >> >> >> I have a nifi flow that reads zip files. For each non-zip file it performs >> some treatment on its content and for each zip file it unzips it and >> performs the treatment on files in the archive. There is a loop in the flow >> so if a zip contains a zip, this zip will be reinjected at the beginning of >> the flow to be processed (and so one). However, when I have several zips in >> an archive, I experience a deadlock in my loop. Is there a solution to >> mitigate this issue in Nifi, such as having a back pressure on the first >> processor of the loop depending on the state on the queues in loop? >> >> >> >> Thank you, >> >> >> >> Aurelien
