Thanks Paul, that was an excellent idea and the line is added to the Groovy code that was already in the flow. Works now :)
Jeremy On Thu, Apr 15, 2021 at 12:44 AM Paul Kelly <[email protected]> wrote: > Whatever processor is putting it into the queue could set a penalty on > failed flow files that enter your retry queue. It's one of the properties > on the General tab. > > If you can't set it there for whatever reason, you could put an > ExecuteScript processor within the retry loop with the following Groovy > code in it, and set that ExecuteScript's penalty duration to 20s: > > flowFile = session.get() > if(!flowFile) return > session.penalize(flowFile) > session.transfer(flowFile, REL_SUCCESS) > > Paul > > On Wed, Apr 14, 2021 at 4:22 PM Jeremy Pemberton-Pigott < > [email protected]> wrote: > >> Yes it can. Is that an update attribute which can set that on the flow >> file? >> >> Regards, >> >> Jeremy >> >> >> On 15 Apr 2021, at 00:16, Paul Kelly <[email protected]> wrote: >> >> >> Are you able to penalize the flow file for 20s within the update retry >> queue? Penalizing without yielding would cause a flow file to sit in the >> queue for 20s before the next processor will try acting on it. >> >> Paul >> >> On Wed, Apr 14, 2021 at 5:05 AM Jeremy Pemberton-Pigott < >> [email protected]> wrote: >> >>> Hi, >>> >>> I have a parallel update process in 2 different flows running in a 3 >>> node cluster running 1.6.0, if the insert side has not completed yet the >>> update side moves the flow file to a waiting queue for retry. I want to >>> retry every 20s all the flow files in the queue that have been waiting that >>> long. There may be 10,000s waiting so I don't want to do 1 flow file every >>> 20s. Any idea how I can achieve this? I don't think that I can use the >>> yield mechanism because after n retries it goes into a notification logging >>> flow. And wait/notify won't work because the notification may appear >>> before the update flow file arrives. >>> >>> Jeremy >>> >>
