Hi Shawn,
I think it's a hard question to answer in the general sense. A lot depends
on your specific implementation. What are you using to originate your flow?
Is it a GenerateFlowFile processor, a List, or something else? There are a
couple of different strategies that I've discovered for achieving gating
with Wait/Notify pairs; one of them is having a single flowfile where each
line is some item that you need to process, which is then split line by
line. When you do that, you end up with the fragment.identifier and
fragment.count properties on all the resulting split flowfiles and then you
can feed those into the Wait/Notify logic. Another trick has been that if
you know a single flowfile is going to be somehow transformed into multiple
ones, you can tack on an identifier like a UUID or some other sort of thing
and use that as the identifier in Wait/Notify. I have also built out nested
loops with two pairs as you described, but in general that tends to be
quite cumbersome.
Jerry
On Tue, Apr 23, 2019 at 9:32 PM Shawn Weeks
wrote:
> I’m working to implement a flow where for a given source of data I can
> only be processing one set at a time due to external dependencies. Each set
> needs to go through several different steps so this isn’t just a matter of
> limiting concurrency for a single processor. I’m trying to implement this
> using Wait and Notify as a gate and I’ve ran into a couple of limitations
> that I’m not sure how to get around. I first set my Wait processors to wait
> for a specific counter to be reset to zero before allowing a data source
> through but I quickly discovered that the Wait Processors tries to divide
> the signal counter leading to a divide by zero error. I’m assuming if zero
> isn’t a valid value for the signal counter we should disallow it however
> since you can’t use the Notify processor to set arbitrary values other than
> zero I’m not sure how your supposed to make a 0 or 1 gate. Are you supposed
> to have two Notify Processors back to back where one resets the counter to
> zero and the next increments by one? That seems a bit clunky.
>
>
>
> Thoughts?
>
>
>
> Thanks
>
> Shawn Weeks
>
--
http://www.google.com/profiles/grapesmoker