Elli, Can you share your flow configuration that we could possibly use to replicate this? Perhaps turn this into a JIRA and attach that.
Also anytime you see something that appears 'stuck' please try to get a stack dump (bin/nifi.sh dump) As if there is truly a stuck thread we'll see it there and what it is blocked on. Thanks Joe On Fri, Nov 6, 2015 at 11:25 AM, Elli Schwarz <eliezer_schw...@yahoo.com> wrote: > I had a queue that built up overnight with several thousand flowfiles. The > queue was pointing to a RouteOnAttribute processor, which was running. For > some reason, the RouteOnAttribute processor wasn't emptying the queue so the > queue just built up. Stopping and starting the processor didn't help. > However, simply restarting Nifi got the flow moving again. I can't figure > out why the queue was stuck. > > What could cause a queue to build up like that? How would restarting Nifi > get it going again? There weren't any errors in the log. I did have > backpressure on the relationship to the processor after the RouteOnAttribute > set to 1, but that queue was empty. There was no backpressure on the queue > heading in to the RouteOnAttribute processor. The RouteOnAttribute processor > has a Penalty Duration and a Yield of 1 sec. The relationships all used the > PriorityAttributeAnalyzer as the prioritizer. > > The interesting thing is I had the exact same problem on two different nifis > (one was version 0.2.0 and one 0.3.0). The same type of flow files were > going from one to the other using site-to-site (I was doing this for load > balancing). On the other Nifi, instead of restarting, I added another > RouteOnAttribute processor and rerouted the stuck queue to the new one > instead, and that also got things moving again. (I have a backpressure since > the processor after the RouteOnAttribute is a ControlRate processor, and I > only want to allow 1 flowfile through per second so as not to overwhelm the > system downstream. The backpressure is to assist with load balancing, so if > one pathway fills up, I have the other pathway which uses the site-to-site > to route the flow to the other nifi). > > Is there some combination of backpressure, penalty or yield along with > prioritization that could cause a kind of deadlock-like situation? Any ideas > as to how I can prevent this from occurring? > > Thanks! > -Elli >
