that isn't clearing them out really. it gradually decreases but doesn't clear out 500k items. On Nov 12, 2015 8:22 AM, "Juan Sequeiros" <juan.sequei...@onyxpoint.com> wrote:
> To answer Christopher, > > You can clear out flowfiles on a queue by setting expiration on > relationship line. > > On Thu, Nov 12, 2015 at 7:53 AM, Christopher Hamm <ceham...@gmail.com> > wrote: > >> I will look, but how do I just kill it with fire. What can I delete? >> On Nov 12, 2015 7:42 AM, "Mark Payne" <marka...@hotmail.com> wrote: >> >>> Chris / Elli, >>> >>> What version of NiFi are you running? I am wondering if you may have >>> been bitten by >>> this bug [1]. This should affect only the 'master' branch, not any >>> formally released >>> versions of NiFi. If you built the application from source code, rather >>> than using an >>> already-released version, this may well be what is happening. >>> >>> Thanks >>> -Mark >>> >>> >>> [1] https://issues.apache.org/jira/browse/NIFI-1155 >>> >>> >>> On Nov 12, 2015, at 2:32 AM, Christopher Hamm <em...@christopherhamm.com> >>> wrote: >>> >>> where does it dump? I cant get mine to expire fast enough. It is dev >>> test I want to blow away but cant delete. >>> >>> On Fri, Nov 6, 2015 at 11:33 AM, Joe Witt <joe.w...@gmail.com> wrote: >>> >>>> Elli, >>>> >>>> Can you share your flow configuration that we could possibly use to >>>> replicate this? Perhaps turn this into a JIRA and attach that. >>>> >>>> Also anytime you see something that appears 'stuck' please try to get >>>> a stack dump (bin/nifi.sh dump) As if there is truly a stuck thread >>>> we'll see it there and what it is blocked on. >>>> >>>> Thanks >>>> Joe >>>> >>>> On Fri, Nov 6, 2015 at 11:25 AM, Elli Schwarz < >>>> eliezer_schw...@yahoo.com> wrote: >>>> > I had a queue that built up overnight with several thousand >>>> flowfiles. The >>>> > queue was pointing to a RouteOnAttribute processor, which was >>>> running. For >>>> > some reason, the RouteOnAttribute processor wasn't emptying the queue >>>> so the >>>> > queue just built up. Stopping and starting the processor didn't help. >>>> > However, simply restarting Nifi got the flow moving again. I can't >>>> figure >>>> > out why the queue was stuck. >>>> > >>>> > What could cause a queue to build up like that? How would restarting >>>> Nifi >>>> > get it going again? There weren't any errors in the log. I did have >>>> > backpressure on the relationship to the processor after the >>>> RouteOnAttribute >>>> > set to 1, but that queue was empty. There was no backpressure on the >>>> queue >>>> > heading in to the RouteOnAttribute processor. The RouteOnAttribute >>>> processor >>>> > has a Penalty Duration and a Yield of 1 sec. The relationships all >>>> used the >>>> > PriorityAttributeAnalyzer as the prioritizer. >>>> > >>>> > The interesting thing is I had the exact same problem on two >>>> different nifis >>>> > (one was version 0.2.0 and one 0.3.0). The same type of flow files >>>> were >>>> > going from one to the other using site-to-site (I was doing this for >>>> load >>>> > balancing). On the other Nifi, instead of restarting, I added another >>>> > RouteOnAttribute processor and rerouted the stuck queue to the new one >>>> > instead, and that also got things moving again. (I have a >>>> backpressure since >>>> > the processor after the RouteOnAttribute is a ControlRate processor, >>>> and I >>>> > only want to allow 1 flowfile through per second so as not to >>>> overwhelm the >>>> > system downstream. The backpressure is to assist with load balancing, >>>> so if >>>> > one pathway fills up, I have the other pathway which uses the >>>> site-to-site >>>> > to route the flow to the other nifi). >>>> > >>>> > Is there some combination of backpressure, penalty or yield along with >>>> > prioritization that could cause a kind of deadlock-like situation? >>>> Any ideas >>>> > as to how I can prevent this from occurring? >>>> > >>>> > Thanks! >>>> > -Elli >>>> > >>>> >>> >>> >>> >>> -- >>> Sincerely, >>> Chris Hamm >>> (E) ceham...@gmail.com >>> (Twitter) http://twitter.com/webhamm >>> (Linkedin) http://www.linkedin.com/in/chrishamm >>> >>> >>> >
