ListS3 processor question (duplicate files / maintaining state)

Hi, I had a question on the ListS3 processor. I'm using it to monitor the content of an S3 bucket. The idea is that when new files come in, they need to be processed and sent through the dataflow, using a FetchS3Object to process the file. This all works but I had 2 questions : 1. Where does the

Re: UI can take a very long time to become available

Hi, Just did a fresh build of 1.0.0-snapshot and it seems to have solved the issue. Replayed the 2 scenarios without an issue (Was using an "old" snapshot from 2 weeks ago.) Thanks a lot for the support on this ... -- View this message in context:

Re: UI can take a very long time to become available

ideo). I tried a similar > setup and could not recreate it. > > Let's see if we can get the perspective of a few other folks. > > If you can get a thread dump at all during the spinning that would be > really helpful. > > Thanks > Joe > > On Sat, Jun 25, 2016 at

Re: UI can take a very long time to become available

Here you can see the 2 crash scenarios mentioned in the previous post : https://dl.dropboxusercontent.com/u/13246619/nifi-crash/NifiCrash.mp4 - First one is deleting a processor with a non-empty queue attached - Second one is deleting a processor with a destination that is running both actions

Re: UI can take a very long time to become available

That did the trick... I assumed it would be something like that, but assumed it would apply sufficient back-pressure to stop the flow from getting flooded With the 1 x 100 approach things have stabilised, but I'm still seeing frequent hangs in the UI, forcing me to restart Nifi (This

Re: UI can take a very long time to become available

Thx for the tip ! Flow is up and running again. -- View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/UI-can-take-a-very-long-time-to-become-available-tp12201p12209.html Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Re: UI can take a very long time to become available

I've added a gist <https://gist.github.com/ddewaele/f84c223fff2c18af4efe3b7fabcab946> to show you some logging threaddumps of when the issue occurs. In this particular case I was unable to bring up Nifi I had to kill my mqtt broker so that the ConsumeMQTT processor wouldn't be pick

UI can take a very long time to become available

I have a simple dataflow running on 1.0.0-SNAPSHOT that looks like this ConsumeMQTT -> MergeContent (batch them up by 100 items) -> PutHDFS The ConsumeMQTT is receiving about 100 msgs per 5 minutes. >From time to time I need to restart Nifi because it seems to hang / starts processing