Re: Nifi takes too long to start(~ 30 minutes)

2020-08-16 Thread Mohit Jain
Hi Pierre, No, the election process takes only a minute or two max, reaching the election process takes a lot of time. There are around 25k stopped/invalid components in the UI, I marked all of them as disabled hoping it would resolve the issue but no improvement in the startup time. On a similar

Re: Nifi takes too long to start(~ 30 minutes)

2020-08-16 Thread Mohit Jain
Hi Brandon, There are no flow files in the queue. Thanks, Mohit On Thu, Aug 13, 2020 at 9:56 PM Brandon DeVries wrote: > Mohit, > > How many flowfiles are currently on the instance? Sometimes a very large > number of flowfiles can result in slower start times. > > Brandon > >

Re: Detect duplicate records

2020-08-16 Thread Joe Witt
I believe Robert's case is that he has records flowing through bundled in flowfiles containing one or more of them at a time and he'd like to understand on a per record level (regardless of the flowfile they're contained in) whether that record has already been seen over some time interval. Detect

Re: Detect duplicate records

2020-08-16 Thread Jens M. Kofoed
So Robert too understand it correctly. You have a lot of records in one flow file. And if one record has been seen before that record should be removed? If true: wouldn’t it be a workflow that go through all records, record by record and join the final result. So first you would have to split all