Change internal state of a processor

2021-02-18 Thread Jorge Machado
Hello everyone, I think we need a tool that allows us to change the internal state of a processor. Like for example I would like to got 10 steps back from the state that has a value of 10. The resulting state would be 0 We could just overwrite what is there… Thanks

Re: [E] NIFI show version changed *, but version control show no local changes

2021-02-18 Thread Mark Payne
Jens, Thanks for reporting this. I finally got around to digging in to see what’s happening. I filed a Jira [1] for this. I was able to replicate it pretty easily - thanks for the very well-documented steps! I should have a fix up shortly, I think. Thanks -Mark [1]

Re: Questions about the GetFile processor

2021-02-18 Thread Jean-Sebastien Vachon
OK thanks I missed that part of the documentation. Stupid me Jean-Sébastien Vachon Co-Founder & Architect Brizo Data, Inc. www.brizodata.com From: Arpad Boda Sent: Thursday,

Re: Questions about the GetFile processor

2021-02-18 Thread Arpad Boda
GetFile has no persistence. Actually it has, but it's called your hard drive. :) If you take a look at the documentation: *Keep Source File - *"If true, the file is not deleted after it has been copied to the Content Repository; this causes the file to be picked up continually and is useful for

Re: Questions about the GetFile processor

2021-02-18 Thread Jean-Sebastien Vachon
OK I understand your point.. sorry (early morning)  I am kind of stuck with the GetFile processor for now. Is there a way to know how many files are left to process? Will it go forever? or will it stops streaming once all files have been processed? (there are no new files in the folder...

Re: Questions about the GetFile processor

2021-02-18 Thread Jean-Sebastien Vachon
Thanks for your comment. However, I can't queue everything as the total size of the data is around 560GB. Right now, I am using a GetFile processor and it has been running for a few days. If I look at my end point, it looks like it should be done pretty soon but data is still streaming in at

Re: Questions about the GetFile processor

2021-02-18 Thread Arpad Boda
You can use the combination of listfile and fetchfile. In the queue between the two you are going to see the number of (flow)files left to be processed. On Thu, Feb 18, 2021 at 2:14 PM Jean-Sebastien Vachon < jsvac...@brizodata.com> wrote: > Hi all, > > If I configure a GetFile processor to list

Questions about the GetFile processor

2021-02-18 Thread Jean-Sebastien Vachon
Hi all, If I configure a GetFile processor to list all JSON files under a given folder, will it stops sending flows once it has processed all files? My folder contains thousands of files and the processor reads them by small batch (10) every 30s. Is there a way to know how many files are left