Hi Mark, forgot to ask about VolatileFlowFileRepository you mentioned, if I switch to use VolatileFlowFileRepository, will NiFi swap out all the other FlowFiles to disk if a queue is already full? Is it just simply keeping all FlowFiles in memory?
Regards, Ben 2018-01-12 12:07 GMT+08:00 尹文才 <batman...@gmail.com>: > Thanks Mark, my case is that I'm using NiFi to do some ETL work and it's > possible that NiFi dies unexpectedly due to lack of system resources. After > NiFi restarts itself, > I will re-extract all the data from database and re-perform all the > operations, so I need to clear all possible FlowFiles that might exist in > any queue. > > Regards, > Ben > > 2018-01-12 11:49 GMT+08:00 Mark Payne <marka...@hotmail.com>: > >> Ben, >> >> I have to admit - that’s kind of an odd request :) I’m curious what the >> use case is, if you can share? >> >> Regardless, the easiest way would be to update nifi.properties so that >> the FlowFile repo that is used is the VolatileFlowFileRepository. This >> would avoid writing the FlowFile state to disk, so ok restart you will lose >> all FlowFiles. The content will still be present, but nifi will delete it >> all on startup because there is no FlowFile associated with it. >> >> I’m on my phone right now so can’t easily tell you the exact name of the >> property to change but you’ll probably find it pretty quickly. The Admin >> Guide may well explain the different repositories as well. >> >> Thanks >> -Mark >> >> Sent from my iPhone >> >> > On Jan 11, 2018, at 10:31 PM, 尹文才 <batman...@gmail.com> wrote: >> > >> > Hi guys, I'm trying to clear all FlowFIles in all queues when NiFi is >> > restarted, but I don't know the correct way to do this. I checked all >> > NiFi's guide documentation, >> > it seems there're 2 possible solutions: >> > 1. write a custom notification service: a notification service could be >> > notified when NiFi is restarted and then inside the service, delete all >> the >> > files inside content_repository, flowfile_repository and >> > provenance_repository. >> > I know there're now 2 existing services: email and http. But I'm not >> > quite sure how to correctly write one and deploy it into my NiFi >> > environment, is there a tutorial on writing one notification service? >> > >> > 2. I know from the developer guide that by using the annotation >> @Shutdown >> > in a custom processor, the method could be called when NiFi is >> successfully >> > shut down. The problem with this approach is the method could >> > not be guaranteed to be called when NiFi dies unexpectedly. >> > >> > Does anyone know what is the correct way to implement it? Thanks. >> > >> > Regards, >> > Ben >> > >
