Hi Andrew, sorry I didn't follow your idea, could you please elaborate with more details? What I want to do is to be able to clear all the FlowFiles when NiFi dies unexpectedly and restarts itself.
Regards, Ben 2018-01-12 12:44 GMT+08:00 Andrew Grande <apere...@gmail.com>: > Perhaps you could delete the repository directories when you need to > restart with no data? > > On Thu, Jan 11, 2018, 9:16 PM 尹文才 <batman...@gmail.com> wrote: > > > Hi Mark, forgot to ask about VolatileFlowFileRepository you mentioned, > if I > > switch to use VolatileFlowFileRepository, will NiFi swap out all the > other > > FlowFiles to disk if a queue is already full? > > Is it just simply keeping all FlowFiles in memory? > > > > Regards, > > Ben > > > > 2018-01-12 12:07 GMT+08:00 尹文才 <batman...@gmail.com>: > > > > > Thanks Mark, my case is that I'm using NiFi to do some ETL work and > it's > > > possible that NiFi dies unexpectedly due to lack of system resources. > > After > > > NiFi restarts itself, > > > I will re-extract all the data from database and re-perform all the > > > operations, so I need to clear all possible FlowFiles that might exist > in > > > any queue. > > > > > > Regards, > > > Ben > > > > > > 2018-01-12 11:49 GMT+08:00 Mark Payne <marka...@hotmail.com>: > > > > > >> Ben, > > >> > > >> I have to admit - that’s kind of an odd request :) I’m curious what > the > > >> use case is, if you can share? > > >> > > >> Regardless, the easiest way would be to update nifi.properties so that > > >> the FlowFile repo that is used is the VolatileFlowFileRepository. This > > >> would avoid writing the FlowFile state to disk, so ok restart you will > > lose > > >> all FlowFiles. The content will still be present, but nifi will delete > > it > > >> all on startup because there is no FlowFile associated with it. > > >> > > >> I’m on my phone right now so can’t easily tell you the exact name of > the > > >> property to change but you’ll probably find it pretty quickly. The > Admin > > >> Guide may well explain the different repositories as well. > > >> > > >> Thanks > > >> -Mark > > >> > > >> Sent from my iPhone > > >> > > >> > On Jan 11, 2018, at 10:31 PM, 尹文才 <batman...@gmail.com> wrote: > > >> > > > >> > Hi guys, I'm trying to clear all FlowFIles in all queues when NiFi > is > > >> > restarted, but I don't know the correct way to do this. I checked > all > > >> > NiFi's guide documentation, > > >> > it seems there're 2 possible solutions: > > >> > 1. write a custom notification service: a notification service could > > be > > >> > notified when NiFi is restarted and then inside the service, delete > > all > > >> the > > >> > files inside content_repository, flowfile_repository and > > >> > provenance_repository. > > >> > I know there're now 2 existing services: email and http. But I'm > not > > >> > quite sure how to correctly write one and deploy it into my NiFi > > >> > environment, is there a tutorial on writing one notification > service? > > >> > > > >> > 2. I know from the developer guide that by using the annotation > > >> @Shutdown > > >> > in a custom processor, the method could be called when NiFi is > > >> successfully > > >> > shut down. The problem with this approach is the method could > > >> > not be guaranteed to be called when NiFi dies unexpectedly. > > >> > > > >> > Does anyone know what is the correct way to implement it? Thanks. > > >> > > > >> > Regards, > > >> > Ben > > >> > > > > > > > > >
