Thanks Mark, Andrew and Russell, I think using the Volatile-implementations
repositories mentioned by Mark should be sufficient for me.
Regards,
Ben
2018-01-12 22:59 GMT+08:00 Russell Bateman <r...@windofkeltia.com>:
> Andrew just meant that if you smoke the contents of all the repository
> subdirectories under ${NIFI_ROOT}, it will result in what you seem to be
> asking for.
>
> Hope this helps.
>
>
> On 01/11/2018 09:48 PM, 尹文才 wrote:
>
>> Hi Andrew, sorry I didn't follow your idea, could you please elaborate
>> with
>> more details?
>> What I want to do is to be able to clear all the FlowFiles when NiFi dies
>> unexpectedly and restarts itself.
>>
>> Regards,
>> Ben
>>
>> 2018-01-12 12:44 GMT+08:00 Andrew Grande <apere...@gmail.com>:
>>
>> Perhaps you could delete the repository directories when you need to
>>> restart with no data?
>>>
>>> On Thu, Jan 11, 2018, 9:16 PM 尹文才 <batman...@gmail.com> wrote:
>>>
>>> Hi Mark, forgot to ask about VolatileFlowFileRepository you mentioned,
>>>>
>>> if I
>>>
>>>> switch to use VolatileFlowFileRepository, will NiFi swap out all the
>>>>
>>> other
>>>
>>>> FlowFiles to disk if a queue is already full?
>>>> Is it just simply keeping all FlowFiles in memory?
>>>>
>>>> Regards,
>>>> Ben
>>>>
>>>> 2018-01-12 12:07 GMT+08:00 尹文才 <batman...@gmail.com>:
>>>>
>>>> Thanks Mark, my case is that I'm using NiFi to do some ETL work and
>>>>>
>>>> it's
>>>
>>>> possible that NiFi dies unexpectedly due to lack of system resources.
>>>>>
>>>> After
>>>>
>>>>> NiFi restarts itself,
>>>>> I will re-extract all the data from database and re-perform all the
>>>>> operations, so I need to clear all possible FlowFiles that might exist
>>>>>
>>>> in
>>>
>>>> any queue.
>>>>>
>>>>> Regards,
>>>>> Ben
>>>>>
>>>>> 2018-01-12 11:49 GMT+08:00 Mark Payne <marka...@hotmail.com>:
>>>>>
>>>>> Ben,
>>>>>>
>>>>>> I have to admit - that’s kind of an odd request :) I’m curious what
>>>>>>
>>>>> the
>>>
>>>> use case is, if you can share?
>>>>>>
>>>>>> Regardless, the easiest way would be to update nifi.properties so that
>>>>>> the FlowFile repo that is used is the VolatileFlowFileRepository. This
>>>>>> would avoid writing the FlowFile state to disk, so ok restart you will
>>>>>>
>>>>> lose
>>>>
>>>>> all FlowFiles. The content will still be present, but nifi will delete
>>>>>>
>>>>> it
>>>>
>>>>> all on startup because there is no FlowFile associated with it.
>>>>>>
>>>>>> I’m on my phone right now so can’t easily tell you the exact name of
>>>>>>
>>>>> the
>>>
>>>> property to change but you’ll probably find it pretty quickly. The
>>>>>>
>>>>> Admin
>>>
>>>> Guide may well explain the different repositories as well.
>>>>>>
>>>>>> Thanks
>>>>>> -Mark
>>>>>>
>>>>>> Sent from my iPhone
>>>>>>
>>>>>> On Jan 11, 2018, at 10:31 PM, 尹文才 <batman...@gmail.com> wrote:
>>>>>>>
>>>>>>> Hi guys, I'm trying to clear all FlowFIles in all queues when NiFi
>>>>>>>
>>>>>> is
>>>
>>>> restarted, but I don't know the correct way to do this. I checked
>>>>>>>
>>>>>> all
>>>
>>>> NiFi's guide documentation,
>>>>>>> it seems there're 2 possible solutions:
>>>>>>> 1. write a custom notification service: a notification service could
>>>>>>>
>>>>>> be
>>>>
>>>>> notified when NiFi is restarted and then inside the service, delete
>>>>>>>
>>>>>> all
>>>>
>>>>> the
>>>>>>
>>>>>>> files inside content_repository, flowfile_repository and
>>>>>>> provenance_repository.
>>>>>>> I know there're now 2 existing services: email and http. But I'm
>>>>>>>
>>>>>> not
>>>
>>>> quite sure how to correctly write one and deploy it into my NiFi
>>>>>>> environment, is there a tutorial on writing one notification
>>>>>>>
>>>>>> service?
>>>
>>>> 2. I know from the developer guide that by using the annotation
>>>>>>>
>>>>>> @Shutdown
>>>>>>
>>>>>>> in a custom processor, the method could be called when NiFi is
>>>>>>>
>>>>>> successfully
>>>>>>
>>>>>>> shut down. The problem with this approach is the method could
>>>>>>> not be guaranteed to be called when NiFi dies unexpectedly.
>>>>>>>
>>>>>>> Does anyone know what is the correct way to implement it? Thanks.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Ben
>>>>>>>
>>>>>>
>>>>>
>
