Re: Processor Signaling

[Frank Martínez]

Mark,

Thanks for your suggestions. I will do some experiments

Cheers,

Frank.

On Fri, Jan 22, 2016 at 8:43 AM, Mark Payne  wrote:

> Frank,
>
> The approach that you laid out here would certainly work. However, I feel
> that it may be a bit heavy-handed. If the idea is
> just to reload some information from a local file, then I would recommend
> that you take a look at the ScanContent / ScanAttribute
> Processors. They do something similar. They are configured with a file
> that is the Dictionary of Terms to route on. Then, they
> periodically check if the file has been updated. If so, they will reload
> the information, otherwise they will just keep what they
> already have loaded.
>
> They make use of SynchronousFileWatcher class that exists in the
> nifi-utils module, which helps to make this simpler.
>
> Please feel free to reach out with any additional questions, or if this
> doesn't seem like the right fit for you.
>
> Thanks
> -Mark
>
> On Jan 21, 2016, at 11:08 PM, Frank Martínez  wrote:
>
> Hi Joe,
>
> Thanks for your response.
>
> So the way is to get the instruction set from external source in onTrigger
> method each time? Or is there any way to push immediate commands or
> messages to the Procesor?
>
> Maybe I can create a custom ControllerService that provides and caches the
> current instruction set, then I can create a Custom processor that updates
> the cache of the ControllerService, then use GetFile to poll a directory
> where I can put files (commands) to be executed.
>
>
>
>   get current config
> +> [config source endpoint]
>|
> +---> ConfigLoaderProcessor
> ||
>   GetFileProcessor   | update config cache
> |\/
> DirectoryConfigControllerService
> |   ^
> CommandFile |
> .
> . get current config
> |
>   ...> RouterProcessor .>
>
>
> What do you think?
>
> Thanks
>
>
> On Thu, Jan 21, 2016 at 10:38 PM, Joe Witt  wrote:
>
>> Frank,
>>
>> Absolutely.  Have seen several of these of various types of the years.
>> It usually works best if the processor itself calls out/polls some
>> endpoint to get the latest instruction set.  This approach tends to
>> scale better, means clustered systems work smoothly as well, and it is
>> nice from a reliability perspective.  You can still choose to push new
>> configuration information to the processor but keep in mind you cannot
>> change its properties while it is running.
>>
>> Feel free to fire more questions as you dig deeper into it.
>>
>> Thanks
>> Joe
>>
>> On Thu, Jan 21, 2016 at 10:33 PM, Frank Martínez 
>> wrote:
>> > Hi Guys,
>> >
>> > I want to develop a custom Router processor, But I want that my
>> processor
>> > logic depends on external parameters than can be hot changed. So I have
>> to
>> > send some signals to the processor while it is running, so it will
>> change
>> > its internal routing logic. Is it possible to do that with apache nifi?
>> >
>> > Thanks,
>> >
>> > Frank Martínez.
>> >
>> > --
>> > Frank D. Martínez M.
>>
>
>
>
> --
> Frank D. Martínez M.
>
>
>


-- 
Frank D. Martínez M.

Re: Processor Signaling

[Mark Payne]

Frank,

The approach that you laid out here would certainly work. However, I feel that 
it may be a bit heavy-handed. If the idea is
just to reload some information from a local file, then I would recommend that 
you take a look at the ScanContent / ScanAttribute
Processors. They do something similar. They are configured with a file that is 
the Dictionary of Terms to route on. Then, they
periodically check if the file has been updated. If so, they will reload the 
information, otherwise they will just keep what they
already have loaded.

They make use of SynchronousFileWatcher class that exists in the nifi-utils 
module, which helps to make this simpler.

Please feel free to reach out with any additional questions, or if this doesn't 
seem like the right fit for you.

Thanks
-Mark

> On Jan 21, 2016, at 11:08 PM, Frank Martínez  wrote:
> 
> Hi Joe,
> 
> Thanks for your response.
> 
> So the way is to get the instruction set from external source in onTrigger 
> method each time? Or is there any way to push immediate commands or messages 
> to the Procesor?
> 
> Maybe I can create a custom ControllerService that provides and caches the 
> current instruction set, then I can create a Custom processor that updates 
> the cache of the ControllerService, then use GetFile to poll a directory 
> where I can put files (commands) to be executed.
> 
> 
> 
>   get current config
> +> [config source endpoint]
>   |
> +---> ConfigLoaderProcessor 
> ||
>   GetFileProcessor   | update config cache
> |\/
> DirectoryConfigControllerService
> |   ^
> CommandFile |
> .
> . get current config
> |
>   ...> RouterProcessor .>
>
> 
> What do you think?
> 
> Thanks
> 
> 
> On Thu, Jan 21, 2016 at 10:38 PM, Joe Witt  > wrote:
> Frank,
> 
> Absolutely.  Have seen several of these of various types of the years.
> It usually works best if the processor itself calls out/polls some
> endpoint to get the latest instruction set.  This approach tends to
> scale better, means clustered systems work smoothly as well, and it is
> nice from a reliability perspective.  You can still choose to push new
> configuration information to the processor but keep in mind you cannot
> change its properties while it is running.
> 
> Feel free to fire more questions as you dig deeper into it.
> 
> Thanks
> Joe
> 
> On Thu, Jan 21, 2016 at 10:33 PM, Frank Martínez  > wrote:
> > Hi Guys,
> >
> > I want to develop a custom Router processor, But I want that my processor
> > logic depends on external parameters than can be hot changed. So I have to
> > send some signals to the processor while it is running, so it will change
> > its internal routing logic. Is it possible to do that with apache nifi?
> >
> > Thanks,
> >
> > Frank Martínez.
> >
> > --
> > Frank D. Martínez M.
> 
> 
> 
> -- 
> Frank D. Martínez M.

Re: Processor Signaling

[Joe Witt]

Frank,

Absolutely.  Have seen several of these of various types of the years.
It usually works best if the processor itself calls out/polls some
endpoint to get the latest instruction set.  This approach tends to
scale better, means clustered systems work smoothly as well, and it is
nice from a reliability perspective.  You can still choose to push new
configuration information to the processor but keep in mind you cannot
change its properties while it is running.

Feel free to fire more questions as you dig deeper into it.

Thanks
Joe

On Thu, Jan 21, 2016 at 10:33 PM, Frank Martínez  wrote:
> Hi Guys,
>
> I want to develop a custom Router processor, But I want that my processor
> logic depends on external parameters than can be hot changed. So I have to
> send some signals to the processor while it is running, so it will change
> its internal routing logic. Is it possible to do that with apache nifi?
>
> Thanks,
>
> Frank Martínez.
>
> --
> Frank D. Martínez M.