Re: Run 1 instance of ExecuteStreamCommand constantly

[moe2017]

Thanks for the response. I'll give that a shot. I was hoping to avoid sending
the data through another layer since we're going to have so much of it.

Thanks,
Moe



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Re: Run 1 instance of ExecuteStreamCommand constantly

[Koji Kawamura]

Hi,

If the script encounters a while(1) loop when it called from NiFi,
then NiFi can not do anything until the loop ends.

To achieve what you described (keep using the same instance of a
script), I'd recommend to implement an API endpoint in that script,
e.g. a simple REST endpoint to receive new input, then let NiFi make
HTTP requests using InvokeHTTP processor.
This way, your python script can receive new input and also updating ML model.

Thanks,
Koji

On Fri, Nov 17, 2017 at 4:24 AM, moe2017  wrote:
> Hey,
>
> I have python machine learning model being executed my an
> ExecuteStreamCommand processor. The problem right now is that I need this
> processor to be executed once and then consistently loop so that the model
> can update itself when new data is passed to it from stdin.
>
> I tried putting in a while(1) in my script and then sending the output to
> stdout but NiFi just hangs and the processor won't take in the data from the
> queue. Is NiFi not equipped to handle while loops? Or is there a NiFi way
> that I can constantly run only one instance of a script during the entire
> lifecycle of the pipeline?
>
> Thanks
>
>
>
> --
> Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Run 1 instance of ExecuteStreamCommand constantly

[moe2017]

Hey,

I have python machine learning model being executed my an
ExecuteStreamCommand processor. The problem right now is that I need this
processor to be executed once and then consistently loop so that the model
can update itself when new data is passed to it from stdin.

I tried putting in a while(1) in my script and then sending the output to
stdout but NiFi just hangs and the processor won't take in the data from the
queue. Is NiFi not equipped to handle while loops? Or is there a NiFi way
that I can constantly run only one instance of a script during the entire
lifecycle of the pipeline?

Thanks



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/