Re: Nifi run.as

[John King]

Run nifi as "Nifi" and no it does not require sudo privileges

On Fri, May 17, 2019 at 1:23 PM Noe Detore  wrote:

> Hello,
>
> While using Nifi 1.5+ I want to update run.as on a linux box. Does this
> user require sudo. I have been finding conflicting information. If sudo is
> required what exact permissions are needed?
>
> Thank you
> N\oe
>

Re: Filter flowfiles by size from RouteOnAttribute

[Mike Thomsen]

Andy, FWIW I just edited the question to put a prominent note directing
anyone who sees it to review your answer.

On Fri, May 17, 2019 at 3:44 PM Suman B N  wrote:

> This totally worked for me! It is used to filter out the flow files with 0
> bytes. It allows flow files with size greater than zero.
> [image: Screen Shot 2019-05-18 at 1.00.23 AM.png]
>
> On Fri, May 17, 2019 at 11:14 PM Andy LoPresto 
> wrote:
>
>> Ha, I went to see if we could add a more helpful answer there and I see
>> I’ve been there before. The question’s author selected the incorrect answer
>> as most helpful to them. Content-Length is not a core attribute and would
>> only exist in a scenario where a previous processor populated it.
>>
>> Andy LoPresto
>> alopre...@apache.org
>> *alopresto.apa...@gmail.com *
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>
>> On May 17, 2019, at 10:29 AM, Mike Thomsen 
>> wrote:
>>
>> Thanks, Mark
>>
>> On Fri, May 17, 2019 at 1:29 PM Mark Payne  wrote:
>>
>>> Mike,
>>>
>>> You can reference ${fileSize}. This is the size of a FlowFile in bytes.
>>>
>>> Thanks
>>> -Mark
>>>
>>> On May 17, 2019, at 1:27 PM, Mike Thomsen 
>>> wrote:
>>>
>>> Is there a way to get access to the flowfile size from EL? This said to
>>> use Content-Length, but that didn't appear to work for me on 1.8.0.
>>>
>>>
>>> https://stackoverflow.com/questions/45854584/using-flowfile-size-as-an-argument-in-routeonattribute-nifi
>>>
>>> Thanks,
>>>
>>> Mike
>>>
>>>
>>>
>>
>
> --
> *Suman*
> *OlaCabs*
>

Re: Filter flowfiles by size from RouteOnAttribute

[Suman B N]

This totally worked for me! It is used to filter out the flow files with 0
bytes. It allows flow files with size greater than zero.
[image: Screen Shot 2019-05-18 at 1.00.23 AM.png]

On Fri, May 17, 2019 at 11:14 PM Andy LoPresto  wrote:

> Ha, I went to see if we could add a more helpful answer there and I see
> I’ve been there before. The question’s author selected the incorrect answer
> as most helpful to them. Content-Length is not a core attribute and would
> only exist in a scenario where a previous processor populated it.
>
> Andy LoPresto
> alopre...@apache.org
> *alopresto.apa...@gmail.com *
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On May 17, 2019, at 10:29 AM, Mike Thomsen  wrote:
>
> Thanks, Mark
>
> On Fri, May 17, 2019 at 1:29 PM Mark Payne  wrote:
>
>> Mike,
>>
>> You can reference ${fileSize}. This is the size of a FlowFile in bytes.
>>
>> Thanks
>> -Mark
>>
>> On May 17, 2019, at 1:27 PM, Mike Thomsen  wrote:
>>
>> Is there a way to get access to the flowfile size from EL? This said to
>> use Content-Length, but that didn't appear to work for me on 1.8.0.
>>
>>
>> https://stackoverflow.com/questions/45854584/using-flowfile-size-as-an-argument-in-routeonattribute-nifi
>>
>> Thanks,
>>
>> Mike
>>
>>
>>
>

-- 
*Suman*
*OlaCabs*

RE: Open / Close Gate examples?

[David Gallagher]

Hi Joe – thanks for answering. I can actually live with the message ack at the 
moment. But, I can’t have two messages at once because the underlying process 
will deadlock if two flows try to use it at the same time. This is a problem 
because my nifi node might be brought up after having been shut off for a few 
days, at which point it will read multiple messages off the queue and attempt 
to process them. The problem I’m having is getting flow files to wait for each 
other. If I use a Notify processor to set a release signal as indicated in the 
linked gist, it does release the Wait as designed. But, the next flowfile then 
enters the queue and it will get released when the Notify is triggered again by 
the GenerateFlowFile. What I really want to have happen is for each flowfile to 
block the next flowfile from entering until it has exited the process. I’m not 
really sure how to set that up successfully…

Thanks,

Dave

From: Joe Witt 
Sent: Friday, May 17, 2019 12:33 PM
To: users@nifi.apache.org
Subject: Re: Open / Close Gate examples?

Dave

Using Wait/Notify would ensure you only have one message in flight at a time 
(or it can/should).  But the message will be ack'd before processed.

For Kafka and some of these message queue mechanisms if we want to offer a 'do 
not ack until the whole flow is done' behavior we should update the processors 
to allow for that and actually this is a great use case for the 
nifi-fn/stateless work that SamH in the community has been talking about.

Thanks

On Fri, May 17, 2019 at 12:30 PM David Gallagher 
mailto:dgallag...@cleverdevices.com>> wrote:
Hi – I have a requirement to read one message off RabbitMQ and fully process 
that message before I take another message off the queue. I found some great 
guidance to setting up a ‘traffic light’ pattern here 
(https://gist.github.com/ijokarumawak/9e1a4855934f2bb9661f88ca625bd244),
 but it won’t work for my process because the Notify will get triggered by the 
GenerateFlowFile before the long-running process triggers, which will put two 
or more messages into the process at once. The documentation for Notify makes 
reference to using a 0 value in Signal Counter Delta to simulate 
‘open-close-gate’ flow control, which sounds like what I’m looking for. Can 
someone point me in the right direction?

Thanks,

Dave

Nifi run.as

[Noe Detore]

Hello,

While using Nifi 1.5+ I want to update run.as on a linux box. Does this
user require sudo. I have been finding conflicting information. If sudo is
required what exact permissions are needed?

Thank you
N\oe

Re: Filter flowfiles by size from RouteOnAttribute

[Andy LoPresto]

Ha, I went to see if we could add a more helpful answer there and I see I’ve 
been there before. The question’s author selected the incorrect answer as most 
helpful to them. Content-Length is not a core attribute and would only exist in 
a scenario where a previous processor populated it. 

Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On May 17, 2019, at 10:29 AM, Mike Thomsen  wrote:
> 
> Thanks, Mark
> 
> On Fri, May 17, 2019 at 1:29 PM Mark Payne  > wrote:
> Mike,
> 
> You can reference ${fileSize}. This is the size of a FlowFile in bytes.
> 
> Thanks
> -Mark
> 
>> On May 17, 2019, at 1:27 PM, Mike Thomsen > > wrote:
>> 
>> Is there a way to get access to the flowfile size from EL? This said to use 
>> Content-Length, but that didn't appear to work for me on 1.8.0.
>> 
>> https://stackoverflow.com/questions/45854584/using-flowfile-size-as-an-argument-in-routeonattribute-nifi
>>  
>> 
>> 
>> Thanks,
>> 
>> Mike
> 

Re: Filter flowfiles by size from RouteOnAttribute

[Mike Thomsen]

Thanks, Mark

On Fri, May 17, 2019 at 1:29 PM Mark Payne  wrote:

> Mike,
>
> You can reference ${fileSize}. This is the size of a FlowFile in bytes.
>
> Thanks
> -Mark
>
> On May 17, 2019, at 1:27 PM, Mike Thomsen  wrote:
>
> Is there a way to get access to the flowfile size from EL? This said to
> use Content-Length, but that didn't appear to work for me on 1.8.0.
>
>
> https://stackoverflow.com/questions/45854584/using-flowfile-size-as-an-argument-in-routeonattribute-nifi
>
> Thanks,
>
> Mike
>
>
>

Re: Filter flowfiles by size from RouteOnAttribute

[Mark Payne]

Mike,

You can reference ${fileSize}. This is the size of a FlowFile in bytes.

Thanks
-Mark

On May 17, 2019, at 1:27 PM, Mike Thomsen 
mailto:mikerthom...@gmail.com>> wrote:

Is there a way to get access to the flowfile size from EL? This said to use 
Content-Length, but that didn't appear to work for me on 1.8.0.

https://stackoverflow.com/questions/45854584/using-flowfile-size-as-an-argument-in-routeonattribute-nifi

Thanks,

Mike

Filter flowfiles by size from RouteOnAttribute

[Mike Thomsen]

Is there a way to get access to the flowfile size from EL? This said to use
Content-Length, but that didn't appear to work for me on 1.8.0.

https://stackoverflow.com/questions/45854584/using-flowfile-size-as-an-argument-in-routeonattribute-nifi

Thanks,

Mike

Re: Open / Close Gate examples?

[Joe Witt]

Dave

Using Wait/Notify would ensure you only have one message in flight at a
time (or it can/should).  But the message will be ack'd before processed.

For Kafka and some of these message queue mechanisms if we want to offer a
'do not ack until the whole flow is done' behavior we should update the
processors to allow for that and actually this is a great use case for the
nifi-fn/stateless work that SamH in the community has been talking about.

Thanks

On Fri, May 17, 2019 at 12:30 PM David Gallagher <
dgallag...@cleverdevices.com> wrote:

> Hi – I have a requirement to read one message off RabbitMQ and fully
> process that message before I take another message off the queue. I found
> some great guidance to setting up a ‘traffic light’ pattern here (
> https://gist.github.com/ijokarumawak/9e1a4855934f2bb9661f88ca625bd244),
> but it won’t work for my process because the Notify will get triggered by
> the GenerateFlowFile before the long-running process triggers, which will
> put two or more messages into the process at once. The documentation for
> Notify makes reference to using a 0 value in Signal Counter Delta to
> simulate ‘open-close-gate’ flow control, which sounds like what I’m looking
> for. Can someone point me in the right direction?
>
>
>
> Thanks,
>
>
>
> Dave
>
>
>
>
>

Open / Close Gate examples?

[David Gallagher]

Hi - I have a requirement to read one message off RabbitMQ and fully process 
that message before I take another message off the queue. I found some great 
guidance to setting up a 'traffic light' pattern here 
(https://gist.github.com/ijokarumawak/9e1a4855934f2bb9661f88ca625bd244), but it 
won't work for my process because the Notify will get triggered by the 
GenerateFlowFile before the long-running process triggers, which will put two 
or more messages into the process at once. The documentation for Notify makes 
reference to using a 0 value in Signal Counter Delta to simulate 
'open-close-gate' flow control, which sounds like what I'm looking for. Can 
someone point me in the right direction?

Thanks,

Dave