Re: [EXTERNAL] Re: Processor ID/Processor Name as a default attribute

[Lars Winderling]

Hi James, 

you might want to check the official user guide 
https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#data_provenance. In 
case you have specific questions, feel free to reach out to me directly!

Best, Lars

On 26 May 2023 12:20:38 CEST, James  wrote:
>Hi again
>
>It definitely could be a training issue on my part, working with Nifi for
>so long and STILL figuring things out and finding new cool things to try.
>
>My methodology is more about routing all errors (as an example) to a single
>point (processor) so there's 1 log file to monitor, and having an easy to
>trace "source processor" to look for.
>
>BUT, if you do things differently/better and feel like sharing some tips,
>I'd be very appreciative, even if you took this off the mailing list/direct
>mail.
>
>thanks again
>
>On Fri, May 26, 2023 at 11:54 AM Lars Winderling 
>wrote:
>
>> Hi folks, I think you can obtain the same information via provenance. Or
>> am I mssing something? It sounds to me like you want to do manual debugging
>> instead of automizing it, so provenance should be easily accessible. Enjoy
>> your day and take care! Best, Lars
>>
>>
>> On 26 May 2023 11:21:13 CEST, James  wrote:
>>
>>> Hi
>>>
>>> This would honestly make life SO much easier!
>>>
>>> I've worked around this by adding my own attributes that can get logged
>>> in a central "LogMessage" processor, but even that can prove tricky
>>> sometimes. And this solution would be so much cleaner.
>>>
>>> Just my 2c
>>>
>>> thanks
>>> James
>>>
>>> On Thu, May 25, 2023 at 5:11 PM Chirthani, Deepak Reddy <
>>> c-deepakreddy.chirth...@charter.com> wrote:
>>>
>>>> In a large dataflow with multiple processors, it will help identifying
>>>> which processor routed flowfile to failure relationship.
>>>>
>>>>
>>>>
>>>>
>>>> https://community.cloudera.com/t5/Support-Questions/NiFi-origin-processor-name-or-id-as-attribute/m-p/175408
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> *[image: image005]*
>>>>
>>>> *Deepak Reddy* | Data Engineer
>>>> ​IT Centers of Excellence
>>>> 13736 Riverport Dr., Maryland Heights, MO 63043
>>>>
>>>>
>>>>
>>>> *From:* Joe Witt 
>>>> *Sent:* Wednesday, May 24, 2023 9:51 AM
>>>> *To:* users@nifi.apache.org
>>>> *Subject:* [EXTERNAL] Re: Processor ID/Processor Name as a default
>>>> attribute
>>>>
>>>>
>>>>
>>>> *CAUTION:* The e-mail below is from an external source. Please exercise
>>>> caution before opening attachments, clicking links, or following guidance.
>>>>
>>>> Hello
>>>>
>>>>
>>>>
>>>> Can you describe how you would use this information?
>>>>
>>>>
>>>>
>>>>  These kinds of details and more are present in provenance data now.
>>>>
>>>>
>>>>
>>>> Thanks
>>>>
>>>>
>>>>
>>>> On Wed, May 24, 2023 at 7:45 AM Chirthani, Deepak Reddy <
>>>> c-deepakreddy.chirth...@charter.com> wrote:
>>>>
>>>> Is there any chance where Processor_ID or Processor_Name can be added as
>>>> a default attribute for each flowfile so that its value should be the
>>>> ID/Name of the most recent processor it was processed by regardless of the
>>>> relationship it sends to?
>>>>
>>>>
>>>>
>>>> https://issues.apache.org/jira/browse/NIFI-4284
>>>>
>>>>
>>>>
>>>> Thanks
>>>>
>>>>
>>>>
>>>> *[image: image005]*
>>>>
>>>> *Deepak Reddy* | Data Engineer
>>>> ​IT Centers of Excellence
>>>> 13736 Riverport Dr., Maryland Heights, MO 63043
>>>> <https://www.google.com/maps/search/13736+Riverport+Dr.,+Maryland+Heights,+MO+63043?entry=gmail&source=g>
>>>>
>>>>
>>>>
>>>> The contents of this e-mail message and
>>>> any attachments are intended solely for the
>>>> addressee(s) and may contain confidential
>>>> and/or legally privileged information. If you
>>>> are not the intended recipient of this message
>>>> or if this message has been addressed to you
>>>> in error, please immediately alert the sender
>>>> by reply e-mail and then delete this message
>>>> and any attachments. If you are not the
>>>> intended recipient, you are notified that
>>>> any use, dissemination, distribution, copying,
>>>> or storage of this message or any attachment
>>>> is strictly prohibited.
>>>>
>>>> The contents of this e-mail message and
>>>> any attachments are intended solely for the
>>>> addressee(s) and may contain confidential
>>>> and/or legally privileged information. If you
>>>> are not the intended recipient of this message
>>>> or if this message has been addressed to you
>>>> in error, please immediately alert the sender
>>>> by reply e-mail and then delete this message
>>>> and any attachments. If you are not the
>>>> intended recipient, you are notified that
>>>> any use, dissemination, distribution, copying,
>>>> or storage of this message or any attachment
>>>> is strictly prohibited.
>>>>
>>>

Re: [EXTERNAL] Re: Processor ID/Processor Name as a default attribute

[James]

Hi again

It definitely could be a training issue on my part, working with Nifi for
so long and STILL figuring things out and finding new cool things to try.

My methodology is more about routing all errors (as an example) to a single
point (processor) so there's 1 log file to monitor, and having an easy to
trace "source processor" to look for.

BUT, if you do things differently/better and feel like sharing some tips,
I'd be very appreciative, even if you took this off the mailing list/direct
mail.

thanks again

On Fri, May 26, 2023 at 11:54 AM Lars Winderling 
wrote:

> Hi folks, I think you can obtain the same information via provenance. Or
> am I mssing something? It sounds to me like you want to do manual debugging
> instead of automizing it, so provenance should be easily accessible. Enjoy
> your day and take care! Best, Lars
>
>
> On 26 May 2023 11:21:13 CEST, James  wrote:
>
>> Hi
>>
>> This would honestly make life SO much easier!
>>
>> I've worked around this by adding my own attributes that can get logged
>> in a central "LogMessage" processor, but even that can prove tricky
>> sometimes. And this solution would be so much cleaner.
>>
>> Just my 2c
>>
>> thanks
>> James
>>
>> On Thu, May 25, 2023 at 5:11 PM Chirthani, Deepak Reddy <
>> c-deepakreddy.chirth...@charter.com> wrote:
>>
>>> In a large dataflow with multiple processors, it will help identifying
>>> which processor routed flowfile to failure relationship.
>>>
>>>
>>>
>>>
>>> https://community.cloudera.com/t5/Support-Questions/NiFi-origin-processor-name-or-id-as-attribute/m-p/175408
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> *[image: image005]*
>>>
>>> *Deepak Reddy* | Data Engineer
>>> ​IT Centers of Excellence
>>> 13736 Riverport Dr., Maryland Heights, MO 63043
>>>
>>>
>>>
>>> *From:* Joe Witt 
>>> *Sent:* Wednesday, May 24, 2023 9:51 AM
>>> *To:* users@nifi.apache.org
>>> *Subject:* [EXTERNAL] Re: Processor ID/Processor Name as a default
>>> attribute
>>>
>>>
>>>
>>> *CAUTION:* The e-mail below is from an external source. Please exercise
>>> caution before opening attachments, clicking links, or following guidance.
>>>
>>> Hello
>>>
>>>
>>>
>>> Can you describe how you would use this information?
>>>
>>>
>>>
>>>  These kinds of details and more are present in provenance data now.
>>>
>>>
>>>
>>> Thanks
>>>
>>>
>>>
>>> On Wed, May 24, 2023 at 7:45 AM Chirthani, Deepak Reddy <
>>> c-deepakreddy.chirth...@charter.com> wrote:
>>>
>>> Is there any chance where Processor_ID or Processor_Name can be added as
>>> a default attribute for each flowfile so that its value should be the
>>> ID/Name of the most recent processor it was processed by regardless of the
>>> relationship it sends to?
>>>
>>>
>>>
>>> https://issues.apache.org/jira/browse/NIFI-4284
>>>
>>>
>>>
>>> Thanks
>>>
>>>
>>>
>>> *[image: image005]*
>>>
>>> *Deepak Reddy* | Data Engineer
>>> ​IT Centers of Excellence
>>> 13736 Riverport Dr., Maryland Heights, MO 63043
>>> <https://www.google.com/maps/search/13736+Riverport+Dr.,+Maryland+Heights,+MO+63043?entry=gmail&source=g>
>>>
>>>
>>>
>>> The contents of this e-mail message and
>>> any attachments are intended solely for the
>>> addressee(s) and may contain confidential
>>> and/or legally privileged information. If you
>>> are not the intended recipient of this message
>>> or if this message has been addressed to you
>>> in error, please immediately alert the sender
>>> by reply e-mail and then delete this message
>>> and any attachments. If you are not the
>>> intended recipient, you are notified that
>>> any use, dissemination, distribution, copying,
>>> or storage of this message or any attachment
>>> is strictly prohibited.
>>>
>>> The contents of this e-mail message and
>>> any attachments are intended solely for the
>>> addressee(s) and may contain confidential
>>> and/or legally privileged information. If you
>>> are not the intended recipient of this message
>>> or if this message has been addressed to you
>>> in error, please immediately alert the sender
>>> by reply e-mail and then delete this message
>>> and any attachments. If you are not the
>>> intended recipient, you are notified that
>>> any use, dissemination, distribution, copying,
>>> or storage of this message or any attachment
>>> is strictly prohibited.
>>>
>>

Re: [EXTERNAL] Re: Processor ID/Processor Name as a default attribute

[Lars Winderling]

Hi folks, I think you can obtain the same information via provenance. Or am I 
mssing something? It sounds to me like you want to do manual debugging instead 
of automizing it, so provenance should be easily accessible. Enjoy your day and 
take care! Best, Lars

On 26 May 2023 11:21:13 CEST, James  wrote:
>Hi
>
>This would honestly make life SO much easier!
>
>I've worked around this by adding my own attributes that can get logged in
>a central "LogMessage" processor, but even that can prove tricky sometimes.
>And this solution would be so much cleaner.
>
>Just my 2c
>
>thanks
>James
>
>On Thu, May 25, 2023 at 5:11 PM Chirthani, Deepak Reddy <
>c-deepakreddy.chirth...@charter.com> wrote:
>
>> In a large dataflow with multiple processors, it will help identifying
>> which processor routed flowfile to failure relationship.
>>
>>
>>
>>
>> https://community.cloudera.com/t5/Support-Questions/NiFi-origin-processor-name-or-id-as-attribute/m-p/175408
>>
>>
>>
>>
>>
>>
>>
>> *[image: image005]*
>>
>> *Deepak Reddy* | Data Engineer
>> ​IT Centers of Excellence
>> 13736 Riverport Dr., Maryland Heights, MO 63043
>>
>>
>>
>> *From:* Joe Witt 
>> *Sent:* Wednesday, May 24, 2023 9:51 AM
>> *To:* users@nifi.apache.org
>> *Subject:* [EXTERNAL] Re: Processor ID/Processor Name as a default
>> attribute
>>
>>
>>
>> *CAUTION:* The e-mail below is from an external source. Please exercise
>> caution before opening attachments, clicking links, or following guidance.
>>
>> Hello
>>
>>
>>
>> Can you describe how you would use this information?
>>
>>
>>
>>  These kinds of details and more are present in provenance data now.
>>
>>
>>
>> Thanks
>>
>>
>>
>> On Wed, May 24, 2023 at 7:45 AM Chirthani, Deepak Reddy <
>> c-deepakreddy.chirth...@charter.com> wrote:
>>
>> Is there any chance where Processor_ID or Processor_Name can be added as a
>> default attribute for each flowfile so that its value should be the ID/Name
>> of the most recent processor it was processed by regardless of the
>> relationship it sends to?
>>
>>
>>
>> https://issues.apache.org/jira/browse/NIFI-4284
>>
>>
>>
>> Thanks
>>
>>
>>
>> *[image: image005]*
>>
>> *Deepak Reddy* | Data Engineer
>> ​IT Centers of Excellence
>> 13736 Riverport Dr., Maryland Heights, MO 63043
>> <https://www.google.com/maps/search/13736+Riverport+Dr.,+Maryland+Heights,+MO+63043?entry=gmail&source=g>
>>
>>
>>
>> The contents of this e-mail message and
>> any attachments are intended solely for the
>> addressee(s) and may contain confidential
>> and/or legally privileged information. If you
>> are not the intended recipient of this message
>> or if this message has been addressed to you
>> in error, please immediately alert the sender
>> by reply e-mail and then delete this message
>> and any attachments. If you are not the
>> intended recipient, you are notified that
>> any use, dissemination, distribution, copying,
>> or storage of this message or any attachment
>> is strictly prohibited.
>>
>> The contents of this e-mail message and
>> any attachments are intended solely for the
>> addressee(s) and may contain confidential
>> and/or legally privileged information. If you
>> are not the intended recipient of this message
>> or if this message has been addressed to you
>> in error, please immediately alert the sender
>> by reply e-mail and then delete this message
>> and any attachments. If you are not the
>> intended recipient, you are notified that
>> any use, dissemination, distribution, copying,
>> or storage of this message or any attachment
>> is strictly prohibited.
>>

Re: [EXTERNAL] Re: Processor ID/Processor Name as a default attribute

[James]

Hi

This would honestly make life SO much easier!

I've worked around this by adding my own attributes that can get logged in
a central "LogMessage" processor, but even that can prove tricky sometimes.
And this solution would be so much cleaner.

Just my 2c

thanks
James

On Thu, May 25, 2023 at 5:11 PM Chirthani, Deepak Reddy <
c-deepakreddy.chirth...@charter.com> wrote:

> In a large dataflow with multiple processors, it will help identifying
> which processor routed flowfile to failure relationship.
>
>
>
>
> https://community.cloudera.com/t5/Support-Questions/NiFi-origin-processor-name-or-id-as-attribute/m-p/175408
>
>
>
>
>
>
>
> *[image: image005]*
>
> *Deepak Reddy* | Data Engineer
> ​IT Centers of Excellence
> 13736 Riverport Dr., Maryland Heights, MO 63043
>
>
>
> *From:* Joe Witt 
> *Sent:* Wednesday, May 24, 2023 9:51 AM
> *To:* users@nifi.apache.org
> *Subject:* [EXTERNAL] Re: Processor ID/Processor Name as a default
> attribute
>
>
>
> *CAUTION:* The e-mail below is from an external source. Please exercise
> caution before opening attachments, clicking links, or following guidance.
>
> Hello
>
>
>
> Can you describe how you would use this information?
>
>
>
>  These kinds of details and more are present in provenance data now.
>
>
>
> Thanks
>
>
>
> On Wed, May 24, 2023 at 7:45 AM Chirthani, Deepak Reddy <
> c-deepakreddy.chirth...@charter.com> wrote:
>
> Is there any chance where Processor_ID or Processor_Name can be added as a
> default attribute for each flowfile so that its value should be the ID/Name
> of the most recent processor it was processed by regardless of the
> relationship it sends to?
>
>
>
> https://issues.apache.org/jira/browse/NIFI-4284
>
>
>
> Thanks
>
>
>
> *[image: image005]*
>
> *Deepak Reddy* | Data Engineer
> ​IT Centers of Excellence
> 13736 Riverport Dr., Maryland Heights, MO 63043
> <https://www.google.com/maps/search/13736+Riverport+Dr.,+Maryland+Heights,+MO+63043?entry=gmail&source=g>
>
>
>
> The contents of this e-mail message and
> any attachments are intended solely for the
> addressee(s) and may contain confidential
> and/or legally privileged information. If you
> are not the intended recipient of this message
> or if this message has been addressed to you
> in error, please immediately alert the sender
> by reply e-mail and then delete this message
> and any attachments. If you are not the
> intended recipient, you are notified that
> any use, dissemination, distribution, copying,
> or storage of this message or any attachment
> is strictly prohibited.
>
> The contents of this e-mail message and
> any attachments are intended solely for the
> addressee(s) and may contain confidential
> and/or legally privileged information. If you
> are not the intended recipient of this message
> or if this message has been addressed to you
> in error, please immediately alert the sender
> by reply e-mail and then delete this message
> and any attachments. If you are not the
> intended recipient, you are notified that
> any use, dissemination, distribution, copying,
> or storage of this message or any attachment
> is strictly prohibited.
>

RE: [EXTERNAL] Re: Processor ID/Processor Name as a default attribute

[Chirthani, Deepak Reddy]

In a large dataflow with multiple processors, it will help identifying which 
processor routed flowfile to failure relationship.

https://community.cloudera.com/t5/Support-Questions/NiFi-origin-processor-name-or-id-as-attribute/m-p/175408



[image005]
Deepak Reddy | Data Engineer
​IT Centers of Excellence
13736 Riverport Dr., Maryland Heights, MO 63043

From: Joe Witt 
Sent: Wednesday, May 24, 2023 9:51 AM
To: users@nifi.apache.org
Subject: [EXTERNAL] Re: Processor ID/Processor Name as a default attribute

CAUTION: The e-mail below is from an external source. Please exercise caution 
before opening attachments, clicking links, or following guidance.
Hello

Can you describe how you would use this information?

 These kinds of details and more are present in provenance data now.

Thanks

On Wed, May 24, 2023 at 7:45 AM Chirthani, Deepak Reddy 
mailto:c-deepakreddy.chirth...@charter.com>>
 wrote:
Is there any chance where Processor_ID or Processor_Name can be added as a 
default attribute for each flowfile so that its value should be the ID/Name of 
the most recent processor it was processed by regardless of the relationship it 
sends to?

https://issues.apache.org/jira/browse/NIFI-4284

Thanks

[image005]
Deepak Reddy | Data Engineer
​IT Centers of Excellence
13736 Riverport Dr., Maryland Heights, MO 
63043<https://www.google.com/maps/search/13736+Riverport+Dr.,+Maryland+Heights,+MO+63043?entry=gmail&source=g>

The contents of this e-mail message and
any attachments are intended solely for the
addressee(s) and may contain confidential
and/or legally privileged information. If you
are not the intended recipient of this message
or if this message has been addressed to you
in error, please immediately alert the sender
by reply e-mail and then delete this message
and any attachments. If you are not the
intended recipient, you are notified that
any use, dissemination, distribution, copying,
or storage of this message or any attachment
is strictly prohibited.
E-MAIL CONFIDENTIALITY NOTICE: 
The contents of this e-mail message and any attachments are intended solely for 
the addressee(s) and may contain confidential and/or legally privileged 
information. If you are not the intended recipient of this message or if this 
message has been addressed to you in error, please immediately alert the sender 
by reply e-mail and then delete this message and any attachments. If you are 
not the intended recipient, you are notified that any use, dissemination, 
distribution, copying, or storage of this message or any attachment is strictly 
prohibited.

Re: Processor ID/Processor Name as a default attribute

[Joe Witt]

Hello

Can you describe how you would use this information?

 These kinds of details and more are present in provenance data now.

Thanks

On Wed, May 24, 2023 at 7:45 AM Chirthani, Deepak Reddy <
c-deepakreddy.chirth...@charter.com> wrote:

> Is there any chance where Processor_ID or Processor_Name can be added as a
> default attribute for each flowfile so that its value should be the ID/Name
> of the most recent processor it was processed by regardless of the
> relationship it sends to?
>
>
>
> https://issues.apache.org/jira/browse/NIFI-4284
>
>
>
> Thanks
>
>
>
> *[image: image005]*
>
> *Deepak Reddy* | Data Engineer
> ​IT Centers of Excellence
> 13736 Riverport Dr., Maryland Heights, MO 63043
> 
>
>
> The contents of this e-mail message and
> any attachments are intended solely for the
> addressee(s) and may contain confidential
> and/or legally privileged information. If you
> are not the intended recipient of this message
> or if this message has been addressed to you
> in error, please immediately alert the sender
> by reply e-mail and then delete this message
> and any attachments. If you are not the
> intended recipient, you are notified that
> any use, dissemination, distribution, copying,
> or storage of this message or any attachment
> is strictly prohibited.
>

Processor ID/Processor Name as a default attribute

[Chirthani, Deepak Reddy]

Is there any chance where Processor_ID or Processor_Name can be added as a 
default attribute for each flowfile so that its value should be the ID/Name of 
the most recent processor it was processed by regardless of the relationship it 
sends to?

https://issues.apache.org/jira/browse/NIFI-4284

Thanks

[image005]
Deepak Reddy | Data Engineer
​IT Centers of Excellence
13736 Riverport Dr., Maryland Heights, MO 63043

E-MAIL CONFIDENTIALITY NOTICE: 
The contents of this e-mail message and any attachments are intended solely for 
the addressee(s) and may contain confidential and/or legally privileged 
information. If you are not the intended recipient of this message or if this 
message has been addressed to you in error, please immediately alert the sender 
by reply e-mail and then delete this message and any attachments. If you are 
not the intended recipient, you are notified that any use, dissemination, 
distribution, copying, or storage of this message or any attachment is strictly 
prohibited.