Re: Can we access Queued Duration as an attribute?

2024-02-16 Thread James McMahon
Good to know - very helpful. Thank you both, Joe W. and Mark P. On Thu, Feb 15, 2024 at 10:42 AM Joe Witt wrote: > This [1] blog seems amazingly appropriate and wow do we need these/any > such fields we intend to truly honor in a prominent place in the docs. > Super useful... > > [1]

Re: Can we access Queued Duration as an attribute?

2024-02-15 Thread Joe Witt
This [1] blog seems amazingly appropriate and wow do we need these/any such fields we intend to truly honor in a prominent place in the docs. Super useful... [1] https://jameswing.net/nifi/nifi-internal-fields.html Thanks On Thu, Feb 15, 2024 at 8:35 AM Mark Payne wrote: > Jim, > > You can

Re: Can we access Queued Duration as an attribute?

2024-02-15 Thread Mark Payne
Jim, You can actually reference “lastQueueDate” in Expression Language. It is formatted as number of milliseconds since epoch. So you might have a RouteOnAttribute that has a property named “old” with a value of: ${lastQueueDate:lt( ${now():minus(1)} )} So any FlowFile that has been

Re: Can we access Queued Duration as an attribute?

2024-02-15 Thread James McMahon
That would work - what a good suggestion. I'll do that. I can format the resulting number and then RouteOnAttribute by the desired subset of the result. Something like this to set attribute dt.failure: ${now():toNumber():toDate("-MM-ddHH:mm:ss"):format("MMddHHmmss","EST")} Then I can

Re: Can we access Queued Duration as an attribute?

2024-02-15 Thread Jim Steinebrey
You could add an UpdateAttribute processor first in the failure path to add a new attribute which contains the time the error occurred by using the ${now()} or ${now():toNumber()} expression language function. https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#now Then

Re: Can we access Queued Duration as an attribute?

2024-02-15 Thread James McMahon
As it turns out lineageStartDate and Queued Duration are very different. Without being able to get at Queued Duration as an attribute, it appears we cannot RouteOnAttribute to filter thousands in a queue by anything like hours they have been in queue. Why would this be helpful? Let us say we have

Re: Can we access Queued Duration as an attribute?

2024-02-14 Thread James McMahon
What a great workaround, thank you once again Mike. I'll put this in and use it now. Jim On Tue, Feb 13, 2024 at 4:41 PM Michael Moser wrote: > Hello James, > > I'm not aware of a way to access Queued Duration using expression > language, but you can access the Lineage Duration information.

Re: Can we access Queued Duration as an attribute?

2024-02-13 Thread Michael Moser
Hello James, I'm not aware of a way to access Queued Duration using expression language, but you can access the Lineage Duration information. The Getting Started Guide mentions both entryDate and lineageStartDate as immutable attributes on all flowfiles. These are numbers of milliseconds since

Can we access Queued Duration as an attribute?

2024-02-12 Thread James McMahon
When we examine the contents of a queue through the UI and select a flowfile from the resulting list, we see FlowFile Details in the Details tab. Are those key/values accessible from nifi expression language? I would like to access Queued Duration. I have a queue that holds flowfiles with