Mark, While Pierre’s solution will probably be more helpful in the long run, this provides an easy way for me to examine the data by hand right now. I didn’t know that metric was on the Status History window, good to now!
Thanks, Peter From: Mark Payne [mailto:marka...@hotmail.com] Sent: Tuesday, November 07, 2017 9:26 PM To: users@nifi.apache.org Subject: [EXT] Re: Metrics on Lineage Duration Peter, In the UI you can right-click on a Processor and go to "View Status History" to see a chart of different metrics over time. One of those metrics is the Average Lineage Duration (averaged over a 5-minute period). Of course, you are asking for a reporting task/processor that does this. There's nothing that provides this info directly, but you could always use InvokeHTTP, or something like that to poll the REST API, using the path /nifi-api/flow/processors/<Processor ID>/status/history Or, another approach that may be valuable would be to use the SiteToSiteProvenanceReportingTask to send Provenance Events via Site-to-Site back to the NiFi instance. For a given Provenance Event, you can get Event Time and Lineage Start Date, so to calculate the Lineage Duration / latency for the FlowFile, you can just use EventTime - LineageStartDate. This approach would be a little different, as it provides lineage duration of each event. So this could be used, for instance, to detect any particular FlowFiles that exceed some SLA or calculate a 90th percentile type of thing, rather than getting an average. Does this help at all? Thanks -Mark On Nov 7, 2017, at 6:55 AM, Peter Wicks (pwicks) <pwi...@micron.com<mailto:pwi...@micron.com>> wrote: Is there a reporting task/processor that will let me aggregate Lineage Duration’s for a point in time so I can monitor my flow using this metric? Thanks, Peter
