[
https://issues.apache.org/jira/browse/NIFI-16380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-16380:
----------------------------------
Fix Version/s: 2.13.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Add Records Metrics to ConsumeKinesis
> -------------------------------------
>
> Key: NIFI-16380
> URL: https://issues.apache.org/jira/browse/NIFI-16380
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: David Handermann
> Assignee: David Handermann
> Priority: Major
> Fix For: 2.13.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Following the pattern of {{{}ConsumeKafka{}}}, the {{ConsumeKinesis}}
> Processor should track lag and record metrics for each invocation. The Map of
> attributes for Counters and Gauges should include the Kinesis Stream and
> Shard information.
> The metrics should include the following:
> * kinesis.bytes.consumed
> ** Type: Counter
> * kinesis.records.consumed
> ** Type: Counter
> * kinesis.consumer.milliseconds.behind
> ** Type: Gauge
> * kinesis.records.parsed.errors
> ** Type: Counter
> Kinesis supports the concept of
> [MillisBehindLatest|https://docs.aws.amazon.com/kinesis/latest/APIReference/API_GetRecords.html#API_GetRecords_ResponseSyntax]
> to indicate the number milliseconds behind the consumer is from the latest
> record in the stream. This is a loose indicator of lag because it does not
> provide the number of records and interpreting the number requires an
> understanding of expected periodicity for records added to the Kinesis Stream.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
