[ https://issues.apache.org/jira/browse/NIFI-13487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zsihovszki Krisztina reassigned NIFI-13487: ------------------------------------------- Assignee: Zsihovszki Krisztina > ConsumeKinesisStream processor does not consume records inserted before its > start > --------------------------------------------------------------------------------- > > Key: NIFI-13487 > URL: https://issues.apache.org/jira/browse/NIFI-13487 > Project: Apache NiFi > Issue Type: Bug > Affects Versions: 2.0.0-M4 > Reporter: Zsihovszki Krisztina > Assignee: Zsihovszki Krisztina > Priority: Major > > Noticed a changed behaviour for ConsumeKinesisStream processor when > "Initial Stream Position" is set to "TRIM_HORIZON". > ConsumeKinesisStream consumes only records which were inserted to Kinesis > stream after the processor start and it does not consume the records which > were insterted before processor start. > In case of "TRIM_HORIZON" the records inserted before processor start (app > registration) should be consumed as well. > > The processor behaved as it used "LATEST" initial stream position. > The issue occurs for NiFi 2.0 only and the root cause seems to be related to > NIFI-8531: Upgrading Kinesis processors to AWS SDK 2.x > > According to > [https://docs.aws.amazon.com/streams/latest/dev/kcl-migration.html,] setting > initialPositionInStreamExtended was moved from LeaseManagementConfig to > RetrievalConfig. > The [existing > implementation|https://github.com/apache/nifi/blob/main/nifi-extension-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/kinesis/stream/ConsumeKinesisStream.java] > sets the value still in LeaseManagementConfig, this is the reason why the > default value, "LATEST" was used as initial position. -- This message was sent by Atlassian Jira (v8.20.10#820010)