Jonathan Du created FLINK-37908:
-----------------------------------
Summary: KinesisStreamsSource connector de/registration strategy
fails on automated restarts
Key: FLINK-37908
URL: https://issues.apache.org/jira/browse/FLINK-37908
Project: Flink
Issue Type: Bug
Components: Connectors / Kinesis
Reporter: Jonathan Du
Attachments: jobmanager_log_filtered.txt
The KinesisStreamsSource Flink-Kinesis connector can encounter a race condition
if the EFO is enabled on the source stream and the application is started soon
after termination (such as in cases of autoscaling or automatic job restarts
from failover).
Failure conditions
* Source operator uses KinesisStreamsSource Flink-Kinesis connector
(org.apache.flink:flink-connector-kinesis:5.0.0-1.20)
* Reader type: EFO
* Trigger: Automated flink application restart
I've attached a few lines from the Job Manager logs showing this issue.
This issue was previously observed and resolved in the old
StreamConsumerRegistrar connector
(https://issues.apache.org/jira/browse/FLINK-24431), but has been reintroduced
in the new KinesisStreamsSource connector.
There are no existing workarounds because the KinesisStreamsSource defaults to
an eager registration/deregistration strategy.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
