Abhi Gupta created FLINK-40184:
----------------------------------
Summary: Remove inconsistency checks from DynamoDB Connector
Key: FLINK-40184
URL: https://issues.apache.org/jira/browse/FLINK-40184
Project: Flink
Issue Type: Improvement
Components: Connectors / DynamoDB
Affects Versions: aws-connector-6.0.1, aws-connector-6.0.0,
aws-connector-5.1.0, aws-connector-5.0.0, aws-connector-6.1.0
Reporter: Abhi Gupta
The
[discoverSplits|https://github.com/apache/flink-connector-aws/blob/main/flink-connector-aws/flink-connector-dynamodb/src/main/java/org/apache/flink/connector/dynamodb/source/enumerator/DynamoDbStreamsSourceEnumerator.java#L284C30-L284C44]
function is currently broken in the DDB Connector, where if it discovers an
inconsistency, it returns an empty list to its caller function, which can cause
bootstrap delays. As seen in KCLv3 version, we dont need to resolve
inconsistencies and can rely on periodic shard syncing. We can remove this
complex inconsistency resolution logic from the flink connector altogether
similar to DynamoDB Streams Kinesis Adapter
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
