[
https://issues.apache.org/jira/browse/FLINK-36296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ferenc Csaky closed FLINK-36296.
--------------------------------
Resolution: Fixed
[{{68acdbd}}|https://github.com/apache/flink-connector-aws/commit/68acdbd69b9ba5beb00e26abf277b6ec5c660e92]
on main
> Add incremental shard discovery for DDB Streams source
> ------------------------------------------------------
>
> Key: FLINK-36296
> URL: https://issues.apache.org/jira/browse/FLINK-36296
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / DynamoDB
> Reporter: Abhi Gupta
> Assignee: Abhi Gupta
> Priority: Major
> Labels: pull-request-available
> Fix For: aws-connector-6.0.0
>
>
> DDB Streams source does a full shard discovery every minute, on large
> streams, this discovery can take more than 20 minutes to discover all shards,
> we can optimize this to do an incremental shard discovery periodically while
> also doing periodic shard discovery. This makes having the children shards
> faster and reduces the millisBehindLatest
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
