Hi devs, With the delivery of FLINK-31813 <https://issues.apache.org/jira/browse/FLINK-31813>[1] that offers a new Datastream Kinesis Streams source paving the way to deprecate the KCL backed `FlinkKinesisConsumer`, I would like to start working on migrating Kinesis Source Table API to the new source (FLINK-31987 <https://issues.apache.org/jira/browse/FLINK-31987>[2]) in a similar approach to what I did with the sink migration. This work however is blocked on migration of additional features like EFO support and deaggregation which seems to be idle for some time now. I want to discuss an alternative approach for migration ensuring backward compatibility.
Proposal 1: Given the pre-existing dependency from `flink-connector-kinesis` to `flink-connector-aws-kinesis-streams` I suggest we implement a `KinesisStreamsDynamicSource` backed by the new `KinesisStreamsSource` and add a `KinesisSourceSelector` to the existing source factory to dynamically select a source according to features enabled from the DDL, i.e shall the unsupported features like EFO or aggregation be enabled the legacy dynamic source would be provided, otherwise the new dynamic source is to be provided. Once all other features are implemented we migrate the factory as well. Proposal 2: We do nothing and wait for all features to be implemented. Let me know your thoughts. 1- https://issues.apache.org/jira/browse/FLINK-31813 2- https://issues.apache.org/jira/browse/FLINK-31987 Best Regards Ahmed Hamdy
