Hi all,

I'd like to start a discussion about improving the new TableSource and
TableSink interfaces.

Most connectors have been migrated to FLIP-95, but there are still the
Filesystem and Hive that have not been migrated. They have some
requirements on table connector API. And users also have some additional
requirements:
- Some connectors have the ability to infer parallelism, the parallelism is
good for most cases.
- Users have customized parallelism configuration requirements for source
and sink.
- The connectors need to use topology to build their source/sink instead of
a single function. Like JIRA[1], Partition Commit feature and File
Compaction feature.

Details are in [2].

[1]https://issues.apache.org/jira/browse/FLINK-18674
[2]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-146%3A+Improve+new+TableSource+and+TableSink+interfaces

Best,
Jingsong

Reply via email to