Hi all, I'd like to start a discussion about improving the new TableSource and TableSink interfaces.
Most connectors have been migrated to FLIP-95, but there are still the Filesystem and Hive that have not been migrated. They have some requirements on table connector API. And users also have some additional requirements: - Some connectors have the ability to infer parallelism, the parallelism is good for most cases. - Users have customized parallelism configuration requirements for source and sink. - The connectors need to use topology to build their source/sink instead of a single function. Like JIRA[1], Partition Commit feature and File Compaction feature. Details are in [2]. [1]https://issues.apache.org/jira/browse/FLINK-18674 [2] https://cwiki.apache.org/confluence/display/FLINK/FLIP-146%3A+Improve+new+TableSource+and+TableSink+interfaces Best, Jingsong