Very thanks Timo for starting this discussion. Big +1 for this.
The design looks good to me! We can add some documentation for connector developers. For example: for sink, If there needs some keyby, please finish the keyby by the connector itself. SupportsBucketing is just a marker interface. Best, Jingsong On Thu, Oct 26, 2023 at 5:00 PM Timo Walther <twal...@apache.org> wrote: > > Hi everyone, > > I would like to start a discussion on FLIP-376: Add DISTRIBUTED BY > clause [1]. > > Many SQL vendors expose the concepts of Partitioning, Bucketing, and > Clustering. This FLIP continues the work of previous FLIPs and would > like to introduce the concept of "Bucketing" to Flink. > > This is a pure connector characteristic and helps both Apache Kafka and > Apache Paimon connectors in avoiding a complex WITH clause by providing > improved syntax. > > Here is an example: > > CREATE TABLE MyTable > ( > uid BIGINT, > name STRING > ) > DISTRIBUTED BY (uid) INTO 6 BUCKETS > WITH ( > 'connector' = 'kafka' > ) > > The full syntax specification can be found in the document. The clause > should be optional and fully backwards compatible. > > Regards, > Timo > > [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-376%3A+Add+DISTRIBUTED+BY+clause
