Re: [DISCUSS] SQL CTAS Syntax

2021-06-01 Thread Jingsong Li
Thanks Danny for starting the discussion. +1 for this feature. I like schema evolution. A question: Can we support multiple pipelines? For example, I have three source tables and just one sink table. So, I will write three CTAS: - CTAS IF NOT EXISTS 1 - CTAS IF NOT EXISTS 2 - CTAS IF NOT

Re: [DISCUSS] SQL CTAS Syntax

2021-05-28 Thread Kurt Young
Hi Konstantin, >From my understanding, this syntax has 2 major benefits: 1. Just like you said, it saves the effort to specify the schema, especially when involving hundreds of fields. 2. When using CREATE TABLE xx AS TABLE yy, it gives us the possibility to enable schema evolution, and it seems

Re: [DISCUSS] SQL CTAS Syntax

2021-05-28 Thread Konstantin Knauf
Hi everyone, quick question for my understanding: how is this different to CREATE TABLE IF NOT EXISTS my_table ( ... ) WITH ( ... ); INSERT INTO my_table SELECT ...; ? Is it only about a) not having to specify the schema and b) a more condensed syntax? Cheers, Konstantin On Fri, May 28,

Re: [DISCUSS] SQL CTAS Syntax

2021-05-28 Thread Jark Wu
Thanks Danny for starting the discussion of extending CTAS syntax. I think this is a very useful feature for data integration and ETL jobs (a big use case of Flink). Many users complain a lot that manually defining schemas for sources and sinks is hard. CTAS helps users to write ETL jobs without

Re: [DISCUSS] SQL CTAS Syntax

2021-05-28 Thread JING ZHANG
Hi Danny, Thanks for starting this discussion. Big +1 for this feature. Both CTAS AND CREATE TABLE LIKE are very useful features. IMO, it is clear to separate them into two parts in the `syntax` character.  First, I have two related problems: 1. Would `create table` in CTAS trigger to

[DISCUSS] SQL CTAS Syntax

2021-05-27 Thread Danny Chan
Hi, dear Flink community ~ The data integration is one of the most popular use case for Flink, from the point of SQL, we aim to design an API for SQL users in data ingestion. We have referenced some RDBMS and there is a classic CTAS syntax for table data copy [1][2][3][4]. We decide to use the