Re: [DISCUSS] FLIP 69 - Flink SQL DDL Enhancement

[Jingsong Li]

Thanks Kurt and Terry for bringing this to my attention.
Agree each topic of this FLIP touched is super big.

But because it's too big, we can't cover everything in one FLIP. So I think
there are two ways:
1. Decompose this big thing into an independent topic(FLIP), so that each
FLIP can be explained very clearly.
2. Only some basic concepts are put forward in this FLIP. It should be
noted that these basic things cannot conflict with the overall design (nor
with SQL standards), nor can API change in the future development. Then we
will improve each point step by step.

I think our road is #2 at present. This path may be more necessary for
users, because most users only need a basic framework for their common
cases.

Best,
Jingsong Lee

On Fri, Nov 8, 2019 at 2:20 PM Kurt Young  wrote:

> Hi Terry,
>
> I wouldn't say it's a mistake, I also don't have any suggestions about
> the issue. I just saw this and want to point it out to bring more
> attention.
> Maybe someone has some good opinion on that part and we can
> discuss around it and have some good advises for the community in
> the future.
>
> Best,
> Kurt
>
>
> On Fri, Nov 8, 2019 at 2:16 PM Terry Wang  wrote:
>
> > Hi, Kurt~
> >
> > Thanks for your vote and pointing out some deficiency of this flip. I’ll
> > try to avoid making similar mistakes.
> >
> > Best,
> > Terry Wang
> >
> >
> >
> > > 2019年11月8日 11:28，Kurt Young  写道：
> > >
> > > Hi,
> > >
> > > Sorry to join this so late and thanks for proposing this FLIP. After
> > > going through the proposal details, I would +1 for the changes.
> > >
> > > However, the FLIP name is kind of confusing me. It says will do
> > > DDL enhancement, and picked up a few new features to do. It looks
> > > to me the goal and content of this FLIP is kind of random.
> > >
> > > Each topic of this FLIP touched is super big, e.g. to enhance
> > > alter table command. According to SQL 2011 standard, it would contains
> > > at least so many features like:
> > >
> > >  ::=
> > >  ALTER TABLE  
> > >  ::=
> > >
> > >  | 
> > >  | 
> > >  | 
> > >  | 
> > >  | 
> > >  | 
> > >  | 
> > >  | 
> > >  | 
> > >
> > > I'm not suggesting to do all these at once, but I also didn't see any
> > > future plan or goals in the FLIP to describe the full picture here. We
> > just
> > > picked up some random chosen features to start.
> > >
> > > But still I'm +1 to this FLIP since they are all good enhancements.
> > >
> > > Best,
> > > Kurt
> > >
> > >
> > > On Tue, Nov 5, 2019 at 10:32 PM Terry Wang   > zjuwa...@gmail.com>> wrote:
> > >
> > >> Hi Bowen~
> > >>
> > >> We don’t intend to support create/drop catalog  syntax in this flip,
> we
> > >> may support it if there indeed has a strong desire.
> > >> And I’m going to kick off a vote for this flip, feel free to review
> > again.
> > >>
> > >> Best,
> > >> Terry Wang
> > >>
> > >>
> > >>
> > >>> 2019年9月26日 00:44，Xuefu Z  写道：
> > >>>
> > >>> Actually catalogs are more of system settings than of user objects
> > that a
> > >>> user might create or drop constantly. Thus, it's probably sufficient
> to
> > >> set
> > >>> up catalog information in the config file, at least for now.
> > >>>
> > >>> Thanks,
> > >>> Xuefu
> > >>>
> > >>> On Tue, Sep 24, 2019 at 7:10 PM Terry Wang  >   > >> zjuwa...@gmail.com >> wrote:
> > >>>
> >  Thanks Bowen for your insightful comments, I’ll think twice and do
> >  corresponding improvement.
> >  After finished, I’ll update in this mailing thread again.
> >  Best,
> >  Terry Wang
> > 
> > 
> > 
> > > 在 2019年9月25日，上午8:28，Bowen Li  > bowenl...@gmail.com>> 写道：
> > >
> > > BTW, will there be a "CREATE/DROP CATALOG" DDL?
> > >
> > > Though it's not SQL standard, I can see it'll be useful and handy
> for
> >  our end users in many cases.
> > >
> > > On Mon, Sep 23, 2019 at 12:28 PM Bowen Li  > 
> > >> >  >  bowenl...@gmail.com   > bowenl...@gmail.com  > > Hi Terry,
> > >
> > > Thanks for driving the effort! I left some comments in the doc.
> > >
> > > AFAIU, the biggest motivation is to support DDLs in sql parser so
> > that
> >  both Table API and SQL CLI can share the stack, despite that SQL CLI
> > has
> >  already supported some commands itself. However, I don't see details
> > on
> > >> how
> >  SQL CLI would migrate and depend on sql parser, and how Table API
> and
> > >> SQL
> >  CLI would actually share SQL parser. I'm not sure yet how much work
> > that
> >  will take, just want to double check that you didn't include them
> > >> because
> >  they are very trivial according to your estimate?
> > >
> > >
> > > On Mon, Sep 16, 2019 at 1:46 AM Terry Wang  > 
> > >> 

Re: [DISCUSS] FLIP 69 - Flink SQL DDL Enhancement

[Kurt Young]

Hi Terry,

I wouldn't say it's a mistake, I also don't have any suggestions about
the issue. I just saw this and want to point it out to bring more attention.
Maybe someone has some good opinion on that part and we can
discuss around it and have some good advises for the community in
the future.

Best,
Kurt


On Fri, Nov 8, 2019 at 2:16 PM Terry Wang  wrote:

> Hi, Kurt~
>
> Thanks for your vote and pointing out some deficiency of this flip. I’ll
> try to avoid making similar mistakes.
>
> Best,
> Terry Wang
>
>
>
> > 2019年11月8日 11:28，Kurt Young  写道：
> >
> > Hi,
> >
> > Sorry to join this so late and thanks for proposing this FLIP. After
> > going through the proposal details, I would +1 for the changes.
> >
> > However, the FLIP name is kind of confusing me. It says will do
> > DDL enhancement, and picked up a few new features to do. It looks
> > to me the goal and content of this FLIP is kind of random.
> >
> > Each topic of this FLIP touched is super big, e.g. to enhance
> > alter table command. According to SQL 2011 standard, it would contains
> > at least so many features like:
> >
> >  ::=
> >  ALTER TABLE  
> >  ::=
> >
> >  | 
> >  | 
> >  | 
> >  | 
> >  | 
> >  | 
> >  | 
> >  | 
> >  | 
> >
> > I'm not suggesting to do all these at once, but I also didn't see any
> > future plan or goals in the FLIP to describe the full picture here. We
> just
> > picked up some random chosen features to start.
> >
> > But still I'm +1 to this FLIP since they are all good enhancements.
> >
> > Best,
> > Kurt
> >
> >
> > On Tue, Nov 5, 2019 at 10:32 PM Terry Wang  zjuwa...@gmail.com>> wrote:
> >
> >> Hi Bowen~
> >>
> >> We don’t intend to support create/drop catalog  syntax in this flip, we
> >> may support it if there indeed has a strong desire.
> >> And I’m going to kick off a vote for this flip, feel free to review
> again.
> >>
> >> Best,
> >> Terry Wang
> >>
> >>
> >>
> >>> 2019年9月26日 00:44，Xuefu Z  写道：
> >>>
> >>> Actually catalogs are more of system settings than of user objects
> that a
> >>> user might create or drop constantly. Thus, it's probably sufficient to
> >> set
> >>> up catalog information in the config file, at least for now.
> >>>
> >>> Thanks,
> >>> Xuefu
> >>>
> >>> On Tue, Sep 24, 2019 at 7:10 PM Terry Wang    >> zjuwa...@gmail.com >> wrote:
> >>>
>  Thanks Bowen for your insightful comments, I’ll think twice and do
>  corresponding improvement.
>  After finished, I’ll update in this mailing thread again.
>  Best,
>  Terry Wang
> 
> 
> 
> > 在 2019年9月25日，上午8:28，Bowen Li  bowenl...@gmail.com>> 写道：
> >
> > BTW, will there be a "CREATE/DROP CATALOG" DDL?
> >
> > Though it's not SQL standard, I can see it'll be useful and handy for
>  our end users in many cases.
> >
> > On Mon, Sep 23, 2019 at 12:28 PM Bowen Li  
> >> >   bowenl...@gmail.com   bowenl...@gmail.com  > Hi Terry,
> >
> > Thanks for driving the effort! I left some comments in the doc.
> >
> > AFAIU, the biggest motivation is to support DDLs in sql parser so
> that
>  both Table API and SQL CLI can share the stack, despite that SQL CLI
> has
>  already supported some commands itself. However, I don't see details
> on
> >> how
>  SQL CLI would migrate and depend on sql parser, and how Table API and
> >> SQL
>  CLI would actually share SQL parser. I'm not sure yet how much work
> that
>  will take, just want to double check that you didn't include them
> >> because
>  they are very trivial according to your estimate?
> >
> >
> > On Mon, Sep 16, 2019 at 1:46 AM Terry Wang  
> >> >   zjuwa...@gmail.com   zjuwa...@gmail.com  > Hi everyone,
> >
> > In flink 1.9, we have introduced some awesome features such as
> complete
>  catalog support[1] and sql ddl support[2]. These features have been a
>  critical integration for Flink to be able to manage data and metadata
> >> like
>  a classic RDBMS and make developers more easy to construct their
>  real-time/off-line warehouse or sth similar base on flink.
> >
> > But there is still a lack of support on how Flink SQL DDL to manage
>  metadata and data like classic RDBMS such as `alter table rename` and
> >> so on.
> >
> > So I’d like to kick off a discussion on enhancing Flink Sql Ddls:
> >
> 
> >>
> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
> <
> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
> >
> >> <
> >>
> 

Re: [DISCUSS] FLIP 69 - Flink SQL DDL Enhancement

[Terry Wang]

Hi, Kurt~

Thanks for your vote and pointing out some deficiency of this flip. I’ll try to 
avoid making similar mistakes.

Best,
Terry Wang



> 2019年11月8日 11:28，Kurt Young  写道：
> 
> Hi,
> 
> Sorry to join this so late and thanks for proposing this FLIP. After
> going through the proposal details, I would +1 for the changes.
> 
> However, the FLIP name is kind of confusing me. It says will do
> DDL enhancement, and picked up a few new features to do. It looks
> to me the goal and content of this FLIP is kind of random.
> 
> Each topic of this FLIP touched is super big, e.g. to enhance
> alter table command. According to SQL 2011 standard, it would contains
> at least so many features like:
> 
>  ::=
>  ALTER TABLE  
>  ::=
>
>  | 
>  | 
>  | 
>  | 
>  | 
>  | 
>  | 
>  | 
>  | 
> 
> I'm not suggesting to do all these at once, but I also didn't see any
> future plan or goals in the FLIP to describe the full picture here. We just
> picked up some random chosen features to start.
> 
> But still I'm +1 to this FLIP since they are all good enhancements.
> 
> Best,
> Kurt
> 
> 
> On Tue, Nov 5, 2019 at 10:32 PM Terry Wang  > wrote:
> 
>> Hi Bowen~
>> 
>> We don’t intend to support create/drop catalog  syntax in this flip, we
>> may support it if there indeed has a strong desire.
>> And I’m going to kick off a vote for this flip, feel free to review again.
>> 
>> Best,
>> Terry Wang
>> 
>> 
>> 
>>> 2019年9月26日 00:44，Xuefu Z  写道：
>>> 
>>> Actually catalogs are more of system settings than of user objects that a
>>> user might create or drop constantly. Thus, it's probably sufficient to
>> set
>>> up catalog information in the config file, at least for now.
>>> 
>>> Thanks,
>>> Xuefu
>>> 
>>> On Tue, Sep 24, 2019 at 7:10 PM Terry Wang >>  > zjuwa...@gmail.com >> wrote:
>>> 
 Thanks Bowen for your insightful comments, I’ll think twice and do
 corresponding improvement.
 After finished, I’ll update in this mailing thread again.
 Best,
 Terry Wang
 
 
 
> 在 2019年9月25日，上午8:28，Bowen Li  > 写道：
> 
> BTW, will there be a "CREATE/DROP CATALOG" DDL?
> 
> Though it's not SQL standard, I can see it'll be useful and handy for
 our end users in many cases.
> 
> On Mon, Sep 23, 2019 at 12:28 PM Bowen Li  
>> > >>> bowenl...@gmail.com  
  Hi Terry,
> 
> Thanks for driving the effort! I left some comments in the doc.
> 
> AFAIU, the biggest motivation is to support DDLs in sql parser so that
 both Table API and SQL CLI can share the stack, despite that SQL CLI has
 already supported some commands itself. However, I don't see details on
>> how
 SQL CLI would migrate and depend on sql parser, and how Table API and
>> SQL
 CLI would actually share SQL parser. I'm not sure yet how much work that
 will take, just want to double check that you didn't include them
>> because
 they are very trivial according to your estimate?
> 
> 
> On Mon, Sep 16, 2019 at 1:46 AM Terry Wang  
>> > >>> zjuwa...@gmail.com   Hi everyone,
> 
> In flink 1.9, we have introduced some awesome features such as complete
 catalog support[1] and sql ddl support[2]. These features have been a
 critical integration for Flink to be able to manage data and metadata
>> like
 a classic RDBMS and make developers more easy to construct their
 real-time/off-line warehouse or sth similar base on flink.
> 
> But there is still a lack of support on how Flink SQL DDL to manage
 metadata and data like classic RDBMS such as `alter table rename` and
>> so on.
> 
> So I’d like to kick off a discussion on enhancing Flink Sql Ddls:
> 
 
>> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
>>  
>> 
>> <
>> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
>>  
>> 
>>> 
 <
 
>> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
>>  
>> 
>> <
>> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
>>  
>> 

Re: [DISCUSS] FLIP 69 - Flink SQL DDL Enhancement

[Kurt Young]

Hi,

Sorry to join this so late and thanks for proposing this FLIP. After
going through the proposal details, I would +1 for the changes.

However, the FLIP name is kind of confusing me. It says will do
DDL enhancement, and picked up a few new features to do. It looks
to me the goal and content of this FLIP is kind of random.

Each topic of this FLIP touched is super big, e.g. to enhance
alter table command. According to SQL 2011 standard, it would contains
at least so many features like:

 ::=
  ALTER TABLE  
 ::=

  | 
  | 
  | 
  | 
  | 
  | 
  | 
  | 
  | 

I'm not suggesting to do all these at once, but I also didn't see any
future plan or goals in the FLIP to describe the full picture here. We just
picked up some random chosen features to start.

But still I'm +1 to this FLIP since they are all good enhancements.

Best,
Kurt


On Tue, Nov 5, 2019 at 10:32 PM Terry Wang  wrote:

> Hi Bowen~
>
> We don’t intend to support create/drop catalog  syntax in this flip, we
> may support it if there indeed has a strong desire.
> And I’m going to kick off a vote for this flip, feel free to review again.
>
> Best,
> Terry Wang
>
>
>
> > 2019年9月26日 00:44，Xuefu Z  写道：
> >
> > Actually catalogs are more of system settings than of user objects that a
> > user might create or drop constantly. Thus, it's probably sufficient to
> set
> > up catalog information in the config file, at least for now.
> >
> > Thanks,
> > Xuefu
> >
> > On Tue, Sep 24, 2019 at 7:10 PM Terry Wang  zjuwa...@gmail.com>> wrote:
> >
> >> Thanks Bowen for your insightful comments, I’ll think twice and do
> >> corresponding improvement.
> >> After finished, I’ll update in this mailing thread again.
> >> Best,
> >> Terry Wang
> >>
> >>
> >>
> >>> 在 2019年9月25日，上午8:28，Bowen Li  写道：
> >>>
> >>> BTW, will there be a "CREATE/DROP CATALOG" DDL?
> >>>
> >>> Though it's not SQL standard, I can see it'll be useful and handy for
> >> our end users in many cases.
> >>>
> >>> On Mon, Sep 23, 2019 at 12:28 PM Bowen Li    >> bowenl...@gmail.com >> wrote:
> >>> Hi Terry,
> >>>
> >>> Thanks for driving the effort! I left some comments in the doc.
> >>>
> >>> AFAIU, the biggest motivation is to support DDLs in sql parser so that
> >> both Table API and SQL CLI can share the stack, despite that SQL CLI has
> >> already supported some commands itself. However, I don't see details on
> how
> >> SQL CLI would migrate and depend on sql parser, and how Table API and
> SQL
> >> CLI would actually share SQL parser. I'm not sure yet how much work that
> >> will take, just want to double check that you didn't include them
> because
> >> they are very trivial according to your estimate?
> >>>
> >>>
> >>> On Mon, Sep 16, 2019 at 1:46 AM Terry Wang    >> zjuwa...@gmail.com >> wrote:
> >>> Hi everyone,
> >>>
> >>> In flink 1.9, we have introduced some awesome features such as complete
> >> catalog support[1] and sql ddl support[2]. These features have been a
> >> critical integration for Flink to be able to manage data and metadata
> like
> >> a classic RDBMS and make developers more easy to construct their
> >> real-time/off-line warehouse or sth similar base on flink.
> >>>
> >>> But there is still a lack of support on how Flink SQL DDL to manage
> >> metadata and data like classic RDBMS such as `alter table rename` and
> so on.
> >>>
> >>> So I’d like to kick off a discussion on enhancing Flink Sql Ddls:
> >>>
> >>
> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
> <
> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
> >
> >> <
> >>
> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
> <
> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
> >>
> >> <
> >>
> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
> <
> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
> >
> >> <
> >>
> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
> <
> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
> >
> 
> >>>
> >>> In short, it:
> >>>- Add Catalog DDL enhancement support:  show catalogs / describe
> >> catalog / use catalog
> >>>- Add Database DDL enhancement support:  show databses / create
> >> database / drop database/ alter database
> >>>- Add Table DDL enhancement support:show tables/ describe
> >> table / alter table
> >>>- Add Function DDL enhancement support: show functions/ create
> >> function /drop function
> >>>
> >>> Looking forward to your opinions.
> >>>
> >>> Best,
> >>> Terry Wang
> >>>
> >>>
> >>>
> >>> 

Re: [DISCUSS] FLIP 69 - Flink SQL DDL Enhancement

[Terry Wang]

Hi Bowen~

We don’t intend to support create/drop catalog  syntax in this flip, we may 
support it if there indeed has a strong desire.
And I’m going to kick off a vote for this flip, feel free to review again.

Best,
Terry Wang



> 2019年9月26日 00:44，Xuefu Z  写道：
> 
> Actually catalogs are more of system settings than of user objects that a
> user might create or drop constantly. Thus, it's probably sufficient to set
> up catalog information in the config file, at least for now.
> 
> Thanks,
> Xuefu
> 
> On Tue, Sep 24, 2019 at 7:10 PM Terry Wang  > wrote:
> 
>> Thanks Bowen for your insightful comments, I’ll think twice and do
>> corresponding improvement.
>> After finished, I’ll update in this mailing thread again.
>> Best,
>> Terry Wang
>> 
>> 
>> 
>>> 在 2019年9月25日，上午8:28，Bowen Li  写道：
>>> 
>>> BTW, will there be a "CREATE/DROP CATALOG" DDL?
>>> 
>>> Though it's not SQL standard, I can see it'll be useful and handy for
>> our end users in many cases.
>>> 
>>> On Mon, Sep 23, 2019 at 12:28 PM Bowen Li >>  > bowenl...@gmail.com >> wrote:
>>> Hi Terry,
>>> 
>>> Thanks for driving the effort! I left some comments in the doc.
>>> 
>>> AFAIU, the biggest motivation is to support DDLs in sql parser so that
>> both Table API and SQL CLI can share the stack, despite that SQL CLI has
>> already supported some commands itself. However, I don't see details on how
>> SQL CLI would migrate and depend on sql parser, and how Table API and SQL
>> CLI would actually share SQL parser. I'm not sure yet how much work that
>> will take, just want to double check that you didn't include them because
>> they are very trivial according to your estimate?
>>> 
>>> 
>>> On Mon, Sep 16, 2019 at 1:46 AM Terry Wang >>  > zjuwa...@gmail.com >> wrote:
>>> Hi everyone,
>>> 
>>> In flink 1.9, we have introduced some awesome features such as complete
>> catalog support[1] and sql ddl support[2]. These features have been a
>> critical integration for Flink to be able to manage data and metadata like
>> a classic RDBMS and make developers more easy to construct their
>> real-time/off-line warehouse or sth similar base on flink.
>>> 
>>> But there is still a lack of support on how Flink SQL DDL to manage
>> metadata and data like classic RDBMS such as `alter table rename` and so on.
>>> 
>>> So I’d like to kick off a discussion on enhancing Flink Sql Ddls:
>>> 
>> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
>>  
>> 
>> <
>> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
>>  
>> >
>> <
>> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
>>  
>> 
>> <
>> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
>>  
>> 
 
>>> 
>>> In short, it:
>>>- Add Catalog DDL enhancement support:  show catalogs / describe
>> catalog / use catalog
>>>- Add Database DDL enhancement support:  show databses / create
>> database / drop database/ alter database
>>>- Add Table DDL enhancement support:show tables/ describe
>> table / alter table
>>>- Add Function DDL enhancement support: show functions/ create
>> function /drop function
>>> 
>>> Looking forward to your opinions.
>>> 
>>> Best,
>>> Terry Wang
>>> 
>>> 
>>> 
>>> [1]:https://issues.apache.org/jira/browse/FLINK-11275 
>>>  <
>> https://issues.apache.org/jira/browse/FLINK-11275 
>> > <
>> https://issues.apache.org/jira/browse/FLINK-11275 
>>  <
>> https://issues.apache.org/jira/browse/FLINK-11275 
>> >>
>>> [2]:https://issues.apache.org/jira/browse/FLINK-1 
>>>  <
>> https://issues.apache.org/jira/browse/FLINK-1 
>> > <
>> https://issues.apache.org/jira/browse/FLINK-11275 
>>  <
>> https://issues.apache.org/jira/browse/FLINK-11275 
>> >>0232
>>> >>  <
>> https://issues.apache.org/jira/browse/FLINK-11275 
>> >>

Re: [DISCUSS] FLIP 69 - Flink SQL DDL Enhancement

[Terry Wang]

Hi, everyone~

Sorry so late to reply this this thread again. 
I am working on FLIP-84 recently  to make sql api support return value which 
this flip depends on.
I think it’s time to pick up this discussion again and there are some new 
updates in this flip design:
https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
 


1. Remove function ddl section. For Peter Huang are working on FLIP-79 to 
convege function ddl support in a more overall way
2. Remove the proposed TableEnvironment SQL API Changes which is covered in 
FLIP-84
3. Update the design doc accoding to review comments.

Looking forward to receiving more comments ~

Best,
Terry Wang



> 2019年9月26日 00:44，Xuefu Z  写道：
> 
> Actually catalogs are more of system settings than of user objects that a
> user might create or drop constantly. Thus, it's probably sufficient to set
> up catalog information in the config file, at least for now.
> 
> Thanks,
> Xuefu
> 
> On Tue, Sep 24, 2019 at 7:10 PM Terry Wang  > wrote:
> 
>> Thanks Bowen for your insightful comments, I’ll think twice and do
>> corresponding improvement.
>> After finished, I’ll update in this mailing thread again.
>> Best,
>> Terry Wang
>> 
>> 
>> 
>>> 在 2019年9月25日，上午8:28，Bowen Li  写道：
>>> 
>>> BTW, will there be a "CREATE/DROP CATALOG" DDL?
>>> 
>>> Though it's not SQL standard, I can see it'll be useful and handy for
>> our end users in many cases.
>>> 
>>> On Mon, Sep 23, 2019 at 12:28 PM Bowen Li >>  > bowenl...@gmail.com >> wrote:
>>> Hi Terry,
>>> 
>>> Thanks for driving the effort! I left some comments in the doc.
>>> 
>>> AFAIU, the biggest motivation is to support DDLs in sql parser so that
>> both Table API and SQL CLI can share the stack, despite that SQL CLI has
>> already supported some commands itself. However, I don't see details on how
>> SQL CLI would migrate and depend on sql parser, and how Table API and SQL
>> CLI would actually share SQL parser. I'm not sure yet how much work that
>> will take, just want to double check that you didn't include them because
>> they are very trivial according to your estimate?
>>> 
>>> 
>>> On Mon, Sep 16, 2019 at 1:46 AM Terry Wang >>  > zjuwa...@gmail.com >> wrote:
>>> Hi everyone,
>>> 
>>> In flink 1.9, we have introduced some awesome features such as complete
>> catalog support[1] and sql ddl support[2]. These features have been a
>> critical integration for Flink to be able to manage data and metadata like
>> a classic RDBMS and make developers more easy to construct their
>> real-time/off-line warehouse or sth similar base on flink.
>>> 
>>> But there is still a lack of support on how Flink SQL DDL to manage
>> metadata and data like classic RDBMS such as `alter table rename` and so on.
>>> 
>>> So I’d like to kick off a discussion on enhancing Flink Sql Ddls:
>>> 
>> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
>>  
>> 
>> <
>> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
>>  
>> >
>> <
>> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
>>  
>> 
>> <
>> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
>>  
>> 
 
>>> 
>>> In short, it:
>>>- Add Catalog DDL enhancement support:  show catalogs / describe
>> catalog / use catalog
>>>- Add Database DDL enhancement support:  show databses / create
>> database / drop database/ alter database
>>>- Add Table DDL enhancement support:show tables/ describe
>> table / alter table
>>>- Add Function DDL enhancement support: show functions/ create
>> function /drop function
>>> 
>>> Looking forward to your opinions.
>>> 
>>> Best,
>>> Terry Wang
>>> 
>>> 
>>> 
>>> [1]:https://issues.apache.org/jira/browse/FLINK-11275 
>>>  <
>> https://issues.apache.org/jira/browse/FLINK-11275 
>> > <
>> https://issues.apache.org/jira/browse/FLINK-11275 
>>  <
>> https://issues.apache.org/jira/browse/FLINK-11275 
>> >>
>>> [2]:https://issues.apache.org/jira/browse/FLINK-1 
>>> 

Re: [DISCUSS] FLIP 69 - Flink SQL DDL Enhancement

[Xuefu Z]

Actually catalogs are more of system settings than of user objects that a
user might create or drop constantly. Thus, it's probably sufficient to set
up catalog information in the config file, at least for now.

Thanks,
Xuefu

On Tue, Sep 24, 2019 at 7:10 PM Terry Wang  wrote:

> Thanks Bowen for your insightful comments, I’ll think twice and do
> corresponding improvement.
> After finished, I’ll update in this mailing thread again.
> Best,
> Terry Wang
>
>
>
> > 在 2019年9月25日，上午8:28，Bowen Li  写道：
> >
> > BTW, will there be a "CREATE/DROP CATALOG" DDL?
> >
> > Though it's not SQL standard, I can see it'll be useful and handy for
> our end users in many cases.
> >
> > On Mon, Sep 23, 2019 at 12:28 PM Bowen Li  bowenl...@gmail.com>> wrote:
> > Hi Terry,
> >
> > Thanks for driving the effort! I left some comments in the doc.
> >
> > AFAIU, the biggest motivation is to support DDLs in sql parser so that
> both Table API and SQL CLI can share the stack, despite that SQL CLI has
> already supported some commands itself. However, I don't see details on how
> SQL CLI would migrate and depend on sql parser, and how Table API and SQL
> CLI would actually share SQL parser. I'm not sure yet how much work that
> will take, just want to double check that you didn't include them because
> they are very trivial according to your estimate?
> >
> >
> > On Mon, Sep 16, 2019 at 1:46 AM Terry Wang  zjuwa...@gmail.com>> wrote:
> > Hi everyone,
> >
> > In flink 1.9, we have introduced some awesome features such as complete
> catalog support[1] and sql ddl support[2]. These features have been a
> critical integration for Flink to be able to manage data and metadata like
> a classic RDBMS and make developers more easy to construct their
> real-time/off-line warehouse or sth similar base on flink.
> >
> > But there is still a lack of support on how Flink SQL DDL to manage
> metadata and data like classic RDBMS such as `alter table rename` and so on.
> >
> > So I’d like to kick off a discussion on enhancing Flink Sql Ddls:
> >
> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
> <
> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing>
> <
> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
> <
> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
> >>
> >
> > In short, it:
> > - Add Catalog DDL enhancement support:  show catalogs / describe
> catalog / use catalog
> > - Add Database DDL enhancement support:  show databses / create
> database / drop database/ alter database
> > - Add Table DDL enhancement support:show tables/ describe
> table / alter table
> > - Add Function DDL enhancement support: show functions/ create
> function /drop function
> >
> > Looking forward to your opinions.
> >
> > Best,
> > Terry Wang
> >
> >
> >
> > [1]:https://issues.apache.org/jira/browse/FLINK-11275 <
> https://issues.apache.org/jira/browse/FLINK-11275> <
> https://issues.apache.org/jira/browse/FLINK-11275 <
> https://issues.apache.org/jira/browse/FLINK-11275>>
> > [2]:https://issues.apache.org/jira/browse/FLINK-1 <
> https://issues.apache.org/jira/browse/FLINK-1> <
> https://issues.apache.org/jira/browse/FLINK-11275 <
> https://issues.apache.org/jira/browse/FLINK-11275>>0232
> >   https://issues.apache.org/jira/browse/FLINK-11275>>
>
>

-- 
Xuefu Zhang

"In Honey We Trust!"

Re: [DISCUSS] FLIP 69 - Flink SQL DDL Enhancement

[Terry Wang]

Thanks Bowen for your insightful comments, I’ll think twice and do 
corresponding improvement.
After finished, I’ll update in this mailing thread again.
Best,
Terry Wang



> 在 2019年9月25日，上午8:28，Bowen Li  写道：
> 
> BTW, will there be a "CREATE/DROP CATALOG" DDL?
> 
> Though it's not SQL standard, I can see it'll be useful and handy for our end 
> users in many cases.
> 
> On Mon, Sep 23, 2019 at 12:28 PM Bowen Li  > wrote:
> Hi Terry,
> 
> Thanks for driving the effort! I left some comments in the doc.
> 
> AFAIU, the biggest motivation is to support DDLs in sql parser so that both 
> Table API and SQL CLI can share the stack, despite that SQL CLI has already 
> supported some commands itself. However, I don't see details on how SQL CLI 
> would migrate and depend on sql parser, and how Table API and SQL CLI would 
> actually share SQL parser. I'm not sure yet how much work that will take, 
> just want to double check that you didn't include them because they are very 
> trivial according to your estimate?
> 
> 
> On Mon, Sep 16, 2019 at 1:46 AM Terry Wang  > wrote:
> Hi everyone,
> 
> In flink 1.9, we have introduced some awesome features such as complete 
> catalog support[1] and sql ddl support[2]. These features have been a 
> critical integration for Flink to be able to manage data and metadata like a 
> classic RDBMS and make developers more easy to construct their 
> real-time/off-line warehouse or sth similar base on flink.
> 
> But there is still a lack of support on how Flink SQL DDL to manage metadata 
> and data like classic RDBMS such as `alter table rename` and so on.
> 
> So I’d like to kick off a discussion on enhancing Flink Sql Ddls:
> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
>  
> 
>  
>   
> >
> 
> In short, it:
> - Add Catalog DDL enhancement support:  show catalogs / describe 
> catalog / use catalog
> - Add Database DDL enhancement support:  show databses / create 
> database / drop database/ alter database 
> - Add Table DDL enhancement support:show tables/ describe table / 
> alter table
> - Add Function DDL enhancement support: show functions/ create 
> function /drop function
> 
> Looking forward to your opinions.
> 
> Best,
> Terry Wang
> 
> 
> 
> [1]:https://issues.apache.org/jira/browse/FLINK-11275 
>  
>  >
> [2]:https://issues.apache.org/jira/browse/FLINK-1 
>  
>  >0232
>   >

Re: [DISCUSS] FLIP 69 - Flink SQL DDL Enhancement

[Bowen Li]

BTW, will there be a "CREATE/DROP CATALOG" DDL?

Though it's not SQL standard, I can see it'll be useful and handy for our
end users in many cases.

On Mon, Sep 23, 2019 at 12:28 PM Bowen Li  wrote:

> Hi Terry,
>
> Thanks for driving the effort! I left some comments in the doc.
>
> AFAIU, the biggest motivation is to support DDLs in sql parser so that
> both Table API and SQL CLI can share the stack, despite that SQL CLI has
> already supported some commands itself. However, I don't see details on how
> SQL CLI would migrate and depend on sql parser, and how Table API and SQL
> CLI would actually share SQL parser. I'm not sure yet how much work that
> will take, just want to double check that you didn't include them because
> they are very trivial according to your estimate?
>
>
> On Mon, Sep 16, 2019 at 1:46 AM Terry Wang  wrote:
>
>> Hi everyone,
>>
>> In flink 1.9, we have introduced some awesome features such as complete
>> catalog support[1] and sql ddl support[2]. These features have been a
>> critical integration for Flink to be able to manage data and metadata like
>> a classic RDBMS and make developers more easy to construct their
>> real-time/off-line warehouse or sth similar base on flink.
>>
>> But there is still a lack of support on how Flink SQL DDL to manage
>> metadata and data like classic RDBMS such as `alter table rename` and so on.
>>
>> So I’d like to kick off a discussion on enhancing Flink Sql Ddls:
>>
>> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
>> <
>> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
>> >
>>
>> In short, it:
>> - Add Catalog DDL enhancement support:  show catalogs / describe
>> catalog / use catalog
>> - Add Database DDL enhancement support:  show databses / create
>> database / drop database/ alter database
>> - Add Table DDL enhancement support:show tables/ describe
>> table / alter table
>> - Add Function DDL enhancement support: show functions/ create
>> function /drop function
>>
>> Looking forward to your opinions.
>>
>> Best,
>> Terry Wang
>>
>>
>>
>> [1]:https://issues.apache.org/jira/browse/FLINK-11275 <
>> https://issues.apache.org/jira/browse/FLINK-11275>
>> [2]:https://issues.apache.org/jira/browse/FLINK-1 <
>> https://issues.apache.org/jira/browse/FLINK-11275>0232
>>  
>
>

Re: [DISCUSS] FLIP 69 - Flink SQL DDL Enhancement

[Bowen Li]

Hi Terry,

Thanks for driving the effort! I left some comments in the doc.

AFAIU, the biggest motivation is to support DDLs in sql parser so that both
Table API and SQL CLI can share the stack, despite that SQL CLI has already
supported some commands itself. However, I don't see details on how SQL CLI
would migrate and depend on sql parser, and how Table API and SQL CLI would
actually share SQL parser. I'm not sure yet how much work that will take,
just want to double check that you didn't include them because they are
very trivial according to your estimate?


On Mon, Sep 16, 2019 at 1:46 AM Terry Wang  wrote:

> Hi everyone,
>
> In flink 1.9, we have introduced some awesome features such as complete
> catalog support[1] and sql ddl support[2]. These features have been a
> critical integration for Flink to be able to manage data and metadata like
> a classic RDBMS and make developers more easy to construct their
> real-time/off-line warehouse or sth similar base on flink.
>
> But there is still a lack of support on how Flink SQL DDL to manage
> metadata and data like classic RDBMS such as `alter table rename` and so on.
>
> So I’d like to kick off a discussion on enhancing Flink Sql Ddls:
>
> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
> <
> https://docs.google.com/document/d/1mhZmx1h2ecfL0x8OzYD1n-nVRn4yE7pwk4jGed4k7kc/edit?usp=sharing
> >
>
> In short, it:
> - Add Catalog DDL enhancement support:  show catalogs / describe
> catalog / use catalog
> - Add Database DDL enhancement support:  show databses / create
> database / drop database/ alter database
> - Add Table DDL enhancement support:show tables/ describe
> table / alter table
> - Add Function DDL enhancement support: show functions/ create
> function /drop function
>
> Looking forward to your opinions.
>
> Best,
> Terry Wang
>
>
>
> [1]:https://issues.apache.org/jira/browse/FLINK-11275 <
> https://issues.apache.org/jira/browse/FLINK-11275>
> [2]:https://issues.apache.org/jira/browse/FLINK-1 <
> https://issues.apache.org/jira/browse/FLINK-11275>0232
>