Re: [DISCUSS] FLIP-129: Refactor Descriptor API to register connector in Table API

2020-07-24 Thread Aljoscha Krettek
I'm jumping in quite late but I think overall this is a very good effort and it's in very good shape now. Best, Aljoscha On 24.07.20 10:24, Jark Wu wrote: Thanks Dawid, Regarding (3), I think you mentioned it will affect other behavior, e.g. listing tables, is a strong reason. I will at

Re: [DISCUSS] FLIP-129: Refactor Descriptor API to register connector in Table API

2020-07-24 Thread Jark Wu
Thanks Dawid, Regarding (3), I think you mentioned it will affect other behavior, e.g. listing tables, is a strong reason. I will at least consider implementing it in the `TableSourceQueryOperation` way to avoid affecting other behaviors. I have updated the FLIP. Anyway, this is implementation

Re: [DISCUSS] FLIP-129: Refactor Descriptor API to register connector in Table API

2020-07-24 Thread Leonard Xu
Thanks Jark for the update. The latest FLIP looks well. I like Dawid’s proposal of TableDescriptor. Best Leonard Xu > 在 2020年7月23日,22:56,Jark Wu 写道: > > Hi Timo, > > That's a good point I missed in the design. I have updated the FLIP and added > a note under the `KafkaConnector` to mention

Re: [DISCUSS] FLIP-129: Refactor Descriptor API to register connector in Table API

2020-07-24 Thread Jingsong Li
Thanks for the update. The FLIP looks good, I think it is time to vote. Best, Jingsong On Fri, Jul 24, 2020 at 3:21 PM Dawid Wysakowicz wrote: > Hi Jark, > > Ad. 1 > > Personally I don't see any benefit of having the specific descriptors > (KafkaDescriptor/ElasticDescriptor/...). E.g. the

Re: [DISCUSS] FLIP-129: Refactor Descriptor API to register connector in Table API

2020-07-24 Thread Dawid Wysakowicz
Hi Jark, Ad. 1 Personally I don't see any benefit of having the specific descriptors (KafkaDescriptor/ElasticDescriptor/...). E.g. the KafkaDescriptor would not differ at all from other descriptors. I don't think returning always a TableDescriptor would be confusing for users. Therefore I am

Re: [DISCUSS] FLIP-129: Refactor Descriptor API to register connector in Table API

2020-07-23 Thread Jark Wu
Hi Timo, That's a good point I missed in the design. I have updated the FLIP and added a note under the `KafkaConnector` to mention this. I will not list all the method names in the FLIP as the design doc is super long now. Hi

Re: [DISCUSS] FLIP-129: Refactor Descriptor API to register connector in Table API

2020-07-23 Thread Dawid Wysakowicz
Hi Jark, Thanks for the update. I think the FLIP looks really well on the high level. I have a few comments to the code structure in the FLIP: 1) I really don't like how the TableDescriptor exposes protected fields. Moreover why do we need to extend from it? I don't think we need KafkaConnector

Re: [DISCUSS] FLIP-129: Refactor Descriptor API to register connector in Table API

2020-07-23 Thread Timo Walther
Hi Jark, thanks for the update. I think the FLIP is in a really good shape now and ready to be voted. If others have no further comments? I have one last comment around the methods of the descriptor builders. When refactoring classes such as `KafkaConnector` or `ElasticsearchConnector`. We

Re: [DISCUSS] FLIP-129: Refactor Descriptor API to register connector in Table API

2020-07-22 Thread Jark Wu
Hi all, After some offline discussion with other people, I'm also fine with using the builder pattern now, even though I still think the `.build()` method is a little verbose in the user code. I have updated the FLIP with following changes: 1) use builder pattern instead of "new" keyword. In

Re: [DISCUSS] FLIP-129: Refactor Descriptor API to register connector in Table API

2020-07-16 Thread Jark Wu
Thank you all for the discussion! Here are my comments: 2) I agree we should support Expression as a computed column. But I'm in favor of Leonard's point that maybe we can also support SQL string expression as a computed column. Because it also keeps aligned with DDL. The concern for Expression

Re: [DISCUSS] FLIP-129: Refactor Descriptor API to register connector in Table API

2020-07-16 Thread Jingsong Li
Thanks for the discussion. Descriptor lacks the watermark and the computed column is too long. 1) +1 for just `column(...)` 2) +1 for being consistent with Table API, the Java Table API should be Expression DSL. We don't need pure string support, users should just use DDL instead. I think this

Re: [DISCUSS] FLIP-129: Refactor Descriptor API to register connector in Table API

2020-07-15 Thread Leonard Xu
Thanks Jark bring this discussion and organize the FLIP document. Thanks Dawid and Timo for the feedback. Here are my thoughts. 1) I’m +1 with using column() for both cases. 2) Expression DSL vs pure SQL string for computed columns I think we can support them both and implement the pure SQL

Re: [DISCUSS] FLIP-129: Refactor Descriptor API to register connector in Table API

2020-07-15 Thread Timo Walther
Hi Jark, thanks for working on this issue. It is time to fix this last part of inconsistency in the API. I also like the core parts of the FLIP, esp. that TableDescriptor is one entity that can be passed to different methods. Here is some feedback from my side: 1) +1 for just `column(...)`

Re: [DISCUSS] FLIP-129: Refactor Descriptor API to register connector in Table API

2020-07-09 Thread Jark Wu
Hi Dawid, Thanks for the great feedback! Here are my responses: 1) computedColumn(..) vs column(..) I'm fine to use `column(..)` in both cases. 2) Expression DSL vs pure SQL string for computed columns This is a good point. Actually, I also prefer to use Expression DSL because this is more

Re: [DISCUSS] FLIP-129: Refactor Descriptor API to register connector in Table API

2020-07-09 Thread Dawid Wysakowicz
Correction to my point 4. The example is correct. I did not read it carefully enough. Sorry for the confusion. Nevertheless I'd still like to see a bit more explanation on the LikeOptions. On 07/07/2020 04:32, Jark Wu wrote: > Hi everyone, > > Leonard and I prepared a FLIP about refactoring

Re: [DISCUSS] FLIP-129: Refactor Descriptor API to register connector in Table API

2020-07-09 Thread Dawid Wysakowicz
Hi Jark, Thanks for starting the discussion. I think this is an important effort. I really like the general concept, but I have a few comments to the details of the FLIP. 1) I don't see any benefit in differentiating the computedColumn vs column in the method name. It does add cognitive burden.

[DISCUSS] FLIP-129: Refactor Descriptor API to register connector in Table API

2020-07-06 Thread Jark Wu
Hi everyone, Leonard and I prepared a FLIP about refactoring current Descriptor API, i.e. TableEnvironment#connect(). We would like to propose a new descriptor API to register connectors in Table API. Since Flink 1.9, the community focused more on the new SQL DDL feature. After a series of