[RESULT][VOTE] FLIP-95: New TableSource and TableSink interfaces

2020-04-03 Thread Timo Walther
Hi all, The voting time for FLIP-95 has passed. I'm closing the vote now. There were 6 +1 votes, 4 of which are binding: - Jark (binding) - Benchao (non-binding) - Kurt (binding) - Jingsong (binding) - Leonard (non-binding) - Dawid (binding) There were no disapproving votes. Thus, FLIP-95 has

Re: [VOTE] FLIP-95: New TableSource and TableSink interfaces

2020-04-03 Thread Timo Walther
Hi Dawid, thanks for your response. Let's postpone the discussion around a single interface into a later issue. It seems that we reached a consensus on the remaining interfaces. Given the size of the FLIP, I think it is fine to conclude the voting now. Thanks, Timo On 03.04.20 10:42, Dawid

Re: [VOTE] FLIP-95: New TableSource and TableSink interfaces

2020-04-03 Thread Dawid Wysakowicz
Hi all, Just to make it clear. I don't want to block the whole effort. I'm big +1 on the whole document and -0 on using the TableSchema for a projection pushdown. My personal opinion was that TableSchema is misleading for Projection pushdown and I still think that way. That's why I wanted to

Re: [VOTE] FLIP-95: New TableSource and TableSink interfaces

2020-04-02 Thread Jark Wu
Hi Timo, I don't think source should work with `CatalogTableSchema`. So far, a table source doesn't need to know the logic information of computed column and watermark. IMO, we should provide a method to convert from `CatalogTableSchema` into `TableSchema` without computed columns in source

Re: [VOTE] FLIP-95: New TableSource and TableSink interfaces

2020-04-02 Thread Timo Walther
Hi Jark, if catalogs use `CatalogTableSchema` in the future. The source would internally also work with `CatalogTableSchema`. I'm fine with cleaning up the `TableSchema` class but should a source deal with two different schema classes then? Another problem that I see is that connectors

Re: [VOTE] FLIP-95: New TableSource and TableSink interfaces

2020-04-02 Thread Jark Wu
Hi Dawid, > How to express projections with TableSchema? The TableSource holds the original TableSchema (i.e. from DDL) and the pushed TableSchema represents the schema after projection. Thus the table source can compare them to figure out changed field orders or not matched types. For most

Re: [VOTE] FLIP-95: New TableSource and TableSink interfaces

2020-04-02 Thread Timo Walther
Hi Dawid, thanks for your feedback. I agree with your concerns. I also observed that TableSchema might be used at too many different places for different responsibilities. How about we introduce a helper class for `SupportsProjectionPushDown` and also `LookupTableSource#Context#getKeys()`

Re: [VOTE] FLIP-95: New TableSource and TableSink interfaces

2020-04-02 Thread Dawid Wysakowicz
Generally +1 One slight concern I have is about the |SupportsProjectionPushDown.|I don't necessarily understand how can we express projections with TableSchema. It's unclear for me what happens when a type of a field changes, fields are in a different order, when types do not match. How do we

Re: [VOTE] FLIP-95: New TableSource and TableSink interfaces

2020-03-30 Thread Leonard Xu
+1(non-binding) Best, Leonard Xu > 在 2020年3月30日,16:43,Jingsong Li 写道: > > +1 > > Best, > Jingsong Lee > > On Mon, Mar 30, 2020 at 4:41 PM Kurt Young wrote: > >> +1 >> >> Best, >> Kurt >> >> >> On Mon, Mar 30, 2020 at 4:08 PM Benchao Li wrote: >> >>> +1 (non-binding) >>> >>> Jark Wu

Re: [VOTE] FLIP-95: New TableSource and TableSink interfaces

2020-03-30 Thread Jingsong Li
+1 Best, Jingsong Lee On Mon, Mar 30, 2020 at 4:41 PM Kurt Young wrote: > +1 > > Best, > Kurt > > > On Mon, Mar 30, 2020 at 4:08 PM Benchao Li wrote: > > > +1 (non-binding) > > > > Jark Wu 于2020年3月30日周一 下午3:57写道: > > > > > +1 from my side. > > > > > > Thanks Timo for driving this. > > > > >

Re: [VOTE] FLIP-95: New TableSource and TableSink interfaces

2020-03-30 Thread Kurt Young
+1 Best, Kurt On Mon, Mar 30, 2020 at 4:08 PM Benchao Li wrote: > +1 (non-binding) > > Jark Wu 于2020年3月30日周一 下午3:57写道: > > > +1 from my side. > > > > Thanks Timo for driving this. > > > > Best, > > Jark > > > > On Mon, 30 Mar 2020 at 15:36, Timo Walther wrote: > > > > > Hi all, > > > > > >

Re: [VOTE] FLIP-95: New TableSource and TableSink interfaces

2020-03-30 Thread Benchao Li
+1 (non-binding) Jark Wu 于2020年3月30日周一 下午3:57写道: > +1 from my side. > > Thanks Timo for driving this. > > Best, > Jark > > On Mon, 30 Mar 2020 at 15:36, Timo Walther wrote: > > > Hi all, > > > > I would like to start the vote for FLIP-95 [1], which is discussed and > > reached a consensus in

Re: [VOTE] FLIP-95: New TableSource and TableSink interfaces

2020-03-30 Thread Jark Wu
+1 from my side. Thanks Timo for driving this. Best, Jark On Mon, 30 Mar 2020 at 15:36, Timo Walther wrote: > Hi all, > > I would like to start the vote for FLIP-95 [1], which is discussed and > reached a consensus in the discussion thread [2]. > > The vote will be open until April 2nd (72h),

[VOTE] FLIP-95: New TableSource and TableSink interfaces

2020-03-30 Thread Timo Walther
Hi all, I would like to start the vote for FLIP-95 [1], which is discussed and reached a consensus in the discussion thread [2]. The vote will be open until April 2nd (72h), unless there is an objection or not enough votes. Thanks, Timo [1]