Thanks Zhenghua for starting this discussion. Currently, all the UpsertStreamTableSinks can't upgrade to the new type system which affects usability a lot. I hope we can fix that in 1.11.
I'm find with *getRecordDataType* for a temporary solution. IIUC, the framework will only recognize getRecordDataType and ignore getConsumedDataType for UpsertStreamTableSink, is that right? I guess Timo are planning to design a new source/sink interface which will also fix this problem, but I'm not sure the timelines. cc @Timo It would be better if we can have a new and complete interface, because getRecordDataType is little confused as UpsertStreamTableSink already has three getXXXType(). Best, Jark On Mon, 3 Feb 2020 at 17:48, Zhenghua Gao <doc...@gmail.com> wrote: > Hi Jingsong, For now, only UpsertStreamTableSink and > RetractStreamTableSink consumes JTuple2 > So the 'getConsumedDataType' interface is not necessary in validate & > codegen phase. > See > > https://github.com/apache/flink/pull/10880/files#diff-137d090bd719f3a99ae8ba6603ed81ebR52 > and > > https://github.com/apache/flink/pull/10880/files#diff-8405c17e5155aa63d16e497c4c96a842R304 > > What about stay the same to use RAW type? > > *Best Regards,* > *Zhenghua Gao* > > > On Mon, Feb 3, 2020 at 4:59 PM Jingsong Li <jingsongl...@gmail.com> wrote: > > > Hi Zhenghua, > > > > The *getRecordDataType* looks good to me. > > > > But the main problem is how to represent the tuple type in DataType. I > > understand that it is necessary to use StructuredType, but at present, > > planner does not support StructuredType, so the other way is to support > > StructuredType. > > > > Best, > > Jingsong Lee > > > > On Mon, Feb 3, 2020 at 4:49 PM Kurt Young <ykt...@gmail.com> wrote: > > > > > Would overriding `getConsumedDataType` do the job? > > > > > > Best, > > > Kurt > > > > > > > > > On Mon, Feb 3, 2020 at 3:52 PM Zhenghua Gao <doc...@gmail.com> wrote: > > > > > >> Hi all, > > >> > > >> FLINK-12254[1] [2] updated TableSink and related interfaces to new > type > > >> system which > > >> allows connectors use the new type system based on DataTypes. > > >> > > >> But FLINK-12911 port UpsertStreamTableSink and RetractStreamTableSink > to > > >> flink-api-java-bridge and returns TypeInformation of the requested > > record > > >> type which > > >> can't support types with precision and scale, e.g. TIMESTAMP(p), > > >> DECIMAL(p,s). > > >> > > >> /** > > >> * Returns the requested record type. > > >> */ > > >> TypeInformation<T> getRecordType(); > > >> > > >> > > >> A proposal is deprecating the *getRecordType* API and adding a > > >> *getRecordDataType* API instead to return the data type of the > requested > > >> record. I have filed the issue FLINK-15469 and > > >> an initial PR to verify it. > > >> > > >> What do you think about this API changes? Any feedback are > appreciated. > > >> [1] https://issues.apache.org/jira/browse/FLINK-12254 > > >> [2] https://github.com/apache/flink/pull/8596 > > >> [3] https://issues.apache.org/jira/browse/FLINK-15469 > > >> > > >> *Best Regards,* > > >> *Zhenghua Gao* > > >> > > > > > > > -- > > Best, Jingsong Lee > > >
