Hi Lincoln,
Thank you for sharing this FLIP. Overall, it looks good to me. I have
one question: with the introduction of this interface,
will any existing Flink connectors need to be updated in order to take
advantage of its capabilities? For example, HBase.
yuxia <luoyu...@alumni.sjtu.edu.cn> 于2023年3月7日周二 10:01写道：
> Thanks. It makes sense to me.
>
> Best regards,
> Yuxia
>
> ----- 原始邮件 -----
> 发件人: "Lincoln Lee" <lincoln.8...@gmail.com>
> 收件人: "dev" <dev@flink.apache.org>
> 发送时间: 星期一, 2023年 3 月 06日 下午 10:26:26
> 主题: Re: [DISCUSS] FLIP-300: Add targetColumns to DynamicTableSink#Context
> to solve the null overwrite problem of partial-insert
>
> hi yuxia,
>
> Thanks for your feedback and tracking the issue of update statement! I've
> updated the FLIP[1] and also the poc[2].
> Since the bug and flip are orthogonal, we can focus on finalizing the api
> changes first, and then work on the flip implementation and bugfix
> separately, WDYT?
>
> [1]
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=240885081
> [2] https://github.com/apache/flink/pull/22041
>
> Best,
> Lincoln Lee
>
>
> yuxia <luoyu...@alumni.sjtu.edu.cn> 于2023年3月6日周一 21:21写道：
>
> > Hi, Lincoln.
> > Thanks for bringing this up. +1 for this FLIP, it's helpful for external
> > storage system to implement partial update.
> > The FLIP looks good to me. I only want to add one comment, update
> > statement also doesn't support updating nested column, I have created
> > FLINK-31344[1] to track it.
> > Maybe we also need to explain it in this FLIP.
> >
> > [1] https://issues.apache.org/jira/browse/FLINK-31344
> >
> > Best regards,
> > Yuxia
> >
> > ----- 原始邮件 -----
> > 发件人: "Lincoln Lee" <lincoln.8...@gmail.com>
> > 收件人: "dev" <dev@flink.apache.org>
> > 发送时间: 星期五, 2023年 3 月 03日 下午 12:22:19
> > 主题: [DISCUSS] FLIP-300: Add targetColumns to DynamicTableSink#Context to
> > solve the null overwrite problem of partial-insert
> >
> > Hi everyone,
> >
> > This FLIP[1] aims to support connectors in avoiding overwriting
> non-target
> > columns with null values when processing partial column updates, we
> propose
> > adding information on the target column list to DynamicTableSink#Context.
> >
> > FLINK-18726[2] supports inserting statements with specified column list,
> it
> > fills null values (or potentially declared default values in the future)
> > for columns not appearing in the column list of insert statement to the
> > target table.
> > But this behavior does not satisfy some partial column update
> requirements
> > of some storage systems which allow storing null values. The problem is
> > that connectors cannot distinguish whether the null value of a column is
> > really from the user's data or whether it is a null value populated
> because
> > of partial insert behavior.
> >
> > Looking forward to your comments or feedback.
> >
> > [1]
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=240885081
> > [2] https://issues.apache.org/jira/browse/FLINK-18726
> >
> > Best,
> > Lincoln Lee
> >
>
