整库入湖的需求分为两部分:存量和增量 存量数据可以通过离线spark作业写hudi表,我提问的重点是增量数据怎样通过flink cdc方式写hudi表。如果一个库有成百上千张表的话,我不希望每一张表写一个flink作业,有没有方法可以只写一个作业就能同步一个库下所有表的增量数据呢?
At 2021-12-08 10:00:37, "su wenwen" <zhiyuan.frank...@outlook.com> wrote: >hi, I think Chengyanan is talking about hudig website >https://hudi.apache.org/docs/flink-quick-start-guide >Flink Guide | Apache >Hudi!<https://hudi.apache.org/docs/flink-quick-start-guide> >Notice that the save mode is now Append.In general, always use append mode >unless you are trying to create the table for the first time. Querying the >data again will now show updated records. Each write operation generates a new >commit denoted by the timestamp. Look for changes in _hoodie_commit_time, age >fields for the same _hoodie_record_keys in previous commit. {: .notice--info} >hudi.apache.org > >________________________________ >发件人: casel.chen <casel_c...@126.com> >发送时间: 2021年12月8日 0:19 >收件人: user-zh@flink.apache.org <user-zh@flink.apache.org> >主题: Re:回复: flink cdc支持mysql整库同步进hudi湖吗? > >“例子参考hudi官网 ” -----> 这个没有链接吗?官网哪里有介绍cdc整库入湖了? > > > > > > > > > > > > > > > > > >在 2021-12-07 10:23:03,"chengyanan1...@foxmail.com" ><chengyanan1...@foxmail.com> 写道: >>支持,例子参考hudi官网 >> >> >> >>chengyanan1...@foxmail.com >> >>发件人: casel.chen >>发送时间: 2021-12-06 23:55 >>收件人: user-zh@flink.apache.org >>主题: flink cdc支持mysql整库同步进hudi湖吗? >>flink cdc支持mysql整库同步进hudi湖吗?如果支持的话,希望能给一个例子,还要求能够支持schema变更。谢谢!