+1, SeaTunnel the next version will strive to reduce user study costs, this feature can reduce the difficulty of getting started with seatunnel.
hailin0 <[email protected]> 于2023年7月22日周六 15:50写道： > > +1 > > Agree with you > > TaoZex <[email protected]> 于2023年7月22日周六 13:37写道： > > > Agree with you. We should also consider what metadata information is > > necessary to count in the catalog. > > > > > > Best Regards > > TaoZex > > ------------------ 原始邮件 ------------------ > > 发件人: > > "dev" > > < > > [email protected]>; > > 发送时间: 2023年7月22日(星期六) 上午10:17 > > 收件人: "dev"<[email protected]>; > > > > 主题: [Proposal] Spark&Flink compatible catalog > > > > > > > > *background* > > > > Due to the existence of commercial versions and open source web, it is > > necessary to clearly know the data source table and the table structure > > processed by each node when defining tasks. So the entire architecture > > layer needs to use a catalog approach, and the previous description of > > SeaTunnelRowType cannot meet the synchronization requirements. > > At present, the Zeta engine has supported the creation of connectors using > > the catalog method, but spark/link still uses the SeaTunnelRowType > > interface, which requires the connector to implement both the Catalog and > > SeaTunnelRowType interfaces in order to execute on different engines. This > > will require a lot of work and is meaningless. > > To solve this problem, we need spark/link to first support the Catalog > > method, so that the new connector can only implement the Catalog interface, > > while the old connector can gradually remove the SeaTunnelRowType > > interface. > > > > -- > > > > Best Regards > > > > ------------ > > > > EricJoy2048 > > [email protected]
