Hi Abid, We should have a FLIP as this would be a code contribution. If you provide your Confluence user name, we can grant you access to create one.
Is there also something from an Iceberg point of view needed to agree with the code contribution? Best regards, Martijn Op wo 19 okt. 2022 om 19:11 schreef <abmo.w...@icloud.com.invalid> > Thanks Martijn! > > Thanks for all the support and positive responses. I will start a vote > thread and send it out to the dev list. > > Also, we need help with creation of a new repo for the Iceberg Connector. > > Can someone help with the creation of a repo? Please let me know if I need > to create an issue or flip for that. > Following similar naming for other connectors, I propose > https://github.com/apache/flink-connector-iceberg (doesn’t exist) > > Thanks > Abid > > On 2022/10/19 08:41:02 Martijn Visser wrote: > > Hi all, > > > > Thanks for the info and also thanks Peter and Steven for offering to > > volunteer. I think that's a great idea and a necessity. > > > > Overall +1 given the current ideas to make this contribution happen. > > > > BTW congrats on reaching Iceberg 1.0, a great accomplishment :) > > > > Thanks, > > > > Martijn > > > > On Tue, Oct 18, 2022 at 12:31 AM Steven Wu <st...@gmail.com> wrote: > > > > > I was one of the maintainers for the Flink Iceberg connector in Iceberg > > > repo. I can volunteer as one of the initial maintainers if we decide to > > > move forward. > > > > > > On Mon, Oct 17, 2022 at 3:26 PM <ab...@icloud.com.invalid> wrote: > > > > > > > Hi Martijn, > > > > > > > > Yes, It is considered a connector in Flink terms. > > > > > > > > We wanted to join the Flink connector externalization effort so that > we > > > > can bring the Iceberg connector closer to the Flink community. We are > > > > hoping any issues with the APIs for Iceberg connector will surface > sooner > > > > and get more attention from the Flink community when the connector is > > > > within Flink umbrella rather than in Iceberg repo. Also to get better > > > > feedback from Flink experts when it comes to things related to adding > > > > things in a connector vs Flink itself. > > > > > > > > Thanks everyone for all your responses! Looking forward to the next > > > steps. > > > > > > > > Thanks > > > > Abid > > > > > > > > On 2022/10/14 03:37:09 Jark Wu wrote: > > > > > Thank Abid for the discussion, > > > > > > > > > > I'm also fine with maintaining it under the Flink project. > > > > > But I'm also interested in the response to Martijn's question. > > > > > > > > > > Besides, once the code is moved to the Flink project, are there any > > > > initial > > > > > maintainers for the connector we can find? > > > > > In addition, do we still maintain documentation under Iceberg > > > > > https://iceberg.apache.org/docs/latest/flink/ ? > > > > > > > > > > Best, > > > > > Jark > > > > > > > > > > > > > > > On Thu, 13 Oct 2022 at 17:52, yuxia <lu...@alumni.sjtu.edu.cn> > wrote: > > > > > > > > > > > +1. Thanks for driving it. Hope I can find some chances to take > part > > > in > > > > > > the future development of Iceberg Flink Connector. > > > > > > > > > > > > Best regards, > > > > > > Yuxia > > > > > > > > > > > > ----- 原始邮件 ----- > > > > > > 发件人: "Zheng Yu Chen" <ja...@gmail.com> > > > > > > 收件人: "dev" <de...@flink.apache.org> > > > > > > 发送时间: 星期四, 2022年 10 月 13日 上午 11:26:29 > > > > > > 主题: Re: [Discuss]- Donate Iceberg Flink Connector > > > > > > > > > > > > +1, thanks to drive it > > > > > > > > > > > > Abid Mohammed <ab...@icloud.com.invalid> 于2022年10月10日周一 09:22写道： > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > I would like to start a discussion about contributing Iceberg > Flink > > > > > > > Connector to Flink. > > > > > > > > > > > > > > I created a doc < > > > > > > > > > > > > > > > > > > > > > https://docs.google.com/document/d/1WC8xkPiVdwtsKL2VSPAUgzm9EjrPs8ZRjEtcwv93ISI/edit?usp=sharing > > > > > > > > > > > > > > with all the details following the Flink Connector template as > I > > > > don’t > > > > > > have > > > > > > > permissions to create a FLIP yet. > > > > > > > High level details are captured below: > > > > > > > > > > > > > > Motivation: > > > > > > > > > > > > > > This FLIP aims to contribute the existing Apache Iceberg Flink > > > > Connector > > > > > > > to Flink. > > > > > > > > > > > > > > Apache Iceberg is an open table format for huge analytic > datasets. > > > > > > Iceberg > > > > > > > adds tables to compute engines including Spark, Trino, > PrestoDB, > > > > Flink, > > > > > > > Hive and Impala using a high-performance table format that > works > > > just > > > > > > like > > > > > > > a SQL table. > > > > > > > Iceberg avoids unpleasant surprises. Schema evolution works and > > > won’t > > > > > > > inadvertently un-delete data. Users don’t need to know about > > > > partitioning > > > > > > > to get fast queries. Iceberg was designed to solve correctness > > > > problems > > > > > > in > > > > > > > eventually-consistent cloud object stores. > > > > > > > > > > > > > > Iceberg supports both Flink’s DataStream API and Table API. > Based > > > on > > > > the > > > > > > > guideline of the Flink community, only the latest 2 minor > versions > > > > are > > > > > > > actively maintained. See the Multi-Engine Support#apache-flink > for > > > > > > further > > > > > > > details. > > > > > > > > > > > > > > > > > > > > > Iceberg connector supports: > > > > > > > > > > > > > > • Source: detailed Source design < > > > > > > > > > > > > > > > > > > > > > https://docs.google.com/document/d/1q6xaBxUPFwYsW9aXWxYUh7die6O7rDeAPFQcTAMQ0GM/edit# > > > > > > >, > > > > > > > based on FLIP-27 > > > > > > > • Sink: detailed Sink design and interfaces used < > > > > > > > > > > > > > > > > > > > > > https://docs.google.com/document/d/1O-dPaFct59wUWQECXEEYIkl9_MOoG3zTbC2V-fZRwrg/edit# > > > > > > > > > > > > > > > • Usable in both DataStream and Table API/SQL > > > > > > > • DataStream read/append/overwrite > > > > > > > • SQL create/alter/drop table, select, insert into, > insert > > > > > > > overwrite > > > > > > > • Streaming or batch read in Java API > > > > > > > • Support for Flink’s Python API > > > > > > > > > > > > > > See Iceberg Flink < > > > > https://iceberg.apache.org/docs/latest/flink/#flink > > > > > > >for > > > > > > > detailed usage instructions. > > > > > > > > > > > > > > Looking forward to the discussion! > > > > > > > > > > > > > > Thanks > > > > > > > Abid > > > > > > > > > > > > > > > > -- Martijn https://twitter.com/MartijnVisser82 https://github.com/MartijnVisser
