Hi, Thanks a lot Jean-Baptiste Onofré.
> how the users implement the ETL pipeline ? Users implement the ETL pipeline by configuring a json file or SQL. the following is one of examples: https://github.com/DTStack/chunjun/blob/master/flinkx-examples/json/doris/mysql_doris.json > Is it using a kind of SDK/DSL or do you plan an UI ? It is using a json file or SQL file. We had a plan to implement an UI a year ago, but current core developers have no time to implement an UI and are not good at Web frontend skills. > Is it very coupled to Flink or the runtime is "pluggable" ? Do you plan to > have some kind of optional storage, similar to Apache Druid. Nowadays, the ChunJun is coupled to Flink. We plan to support Spark or other runtimes, but no person in the ChunJun community is good at Apache Spark. We need to bring in more contributors. Best, LuNing Wang 王鲁宁 Daniel Widdis <wid...@gmail.com> 于2022年2月25日周五 11:11写道： > > Hi, LuNing. > > I am not an IPC member, just an interested open source enthusiast looking for > a project to contribute to. > > I was excited about contributing to another project that recently joined the > incubator but as someone who only speaks English, I have had a challenge when > a large number of issues are written in Chinese without enough translation > for me to help. > > Reading your proposal I thought this may be another opportunity for me to > help, but I just visited your github site and found that most of the current > open issues are not understandable to me, and while I am enthusiastic and > want to help, I do not see how I can do so. > > I am concerned about your ability to gain members of your project who do not > speak your language. > > On 2/23/22, 9:15 PM, "LuNing Wang" <wang4lun...@gmail.com> wrote: > > Hi, > > I am LuNing Wang who sent the Chunjun proposal using 'apa...@dtstack.com' > and I'm one of the maintainer of Chunjun project. > As Apache is a community of peers, I will use this email to reply to all > questions and issues in this thread, after I read The Apache Incubator > Cookbook. > > May I use this email to continue to communicate with the Apache community > in this thread? > > Best, > LuNing Wang 王鲁宁 > > > Apache <apa...@dtstack.com> 于2022年2月24日周四 09:57写道： > > > > > > > ------------------------------------------------------------------ > > 发件人：Calvin Kirs <k...@apache.org> > > 发送时间：2022年2月23日(星期三) 00:07 > > 收件人：general <general@incubator.apache.org> > > 主 题：Re: Chunjun Proposal > > > > Hi, > > > > I second with Tison and I'm glad to see your proposal, > > you must have put a lot of effort into drafting this proposal, > > but it needs to be clear what you expect and what you know about > Apache[1], > > and are we are on the same page? > > > > > > You can see the following information: > > The Apache Incubator Cookbook[2] > > The Apache Way[3] > > > > I'd be happy to help you if you need it. > > > > [1]https://www.apache.org/ > > [2]https://incubator.apache.org/cookbook/ > > [3]https://www.apache.org/theapacheway/index.html > > > > tison <wander4...@gmail.com> 于2022年2月22日周二 23:17写道： > > > > > Hi, > > > > > > I have two questions here: > > > > > > 1. How should I name you in this thread? Apache is a community of > peers. > > I > > > can't image I'm talking to the whole Chunjun community or "Apache" > expect > > > its your name. > > > 2. What's your expectations on going into the incubator? Among the > whole > > > proposal it's almost about what the current state of Chunjun and the > only > > > statement about your expectations is: > > > > > > > we seek to further prosper the community with the aid of Apache > > > > > > Could you elaborate a bit the motivation here? What help are you > seeking? > > > > > > Also I second to Sheng's comment that it's confused about your > > expressions > > > of contributors and initial committers. If your community continuously > > > promote contributors , why the initial committer list is quite a bit > > > limited? > > > > > > Best, > > > tison. > > > > > > > > > Sheng Wu <wu.sheng.841...@gmail.com> 于2022年2月22日周二 20:54写道： > > > > > > > I think this description is incorrect. > > > > > > > > > Our initial committers will submit iCLA(s), SGA, and CCLA(s). > > > > > > > > Committers are individuals, who should only submit ICLA, their > > > > employers are recommended to submit CCLA, the owner of the project > > > > should sign the SGA. > > > > ___ > > > > > > > > Also, I noticed a conflict in your description > > > > On one side, you mentioned `The initial committers are employees of > > > > DTStack.` with only 5 initial committers, and on the other hand, you > > > > gave a very long vendor list and core contributors list. > > > > So, which is an accurate description? If you have those > contributors, > > > > why were all of them invited as PPMC members? Do you have any public > > > > discussion about this decision? > > > > Such as GitHub ID(demotto) is the #8 in the contributor list, and > also > > > > listed in the core contributor list, but can't find it in the > initial > > > > committer list. > > > > > > > > > > > > Sheng Wu 吴晟 > > > > Twitter, wusheng1108 > > > > > > > > Lidong Dai <lidong...@apache.org> 于2022年2月22日周二 20:41写道： > > > > > > > > > > I am curious that Chunjun(was Flinkx) is built on the Flink CDC, > so > > > what > > > > is > > > > > its innovation? > > > > > > > > > > BTW, you shouldn't use the mail(apa...@dtstack.com) as your user > > > > account, > > > > > apache is a registered trademark of ASF > > > > > > > > > > > > > > > > > > > > Best Regards > > > > > > > > > > > > > > > > > > > > --------------- > > > > > Apache DolphinScheduler PMC Chair > > > > > Lidong Dai > > > > > lidong...@apache.org > > > > > Linkedin: https://www.linkedin.com/in/dailidong > > > > > Twitter: @WorkflowEasy <https://twitter.com/WorkflowEasy> > > > > > > > > > > --------------- > > > > > > > > > > > > > > > On Tue, Feb 22, 2022 at 3:10 PM Jean-Baptiste Onofré < > > j...@nanthrax.net> > > > > > wrote: > > > > > > > > > > > By the way, if you need a mentor, you can ping me. > > > > > > > > > > > > Regards > > > > > > JB > > > > > > > > > > > > On Tue, Feb 22, 2022 at 8:08 AM Jean-Baptiste Onofré < > > > j...@nanthrax.net> > > > > > > wrote: > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > It looks interesting. Quick questions about the current state > of > > > the > > > > > > > project/roadmap: how the users implement the ETL pipeline ? > Is it > > > > > > > using a kind of SDK/DSL or do you plan an UI ? > > > > > > > Is it very coupled to Flink or the runtime is "pluggable" ? Do > > you > > > > > > > plan to have some kind of optional storage, similar to Apache > > > Druid, > > > > > > > ... ? > > > > > > > > > > > > > > Regards > > > > > > > JB > > > > > > > > > > > > > > On Tue, Feb 22, 2022 at 7:41 AM Apache <apa...@dtstack.com> > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > Dear Apache Incubator Community, > > > > > > > > We propose to contribute Chunjun as an Apache Incubator > > project. > > > > > > > > We are still looking for possible Champion and Mentors if > > anyone > > > > would > > > > > > like to volunteer. Thanks a lot. > > > > > > > > Best Regards, > > > > > > > > Real-time computing engine team of DTStack. > > > > > > > > > > > > > > > > #Chunjun Proposal > > > > > > > > > > > > > > > > ##Abstract > > > > > > > > Chunjun is a distributed ETL tool and distributed data > > > integration > > > > > > tool. Currently Chunjun is based on Apache Flink. It was > initially > > > > known as > > > > > > FlinkX and renamed Chunjun on February 22, 2022. > > > > > > > > - Chunjun codebase: https://github.com/DTStack/chunjun > > > > > > > > > > > > > > > > ##Proposal > > > > > > > > We propose to contribute the Chunjun codebase to the Apache > > > > Software > > > > > > Foundation with the intent of forming a productive, meritocratic > > and > > > > open > > > > > > community around Chunjun’s continued development, according to > the > > > > 'Apache > > > > > > Way'. The Chunjun's source code is already under the Apache > License > > > > Version > > > > > > 2.0. > > > > > > > > > > > > > > > > ##Background > > > > > > > > We developed Chunjun in DTStack company in 2017, when we > > needed a > > > > > > low-code development and high-performance data integration > tool. It > > > has > > > > > > been an open-source project on GitHub since April 2018. Chunjun > is > > > > running > > > > > > in DTStack production environment all the time. Chunjun has also > > been > > > > > > widely used by companies in China, including DTStack ( > > > > > > https://www.dtstack.com/), Qihu360(https://www.360.cn/), > Iflytek ( > > > > > > https://www.iflytek.com/), XPeng Motors > (https://en.xiaopeng.com/ > > ), > > > > > > WeBank (https://www.webank.com/), > Asiainfo(https://asiainfo.com/), > > > > Guazi( > > > > > > https://www.guazi.com/), Hello Inc (https://www.hello-inc.com/), > > > etc. > > > > > > Nowadays, Chunjun has a strong community in China. > > > > > > > > > > > > > > > > ##Rationale > > > > > > > > High-performance of Chunjun is based on Apache Flink, and > > Chunjun > > > > can > > > > > > integrate data from different data source. Users only need to > > > > configure a > > > > > > JSON file to complete the data reading, transformer, and > writing. > > > > Users can > > > > > > implement new reader/writer plugins to meet their requirements. > > > Chunjun > > > > > > have implemented plugins that can capture data change for MySQL > to > > > > restore > > > > > > data for Apache Doris. > > > > > > > > Chunjun has the following feature: > > > > > > > > real-time and offline integrate data from different data > > sources. > > > > > > > > change data capture(CDC) to merge restore data. > > > > > > > > resume from broken-point. > > > > > > > > capture and collect dirty data. > > > > > > > > limit data transferring rate. > > > > > > > > thoughput metrics. > > > > > > > > capture and restore schema evolution. (TODO) > > > > > > > > > > > > > > > > ##Current Status###Meritocracy > > > > > > > > Since Chunjun was open-sourced, many enterprises have > adopted > > > > Chunjun > > > > > > to build up their data integration system. In return, we have > > > received > > > > many > > > > > > issue reports or enhancements from them simultaneously. The > > codebase > > > > is now > > > > > > mainly managed by the development team inside DTStack who's > > > > responsible for > > > > > > building internal data integration system too.###Community > > > > > > > > Chunjun has been building a community around contributors > and > > > > users to > > > > > > this framework for the last five years. We organized one meetup > in > > > > 2020. > > > > > > Currently, we communicate in Github issues and in chinese > DingTalk > > > > group. > > > > > > There are about 3000 people in this group. And we believe that > we > > can > > > > get a > > > > > > lot of help from the Apache Flink community too. We will > organize a > > > > meetup > > > > > > again in 2022.###Core Developers > > > > > > > > (In alphabetical order) > > > > > > > > Chao Xu (https://github.com/zoudaokoulife) > > > > > > > > Gongjiang Tang, (https://github.com/kyo-tom) > > > > > > > > Huai Yang, (https://github.com/yanghuaiGit) > > > > > > > > Jiangbo Li, (https://github.com/lijiangbo) > > > > > > > > Luning Wong, (https://github.com/deadwind4) > > > > > > > > Luo Li, (https://github.com/kanata163) > > > > > > > > Sishu Yang, (https://github.com/yangsishu) > > > > > > > > Tianzhu Wen, (https://github.com/WTZ468071157) > > > > > > > > Weiliang Hao, (https://github.com/xiuzhu9527) > > > > > > > > Wenqiang Liu, (https://github.com/meng1222) > > > > > > > > Xing Liu, (https://github.com/simenliuxing) > > > > > > > > Yang Lan, (https://github.com/HiLany) > > > > > > > > Yanquan Lv, (https://github.com/lvyanquan) > > > > > > > > Yifan Hu, (https://github.com/demotto) > > > > > > > > Zaiyue Yu, (https://github.com/tonybobam) > > > > > > > > Zhangwan Zhao, (https://github.com/jiemotongxue) > > > > > > > > Zhiqiang Li, (https://github.com/ChestnutQiang) > > > > > > > > They are almost working in real-time computing engine team > of > > > > DTStack. > > > > > > Only Yifan Hu working for CaoCao Tech. Most of them are Apache > > Flink > > > > > > contributor. > > > > > > > > > > > > > > > > ##Known Risks###Project Name > > > > > > > > The name of the project is Chunjun. Chunjun comes from > mandarin > > > > > > chinese Pinyin "Chun Jun", and it is one of the top ten famous > > swords > > > > in > > > > > > China.###Orphaned products > > > > > > > > More than 20 contributors and thousands of forks and star > > further > > > > show > > > > > > that Chunjun is actively supported, and we seek to further > prosper > > > the > > > > > > community with the aid of Apache. As a consequence, Chunjun is > > > > unlikely to > > > > > > be reduced to an orphaned project.###Inexperience with Open > Source > > > > > > > > Many of the Chunjun committers have experience working on > open > > > > source > > > > > > projects. They are also active contributors to other Apache > > projects. > > > > > > > > ###Homogenous Developers > > > > > > > > The most of core developers are from DTStack, and Chunjun > > > received > > > > > > some bug fixes and enhancements from other developers not > working > > at > > > > > > DTStack. ###Reliance on Salaried Developers > > > > > > > > Currently, most of core developers are paid to work on > Chunjun > > > > project > > > > > > by DTStack. We look forward to attracting more people outside > > DTStack > > > > to > > > > > > join this project.###Relationships with Other Apache Products > > > > > > > > We have integrated with Apache Flink, Apache Hadoop, Apache > > > Common > > > > and > > > > > > Apache HttpComponents, Log4J and Maven. > > > > > > > > Usage of Apache projects related to Chunjun plugin > > > > > > > > Apache Hive > > > > > > > > Apache Solr > > > > > > > > Apache Doris > > > > > > > > Apache HBase > > > > > > > > Apache Kudu > > > > > > > > Apache Kafka > > > > > > > > Apache Pulsar (TODO)###An Excessive Fascination with the > Apache > > > > Brand > > > > > > > > We acknowledge the value and reputation that the Apache > brand > > > would > > > > > > bring to Chunjun. However, our primary interest is in the > excellent > > > > > > community provided by Apache Software Foundation, in which all > the > > > > projects > > > > > > could gain stability for long-term development. > > > > > > > > > > > > > > > > ##Documentation > > > > > > > > A complete set of documents is provided on GitHub, including > > > > English > > > > > > and Simplified Chinese versions. > > > > > > > > English: > > > https://github.com/DTStack/chunjun/blob/master/README.md > > > > > > > > Chinese: > > > > https://github.com/DTStack/chunjun/blob/master/README_CH.md > > > > > > > > > > > > > > > > ##Initial Code > > > > > > > > https://github.com/DTStack/chunjun > > > > > > > > > > > > > > > > ##Initial Source and Intellectual Property Submission Plan > > > > > > > > The codebase is already licensed under the Apache License > 2.0 > > and > > > > the > > > > > > copyright is assigned to DTStack. If the project enters > incubator, > > > > DTStack > > > > > > will transfer the source code & trademark ownership to ASF via a > > > > Software > > > > > > Grant Agreement. Our initial committers will submit iCLA(s), > SGA, > > and > > > > > > CCLA(s). ##External DependenciesApache-2.0 licenses > > > > > > > > Apache Avro > > > > > > > > Apache Commons > > > > > > > > Apache Curator > > > > > > > > Apache Flink > > > > > > > > Apache Hadoop > > > > > > > > Apache HttpComponents > > > > > > > > Apache Log4j > > > > > > > > Gson > > > > > > > > Guava > > > > > > > > Jackson > > > > > > > > Powermock > > > > > > > > PrometheusEclipse Distribution License > > > > > > > > JUnitEPL licenses > > > > > > > > LogbackMIT licenses > > > > > > > > Mockito > > > > > > > > SLF4J > > > > > > > > > > > > > > > > ##Required Resources ###Git Repositories > > > > > > > > https://github.com/apache/incubator-chunjun###Issue Tracking > > > > > > > > The community would like to continue using GitHub > > > Issues.###Mailing > > > > > > List > > > > > > > > priv...@chunjun.incubator.apache.org > > > > > > > > d...@chunjun.incubator.apache.org > > > > > > > > comm...@chunjun.incubator.apache.org###Continuous > > <http://comm...@chunjun.incubator.apache.org#%23%23Continuous> > > > <http://comm...@chunjun.incubator.apache.org#%23%23Continuous> > > > > <http://comm...@chunjun.incubator.apache.org#%23%23Continuous> > > > > > > <http://comm...@chunjun.incubator.apache.org#%23%23Continuous> > > > > > > Integration tool > > > > > > > > GitHub Action > > > > > > > > > > > > > > > > ##Initial Committers > > > > > > > > (In alphabetical order) > > > > > > > > Chao Xu (https://github.com/zoudaokoulife, xuchao at dtstack > > dot > > > > com) > > > > > > > > Luning Wong (https://github.com/deadwind4, gfeng48 at gmail > > dot > > > > com) > > > > > > > > Sishu Yang (https://github.com/yangsishu, sishu at dtstack > dot > > > > com) > > > > > > > > Yang Huai (https://github.com/yanghuaiGit, dujie at dtstack > > dot > > > > com) > > > > > > > > Zhiqiang Li (https://github.com/ChestnutQiang, wujuan at > > dtstack > > > > dot > > > > > > com) > > > > > > > > > > > > > > > > > > > > > > > > ##Affiliations > > > > > > > > The initial committers are employees of DTStack. The > nominated > > > > mentors > > > > > > and champion are employees of TODO. > > > > > > > > > > > > > > > > ##Sponsors > > > > > > > > > > > > > > > > ###Champion > > > > > > > > TODO ###Nominated Mentors > > > > > > > > TODO > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > > > > > > For additional commands, e-mail: > general-h...@incubator.apache.org > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > > > > For additional commands, e-mail: general-h...@incubator.apache.org > > > > > > > > > > > > > > > > > -- > > Best wishes！ > > CalvinKirs > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > For additional commands, e-mail: general-h...@incubator.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
