Re: Chunjun Proposal
Hi LuNing I can see several similar concerns from IPMC members and committers of ASF TLPs. This seems to become a serious concern of this new proposal. We should resolve these concerns(if possible) first. In my understanding, from the ASF incubator perspective, we prefer `Do NOT confuse users` because we accepted projects nearly doing the same thing. We always encourage more people could join together and build a more powerful project and community, rather than building several similar projects. I would like you to recommend you talk with some IPMC members or Apache members, who have enough technology background of FlinkCDC, Camel, and SeeTunnel projects. They could help you to provide a more objective review, and discuss with you privately whether this new proposed project is suitable for the ASF incubator. Sheng Wu 吴晟 Twitter, wusheng1108 Lidong Dai 于2022年2月26日周六 19:12写道： > > hi, > > if ChunJun also plan to support Spark or other runtimes, it will be highly > similar to Apache SeaTunnel, have you considered contributing to Apache > SeaTunnel? > > > > > Best Regards > > > > --- > Apache DolphinScheduler PMC Chair > Lidong Dai > lidong...@apache.org > Linkedin: https://www.linkedin.com/in/dailidong > Twitter: @WorkflowEasy <https://twitter.com/WorkflowEasy> > > --- > > > On Fri, Feb 25, 2022 at 2:29 PM LuNing Wang wrote: > > > Hi, > > > > Thanks a lot Jean-Baptiste Onofré. > > > > > how the users implement the ETL pipeline ? > > > > Users implement the ETL pipeline by configuring a json file or SQL. > > the following is one of examples: > > > > https://github.com/DTStack/chunjun/blob/master/flinkx-examples/json/doris/mysql_doris.json > > > > > > > Is it using a kind of SDK/DSL or do you plan an UI ? > > > > It is using a json file or SQL file. We had a plan to implement an UI > > a year ago, but current core developers have no time to implement an > > UI and are not good at Web frontend skills. > > > > > > > Is it very coupled to Flink or the runtime is "pluggable" ? Do you plan > > to have some kind of optional storage, similar to Apache Druid. > > > > Nowadays, the ChunJun is coupled to Flink. We plan to support Spark or > > other runtimes, but no person in the ChunJun community is good at > > Apache Spark. We need to bring in more contributors. > > > > Best, > > LuNing Wang 王鲁宁 > > > > Daniel Widdis 于2022年2月25日周五 11:11写道： > > > > > > Hi, LuNing. > > > > > > I am not an IPC member, just an interested open source enthusiast > > looking for a project to contribute to. > > > > > > I was excited about contributing to another project that recently joined > > the incubator but as someone who only speaks English, I have had a > > challenge when a large number of issues are written in Chinese without > > enough translation for me to help. > > > > > > Reading your proposal I thought this may be another opportunity for me > > to help, but I just visited your github site and found that most of the > > current open issues are not understandable to me, and while I am > > enthusiastic and want to help, I do not see how I can do so. > > > > > > I am concerned about your ability to gain members of your project who > > do not speak your language. > > > > > > On 2/23/22, 9:15 PM, "LuNing Wang" wrote: > > > > > > Hi, > > > > > > I am LuNing Wang who sent the Chunjun proposal using ' > > apa...@dtstack.com' > > > and I'm one of the maintainer of Chunjun project. > > > As Apache is a community of peers, I will use this email to reply to > > all > > > questions and issues in this thread, after I read The Apache > > Incubator > > > Cookbook. > > > > > > May I use this email to continue to communicate with the Apache > > community > > > in this thread? > > > > > > Best, > > > LuNing Wang 王鲁宁 > > > > > > > > > Apache 于2022年2月24日周四 09:57写道： > > > > > > > > > > > > > > > -- > > > > 发件人：Calvin Kirs > > > > 发送时间：2022年2月23日(星期三) 00:07 > > > > 收件人：general > > > > 主 题：Re: Chunjun Proposal > > > > > > > > Hi, > > > > > > > > I second with Tison and I'm glad to see your proposal, > > >
Re: Chunjun Proposal
hi, if ChunJun also plan to support Spark or other runtimes, it will be highly similar to Apache SeaTunnel, have you considered contributing to Apache SeaTunnel? Best Regards --- Apache DolphinScheduler PMC Chair Lidong Dai lidong...@apache.org Linkedin: https://www.linkedin.com/in/dailidong Twitter: @WorkflowEasy <https://twitter.com/WorkflowEasy> --- On Fri, Feb 25, 2022 at 2:29 PM LuNing Wang wrote: > Hi, > > Thanks a lot Jean-Baptiste Onofré. > > > how the users implement the ETL pipeline ? > > Users implement the ETL pipeline by configuring a json file or SQL. > the following is one of examples: > > https://github.com/DTStack/chunjun/blob/master/flinkx-examples/json/doris/mysql_doris.json > > > > Is it using a kind of SDK/DSL or do you plan an UI ? > > It is using a json file or SQL file. We had a plan to implement an UI > a year ago, but current core developers have no time to implement an > UI and are not good at Web frontend skills. > > > > Is it very coupled to Flink or the runtime is "pluggable" ? Do you plan > to have some kind of optional storage, similar to Apache Druid. > > Nowadays, the ChunJun is coupled to Flink. We plan to support Spark or > other runtimes, but no person in the ChunJun community is good at > Apache Spark. We need to bring in more contributors. > > Best, > LuNing Wang 王鲁宁 > > Daniel Widdis 于2022年2月25日周五 11:11写道： > > > > Hi, LuNing. > > > > I am not an IPC member, just an interested open source enthusiast > looking for a project to contribute to. > > > > I was excited about contributing to another project that recently joined > the incubator but as someone who only speaks English, I have had a > challenge when a large number of issues are written in Chinese without > enough translation for me to help. > > > > Reading your proposal I thought this may be another opportunity for me > to help, but I just visited your github site and found that most of the > current open issues are not understandable to me, and while I am > enthusiastic and want to help, I do not see how I can do so. > > > > I am concerned about your ability to gain members of your project who > do not speak your language. > > > > On 2/23/22, 9:15 PM, "LuNing Wang" wrote: > > > > Hi, > > > > I am LuNing Wang who sent the Chunjun proposal using ' > apa...@dtstack.com' > > and I'm one of the maintainer of Chunjun project. > > As Apache is a community of peers, I will use this email to reply to > all > > questions and issues in this thread, after I read The Apache > Incubator > > Cookbook. > > > > May I use this email to continue to communicate with the Apache > community > > in this thread? > > > > Best, > > LuNing Wang 王鲁宁 > > > > > > Apache 于2022年2月24日周四 09:57写道： > > > > > > > > > > > -- > > > 发件人：Calvin Kirs > > > 发送时间：2022年2月23日(星期三) 00:07 > > > 收件人：general > > > 主 题：Re: Chunjun Proposal > > > > > > Hi, > > > > > > I second with Tison and I'm glad to see your proposal, > > > you must have put a lot of effort into drafting this proposal, > > > but it needs to be clear what you expect and what you know about > Apache[1], > > > and are we are on the same page? > > > > > > > > > You can see the following information: > > > The Apache Incubator Cookbook[2] > > > The Apache Way[3] > > > > > > I'd be happy to help you if you need it. > > > > > > [1]https://www.apache.org/ > > > [2]https://incubator.apache.org/cookbook/ > > > [3]https://www.apache.org/theapacheway/index.html > > > > > > tison 于2022年2月22日周二 23:17写道： > > > > > > > Hi, > > > > > > > > I have two questions here: > > > > > > > > 1. How should I name you in this thread? Apache is a community > of peers. > > > I > > > > can't image I'm talking to the whole Chunjun community or > "Apache" expect > > > > its your name. > > > > 2. What's your expectations on going into the incubator? Among > the whole > > > > proposal it's almost about what the current state of Chunjun and > the only > > > > statement about your expectatio
Re: Chunjun Proposal
Hi > Until now ChunJun is the only tool that can capture all tables data change of > the whole database With the above feature description, I am curious too about is this better to be an extension of Camel to provide data changes into a pipeline? Could you share why did you think `Establishing a new project and rebuilding all things from new` is better? Have you considered integrating with Camel? Sheng Wu 吴晟 Twitter, wusheng1108 LuNing Wang 于2022年2月25日周五 16:38写道： > > Hi, > > Thank a lot Lidong Dai 代立冬老师. > > > I am curious that Chunjun(was Flinkx) is built on the Flink CDC, so what is > > its innovation? > > The ChunJun is not built on the Flink CDC. When we built the ChunJun > CDC code, the Flink CDC did not exist. We independently created a new > data structure that is different from Flink CDC. Every CDC record of > this data structure owns all metadata. > > Until now ChunJun is the only tool that can capture all tables data > change of the whole database by configuring a simple file rather than > writing java code. The Flink CDC can capture one table by using SQL, > and capture all tables by using java code. > > We will also implement capture schema evolution, it is also based on a > new data structure. Now we have implemented capture schema evolution > in the reader/source part, we will implement the writer/sink part. > > Best, > LuNing Wang 王鲁宁 > > Jean-Baptiste Onofré 于2022年2月25日周五 14:44写道： > > > > Hi > > > > Thanks for the details. > > > > Another question: how do you compare Chunjun with camel, nifi, goblin, and > > others ? > > What kind of community you already have ? > > > > Thanks > > Regards > > JB > > > > Le ven. 25 févr. 2022 à 07:28, LuNing Wang a écrit : > >> > >> Hi, > >> > >> Thanks a lot Jean-Baptiste Onofré. > >> > >> > how the users implement the ETL pipeline ? > >> > >> Users implement the ETL pipeline by configuring a json file or SQL. > >> the following is one of examples: > >> https://github.com/DTStack/chunjun/blob/master/flinkx-examples/json/doris/mysql_doris.json > >> > >> > >> > Is it using a kind of SDK/DSL or do you plan an UI ? > >> > >> It is using a json file or SQL file. We had a plan to implement an UI > >> a year ago, but current core developers have no time to implement an > >> UI and are not good at Web frontend skills. > >> > >> > >> > Is it very coupled to Flink or the runtime is "pluggable" ? Do you plan > >> > to have some kind of optional storage, similar to Apache Druid. > >> > >> Nowadays, the ChunJun is coupled to Flink. We plan to support Spark or > >> other runtimes, but no person in the ChunJun community is good at > >> Apache Spark. We need to bring in more contributors. > >> > >> Best, > >> LuNing Wang 王鲁宁 > >> > >> Daniel Widdis 于2022年2月25日周五 11:11写道： > >> > > >> > Hi, LuNing. > >> > > >> > I am not an IPC member, just an interested open source enthusiast > >> > looking for a project to contribute to. > >> > > >> > I was excited about contributing to another project that recently joined > >> > the incubator but as someone who only speaks English, I have had a > >> > challenge when a large number of issues are written in Chinese without > >> > enough translation for me to help. > >> > > >> > Reading your proposal I thought this may be another opportunity for me > >> > to help, but I just visited your github site and found that most of the > >> > current open issues are not understandable to me, and while I am > >> > enthusiastic and want to help, I do not see how I can do so. > >> > > >> > I am concerned about your ability to gain members of your project who > >> > do not speak your language. > >> > > >> > On 2/23/22, 9:15 PM, "LuNing Wang" wrote: > >> > > >> > Hi, > >> > > >> > I am LuNing Wang who sent the Chunjun proposal using > >> > 'apa...@dtstack.com' > >> > and I'm one of the maintainer of Chunjun project. > >> > As Apache is a community of peers, I will use this email to reply to > >> > all > >> > questions and issues in this thread, after I read The Apache > >> > Incubator > >> > Cookbook. > >> > > >> > May I use this email t
Re: Chunjun Proposal
Hi, Thank a lot Lidong Dai 代立冬老师. > I am curious that Chunjun(was Flinkx) is built on the Flink CDC, so what is > its innovation? The ChunJun is not built on the Flink CDC. When we built the ChunJun CDC code, the Flink CDC did not exist. We independently created a new data structure that is different from Flink CDC. Every CDC record of this data structure owns all metadata. Until now ChunJun is the only tool that can capture all tables data change of the whole database by configuring a simple file rather than writing java code. The Flink CDC can capture one table by using SQL, and capture all tables by using java code. We will also implement capture schema evolution, it is also based on a new data structure. Now we have implemented capture schema evolution in the reader/source part, we will implement the writer/sink part. Best, LuNing Wang 王鲁宁 Jean-Baptiste Onofré 于2022年2月25日周五 14:44写道： > > Hi > > Thanks for the details. > > Another question: how do you compare Chunjun with camel, nifi, goblin, and > others ? > What kind of community you already have ? > > Thanks > Regards > JB > > Le ven. 25 févr. 2022 à 07:28, LuNing Wang a écrit : >> >> Hi, >> >> Thanks a lot Jean-Baptiste Onofré. >> >> > how the users implement the ETL pipeline ? >> >> Users implement the ETL pipeline by configuring a json file or SQL. >> the following is one of examples: >> https://github.com/DTStack/chunjun/blob/master/flinkx-examples/json/doris/mysql_doris.json >> >> >> > Is it using a kind of SDK/DSL or do you plan an UI ? >> >> It is using a json file or SQL file. We had a plan to implement an UI >> a year ago, but current core developers have no time to implement an >> UI and are not good at Web frontend skills. >> >> >> > Is it very coupled to Flink or the runtime is "pluggable" ? Do you plan to >> > have some kind of optional storage, similar to Apache Druid. >> >> Nowadays, the ChunJun is coupled to Flink. We plan to support Spark or >> other runtimes, but no person in the ChunJun community is good at >> Apache Spark. We need to bring in more contributors. >> >> Best, >> LuNing Wang 王鲁宁 >> >> Daniel Widdis 于2022年2月25日周五 11:11写道： >> > >> > Hi, LuNing. >> > >> > I am not an IPC member, just an interested open source enthusiast looking >> > for a project to contribute to. >> > >> > I was excited about contributing to another project that recently joined >> > the incubator but as someone who only speaks English, I have had a >> > challenge when a large number of issues are written in Chinese without >> > enough translation for me to help. >> > >> > Reading your proposal I thought this may be another opportunity for me to >> > help, but I just visited your github site and found that most of the >> > current open issues are not understandable to me, and while I am >> > enthusiastic and want to help, I do not see how I can do so. >> > >> > I am concerned about your ability to gain members of your project who do >> > not speak your language. >> > >> > On 2/23/22, 9:15 PM, "LuNing Wang" wrote: >> > >> > Hi, >> > >> > I am LuNing Wang who sent the Chunjun proposal using >> > 'apa...@dtstack.com' >> > and I'm one of the maintainer of Chunjun project. >> > As Apache is a community of peers, I will use this email to reply to >> > all >> > questions and issues in this thread, after I read The Apache Incubator >> > Cookbook. >> > >> > May I use this email to continue to communicate with the Apache >> > community >> > in this thread? >> > >> > Best, >> > LuNing Wang 王鲁宁 >> > >> > >> > Apache 于2022年2月24日周四 09:57写道： >> > >> > > >> > > >> > > -- >> > > 发件人：Calvin Kirs >> > > 发送时间：2022年2月23日(星期三) 00:07 >> > > 收件人：general >> > > 主 题：Re: Chunjun Proposal >> > > >> > > Hi, >> > > >> > > I second with Tison and I'm glad to see your proposal, >> > > you must have put a lot of effort into drafting this proposal, >> > > but it needs to be clear what you expect and what you know about >> > Apache[1], >> > > and are we are on the same page? >> > > >&
Re: Chunjun Proposal
Hi Thanks for the details. Another question: how do you compare Chunjun with camel, nifi, goblin, and others ? What kind of community you already have ? Thanks Regards JB Le ven. 25 févr. 2022 à 07:28, LuNing Wang a écrit : > Hi, > > Thanks a lot Jean-Baptiste Onofré. > > > how the users implement the ETL pipeline ? > > Users implement the ETL pipeline by configuring a json file or SQL. > the following is one of examples: > > https://github.com/DTStack/chunjun/blob/master/flinkx-examples/json/doris/mysql_doris.json > > > > Is it using a kind of SDK/DSL or do you plan an UI ? > > It is using a json file or SQL file. We had a plan to implement an UI > a year ago, but current core developers have no time to implement an > UI and are not good at Web frontend skills. > > > > Is it very coupled to Flink or the runtime is "pluggable" ? Do you plan > to have some kind of optional storage, similar to Apache Druid. > > Nowadays, the ChunJun is coupled to Flink. We plan to support Spark or > other runtimes, but no person in the ChunJun community is good at > Apache Spark. We need to bring in more contributors. > > Best, > LuNing Wang 王鲁宁 > > Daniel Widdis 于2022年2月25日周五 11:11写道： > > > > Hi, LuNing. > > > > I am not an IPC member, just an interested open source enthusiast > looking for a project to contribute to. > > > > I was excited about contributing to another project that recently joined > the incubator but as someone who only speaks English, I have had a > challenge when a large number of issues are written in Chinese without > enough translation for me to help. > > > > Reading your proposal I thought this may be another opportunity for me > to help, but I just visited your github site and found that most of the > current open issues are not understandable to me, and while I am > enthusiastic and want to help, I do not see how I can do so. > > > > I am concerned about your ability to gain members of your project who > do not speak your language. > > > > On 2/23/22, 9:15 PM, "LuNing Wang" wrote: > > > > Hi, > > > > I am LuNing Wang who sent the Chunjun proposal using ' > apa...@dtstack.com' > > and I'm one of the maintainer of Chunjun project. > > As Apache is a community of peers, I will use this email to reply to > all > > questions and issues in this thread, after I read The Apache > Incubator > > Cookbook. > > > > May I use this email to continue to communicate with the Apache > community > > in this thread? > > > > Best, > > LuNing Wang 王鲁宁 > > > > > > Apache 于2022年2月24日周四 09:57写道： > > > > > > > > > > > -- > > > 发件人：Calvin Kirs > > > 发送时间：2022年2月23日(星期三) 00:07 > > > 收件人：general > > > 主 题：Re: Chunjun Proposal > > > > > > Hi, > > > > > > I second with Tison and I'm glad to see your proposal, > > > you must have put a lot of effort into drafting this proposal, > > > but it needs to be clear what you expect and what you know about > Apache[1], > > > and are we are on the same page? > > > > > > > > > You can see the following information: > > > The Apache Incubator Cookbook[2] > > > The Apache Way[3] > > > > > > I'd be happy to help you if you need it. > > > > > > [1]https://www.apache.org/ > > > [2]https://incubator.apache.org/cookbook/ > > > [3]https://www.apache.org/theapacheway/index.html > > > > > > tison 于2022年2月22日周二 23:17写道： > > > > > > > Hi, > > > > > > > > I have two questions here: > > > > > > > > 1. How should I name you in this thread? Apache is a community > of peers. > > > I > > > > can't image I'm talking to the whole Chunjun community or > "Apache" expect > > > > its your name. > > > > 2. What's your expectations on going into the incubator? Among > the whole > > > > proposal it's almost about what the current state of Chunjun and > the only > > > > statement about your expectations is: > > > > > > > > > we seek to further prosper the community with the aid of Apache > > > > > > > > Could you elaborate a bit the
Re: Chunjun Proposal
Hi, Thanks a lot Jean-Baptiste Onofré. > how the users implement the ETL pipeline ? Users implement the ETL pipeline by configuring a json file or SQL. the following is one of examples: https://github.com/DTStack/chunjun/blob/master/flinkx-examples/json/doris/mysql_doris.json > Is it using a kind of SDK/DSL or do you plan an UI ? It is using a json file or SQL file. We had a plan to implement an UI a year ago, but current core developers have no time to implement an UI and are not good at Web frontend skills. > Is it very coupled to Flink or the runtime is "pluggable" ? Do you plan to > have some kind of optional storage, similar to Apache Druid. Nowadays, the ChunJun is coupled to Flink. We plan to support Spark or other runtimes, but no person in the ChunJun community is good at Apache Spark. We need to bring in more contributors. Best, LuNing Wang 王鲁宁 Daniel Widdis 于2022年2月25日周五 11:11写道： > > Hi, LuNing. > > I am not an IPC member, just an interested open source enthusiast looking for > a project to contribute to. > > I was excited about contributing to another project that recently joined the > incubator but as someone who only speaks English, I have had a challenge when > a large number of issues are written in Chinese without enough translation > for me to help. > > Reading your proposal I thought this may be another opportunity for me to > help, but I just visited your github site and found that most of the current > open issues are not understandable to me, and while I am enthusiastic and > want to help, I do not see how I can do so. > > I am concerned about your ability to gain members of your project who do not > speak your language. > > On 2/23/22, 9:15 PM, "LuNing Wang" wrote: > > Hi, > > I am LuNing Wang who sent the Chunjun proposal using 'apa...@dtstack.com' > and I'm one of the maintainer of Chunjun project. > As Apache is a community of peers, I will use this email to reply to all > questions and issues in this thread, after I read The Apache Incubator > Cookbook. > > May I use this email to continue to communicate with the Apache community > in this thread? > > Best, > LuNing Wang 王鲁宁 > > > Apache 于2022年2月24日周四 09:57写道： > > > > > > > ------ > > 发件人：Calvin Kirs > > 发送时间：2022年2月23日(星期三) 00:07 > > 收件人：general > > 主 题：Re: Chunjun Proposal > > > > Hi, > > > > I second with Tison and I'm glad to see your proposal, > > you must have put a lot of effort into drafting this proposal, > > but it needs to be clear what you expect and what you know about > Apache[1], > > and are we are on the same page? > > > > > > You can see the following information: > > The Apache Incubator Cookbook[2] > > The Apache Way[3] > > > > I'd be happy to help you if you need it. > > > > [1]https://www.apache.org/ > > [2]https://incubator.apache.org/cookbook/ > > [3]https://www.apache.org/theapacheway/index.html > > > > tison 于2022年2月22日周二 23:17写道： > > > > > Hi, > > > > > > I have two questions here: > > > > > > 1. How should I name you in this thread? Apache is a community of > peers. > > I > > > can't image I'm talking to the whole Chunjun community or "Apache" > expect > > > its your name. > > > 2. What's your expectations on going into the incubator? Among the > whole > > > proposal it's almost about what the current state of Chunjun and the > only > > > statement about your expectations is: > > > > > > > we seek to further prosper the community with the aid of Apache > > > > > > Could you elaborate a bit the motivation here? What help are you > seeking? > > > > > > Also I second to Sheng's comment that it's confused about your > > expressions > > > of contributors and initial committers. If your community continuously > > > promote contributors , why the initial committer list is quite a bit > > > limited? > > > > > > Best, > > > tison. > > > > > > > > > Sheng Wu 于2022年2月22日周二 20:54写道： > > > > > > > I think this description is incorrect. > > > > > > > > > Our initial committers will submit iCLA(s), SGA, and CCLA(s). > &g
Re: Chunjun Proposal
Hi, LuNing.
I am not an IPC member, just an interested open source enthusiast looking for a
project to contribute to.
I was excited about contributing to another project that recently joined the
incubator but as someone who only speaks English, I have had a challenge when a
large number of issues are written in Chinese without enough translation for me
to help.
Reading your proposal I thought this may be another opportunity for me to help,
but I just visited your github site and found that most of the current open
issues are not understandable to me, and while I am enthusiastic and want to
help, I do not see how I can do so.
I am concerned about your ability to gain members of your project who do not
speak your language.
On 2/23/22, 9:15 PM, "LuNing Wang" wrote:
Hi,
I am LuNing Wang who sent the Chunjun proposal using 'apa...@dtstack.com'
and I'm one of the maintainer of Chunjun project.
As Apache is a community of peers, I will use this email to reply to all
questions and issues in this thread, after I read The Apache Incubator
Cookbook.
May I use this email to continue to communicate with the Apache community
in this thread?
Best,
LuNing Wang 王鲁宁
Apache 于2022年2月24日周四 09:57写道：
>
>
> --
> 发件人：Calvin Kirs
> 发送时间：2022年2月23日(星期三) 00:07
> 收件人：general
> 主 题：Re: Chunjun Proposal
>
> Hi,
>
> I second with Tison and I'm glad to see your proposal,
> you must have put a lot of effort into drafting this proposal,
> but it needs to be clear what you expect and what you know about
Apache[1],
> and are we are on the same page?
>
>
> You can see the following information:
> The Apache Incubator Cookbook[2]
> The Apache Way[3]
>
> I'd be happy to help you if you need it.
>
> [1]https://www.apache.org/
> [2]https://incubator.apache.org/cookbook/
> [3]https://www.apache.org/theapacheway/index.html
>
> tison 于2022年2月22日周二 23:17写道：
>
> > Hi,
> >
> > I have two questions here:
> >
> > 1. How should I name you in this thread? Apache is a community of peers.
> I
> > can't image I'm talking to the whole Chunjun community or "Apache"
expect
> > its your name.
> > 2. What's your expectations on going into the incubator? Among the whole
> > proposal it's almost about what the current state of Chunjun and the
only
> > statement about your expectations is:
> >
> > > we seek to further prosper the community with the aid of Apache
> >
> > Could you elaborate a bit the motivation here? What help are you
seeking?
> >
> > Also I second to Sheng's comment that it's confused about your
> expressions
> > of contributors and initial committers. If your community continuously
> > promote contributors , why the initial committer list is quite a bit
> > limited?
> >
> > Best,
> > tison.
> >
> >
> > Sheng Wu 于2022年2月22日周二 20:54写道：
> >
> > > I think this description is incorrect.
> > >
> > > > Our initial committers will submit iCLA(s), SGA, and CCLA(s).
> > >
> > > Committers are individuals, who should only submit ICLA, their
> > > employers are recommended to submit CCLA, the owner of the project
> > > should sign the SGA.
> > > ___
> > >
> > > Also, I noticed a conflict in your description
> > > On one side, you mentioned `The initial committers are employees of
> > > DTStack.` with only 5 initial committers, and on the other hand, you
> > > gave a very long vendor list and core contributors list.
> > > So, which is an accurate description? If you have those contributors,
> > > why were all of them invited as PPMC members? Do you have any public
> > > discussion about this decision?
> > > Such as GitHub ID(demotto) is the #8 in the contributor list, and also
> > > listed in the core contributor list, but can't find it in the initial
> > > committer list.
> > >
> > >
> > > Sheng Wu 吴晟
> > > Twitter, wusheng1108
> > >
> > > Lidong Dai 于2022年2月22日周二 20:41写道：
> > > >
> > > > I am curious that Chunjun(was Flinkx) is built on the Flink CDC, so
> > what
> > > is
>
Re: Chunjun Proposal
Hi LuNing, It's good to see Chunjun project is interested about ASF incubating process. Please go through the discussion and try to answer the questions of the proposal. It will help you address the concerns and polish the proposal. Willem Jiang Twitter: willemjiang Weibo: 姜宁willem On Thu, Feb 24, 2022 at 11:16 AM LuNing Wang wrote: > > Hi, > > I am LuNing Wang who sent the Chunjun proposal using 'apa...@dtstack.com' > and I'm one of the maintainer of Chunjun project. > As Apache is a community of peers, I will use this email to reply to all > questions and issues in this thread, after I read The Apache Incubator > Cookbook. > > May I use this email to continue to communicate with the Apache community > in this thread? > > Best, > LuNing Wang 王鲁宁 > - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Chunjun Proposal
> May I use this email to continue to communicate with the Apache community in this thread? It is fine to continue on this thread. Sheng Wu 吴晟 Twitter, wusheng1108 LuNing Wang 于2022年2月24日周四 11:16写道： > > Hi, > > I am LuNing Wang who sent the Chunjun proposal using 'apa...@dtstack.com' > and I'm one of the maintainer of Chunjun project. > As Apache is a community of peers, I will use this email to reply to all > questions and issues in this thread, after I read The Apache Incubator > Cookbook. > > May I use this email to continue to communicate with the Apache community > in this thread? > > Best, > LuNing Wang 王鲁宁 > > > Apache 于2022年2月24日周四 09:57写道： > > > > > > > -- > > 发件人：Calvin Kirs > > 发送时间：2022年2月23日(星期三) 00:07 > > 收件人：general > > 主 题：Re: Chunjun Proposal > > > > Hi, > > > > I second with Tison and I'm glad to see your proposal, > > you must have put a lot of effort into drafting this proposal, > > but it needs to be clear what you expect and what you know about Apache[1], > > and are we are on the same page? > > > > > > You can see the following information: > > The Apache Incubator Cookbook[2] > > The Apache Way[3] > > > > I'd be happy to help you if you need it. > > > > [1]https://www.apache.org/ > > [2]https://incubator.apache.org/cookbook/ > > [3]https://www.apache.org/theapacheway/index.html > > > > tison 于2022年2月22日周二 23:17写道： > > > > > Hi, > > > > > > I have two questions here: > > > > > > 1. How should I name you in this thread? Apache is a community of peers. > > I > > > can't image I'm talking to the whole Chunjun community or "Apache" expect > > > its your name. > > > 2. What's your expectations on going into the incubator? Among the whole > > > proposal it's almost about what the current state of Chunjun and the only > > > statement about your expectations is: > > > > > > > we seek to further prosper the community with the aid of Apache > > > > > > Could you elaborate a bit the motivation here? What help are you seeking? > > > > > > Also I second to Sheng's comment that it's confused about your > > expressions > > > of contributors and initial committers. If your community continuously > > > promote contributors , why the initial committer list is quite a bit > > > limited? > > > > > > Best, > > > tison. > > > > > > > > > Sheng Wu 于2022年2月22日周二 20:54写道： > > > > > > > I think this description is incorrect. > > > > > > > > > Our initial committers will submit iCLA(s), SGA, and CCLA(s). > > > > > > > > Committers are individuals, who should only submit ICLA, their > > > > employers are recommended to submit CCLA, the owner of the project > > > > should sign the SGA. > > > > ___ > > > > > > > > Also, I noticed a conflict in your description > > > > On one side, you mentioned `The initial committers are employees of > > > > DTStack.` with only 5 initial committers, and on the other hand, you > > > > gave a very long vendor list and core contributors list. > > > > So, which is an accurate description? If you have those contributors, > > > > why were all of them invited as PPMC members? Do you have any public > > > > discussion about this decision? > > > > Such as GitHub ID(demotto) is the #8 in the contributor list, and also > > > > listed in the core contributor list, but can't find it in the initial > > > > committer list. > > > > > > > > > > > > Sheng Wu 吴晟 > > > > Twitter, wusheng1108 > > > > > > > > Lidong Dai 于2022年2月22日周二 20:41写道： > > > > > > > > > > I am curious that Chunjun(was Flinkx) is built on the Flink CDC, so > > > what > > > > is > > > > > its innovation? > > > > > > > > > > BTW, you shouldn't use the mail(apa...@dtstack.com) as your user > > > > account, > > > > > apache is a registered trademark of ASF > > > > > > > > > > > > > > > > > > > > Best Regards > > > > > > > > > > > > > > > > > > > > --- > > > > > Apache DolphinSched
Re: Chunjun Proposal
Hi, I am LuNing Wang who sent the Chunjun proposal using 'apa...@dtstack.com' and I'm one of the maintainer of Chunjun project. As Apache is a community of peers, I will use this email to reply to all questions and issues in this thread, after I read The Apache Incubator Cookbook. May I use this email to continue to communicate with the Apache community in this thread? Best, LuNing Wang 王鲁宁 Apache 于2022年2月24日周四 09:57写道： > > > -- > 发件人：Calvin Kirs > 发送时间：2022年2月23日(星期三) 00:07 > 收件人：general > 主 题：Re: Chunjun Proposal > > Hi, > > I second with Tison and I'm glad to see your proposal, > you must have put a lot of effort into drafting this proposal, > but it needs to be clear what you expect and what you know about Apache[1], > and are we are on the same page? > > > You can see the following information: > The Apache Incubator Cookbook[2] > The Apache Way[3] > > I'd be happy to help you if you need it. > > [1]https://www.apache.org/ > [2]https://incubator.apache.org/cookbook/ > [3]https://www.apache.org/theapacheway/index.html > > tison 于2022年2月22日周二 23:17写道： > > > Hi, > > > > I have two questions here: > > > > 1. How should I name you in this thread? Apache is a community of peers. > I > > can't image I'm talking to the whole Chunjun community or "Apache" expect > > its your name. > > 2. What's your expectations on going into the incubator? Among the whole > > proposal it's almost about what the current state of Chunjun and the only > > statement about your expectations is: > > > > > we seek to further prosper the community with the aid of Apache > > > > Could you elaborate a bit the motivation here? What help are you seeking? > > > > Also I second to Sheng's comment that it's confused about your > expressions > > of contributors and initial committers. If your community continuously > > promote contributors , why the initial committer list is quite a bit > > limited? > > > > Best, > > tison. > > > > > > Sheng Wu 于2022年2月22日周二 20:54写道： > > > > > I think this description is incorrect. > > > > > > > Our initial committers will submit iCLA(s), SGA, and CCLA(s). > > > > > > Committers are individuals, who should only submit ICLA, their > > > employers are recommended to submit CCLA, the owner of the project > > > should sign the SGA. > > > ___ > > > > > > Also, I noticed a conflict in your description > > > On one side, you mentioned `The initial committers are employees of > > > DTStack.` with only 5 initial committers, and on the other hand, you > > > gave a very long vendor list and core contributors list. > > > So, which is an accurate description? If you have those contributors, > > > why were all of them invited as PPMC members? Do you have any public > > > discussion about this decision? > > > Such as GitHub ID(demotto) is the #8 in the contributor list, and also > > > listed in the core contributor list, but can't find it in the initial > > > committer list. > > > > > > > > > Sheng Wu 吴晟 > > > Twitter, wusheng1108 > > > > > > Lidong Dai 于2022年2月22日周二 20:41写道： > > > > > > > > I am curious that Chunjun(was Flinkx) is built on the Flink CDC, so > > what > > > is > > > > its innovation? > > > > > > > > BTW, you shouldn't use the mail(apa...@dtstack.com) as your user > > > account, > > > > apache is a registered trademark of ASF > > > > > > > > > > > > > > > > Best Regards > > > > > > > > > > > > > > > > --- > > > > Apache DolphinScheduler PMC Chair > > > > Lidong Dai > > > > lidong...@apache.org > > > > Linkedin: https://www.linkedin.com/in/dailidong > > > > Twitter: @WorkflowEasy <https://twitter.com/WorkflowEasy> > > > > > > > > --- > > > > > > > > > > > > On Tue, Feb 22, 2022 at 3:10 PM Jean-Baptiste Onofré < > j...@nanthrax.net> > > > > wrote: > > > > > > > > > By the way, if you need a mentor, you can ping me. > > > > > > > > > > Regards > > > > > JB > > > > > > > > > > On Tue, Feb 22, 2022 at 8:08 AM Jean-Baptiste Onofré < > > j...@nanthrax.net> > > > >
Re: Chunjun Proposal
Hi, I second with Tison and I'm glad to see your proposal, you must have put a lot of effort into drafting this proposal, but it needs to be clear what you expect and what you know about Apache[1], and are we are on the same page? You can see the following information: The Apache Incubator Cookbook[2] The Apache Way[3] I'd be happy to help you if you need it. [1]https://www.apache.org/ [2]https://incubator.apache.org/cookbook/ [3]https://www.apache.org/theapacheway/index.html tison 于2022年2月22日周二 23:17写道： > Hi, > > I have two questions here: > > 1. How should I name you in this thread? Apache is a community of peers. I > can't image I'm talking to the whole Chunjun community or "Apache" expect > its your name. > 2. What's your expectations on going into the incubator? Among the whole > proposal it's almost about what the current state of Chunjun and the only > statement about your expectations is: > > > we seek to further prosper the community with the aid of Apache > > Could you elaborate a bit the motivation here? What help are you seeking? > > Also I second to Sheng's comment that it's confused about your expressions > of contributors and initial committers. If your community continuously > promote contributors , why the initial committer list is quite a bit > limited? > > Best, > tison. > > > Sheng Wu 于2022年2月22日周二 20:54写道： > > > I think this description is incorrect. > > > > > Our initial committers will submit iCLA(s), SGA, and CCLA(s). > > > > Committers are individuals, who should only submit ICLA, their > > employers are recommended to submit CCLA, the owner of the project > > should sign the SGA. > > ___ > > > > Also, I noticed a conflict in your description > > On one side, you mentioned `The initial committers are employees of > > DTStack.` with only 5 initial committers, and on the other hand, you > > gave a very long vendor list and core contributors list. > > So, which is an accurate description? If you have those contributors, > > why were all of them invited as PPMC members? Do you have any public > > discussion about this decision? > > Such as GitHub ID(demotto) is the #8 in the contributor list, and also > > listed in the core contributor list, but can't find it in the initial > > committer list. > > > > > > Sheng Wu 吴晟 > > Twitter, wusheng1108 > > > > Lidong Dai 于2022年2月22日周二 20:41写道： > > > > > > I am curious that Chunjun(was Flinkx) is built on the Flink CDC, so > what > > is > > > its innovation? > > > > > > BTW, you shouldn't use the mail(apa...@dtstack.com) as your user > > account, > > > apache is a registered trademark of ASF > > > > > > > > > > > > Best Regards > > > > > > > > > > > > --- > > > Apache DolphinScheduler PMC Chair > > > Lidong Dai > > > lidong...@apache.org > > > Linkedin: https://www.linkedin.com/in/dailidong > > > Twitter: @WorkflowEasy <https://twitter.com/WorkflowEasy> > > > > > > --- > > > > > > > > > On Tue, Feb 22, 2022 at 3:10 PM Jean-Baptiste Onofré > > > wrote: > > > > > > > By the way, if you need a mentor, you can ping me. > > > > > > > > Regards > > > > JB > > > > > > > > On Tue, Feb 22, 2022 at 8:08 AM Jean-Baptiste Onofré < > j...@nanthrax.net> > > > > wrote: > > > > > > > > > > Hi, > > > > > > > > > > It looks interesting. Quick questions about the current state of > the > > > > > project/roadmap: how the users implement the ETL pipeline ? Is it > > > > > using a kind of SDK/DSL or do you plan an UI ? > > > > > Is it very coupled to Flink or the runtime is "pluggable" ? Do you > > > > > plan to have some kind of optional storage, similar to Apache > Druid, > > > > > ... ? > > > > > > > > > > Regards > > > > > JB > > > > > > > > > > On Tue, Feb 22, 2022 at 7:41 AM Apache wrote: > > > > > > > > > > > > > > > > > > Dear Apache Incubator Community, > > > > > > We propose to contribute Chunjun as an Apache Incubator project. > > > > > > We are still looking for possible Champion and Mentors if anyone > > would > > > > like to volunteer. Thanks a lot. > > > > > > Best Regards, > &g
Re: Chunjun Proposal
Hi, I have two questions here: 1. How should I name you in this thread? Apache is a community of peers. I can't image I'm talking to the whole Chunjun community or "Apache" expect its your name. 2. What's your expectations on going into the incubator? Among the whole proposal it's almost about what the current state of Chunjun and the only statement about your expectations is: > we seek to further prosper the community with the aid of Apache Could you elaborate a bit the motivation here? What help are you seeking? Also I second to Sheng's comment that it's confused about your expressions of contributors and initial committers. If your community continuously promote contributors , why the initial committer list is quite a bit limited? Best, tison. Sheng Wu 于2022年2月22日周二 20:54写道： > I think this description is incorrect. > > > Our initial committers will submit iCLA(s), SGA, and CCLA(s). > > Committers are individuals, who should only submit ICLA, their > employers are recommended to submit CCLA, the owner of the project > should sign the SGA. > ___ > > Also, I noticed a conflict in your description > On one side, you mentioned `The initial committers are employees of > DTStack.` with only 5 initial committers, and on the other hand, you > gave a very long vendor list and core contributors list. > So, which is an accurate description? If you have those contributors, > why were all of them invited as PPMC members? Do you have any public > discussion about this decision? > Such as GitHub ID(demotto) is the #8 in the contributor list, and also > listed in the core contributor list, but can't find it in the initial > committer list. > > > Sheng Wu 吴晟 > Twitter, wusheng1108 > > Lidong Dai 于2022年2月22日周二 20:41写道： > > > > I am curious that Chunjun(was Flinkx) is built on the Flink CDC, so what > is > > its innovation? > > > > BTW, you shouldn't use the mail(apa...@dtstack.com) as your user > account, > > apache is a registered trademark of ASF > > > > > > > > Best Regards > > > > > > > > --- > > Apache DolphinScheduler PMC Chair > > Lidong Dai > > lidong...@apache.org > > Linkedin: https://www.linkedin.com/in/dailidong > > Twitter: @WorkflowEasy <https://twitter.com/WorkflowEasy> > > > > --- > > > > > > On Tue, Feb 22, 2022 at 3:10 PM Jean-Baptiste Onofré > > wrote: > > > > > By the way, if you need a mentor, you can ping me. > > > > > > Regards > > > JB > > > > > > On Tue, Feb 22, 2022 at 8:08 AM Jean-Baptiste Onofré > > > wrote: > > > > > > > > Hi, > > > > > > > > It looks interesting. Quick questions about the current state of the > > > > project/roadmap: how the users implement the ETL pipeline ? Is it > > > > using a kind of SDK/DSL or do you plan an UI ? > > > > Is it very coupled to Flink or the runtime is "pluggable" ? Do you > > > > plan to have some kind of optional storage, similar to Apache Druid, > > > > ... ? > > > > > > > > Regards > > > > JB > > > > > > > > On Tue, Feb 22, 2022 at 7:41 AM Apache wrote: > > > > > > > > > > > > > > > Dear Apache Incubator Community, > > > > > We propose to contribute Chunjun as an Apache Incubator project. > > > > > We are still looking for possible Champion and Mentors if anyone > would > > > like to volunteer. Thanks a lot. > > > > > Best Regards, > > > > > Real-time computing engine team of DTStack. > > > > > > > > > > #Chunjun Proposal > > > > > > > > > > ##Abstract > > > > > Chunjun is a distributed ETL tool and distributed data integration > > > tool. Currently Chunjun is based on Apache Flink. It was initially > known as > > > FlinkX and renamed Chunjun on February 22, 2022. > > > > > - Chunjun codebase: https://github.com/DTStack/chunjun > > > > > > > > > > ##Proposal > > > > > We propose to contribute the Chunjun codebase to the Apache > Software > > > Foundation with the intent of forming a productive, meritocratic and > open > > > community around Chunjun’s continued development, according to the > 'Apache > > > Way'. The Chunjun's source code is already under the Apache License > Version > > > 2.0. > > > > > > > > > > ##Background > > > > > We developed Chunjun in DTSt
Re: Chunjun Proposal
I think this description is incorrect. > Our initial committers will submit iCLA(s), SGA, and CCLA(s). Committers are individuals, who should only submit ICLA, their employers are recommended to submit CCLA, the owner of the project should sign the SGA. ___ Also, I noticed a conflict in your description On one side, you mentioned `The initial committers are employees of DTStack.` with only 5 initial committers, and on the other hand, you gave a very long vendor list and core contributors list. So, which is an accurate description? If you have those contributors, why were all of them invited as PPMC members? Do you have any public discussion about this decision? Such as GitHub ID(demotto) is the #8 in the contributor list, and also listed in the core contributor list, but can't find it in the initial committer list. Sheng Wu 吴晟 Twitter, wusheng1108 Lidong Dai 于2022年2月22日周二 20:41写道： > > I am curious that Chunjun(was Flinkx) is built on the Flink CDC, so what is > its innovation? > > BTW, you shouldn't use the mail(apa...@dtstack.com) as your user account, > apache is a registered trademark of ASF > > > > Best Regards > > > > --- > Apache DolphinScheduler PMC Chair > Lidong Dai > lidong...@apache.org > Linkedin: https://www.linkedin.com/in/dailidong > Twitter: @WorkflowEasy <https://twitter.com/WorkflowEasy> > > --- > > > On Tue, Feb 22, 2022 at 3:10 PM Jean-Baptiste Onofré > wrote: > > > By the way, if you need a mentor, you can ping me. > > > > Regards > > JB > > > > On Tue, Feb 22, 2022 at 8:08 AM Jean-Baptiste Onofré > > wrote: > > > > > > Hi, > > > > > > It looks interesting. Quick questions about the current state of the > > > project/roadmap: how the users implement the ETL pipeline ? Is it > > > using a kind of SDK/DSL or do you plan an UI ? > > > Is it very coupled to Flink or the runtime is "pluggable" ? Do you > > > plan to have some kind of optional storage, similar to Apache Druid, > > > ... ? > > > > > > Regards > > > JB > > > > > > On Tue, Feb 22, 2022 at 7:41 AM Apache wrote: > > > > > > > > > > > > Dear Apache Incubator Community, > > > > We propose to contribute Chunjun as an Apache Incubator project. > > > > We are still looking for possible Champion and Mentors if anyone would > > like to volunteer. Thanks a lot. > > > > Best Regards, > > > > Real-time computing engine team of DTStack. > > > > > > > > #Chunjun Proposal > > > > > > > > ##Abstract > > > > Chunjun is a distributed ETL tool and distributed data integration > > tool. Currently Chunjun is based on Apache Flink. It was initially known as > > FlinkX and renamed Chunjun on February 22, 2022. > > > > - Chunjun codebase: https://github.com/DTStack/chunjun > > > > > > > > ##Proposal > > > > We propose to contribute the Chunjun codebase to the Apache Software > > Foundation with the intent of forming a productive, meritocratic and open > > community around Chunjun’s continued development, according to the 'Apache > > Way'. The Chunjun's source code is already under the Apache License Version > > 2.0. > > > > > > > > ##Background > > > > We developed Chunjun in DTStack company in 2017, when we needed a > > low-code development and high-performance data integration tool. It has > > been an open-source project on GitHub since April 2018. Chunjun is running > > in DTStack production environment all the time. Chunjun has also been > > widely used by companies in China, including DTStack ( > > https://www.dtstack.com/), Qihu360(https://www.360.cn/), Iflytek ( > > https://www.iflytek.com/), XPeng Motors (https://en.xiaopeng.com/), > > WeBank (https://www.webank.com/), Asiainfo(https://asiainfo.com/), Guazi( > > https://www.guazi.com/), Hello Inc (https://www.hello-inc.com/), etc. > > Nowadays, Chunjun has a strong community in China. > > > > > > > > ##Rationale > > > > High-performance of Chunjun is based on Apache Flink, and Chunjun can > > integrate data from different data source. Users only need to configure a > > JSON file to complete the data reading, transformer, and writing. Users can > > implement new reader/writer plugins to meet their requirements. Chunjun > > have implemented plugins that can capture data change for MySQL to restore > > data for Apache Doris. > > > > Chunjun has the following feature: > > > > real-time and offline
Re: Chunjun Proposal
I am curious that Chunjun(was Flinkx) is built on the Flink CDC, so what is its innovation? BTW, you shouldn't use the mail(apa...@dtstack.com) as your user account, apache is a registered trademark of ASF Best Regards --- Apache DolphinScheduler PMC Chair Lidong Dai lidong...@apache.org Linkedin: https://www.linkedin.com/in/dailidong Twitter: @WorkflowEasy <https://twitter.com/WorkflowEasy> --- On Tue, Feb 22, 2022 at 3:10 PM Jean-Baptiste Onofré wrote: > By the way, if you need a mentor, you can ping me. > > Regards > JB > > On Tue, Feb 22, 2022 at 8:08 AM Jean-Baptiste Onofré > wrote: > > > > Hi, > > > > It looks interesting. Quick questions about the current state of the > > project/roadmap: how the users implement the ETL pipeline ? Is it > > using a kind of SDK/DSL or do you plan an UI ? > > Is it very coupled to Flink or the runtime is "pluggable" ? Do you > > plan to have some kind of optional storage, similar to Apache Druid, > > ... ? > > > > Regards > > JB > > > > On Tue, Feb 22, 2022 at 7:41 AM Apache wrote: > > > > > > > > > Dear Apache Incubator Community, > > > We propose to contribute Chunjun as an Apache Incubator project. > > > We are still looking for possible Champion and Mentors if anyone would > like to volunteer. Thanks a lot. > > > Best Regards, > > > Real-time computing engine team of DTStack. > > > > > > #Chunjun Proposal > > > > > > ##Abstract > > > Chunjun is a distributed ETL tool and distributed data integration > tool. Currently Chunjun is based on Apache Flink. It was initially known as > FlinkX and renamed Chunjun on February 22, 2022. > > > - Chunjun codebase: https://github.com/DTStack/chunjun > > > > > > ##Proposal > > > We propose to contribute the Chunjun codebase to the Apache Software > Foundation with the intent of forming a productive, meritocratic and open > community around Chunjun’s continued development, according to the 'Apache > Way'. The Chunjun's source code is already under the Apache License Version > 2.0. > > > > > > ##Background > > > We developed Chunjun in DTStack company in 2017, when we needed a > low-code development and high-performance data integration tool. It has > been an open-source project on GitHub since April 2018. Chunjun is running > in DTStack production environment all the time. Chunjun has also been > widely used by companies in China, including DTStack ( > https://www.dtstack.com/), Qihu360(https://www.360.cn/), Iflytek ( > https://www.iflytek.com/), XPeng Motors (https://en.xiaopeng.com/), > WeBank (https://www.webank.com/), Asiainfo(https://asiainfo.com/), Guazi( > https://www.guazi.com/), Hello Inc (https://www.hello-inc.com/), etc. > Nowadays, Chunjun has a strong community in China. > > > > > > ##Rationale > > > High-performance of Chunjun is based on Apache Flink, and Chunjun can > integrate data from different data source. Users only need to configure a > JSON file to complete the data reading, transformer, and writing. Users can > implement new reader/writer plugins to meet their requirements. Chunjun > have implemented plugins that can capture data change for MySQL to restore > data for Apache Doris. > > > Chunjun has the following feature: > > > real-time and offline integrate data from different data sources. > > > change data capture(CDC) to merge restore data. > > > resume from broken-point. > > > capture and collect dirty data. > > > limit data transferring rate. > > > thoughput metrics. > > > capture and restore schema evolution. (TODO) > > > > > > ##Current Status###Meritocracy > > > Since Chunjun was open-sourced, many enterprises have adopted Chunjun > to build up their data integration system. In return, we have received many > issue reports or enhancements from them simultaneously. The codebase is now > mainly managed by the development team inside DTStack who's responsible for > building internal data integration system too.###Community > > > Chunjun has been building a community around contributors and users to > this framework for the last five years. We organized one meetup in 2020. > Currently, we communicate in Github issues and in chinese DingTalk group. > There are about 3000 people in this group. And we believe that we can get a > lot of help from the Apache Flink community too. We will organize a meetup > again in 2022.###Core Developers > > > (In alphabetical order) > > > Chao Xu (https://github.com/zoudaokoulife) > > > Gongjiang
Re: Chunjun Proposal
By the way, if you need a mentor, you can ping me. Regards JB On Tue, Feb 22, 2022 at 8:08 AM Jean-Baptiste Onofré wrote: > > Hi, > > It looks interesting. Quick questions about the current state of the > project/roadmap: how the users implement the ETL pipeline ? Is it > using a kind of SDK/DSL or do you plan an UI ? > Is it very coupled to Flink or the runtime is "pluggable" ? Do you > plan to have some kind of optional storage, similar to Apache Druid, > ... ? > > Regards > JB > > On Tue, Feb 22, 2022 at 7:41 AM Apache wrote: > > > > > > Dear Apache Incubator Community, > > We propose to contribute Chunjun as an Apache Incubator project. > > We are still looking for possible Champion and Mentors if anyone would like > > to volunteer. Thanks a lot. > > Best Regards, > > Real-time computing engine team of DTStack. > > > > #Chunjun Proposal > > > > ##Abstract > > Chunjun is a distributed ETL tool and distributed data integration tool. > > Currently Chunjun is based on Apache Flink. It was initially known as > > FlinkX and renamed Chunjun on February 22, 2022. > > - Chunjun codebase: https://github.com/DTStack/chunjun > > > > ##Proposal > > We propose to contribute the Chunjun codebase to the Apache Software > > Foundation with the intent of forming a productive, meritocratic and open > > community around Chunjun’s continued development, according to the 'Apache > > Way'. The Chunjun's source code is already under the Apache License Version > > 2.0. > > > > ##Background > > We developed Chunjun in DTStack company in 2017, when we needed a low-code > > development and high-performance data integration tool. It has been an > > open-source project on GitHub since April 2018. Chunjun is running in > > DTStack production environment all the time. Chunjun has also been widely > > used by companies in China, including DTStack (https://www.dtstack.com/), > > Qihu360(https://www.360.cn/), Iflytek (https://www.iflytek.com/), XPeng > > Motors (https://en.xiaopeng.com/), WeBank (https://www.webank.com/), > > Asiainfo(https://asiainfo.com/), Guazi(https://www.guazi.com/), Hello Inc > > (https://www.hello-inc.com/), etc. Nowadays, Chunjun has a strong > > community in China. > > > > ##Rationale > > High-performance of Chunjun is based on Apache Flink, and Chunjun can > > integrate data from different data source. Users only need to configure a > > JSON file to complete the data reading, transformer, and writing. Users can > > implement new reader/writer plugins to meet their requirements. Chunjun > > have implemented plugins that can capture data change for MySQL to restore > > data for Apache Doris. > > Chunjun has the following feature: > > real-time and offline integrate data from different data sources. > > change data capture(CDC) to merge restore data. > > resume from broken-point. > > capture and collect dirty data. > > limit data transferring rate. > > thoughput metrics. > > capture and restore schema evolution. (TODO) > > > > ##Current Status###Meritocracy > > Since Chunjun was open-sourced, many enterprises have adopted Chunjun to > > build up their data integration system. In return, we have received many > > issue reports or enhancements from them simultaneously. The codebase is now > > mainly managed by the development team inside DTStack who's responsible for > > building internal data integration system too.###Community > > Chunjun has been building a community around contributors and users to this > > framework for the last five years. We organized one meetup in 2020. > > Currently, we communicate in Github issues and in chinese DingTalk group. > > There are about 3000 people in this group. And we believe that we can get a > > lot of help from the Apache Flink community too. We will organize a meetup > > again in 2022.###Core Developers > > (In alphabetical order) > > Chao Xu (https://github.com/zoudaokoulife) > > Gongjiang Tang, (https://github.com/kyo-tom) > > Huai Yang, (https://github.com/yanghuaiGit) > > Jiangbo Li, (https://github.com/lijiangbo) > > Luning Wong, (https://github.com/deadwind4) > > Luo Li, (https://github.com/kanata163) > > Sishu Yang, (https://github.com/yangsishu) > > Tianzhu Wen, (https://github.com/WTZ468071157) > > Weiliang Hao, (https://github.com/xiuzhu9527) > > Wenqiang Liu, (https://github.com/meng1222) > > Xing Liu, (https://github.com/simenliuxing) > > Yang Lan, (https://github.com/HiLany) > > Yanquan Lv, (https://github.c
Re: Chunjun Proposal
Hi, It looks interesting. Quick questions about the current state of the project/roadmap: how the users implement the ETL pipeline ? Is it using a kind of SDK/DSL or do you plan an UI ? Is it very coupled to Flink or the runtime is "pluggable" ? Do you plan to have some kind of optional storage, similar to Apache Druid, ... ? Regards JB On Tue, Feb 22, 2022 at 7:41 AM Apache wrote: > > > Dear Apache Incubator Community, > We propose to contribute Chunjun as an Apache Incubator project. > We are still looking for possible Champion and Mentors if anyone would like > to volunteer. Thanks a lot. > Best Regards, > Real-time computing engine team of DTStack. > > #Chunjun Proposal > > ##Abstract > Chunjun is a distributed ETL tool and distributed data integration tool. > Currently Chunjun is based on Apache Flink. It was initially known as FlinkX > and renamed Chunjun on February 22, 2022. > - Chunjun codebase: https://github.com/DTStack/chunjun > > ##Proposal > We propose to contribute the Chunjun codebase to the Apache Software > Foundation with the intent of forming a productive, meritocratic and open > community around Chunjun’s continued development, according to the 'Apache > Way'. The Chunjun's source code is already under the Apache License Version > 2.0. > > ##Background > We developed Chunjun in DTStack company in 2017, when we needed a low-code > development and high-performance data integration tool. It has been an > open-source project on GitHub since April 2018. Chunjun is running in DTStack > production environment all the time. Chunjun has also been widely used by > companies in China, including DTStack (https://www.dtstack.com/), > Qihu360(https://www.360.cn/), Iflytek (https://www.iflytek.com/), XPeng > Motors (https://en.xiaopeng.com/), WeBank (https://www.webank.com/), > Asiainfo(https://asiainfo.com/), Guazi(https://www.guazi.com/), Hello Inc > (https://www.hello-inc.com/), etc. Nowadays, Chunjun has a strong community > in China. > > ##Rationale > High-performance of Chunjun is based on Apache Flink, and Chunjun can > integrate data from different data source. Users only need to configure a > JSON file to complete the data reading, transformer, and writing. Users can > implement new reader/writer plugins to meet their requirements. Chunjun have > implemented plugins that can capture data change for MySQL to restore data > for Apache Doris. > Chunjun has the following feature: > real-time and offline integrate data from different data sources. > change data capture(CDC) to merge restore data. > resume from broken-point. > capture and collect dirty data. > limit data transferring rate. > thoughput metrics. > capture and restore schema evolution. (TODO) > > ##Current Status###Meritocracy > Since Chunjun was open-sourced, many enterprises have adopted Chunjun to > build up their data integration system. In return, we have received many > issue reports or enhancements from them simultaneously. The codebase is now > mainly managed by the development team inside DTStack who's responsible for > building internal data integration system too.###Community > Chunjun has been building a community around contributors and users to this > framework for the last five years. We organized one meetup in 2020. > Currently, we communicate in Github issues and in chinese DingTalk group. > There are about 3000 people in this group. And we believe that we can get a > lot of help from the Apache Flink community too. We will organize a meetup > again in 2022.###Core Developers > (In alphabetical order) > Chao Xu (https://github.com/zoudaokoulife) > Gongjiang Tang, (https://github.com/kyo-tom) > Huai Yang, (https://github.com/yanghuaiGit) > Jiangbo Li, (https://github.com/lijiangbo) > Luning Wong, (https://github.com/deadwind4) > Luo Li, (https://github.com/kanata163) > Sishu Yang, (https://github.com/yangsishu) > Tianzhu Wen, (https://github.com/WTZ468071157) > Weiliang Hao, (https://github.com/xiuzhu9527) > Wenqiang Liu, (https://github.com/meng1222) > Xing Liu, (https://github.com/simenliuxing) > Yang Lan, (https://github.com/HiLany) > Yanquan Lv, (https://github.com/lvyanquan) > Yifan Hu, (https://github.com/demotto) > Zaiyue Yu, (https://github.com/tonybobam) > Zhangwan Zhao, (https://github.com/jiemotongxue) > Zhiqiang Li, (https://github.com/ChestnutQiang) > They are almost working in real-time computing engine team of DTStack. Only > Yifan Hu working for CaoCao Tech. Most of them are Apache Flink contributor. > > ##Known Risks###Project Name > The name of the project is Chunjun. Chunjun comes from mandarin chinese > Pinyin "Chun Jun", and it is one of the top ten famous swords in > C
Re: Chunjun Proposal
I am happy to sign up to be the champion On Tue, 22 Feb 2022, 12:11 Apache, wrote: > > Dear Apache Incubator Community, > We propose to contribute Chunjun as an Apache Incubator project. > We are still looking for possible Champion and Mentors if anyone would > like to volunteer. Thanks a lot. > Best Regards, > Real-time computing engine team of DTStack. > > #Chunjun Proposal > > ##Abstract > Chunjun is a distributed ETL tool and distributed data integration tool. > Currently Chunjun is based on Apache Flink. It was initially known as > FlinkX and renamed Chunjun on February 22, 2022. > - Chunjun codebase: https://github.com/DTStack/chunjun > > ##Proposal > We propose to contribute the Chunjun codebase to the Apache Software > Foundation with the intent of forming a productive, meritocratic and open > community around Chunjun’s continued development, according to the 'Apache > Way'. The Chunjun's source code is already under the Apache License Version > 2.0. > > ##Background > We developed Chunjun in DTStack company in 2017, when we needed a low-code > development and high-performance data integration tool. It has been an > open-source project on GitHub since April 2018. Chunjun is running in > DTStack production environment all the time. Chunjun has also been widely > used by companies in China, including DTStack (https://www.dtstack.com/), > Qihu360(https://www.360.cn/), Iflytek (https://www.iflytek.com/), XPeng > Motors (https://en.xiaopeng.com/), WeBank (https://www.webank.com/), > Asiainfo(https://asiainfo.com/), Guazi(https://www.guazi.com/), Hello Inc > (https://www.hello-inc.com/), etc. Nowadays, Chunjun has a strong > community in China. > > ##Rationale > High-performance of Chunjun is based on Apache Flink, and Chunjun can > integrate data from different data source. Users only need to configure a > JSON file to complete the data reading, transformer, and writing. Users can > implement new reader/writer plugins to meet their requirements. Chunjun > have implemented plugins that can capture data change for MySQL to restore > data for Apache Doris. > Chunjun has the following feature: > real-time and offline integrate data from different data sources. > change data capture(CDC) to merge restore data. > resume from broken-point. > capture and collect dirty data. > limit data transferring rate. > thoughput metrics. > capture and restore schema evolution. (TODO) > > ##Current Status###Meritocracy > Since Chunjun was open-sourced, many enterprises have adopted Chunjun to > build up their data integration system. In return, we have received many > issue reports or enhancements from them simultaneously. The codebase is now > mainly managed by the development team inside DTStack who's responsible for > building internal data integration system too.###Community > Chunjun has been building a community around contributors and users to > this framework for the last five years. We organized one meetup in 2020. > Currently, we communicate in Github issues and in chinese DingTalk group. > There are about 3000 people in this group. And we believe that we can get a > lot of help from the Apache Flink community too. We will organize a meetup > again in 2022.###Core Developers > (In alphabetical order) > Chao Xu (https://github.com/zoudaokoulife) > Gongjiang Tang, (https://github.com/kyo-tom) > Huai Yang, (https://github.com/yanghuaiGit) > Jiangbo Li, (https://github.com/lijiangbo) > Luning Wong, (https://github.com/deadwind4) > Luo Li, (https://github.com/kanata163) > Sishu Yang, (https://github.com/yangsishu) > Tianzhu Wen, (https://github.com/WTZ468071157) > Weiliang Hao, (https://github.com/xiuzhu9527) > Wenqiang Liu, (https://github.com/meng1222) > Xing Liu, (https://github.com/simenliuxing) > Yang Lan, (https://github.com/HiLany) > Yanquan Lv, (https://github.com/lvyanquan) > Yifan Hu, (https://github.com/demotto) > Zaiyue Yu, (https://github.com/tonybobam) > Zhangwan Zhao, (https://github.com/jiemotongxue) > Zhiqiang Li, (https://github.com/ChestnutQiang) > They are almost working in real-time computing engine team of DTStack. > Only Yifan Hu working for CaoCao Tech. Most of them are Apache Flink > contributor. > > ##Known Risks###Project Name > The name of the project is Chunjun. Chunjun comes from mandarin chinese > Pinyin "Chun Jun", and it is one of the top ten famous swords in > China.###Orphaned products > More than 20 contributors and thousands of forks and star further show > that Chunjun is actively supported, and we seek to further prosper the > community with the aid of Apache. As a consequence, Chunjun is unlikely to > be reduced to an orphaned project.###Inexperience with Open Source > Many of the C
Chunjun Proposal
Dear Apache Incubator Community, We propose to contribute Chunjun as an Apache Incubator project. We are still looking for possible Champion and Mentors if anyone would like to volunteer. Thanks a lot. Best Regards, Real-time computing engine team of DTStack. #Chunjun Proposal ##Abstract Chunjun is a distributed ETL tool and distributed data integration tool. Currently Chunjun is based on Apache Flink. It was initially known as FlinkX and renamed Chunjun on February 22, 2022. - Chunjun codebase: https://github.com/DTStack/chunjun ##Proposal We propose to contribute the Chunjun codebase to the Apache Software Foundation with the intent of forming a productive, meritocratic and open community around Chunjun’s continued development, according to the 'Apache Way'. The Chunjun's source code is already under the Apache License Version 2.0. ##Background We developed Chunjun in DTStack company in 2017, when we needed a low-code development and high-performance data integration tool. It has been an open-source project on GitHub since April 2018. Chunjun is running in DTStack production environment all the time. Chunjun has also been widely used by companies in China, including DTStack (https://www.dtstack.com/), Qihu360(https://www.360.cn/), Iflytek (https://www.iflytek.com/), XPeng Motors (https://en.xiaopeng.com/), WeBank (https://www.webank.com/), Asiainfo(https://asiainfo.com/), Guazi(https://www.guazi.com/), Hello Inc (https://www.hello-inc.com/), etc. Nowadays, Chunjun has a strong community in China. ##Rationale High-performance of Chunjun is based on Apache Flink, and Chunjun can integrate data from different data source. Users only need to configure a JSON file to complete the data reading, transformer, and writing. Users can implement new reader/writer plugins to meet their requirements. Chunjun have implemented plugins that can capture data change for MySQL to restore data for Apache Doris. Chunjun has the following feature: real-time and offline integrate data from different data sources. change data capture(CDC) to merge restore data. resume from broken-point. capture and collect dirty data. limit data transferring rate. thoughput metrics. capture and restore schema evolution. (TODO) ##Current Status###Meritocracy Since Chunjun was open-sourced, many enterprises have adopted Chunjun to build up their data integration system. In return, we have received many issue reports or enhancements from them simultaneously. The codebase is now mainly managed by the development team inside DTStack who's responsible for building internal data integration system too.###Community Chunjun has been building a community around contributors and users to this framework for the last five years. We organized one meetup in 2020. Currently, we communicate in Github issues and in chinese DingTalk group. There are about 3000 people in this group. And we believe that we can get a lot of help from the Apache Flink community too. We will organize a meetup again in 2022.###Core Developers (In alphabetical order) Chao Xu (https://github.com/zoudaokoulife) Gongjiang Tang, (https://github.com/kyo-tom) Huai Yang, (https://github.com/yanghuaiGit) Jiangbo Li, (https://github.com/lijiangbo) Luning Wong, (https://github.com/deadwind4) Luo Li, (https://github.com/kanata163) Sishu Yang, (https://github.com/yangsishu) Tianzhu Wen, (https://github.com/WTZ468071157) Weiliang Hao, (https://github.com/xiuzhu9527) Wenqiang Liu, (https://github.com/meng1222) Xing Liu, (https://github.com/simenliuxing) Yang Lan, (https://github.com/HiLany) Yanquan Lv, (https://github.com/lvyanquan) Yifan Hu, (https://github.com/demotto) Zaiyue Yu, (https://github.com/tonybobam) Zhangwan Zhao, (https://github.com/jiemotongxue) Zhiqiang Li, (https://github.com/ChestnutQiang) They are almost working in real-time computing engine team of DTStack. Only Yifan Hu working for CaoCao Tech. Most of them are Apache Flink contributor. ##Known Risks###Project Name The name of the project is Chunjun. Chunjun comes from mandarin chinese Pinyin "Chun Jun", and it is one of the top ten famous swords in China.###Orphaned products More than 20 contributors and thousands of forks and star further show that Chunjun is actively supported, and we seek to further prosper the community with the aid of Apache. As a consequence, Chunjun is unlikely to be reduced to an orphaned project.###Inexperience with Open Source Many of the Chunjun committers have experience working on open source projects. They are also active contributors to other Apache projects. ###Homogenous Developers The most of core developers are from DTStack, and Chunjun received some bug fixes and enhancements from other developers not working at DTStack. ###Reliance on Salaried Developers Currently, most of core developers are paid to work on Chunjun project by DTStack. We look forward to attracting more people outside DTStack to join this project.###Rel
