Re: [DISCUSS] Integrate Hudi with Apache Flink

2019-08-01 Thread vino yang
Hi Vinoth, +1, Sounds good to me. Working in parallel can increase efficiency. Best, Vino taher koitawala 于2019年8月2日周五 上午1:00写道: > Agreed. Scope should be 1 and 2. We will take on 1 while you take on 2 > > On Thu, Aug 1, 2019, 9:54 PM Vinay Patil wrote: > > > Hi Vinoth, > > > > Thank you for

Re: [DISCUSS] Integrate Hudi with Apache Flink

2019-08-01 Thread Vinay Patil
Hi Vinoth, Thank you for proposing this plan, let's keep the scope to 1&2 , as part of v1 let's start with point 1 and you guys can tackle point 2 in parallel. Excited to be a part of this development. Regards, Vinay Patil On Thu, 1 Aug 2019, 21:49 Vinoth Chandar, wrote: > Here are my

Re: [DISCUSS] Integrate Hudi with Apache Flink

2019-08-01 Thread Vinoth Chandar
Here are my thoughts.. Last time, when Flink was brought up, we dug into the use-case and realized that having Flink/Beam support for windowing on physical/arrival time (hoodie_commit_time) would be valuable and that's why Flink was being proposed. I would like to separate two aspects that I

Re: [DISCUSS] Integrate Hudi with Apache Flink

2019-07-31 Thread Vinoth Chandar
>>First of all, we should agree on the plan. +100 . this will be a very involved process.. if we can get a plan agreed upon, then we can start scoping the subtasks.. On Wed, Jul 31, 2019 at 2:11 AM Vinay Patil wrote: > Hi Guys, > > Add me in this as well, missed out on this last time. > >

Re: [DISCUSS] Integrate Hudi with Apache Flink

2019-07-31 Thread Vinay Patil
Hi Guys, Add me in this as well, missed out on this last time. Regards, Vinay Patil On Wed, Jul 31, 2019 at 2:34 PM vino yang wrote: > Hi Jing, > > You are right. Of course, We will split HUDI-184 into several subtasks. > > First of all, we should agree on the plan. > > Taher and I will

Re: [DISCUSS] Integrate Hudi with Apache Flink

2019-07-31 Thread vino yang
Hi Jing, You are right. Of course, We will split HUDI-184 into several subtasks. First of all, we should agree on the plan. Taher and I will propose a HIP then we will discuss more details based on the doc. Best, Vino Jing Chen 于2019年7月31日周三 下午3:54写道: > Shall we separate the story and make

Re: [DISCUSS] Integrate Hudi with Apache Flink

2019-07-31 Thread Jing Chen
Shall we separate the story and make subtasks under HUDI-184. I believe it could keep better track of the progress and get more people involved. Thanks Jing On Tue, Jul 30, 2019 at 9:30 PM vino yang wrote: > Hi all, > > Thanks for your feedback. > > Since there is no objection, I have created

Re: [DISCUSS] Integrate Hudi with Apache Flink

2019-07-30 Thread vino yang
Hi all, Thanks for your feedback. Since there is no objection, I have created an issue on JIRA for more discussion and let us move to HUDI-184.[1] Best, Vino [1]: https://issues.apache.org/jira/browse/HUDI-184 Vinoth Chandar 于2019年7月31日周三 上午2:12写道: > Awesome! > > On Tue, Jul 30, 2019 at

Re: [DISCUSS] Integrate Hudi with Apache Flink

2019-07-30 Thread Vinoth Chandar
Awesome! On Tue, Jul 30, 2019 at 11:05 AM taher koitawala wrote: > Sure would like to start one and see how it goes. > > Regards, > Taher Koitawala > > On Tue, Jul 30, 2019, 11:17 PM Vinoth Chandar wrote: > > > Glad to see this being revived again. Love to get this going. > > > > For

Re: [DISCUSS] Integrate Hudi with Apache Flink

2019-07-30 Thread taher koitawala
Sure would like to start one and see how it goes. Regards, Taher Koitawala On Tue, Jul 30, 2019, 11:17 PM Vinoth Chandar wrote: > Glad to see this being revived again. Love to get this going. > > For reference, the old thread on this topic > >

Re: [DISCUSS] Integrate Hudi with Apache Flink

2019-07-30 Thread Vinoth Chandar
Glad to see this being revived again. Love to get this going. For reference, the old thread on this topic https://lists.apache.org/thread.html/49ba202ce7947eecbf3a800def58afabcaf1f3b30481d8c4e0f88ec7@%3Cdev.hudi.apache.org%3E If we can have some volunteers to scope and plan the work, happy to

Re: [DISCUSS] Integrate Hudi with Apache Flink

2019-07-30 Thread taher koitawala
+1 interested in making the integration with Flink On Tue, Jul 30, 2019, 8:47 PM Aaron Langford wrote: > I am also interested in making Hudi work with Flink. I have no technical > opinions/insights at this time, but I am involved in several projects at > Instructure that could benefit from

Re: [DISCUSS] Integrate Hudi with Apache Flink

2019-07-30 Thread Aaron Langford
I am also interested in making Hudi work with Flink. I have no technical opinions/insights at this time, but I am involved in several projects at Instructure that could benefit from this. Aaron On Tue, Jul 30, 2019, 1:13 AM vino yang wrote: > Hi guys, > > I am a Flink contributor. I have seen

[DISCUSS] Integrate Hudi with Apache Flink

2019-07-30 Thread vino yang
Hi guys, I am a Flink contributor. I have seen the integration between Hudi and Spark. IMO, Making Hudi linkable with Flink is also a good idea. What do you think? Any feedback and opinion are welcome and appreciated. Best, Vino