Re: Upgrade HUDI to Hive 2.x

2019-05-20 Thread Balaji Varadarajan
Thanks Kabeer. That would be very helpful.  Balaji.VOn Sunday, May 19, 2019, 3:26:22 PM PDT, Kabeer Ahmed wrote: Hi Balaji, I never had this issue with 0.4.5 version of the release. The only release I got this issue was with 0.4.6-SNAPSHOT about 3 to 5 weeks ago. I resorted to manua

Re: Upgrade HUDI to Hive 2.x

2019-05-19 Thread Kabeer Ahmed
Hi Balaji, I never had this issue with 0.4.5 version of the release. The only release I got this issue was with 0.4.6-SNAPSHOT about 3 to 5 weeks ago. I resorted to manually making the changes in my local to get through some testing done. I am a bit confused as to how CDH5.7 is working with 0.4

Re: Upgrade HUDI to Hive 2.x

2019-05-19 Thread vbal...@apache.org
+ 1 on deprecating Hive 1.1. On the other note mentioned by Kabeer, Hey Kabeer, The Hive integration with CDH 5.7.x still works fine. We internally use the hive-sync capability of latest version of Hudi to let deltastreamer sync to Hive 1.x tables.  We do not have CH 5.13 setup. Did you notice t

Re: Upgrade HUDI to Hive 2.x

2019-05-19 Thread Kabeer Ahmed
Hi, I think it is OK to deprecate the Hive 1.1 support. As of 0.4.6-SNAPSHOT that I was using the latest build about 3 weeks ago, I did face issues if I did want to work with Hive 1.1 that is bundled as a part of CDH 5.13 docker image. I did have to make manual tweaks listed at: https://github

Re: Upgrade HUDI to Hive 2.x

2019-05-17 Thread Vinoth Chandar
I am in favor of deprecating Hive 1.x unless someone has a strong objection. Most cloud offerings like EMR/Data Proc all support Hive 2.x and Hive 3.x is going to grow. This seems like a move in the right direction /thanks/vinoth On Fri, May 17, 2019 at 11:55 AM nishith agarwal wrote: > All, >

Upgrade HUDI to Hive 2.x

2019-05-17 Thread nishith agarwal
All, Is anyone using Hudi with Hive 1.x ? Currently, Hudi has a dependency on Hive 1.x and works against Hive 2.x by using specific profiles. There are non-backwards compatible changes in the HiveRecordReader for Hive 1.x vs Hive 2.x. I'm planning to upgrade to Hive 2.x which would essentially mea