Yes, there is. About 75% of work in HCatalog is directly tied to work in Hive. This work for the binary type is a perfect example. If we don't tie HCat 0.5 to Hive 0.10 we won't be able to add many of the features we want. We can continue to make releases off the 0.4 branch for those who want bug fixes and features not tied to changes in Hive.
Alan. On Jul 28, 2012, at 9:00 AM, Joe Crobak wrote: > Is there a big reason for depending on hive 0.10-SNAPSHOT? This blocks the > hcatalog 0.5 release on a hive 0.10 release (or extra work will need to be > done to make hcatalog compatible with 0.9 before release). > > In addition, it seems like requiring the latest version of hive with each > release is going to limit adoption. Rather than just installing hcatalog > alongside an existing hive deploy, folks will first have to upgrade hive to > even try it out. This is what we're running into. > > Best, > Joe > > > On Fri, Jul 27, 2012 at 9:55 PM, Rohini Palaniswamy <[email protected]> > wrote: > With server there should be no problem Alan. We can always install hcat 0.5 > with hive 0.10 jars. But tying up hcat 0.5 client installation with hive > 0.10 installation might be little more difficult. May be we can cross the > bridge when we come to that. If we end up deploying hcat 0.5 with hive 0.9 > then we will post a patch and fix it then. > > Thanks, > Rohini > > On Fri, Jul 27, 2012 at 2:34 PM, Alan Gates <[email protected]> wrote: > > > My concern is that this is going to grow too large on us. HCatalog is > > very tightly tied to Hive. Hive almost always changes it's schema between > > versions. If we sign ourselves up for always being backward compatible one > > version I think we are going to seriously complicate our code. > > > > If people feel they want this particular feature to work across versions > > I'm fine with that. I just don't want to sign up to always be backward > > compatible. > > > > Alan. > > > > On Jul 26, 2012, at 3:34 PM, Rohini Palaniswamy wrote: > > > > > My thought is we should at least be backward compatible with one > > previous version unless it becomes really necessary to break backward > > compatibility. Else it makes things very hard on the upgrade process. This > > issue should be a easy one to fix with just a if else condition on hive > > version. I did not mean shims in the sense we need two different classes > > which require compiling with two different versions of hive like we do for > > hadoop. Just a util method would work in this case. > > > > > > Regards, > > > Rohini > > > > > > On Thu, Jul 26, 2012 at 1:52 PM, Travis Crawford < > > [email protected]> wrote: > > > On Thu, Jul 26, 2012 at 1:26 PM, Alan Gates <[email protected]> > > wrote: > > > > We don't want a shims layer. It makes more sense to tie versions of > > HCatalog to versions of Hive. They are much too tightly integrated to try > > to shim. HCat 0.5 should be tied to Hive 0.10. HCat 0.4.x will remain > > tied to Hive 0.9.x. > > > > > > > > > > Rohini had the idea of checking the Hive version at runtime and giving > > > a record in the correct format back (instead of a shims layer). Should > > > I look into that route, or stick with this patch? > > > > > > --travis > > > > > > > > > > > > > Alan. > > > > > > > > On Jul 26, 2012, at 10:55 AM, Travis Crawford wrote: > > > > > > > >> > > > >> > > > >>> On July 26, 2012, 5:23 p.m., Rohini Palaniswamy wrote: > > > >>>> > > hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/HCatBaseStorer.java, > > line 250 > > > >>>> < > > https://reviews.apache.org/r/6156/diff/1/?file=129604#file129604line250> > > > >>>> > > > >>>> Ah. This is result of HIVE-3246. Is it possible to have a shims > > for hive and return ByteArrayRef for hive version < 0.10 and byte[] for > > >=0.10. HCat 0.5 might be released before hive 0.10 happens or even if hive > > 0.10 is released users might still be using hive 0.9 till 0.10 is stable. > > > >> > > > >> What do you think about fixing this so the build is unbroken, and > > opening a separate jira for adding a Hive shims layer. That's going to be a > > bigger change and I'll need to learn how the shims stuff works. > > > >> > > > >> Since our current dependency in trunk is hive 0.10.0-SNAPSHOT I think > > updating to build against that SNAPSHOT version is a legitimate thing to do. > > > >> > > > >> > > > >> - Travis > > > >> > > > >> > > > >> ----------------------------------------------------------- > > > >> This is an automatically generated e-mail. To reply, visit: > > > >> https://reviews.apache.org/r/6156/#review9503 > > > >> ----------------------------------------------------------- > > > >> > > > >> > > > >> On July 26, 2012, 3:26 p.m., Travis Crawford wrote: > > > >>> > > > >>> ----------------------------------------------------------- > > > >>> This is an automatically generated e-mail. To reply, visit: > > > >>> https://reviews.apache.org/r/6156/ > > > >>> ----------------------------------------------------------- > > > >>> > > > >>> (Updated July 26, 2012, 3:26 p.m.) > > > >>> > > > >>> > > > >>> Review request for hcatalog. > > > >>> > > > >>> > > > >>> Description > > > >>> ------- > > > >>> > > > >>> Fix HCatalog breakage due to Hive change from ByteArrayRef to byte[] > > > >>> > > > >>> > > > >>> This addresses bug HCATALOG-455. > > > >>> https://issues.apache.org/jira/browse/HCATALOG-455 > > > >>> > > > >>> > > > >>> Diffs > > > >>> ----- > > > >>> > > > >>> > > > > hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/HCatBaseStorer.java > > 00912ef > > > >>> > > hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/PigHCatUtil.java > > 696081f > > > >>> > > > > hcatalog-pig-adapter/src/test/java/org/apache/hcatalog/pig/TestHCatStorer.java > > 942e93d > > > >>> > > > >>> Diff: https://reviews.apache.org/r/6156/diff/ > > > >>> > > > >>> > > > >>> Testing > > > >>> ------- > > > >>> > > > >>> > > > >>> Thanks, > > > >>> > > > >>> Travis Crawford > > > >>> > > > >>> > > > >> > > > > > > > > > > > >
