Is there a big reason for depending on hive 0.10-SNAPSHOT? This blocks the hcatalog 0.5 release on a hive 0.10 release (or extra work will need to be done to make hcatalog compatible with 0.9 before release).
In addition, it seems like requiring the latest version of hive with each release is going to limit adoption. Rather than just installing hcatalog alongside an existing hive deploy, folks will first have to upgrade hive to even try it out. This is what we're running into. Best, Joe On Fri, Jul 27, 2012 at 9:55 PM, Rohini Palaniswamy <[email protected] > wrote: > With server there should be no problem Alan. We can always install hcat 0.5 > with hive 0.10 jars. But tying up hcat 0.5 client installation with hive > 0.10 installation might be little more difficult. May be we can cross the > bridge when we come to that. If we end up deploying hcat 0.5 with hive 0.9 > then we will post a patch and fix it then. > > Thanks, > Rohini > > On Fri, Jul 27, 2012 at 2:34 PM, Alan Gates <[email protected]> wrote: > > > My concern is that this is going to grow too large on us. HCatalog is > > very tightly tied to Hive. Hive almost always changes it's schema > between > > versions. If we sign ourselves up for always being backward compatible > one > > version I think we are going to seriously complicate our code. > > > > If people feel they want this particular feature to work across versions > > I'm fine with that. I just don't want to sign up to always be backward > > compatible. > > > > Alan. > > > > On Jul 26, 2012, at 3:34 PM, Rohini Palaniswamy wrote: > > > > > My thought is we should at least be backward compatible with one > > previous version unless it becomes really necessary to break backward > > compatibility. Else it makes things very hard on the upgrade process. > This > > issue should be a easy one to fix with just a if else condition on hive > > version. I did not mean shims in the sense we need two different classes > > which require compiling with two different versions of hive like we do > for > > hadoop. Just a util method would work in this case. > > > > > > Regards, > > > Rohini > > > > > > On Thu, Jul 26, 2012 at 1:52 PM, Travis Crawford < > > [email protected]> wrote: > > > On Thu, Jul 26, 2012 at 1:26 PM, Alan Gates <[email protected]> > > wrote: > > > > We don't want a shims layer. It makes more sense to tie versions of > > HCatalog to versions of Hive. They are much too tightly integrated to > try > > to shim. HCat 0.5 should be tied to Hive 0.10. HCat 0.4.x will remain > > tied to Hive 0.9.x. > > > > > > > > > > Rohini had the idea of checking the Hive version at runtime and giving > > > a record in the correct format back (instead of a shims layer). Should > > > I look into that route, or stick with this patch? > > > > > > --travis > > > > > > > > > > > > > Alan. > > > > > > > > On Jul 26, 2012, at 10:55 AM, Travis Crawford wrote: > > > > > > > >> > > > >> > > > >>> On July 26, 2012, 5:23 p.m., Rohini Palaniswamy wrote: > > > >>>> > > > hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/HCatBaseStorer.java, > > line 250 > > > >>>> < > > https://reviews.apache.org/r/6156/diff/1/?file=129604#file129604line250> > > > >>>> > > > >>>> Ah. This is result of HIVE-3246. Is it possible to have a shims > > for hive and return ByteArrayRef for hive version < 0.10 and byte[] for > > >=0.10. HCat 0.5 might be released before hive 0.10 happens or even if > hive > > 0.10 is released users might still be using hive 0.9 till 0.10 is stable. > > > >> > > > >> What do you think about fixing this so the build is unbroken, and > > opening a separate jira for adding a Hive shims layer. That's going to > be a > > bigger change and I'll need to learn how the shims stuff works. > > > >> > > > >> Since our current dependency in trunk is hive 0.10.0-SNAPSHOT I > think > > updating to build against that SNAPSHOT version is a legitimate thing to > do. > > > >> > > > >> > > > >> - Travis > > > >> > > > >> > > > >> ----------------------------------------------------------- > > > >> This is an automatically generated e-mail. To reply, visit: > > > >> https://reviews.apache.org/r/6156/#review9503 > > > >> ----------------------------------------------------------- > > > >> > > > >> > > > >> On July 26, 2012, 3:26 p.m., Travis Crawford wrote: > > > >>> > > > >>> ----------------------------------------------------------- > > > >>> This is an automatically generated e-mail. To reply, visit: > > > >>> https://reviews.apache.org/r/6156/ > > > >>> ----------------------------------------------------------- > > > >>> > > > >>> (Updated July 26, 2012, 3:26 p.m.) > > > >>> > > > >>> > > > >>> Review request for hcatalog. > > > >>> > > > >>> > > > >>> Description > > > >>> ------- > > > >>> > > > >>> Fix HCatalog breakage due to Hive change from ByteArrayRef to > byte[] > > > >>> > > > >>> > > > >>> This addresses bug HCATALOG-455. > > > >>> https://issues.apache.org/jira/browse/HCATALOG-455 > > > >>> > > > >>> > > > >>> Diffs > > > >>> ----- > > > >>> > > > >>> > > > > hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/HCatBaseStorer.java > > 00912ef > > > >>> > > > hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/PigHCatUtil.java > > 696081f > > > >>> > > > > hcatalog-pig-adapter/src/test/java/org/apache/hcatalog/pig/TestHCatStorer.java > > 942e93d > > > >>> > > > >>> Diff: https://reviews.apache.org/r/6156/diff/ > > > >>> > > > >>> > > > >>> Testing > > > >>> ------- > > > >>> > > > >>> > > > >>> Thanks, > > > >>> > > > >>> Travis Crawford > > > >>> > > > >>> > > > >> > > > > > > > > > > > >
