My concern is that this is going to grow too large on us. HCatalog is very tightly tied to Hive. Hive almost always changes it's schema between versions. If we sign ourselves up for always being backward compatible one version I think we are going to seriously complicate our code.
If people feel they want this particular feature to work across versions I'm fine with that. I just don't want to sign up to always be backward compatible. Alan. On Jul 26, 2012, at 3:34 PM, Rohini Palaniswamy wrote: > My thought is we should at least be backward compatible with one previous > version unless it becomes really necessary to break backward compatibility. > Else it makes things very hard on the upgrade process. This issue should be a > easy one to fix with just a if else condition on hive version. I did not mean > shims in the sense we need two different classes which require compiling with > two different versions of hive like we do for hadoop. Just a util method > would work in this case. > > Regards, > Rohini > > On Thu, Jul 26, 2012 at 1:52 PM, Travis Crawford <[email protected]> > wrote: > On Thu, Jul 26, 2012 at 1:26 PM, Alan Gates <[email protected]> wrote: > > We don't want a shims layer. It makes more sense to tie versions of > > HCatalog to versions of Hive. They are much too tightly integrated to try > > to shim. HCat 0.5 should be tied to Hive 0.10. HCat 0.4.x will remain > > tied to Hive 0.9.x. > > > > Rohini had the idea of checking the Hive version at runtime and giving > a record in the correct format back (instead of a shims layer). Should > I look into that route, or stick with this patch? > > --travis > > > > > Alan. > > > > On Jul 26, 2012, at 10:55 AM, Travis Crawford wrote: > > > >> > >> > >>> On July 26, 2012, 5:23 p.m., Rohini Palaniswamy wrote: > >>>> hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/HCatBaseStorer.java, > >>>> line 250 > >>>> <https://reviews.apache.org/r/6156/diff/1/?file=129604#file129604line250> > >>>> > >>>> Ah. This is result of HIVE-3246. Is it possible to have a shims for > >>>> hive and return ByteArrayRef for hive version < 0.10 and byte[] for > >>>> >=0.10. HCat 0.5 might be released before hive 0.10 happens or even if > >>>> hive 0.10 is released users might still be using hive 0.9 till 0.10 is > >>>> stable. > >> > >> What do you think about fixing this so the build is unbroken, and opening > >> a separate jira for adding a Hive shims layer. That's going to be a bigger > >> change and I'll need to learn how the shims stuff works. > >> > >> Since our current dependency in trunk is hive 0.10.0-SNAPSHOT I think > >> updating to build against that SNAPSHOT version is a legitimate thing to > >> do. > >> > >> > >> - Travis > >> > >> > >> ----------------------------------------------------------- > >> This is an automatically generated e-mail. To reply, visit: > >> https://reviews.apache.org/r/6156/#review9503 > >> ----------------------------------------------------------- > >> > >> > >> On July 26, 2012, 3:26 p.m., Travis Crawford wrote: > >>> > >>> ----------------------------------------------------------- > >>> This is an automatically generated e-mail. To reply, visit: > >>> https://reviews.apache.org/r/6156/ > >>> ----------------------------------------------------------- > >>> > >>> (Updated July 26, 2012, 3:26 p.m.) > >>> > >>> > >>> Review request for hcatalog. > >>> > >>> > >>> Description > >>> ------- > >>> > >>> Fix HCatalog breakage due to Hive change from ByteArrayRef to byte[] > >>> > >>> > >>> This addresses bug HCATALOG-455. > >>> https://issues.apache.org/jira/browse/HCATALOG-455 > >>> > >>> > >>> Diffs > >>> ----- > >>> > >>> > >>> hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/HCatBaseStorer.java > >>> 00912ef > >>> > >>> hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/PigHCatUtil.java > >>> 696081f > >>> > >>> hcatalog-pig-adapter/src/test/java/org/apache/hcatalog/pig/TestHCatStorer.java > >>> 942e93d > >>> > >>> Diff: https://reviews.apache.org/r/6156/diff/ > >>> > >>> > >>> Testing > >>> ------- > >>> > >>> > >>> Thanks, > >>> > >>> Travis Crawford > >>> > >>> > >> > > >
