Bit of an update. I'd suggest we go ahead w/ the hbase-thirdparty project [2]. It took a while but in its current form -- a few poms that package a few jars [1]-- it at least enables the below:
+ Allows us to skip checking in protobuf generated files (25MB!); they can be generated inline w/ the build because the hackery patching protobuf has been moved out to hbase-thirdparty. There is a patch up on HBASE-17056. + Update our guava from 12.0 to 22.0 w/o clashing w/ the guava of others. There is a patch at HBASE-17908. It is taking a bit of wrangling getting it to land because I pared back transitive includes from hadoop and it takes a while to work through the failures. Other benefits are the protobuf-util lib is on the classpath now -- its in hbase-thirdparty relocated; depends on pb and guava -- so we have facility to goat "HBASE-18106 Redo ProcedureInfo and LockInfo" and shading netty is almost done so we can do with netty as we wilt independent of hadoop and downstreamers (the hard part -- relocation of the .so -- should be done). Let me figure how to run a vote for a couple of poms..... St.Ack 1. https://repository.apache.org/content/groups/snapshots/org/apache/hbase/thirdparty/ (see hbase-shaded-thirdparty and hbase-shaded-protobuf) 2. https://git-wip-us.apache.org/repos/asf/hbase-thirdparty On Tue, Jun 20, 2017 at 11:04 AM, Josh Elser <[email protected]> wrote: > On 6/20/17 1:28 AM, Stack wrote: > >> On Thu, Apr 13, 2017 at 4:46 PM, Josh Elser<[email protected]> wrote: >> >> ... >>> >>> I think pushing this part forward with some code is the next logical >>> step. >>> Seems to be consensus about taking our known internal dependencies and >>> performing this shade magic. >>> >>> >>> I opened HBASE-18240 "Add hbase-auxillary, a project with hbase utility >> including an hbase-shaded-thirdparty module with guava, netty, etc." >> >> It has a tarball attached that bundles the outline of an hbase-auxillary >> project (groupId:org.apache.hbase.auxillary). This project is intended to >> be standalone, in its own repository, publishing its own artifacts under >> the aegis of this project's PMC. >> >> It includes the first instance of an auxillary utility, a module named >> hbase-thirdparty-shaded (artifactId:hbase-thirdparty-shaded). Herein >> we'll >> pull down 3rd party libs and republish at an offset; e.g. >> com.google.common.* from guava will be at >> org.apache.hbase.thirdparty.shaded.com.google.common.*. Currently it >> builds >> a jar that includes a relocated guava 22.0. >> >> I then messed around making hbase-common use it (You have to build the >> hbase-auxillary into your local repo). I put up a patch on the issue. >> Mostly its mass find-and-replace w/ some clean up of transitive includes >> of >> guava from hadoop-common and some small fixup of methods renamed between >> guava 12.0 and 22.0. >> >> Unless objection, I was going to press on. Sean offered to help set up new >> repo. We can always undo and delete it if this project fails. >> >> When done, the hope is we are on a modern version of guava and our netty >> and protobuf 3 will be be relocated, 'hidden' from downstream (and won't >> clash w/ upstream). I hope to also purge the pre-build we have in our >> modules that do protobuf moving this hackery out and under >> hbase-thirdparty-shaded. >> >> St.Ack >> > > Kudos on the JFDI approach :). I think having something concrete to show > is the best way to judge success of it. > > Will keep an eye on HBASE-18240. > >
