Re: Upgrading Hadoop dependencies and catching potential incompatibilities for HBase
Great suggestion! > On Sep 28, 2016, at 11:55 AM, Enis Söztutar wrote: > > Can Hadoop please shade ALL of the dependencies (including PB) in Hadoop-3 > so that we do not have this mess going forward. > > Enis > >> On Wed, Sep 28, 2016 at 11:19 AM, Jonathan Hsieh wrote: >> >> You can build hbase using the profile by adding this -D setting. >> >> mvn clean test install -Dhadoop.profile=3.0 >> >> At the moment the hbase's hadoop 3 profile has fails at mvn compile, >> install, and test phases. There are some issues that are patch available >> addressing these in HBase currently. >> >> compile https://issues.apache.org/jira/browse/HBASE-16711 >> test https://issues.apache.org/jira/browse/HBASE-6581 >> install https://issues.apache.org/jira/browse/HBASE-16712 >> >> IIRC, currently hbase's hadoop3 profile currently tracks the hadoop-3.0 >> SNAPSHOT artifacts. Sean, would these dependencies changes be targeted for >> this branch and land in the next hadoop 3 alpha/beta? >> Jon. >> >> On Wed, Sep 28, 2016 at 9:56 AM, Sean Mackrory >> wrote: >> >>> HBase folks, >>> >>> I'd like to work on upgrading some of the dependencies in Hadoop that are >>> starting to lag behind: >>> >>> jackson2 (https://issues.apache.org/jira/browse/HADOOP-12705) >>> jaxb-api (https://issues.apache.org/jira/browse/HADOOP-13659) >>> commons-configuration (https://issues.apache.org/ >> jira/browse/HADOOP-13660) >>> htrace (https://issues.apache.org/jira/browse/HADOOP-13661) >>> >>> Some of these (chiefly jackson) are known to be high risk upgrades and >> have >>> caused problems for HBase in the past. I wanted to give you a heads up >>> about this work and coordinate with you to make sure I do all the testing >>> you think is necessary to discover any potential problems well in advance >>> of the breakages. >>> >>> I was going to look at just running HBase's unit tests when built against >>> my Hadoop's trunk with my changes (Should I expect any problems here? Is >>> there a branch already focused on Hadoop 3 compatibility or anything?) >> and >>> Bigtop's integration tests too. Anything else you would add to this? >>> >> >> >> >> -- >> // Jonathan Hsieh (shay) >> // HBase Tech Lead, Software Engineer, Cloudera >> // j...@cloudera.com // @jmhsieh >> - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: Upgrading Hadoop dependencies and catching potential incompatibilities for HBase
> I'm working on this under HADOOP-11804, using HBase as my test application. Cool, glad to see progress on this. Enis On Wed, Sep 28, 2016 at 12:15 PM, Sean Busbey wrote: > On Wed, Sep 28, 2016 at 1:55 PM, Enis Söztutar wrote: > > Can Hadoop please shade ALL of the dependencies (including PB) in > Hadoop-3 > > so that we do not have this mess going forward. > > > > Enis > > > I'm working on this under HADOOP-11804, using HBase as my test application. > > -- > busbey >
Re: Upgrading Hadoop dependencies and catching potential incompatibilities for HBase
On Wed, Sep 28, 2016 at 1:55 PM, Enis Söztutar wrote: > Can Hadoop please shade ALL of the dependencies (including PB) in Hadoop-3 > so that we do not have this mess going forward. > > Enis I'm working on this under HADOOP-11804, using HBase as my test application. -- busbey - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: Upgrading Hadoop dependencies and catching potential incompatibilities for HBase
Can Hadoop please shade ALL of the dependencies (including PB) in Hadoop-3 so that we do not have this mess going forward. Enis On Wed, Sep 28, 2016 at 11:19 AM, Jonathan Hsieh wrote: > You can build hbase using the profile by adding this -D setting. > > mvn clean test install -Dhadoop.profile=3.0 > > At the moment the hbase's hadoop 3 profile has fails at mvn compile, > install, and test phases. There are some issues that are patch available > addressing these in HBase currently. > > compile https://issues.apache.org/jira/browse/HBASE-16711 > test https://issues.apache.org/jira/browse/HBASE-6581 > install https://issues.apache.org/jira/browse/HBASE-16712 > > IIRC, currently hbase's hadoop3 profile currently tracks the hadoop-3.0 > SNAPSHOT artifacts. Sean, would these dependencies changes be targeted for > this branch and land in the next hadoop 3 alpha/beta? > Jon. > > On Wed, Sep 28, 2016 at 9:56 AM, Sean Mackrory > wrote: > > > HBase folks, > > > > I'd like to work on upgrading some of the dependencies in Hadoop that are > > starting to lag behind: > > > > jackson2 (https://issues.apache.org/jira/browse/HADOOP-12705) > > jaxb-api (https://issues.apache.org/jira/browse/HADOOP-13659) > > commons-configuration (https://issues.apache.org/ > jira/browse/HADOOP-13660) > > htrace (https://issues.apache.org/jira/browse/HADOOP-13661) > > > > Some of these (chiefly jackson) are known to be high risk upgrades and > have > > caused problems for HBase in the past. I wanted to give you a heads up > > about this work and coordinate with you to make sure I do all the testing > > you think is necessary to discover any potential problems well in advance > > of the breakages. > > > > I was going to look at just running HBase's unit tests when built against > > my Hadoop's trunk with my changes (Should I expect any problems here? Is > > there a branch already focused on Hadoop 3 compatibility or anything?) > and > > Bigtop's integration tests too. Anything else you would add to this? > > > > > > -- > // Jonathan Hsieh (shay) > // HBase Tech Lead, Software Engineer, Cloudera > // j...@cloudera.com // @jmhsieh >
Re: Upgrading Hadoop dependencies and catching potential incompatibilities for HBase
> On Sep 28, 2016, at 9:56 AM, Sean Mackrory wrote: > and > Bigtop's integration tests too. Anything else you would add to this? Be aware that bigtop does things to the jar layout that will break parts of hadoop 3. - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Upgrading Hadoop dependencies and catching potential incompatibilities for HBase
HBase folks, I'd like to work on upgrading some of the dependencies in Hadoop that are starting to lag behind: jackson2 (https://issues.apache.org/jira/browse/HADOOP-12705) jaxb-api (https://issues.apache.org/jira/browse/HADOOP-13659) commons-configuration (https://issues.apache.org/jira/browse/HADOOP-13660) htrace (https://issues.apache.org/jira/browse/HADOOP-13661) Some of these (chiefly jackson) are known to be high risk upgrades and have caused problems for HBase in the past. I wanted to give you a heads up about this work and coordinate with you to make sure I do all the testing you think is necessary to discover any potential problems well in advance of the breakages. I was going to look at just running HBase's unit tests when built against my Hadoop's trunk with my changes (Should I expect any problems here? Is there a branch already focused on Hadoop 3 compatibility or anything?) and Bigtop's integration tests too. Anything else you would add to this?
