On Wed, Jan 23, 2013 at 5:14 AM, Whitacre,Micah <[email protected]>wrote:
> Thanks. I no longer get that failure when doing the build with > "-Dcrunch.platform=2". I'm still getting the failure on my CDH4 fork and > will keep looking. Your change gave me an idea of where to look. > > I still do not get a completely successful build for "hadoop-2": > > <testcase time="10.516" > classname="org.apache.crunch.io.hbase.WordCountHBaseIT" > name="testWordCount"> > <error > message="org.apache.hadoop.hdfs.MiniDFSCluster.<init>(ILorg/apache/hadoop/conf/Configuration;IZZZLorg/apache/hadoop/hdfs/server/common/HdfsConstants$StartupOption;[Ljava/lang/String;[Ljava/lang/String;[J)V" > type="java.lang.NoSuchMethodError">java.lang.NoSuchMethodError: > org.apache.hadoop.hdfs.MiniDFSCluster.<init>(ILorg/apache/hadoop/conf/Configuration;IZZZLorg/apache/hadoop/hdfs/server/common/HdfsConstants$StartupOption;[Ljava/lang/String;[Ljava/lang/String;[J)V > at > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:430) > at > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:598) > at > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:554) > at > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:523) > at > org.apache.crunch.io.hbase.WordCountHBaseIT.setUp(WordCountHBaseIT.java:160) > > This actually seems to indicate a dependency mismatch between hbase and > hadoop-minicluster. There is a note in the POM about needing to build > hbase 0.94.1 from src. Is that still necessary as a released version is > available in Maven Central[1]. > Yes, it is. Like Crunch, HBase builds its maven jars against Hadoop 1.x, so running the Hadoop 2.x unit tests requires a custom build of hbase from source. That's one of the things bigtop does when they put together one of their package releases. > > [1] - > http://search.maven.org/#artifactdetails%7Corg.apache.hbase%7Chbase%7C0.94.1%7Cjar > > On Jan 22, 2013, at 5:53 PM, Josh Wills wrote: > > I think I have a fix, tracking it at: > https://issues.apache.org/jira/browse/CRUNCH-148<https://urldefense.proofpoint.com/v1/url?u=https://issues.apache.org/jira/browse/CRUNCH-148&k=PmKqfXspAHNo6iYJ48Q45A%3D%3D%0A&r=MwP8zm6sgnnstbiUpAReMZvSqrZXwpejyuwyb6GLlpU%3D%0A&m=OpOZQK3F8FlJaWn6bDsTi1i0sQeqftvdvxB%2BA5kVfzU%3D%0A&s=90d2b9078d65a5f89f5ed618a2a340fc3db5d1dda006d0e8325309a614f3c3c5> > > > CONFIDENTIALITY NOTICE This message and any included attachments are from > Cerner Corporation and are intended only for the addressee. The information > contained in this message is confidential and may constitute inside or > non-public information under international, federal, or state securities > laws. Unauthorized forwarding, printing, copying, distribution, or use of > such information is strictly prohibited and may be unlawful. If you are not > the addressee, please promptly delete this message and notify the sender of > the delivery error by e-mail or you may call Cerner's corporate offices in > Kansas City, Missouri, U.S.A at (+1) (816)221-1024. > -- Director of Data Science Cloudera <http://www.cloudera.com> Twitter: @josh_wills <http://twitter.com/josh_wills>
