I apologize, Suneel is right, Counter breaks the binary compatibility. Well, I can say there is a work in progress for building mahout against hadoop2.
Gokhan On Wed, Dec 11, 2013 at 10:03 PM, Hi There <srudamas...@yahoo.com> wrote: > Here are the full contents of my pom file: > > <project xmlns="http://maven.apache.org/POM/4.0.0"; xmlns:xsi=" > http://www.w3.org/2001/XMLSchema-instance"; > xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 > http://maven.apache.org/xsd/maven-4.0.0.xsd";> > <modelVersion>4.0.0</modelVersion> > > <groupId>clustertest</groupId> > <artifactId>clustertest</artifactId> > <version>1.0</version> > <packaging>jar</packaging> > > <name>clustertest</name> > <url>http://maven.apache.org</url> > <build> > <plugins> > <plugin> > <groupId>org.apache.maven.plugins</groupId> > <artifactId>maven-compiler-plugin</artifactId> > <version>2.3.2</version> > <configuration> > <source>1.7</source> > <target>1.7</target> > </configuration> > </plugin> > </plugins> > </build> > <properties> > <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> > </properties> > > <dependencies> > <dependency> > <groupId>junit</groupId> > <artifactId>junit</artifactId> > <version>3.8.1</version> > <scope>test</scope> > </dependency> > <dependency> > <groupId>org.apache.mahout</groupId> > <artifactId>mahout-core</artifactId> > <version>0.8</version> > <type>jar</type> > <exclusions> > <exclusion> > <artifactId>hadoop-core</artifactId> > <groupId>org.apache.hadoop</groupId> > </exclusion> > </exclusions> > </dependency> > <dependency> > <groupId>com.google.code.gson</groupId> > <artifactId>gson</artifactId> > <version>2.2.4</version> > <type>jar</type> > </dependency> > <dependency> > <groupId>org.apache.mahout</groupId> > <artifactId>mahout-integration</artifactId> > <version>0.8</version> > <type>jar</type> > <exclusions> > <exclusion> > <artifactId>hadoop-core</artifactId> > <groupId>org.apache.hadoop</groupId> > </exclusion> > </exclusions> > </dependency> > <dependency> > <groupId>org.apache.hadoop</groupId> > <artifactId>hadoop-client</artifactId> > <version>2.2.0</version> > </dependency> > </dependencies> > </project> > > How/where do I check to see whether my hadoop cluster is 2.2.0? Sorry, I > am new at this. > > > > > On Wednesday, December 11, 2013 11:56 AM, Gokhan Capan <gkhn...@gmail.com> > wrote: > > Could you check the following? > > Are you sure that your hadoop cluster is hadoop 2.2.0? > Are you sure other dependencies of your project do not have a transitive > dependency to hadoop? > > Gokhan > > > > On Wed, Dec 11, 2013 at 9:46 PM, Hi There <srudamas...@yahoo.com> wrote: > > > I tried to run SparseVectorsFromSequenceFiles, specifying a directory > with > > sequence files, and I got the following error: > > > > java.lang.Exception: java.lang.IncompatibleClassChangeError: Found > > interface org.apache.hadoop.mapreduce.Counter, but class was expected > > > > Here is a relevant snippet of my pom file for maven: > > > > <dependency> > > <groupId>org.apache.mahout</groupId> > > <artifactId>mahout-core</artifactId> > > <version>0.8</version> > > <type>jar</type> > > <exclusions> > > <exclusion> > > <artifactId>hadoop-core</artifactId> > > <groupId>org.apache.hadoop</groupId> > > </exclusion> > > </exclusions> > > </dependency> > > <dependency> > > <groupId>org.apache.mahout</groupId> > > <artifactId>mahout-integration</artifactId> > > <version>0.8</version> > > <type>jar</type> > > <exclusions> > > <exclusion> > > <artifactId>hadoop-core</artifactId> > > <groupId>org.apache.hadoop</groupId> > > </exclusion> > > </exclusions> > > </dependency> > > <dependency> > > <groupId>org.apache.hadoop</groupId> > > <artifactId>hadoop-client</artifactId> > > <version>2.2.0</version> > > </dependency> > > > > > > How do I change this to use the correct version(s) of mahout/hadoop? > > > > Thanks! > > > > > > > > > > On Tuesday, December 10, 2013 1:03 PM, Gokhan Capan <gkhn...@gmail.com> > > wrote: > > > > I meant that you shouldn't need to modify mahout's dependencies, just mvn > > package and it should work against hadoop 2.2.0 (Yeah, 2.2.0 is not > alpha) > > > > Quoting from > > > http://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduce_Compatibility_Hadoop1_Hadoop2.html > > "First, we ensure binary compatibility to the applications that use old > > mapred APIs. This means that applications which were built against MRv1 > > mapred APIs can run directly on YARN without recompilation, merely by > > pointing them to an Apache Hadoop 2.x cluster via configuration." > > > > If you encounter with any problems, just let the list know. > > > > Best > > > > > > > > > On Dec 9, 2013, at 9:40 PM, Hi There <srudamas...@yahoo.com> wrote: > > > > > > Hi Gokhan, > > > > > > My project currently fetches every dependency through Maven--is there > > any way I can grab the version you mentioned that way? > > > > > > In that vein, I am using the following version of hadoop: > > > <dependency> > > > <groupId>org.apache.hadoop</groupId> > > > <artifactId>hadoop-client</artifactId> > > > <version>2.2.0</version> > > > </dependency> > > > > > > > > > That's not alpha, right? > > > > > > Thanks! > > > > > > > > > > > > > > > > > > On Monday, December 9, 2013 10:05 AM, Gokhan Capan <gkhn...@gmail.com> > > wrote: > > > > > > Mahout actually should work with hadoop-2 stable without recompiling, > > > not with hadoop-2 alpha though. > > > > > > We're, by the way, currently in the process of adding support to build > > > mahout with hadoop-2. > > > > > > Please see mahout-1354 for the relevant issue > > > > > > Sent from my iPhone > > > > > > > > >> On Dec 9, 2013, at 19:54, Hi There <srudamas...@yahoo.com> wrote: > > >> > > >> Is Dec 2013 still the intended release date of the next mahout release > > that will be compatible with Hadoop 2.2.0? > > >> > > >> > > >> > > >> > > >> On Thursday, November 21, 2013 12:36 PM, Suneel Marthi < > > suneel_mar...@yahoo.com> wrote: > > >> > > >> Targeted for Dec 2013. > > >> > > >> > > >> > > >> > > >> > > >> > > >> On Thursday, November 21, 2013 3:26 PM, Hi There < > srudamas...@yahoo.com> > > wrote: > > >> > > >> Thanks for the reply! Is there a timeline for then the next release > > will be? > > >> > > >> > > >> Thanks, > > >> Victor > > >> > > >> > > >> > > >> > > >> On Tuesday, November 19, 2013 7:30 PM, Suneel Marthi < > > suneel_mar...@yahoo.com> wrote: > > >> > > >> Hi Victor, > > >> > > >> Future releases of Mahout will support Hadoop 2.x, the present > codebase > > still only supports Hadoop 1.x. > > >> > > >> > > >> > > >> > > >> > > >> > > >> On Tuesday, November 19, 2013 1:42 PM, Hi There < > srudamas...@yahoo.com> > > wrote: > > >> > > >> > > >> > > >> Hello, > > >> > > >> I recently upgraded to hadoop's > > >> newest release, and it seems one of their interfaces has changed, and > > >> when I try to create sparse vectors from sequence files, I get the > > >> following exception: > > >> > > >> java.lang.IncompatibleClassChangeError: Found interface > > org.apache.hadoop.mapreduce.Counter, but class was expected > > >> > > >> I can include more of the stack trace if necessary. > > >> > > >> Are there any plans in the immediate future to upgrade mahout to be > > compatible with the newest hadoop release? > > >> > > >> Thanks, > > >> Victor > > >
