Re: Mahout and Hadoop 2.2.0

2013-12-11 Thread Gokhan Capan
In the meantime, you might apply the patch in MAHOUT-1354, build mahout using mvn package -Phadoop2 -DskipTests=true, use that mahout version and see if that works Gokhan On Wed, Dec 11, 2013 at 10:09 PM, Gokhan Capan wrote: > I apologize, Suneel is right, Counter breaks the binary compatibili

Re: Mahout and Hadoop 2.2.0

2013-12-11 Thread Hi There
Here are the full contents of my pom file: http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";   xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";>   4.0.0   clustertest   clustertest   1.0   jar   cluster

Re: Mahout and Hadoop 2.2.0

2013-12-11 Thread Gokhan Capan
I apologize, Suneel is right, Counter breaks the binary compatibility. Well, I can say there is a work in progress for building mahout against hadoop2. Gokhan On Wed, Dec 11, 2013 at 10:03 PM, Hi There wrote: > Here are the full contents of my pom file: > > http://maven.apache.org/POM/4.0.0";

Re: Mahout and Hadoop 2.2.0

2013-12-11 Thread Suneel Marthi
Per this link, one notability incompatibility is Counter and CounterGroup. http://hadoop.apache.org/docs/r2.2.0/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduce_Compatibility_Hadoop1_Hadoop2.html On Wednesday, December 11, 2013 2:46 PM, Hi There wrote: I tried to run SparseVe

Re: Mahout and Hadoop 2.2.0

2013-12-11 Thread Gokhan Capan
Could you check the following? Are you sure that your hadoop cluster is hadoop 2.2.0? Are you sure other dependencies of your project do not have a transitive dependency to hadoop? Gokhan On Wed, Dec 11, 2013 at 9:46 PM, Hi There wrote: > I tried to run SparseVectorsFromSequenceFiles, specify

Re: Mahout and Hadoop 2.2.0

2013-12-11 Thread Hi There
I tried to run SparseVectorsFromSequenceFiles, specifying a directory with sequence files, and I got the following error: java.lang.Exception: java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.Counter, but class was expected Here is a relevant snippet of my pom

Re: Mahout and Hadoop 2.2.0

2013-12-11 Thread Gokhan Capan
Hi Zoltan, I am saying that hadoop2-stable and hadoop1 are binary compatible. I don't know what version of hadoop is used in cdh4-mr2 but I guess it was hadoop2 alpha, since bigtop was at hadoop 2.0.6 alpha last time I checked, which was last week. Just try it and let us know if you experience

Re: Mahout and Hadoop 2.2.0

2013-12-11 Thread Suneel Marthi
Sebastian, R we still using SplitInputJob, seems like its been replaced by a much newer SplitInput. Do u think this needs to be purged from the codebase for 0.9, its been marked as deprecated anyways? On Wednesday, December 11, 2013 2:08 PM, Suneel Marthi wrote: A quick search thru the

Re: Mahout and Hadoop 2.2.0

2013-12-11 Thread Suneel Marthi
A quick search thru the codebase has the following using old mapred :- DistributedRowMatrix SplitInputJob MatrixMultiplicationJob BtJob TransposeJob TimesSquaredJob ABtJob ABtDenseOutJob BtJob QJob QRFirstStep On Wednesday, December 11, 2013 2:01 PM, Sebastian Schelter wrote: I think t

Re: Mahout and Hadoop 2.2.0

2013-12-11 Thread Sebastian Schelter
I think there are still parts of the code (e.g. in DistributedRowMatrix) that use the old API. --sebastian On 11.12.2013 19:56, Suneel Marthi wrote: > Mahout is using the newer mapreduce API and not the older mapred API. > Was that what u were looking for? > > > > > > On Wednesday, December

Re: Mahout and Hadoop 2.2.0

2013-12-11 Thread Suneel Marthi
Mahout is using the newer mapreduce API and not the older mapred API. Was that what u were looking for? On Wednesday, December 11, 2013 1:53 PM, Zoltan Prekopcsak wrote: Hi Gokhan, Thank you for the clarification. Does it mean that Mahout is using the mapred API everywhere and there is

Re: Mahout and Hadoop 2.2.0

2013-12-11 Thread Zoltan Prekopcsak
Hi Gokhan, Thank you for the clarification. Does it mean that Mahout is using the mapred API everywhere and there is no mapreduce API left? As far as I know, the mapreduce API needs to be recompiled and I remember needing to recompile Mahout for CDH4 when it first came out. Thanks, Zoltan

Re: Mahout and Hadoop 2.2.0

2013-12-10 Thread Gokhan Capan
I meant that you shouldn't need to modify mahout's dependencies, just mvn package and it should work against hadoop 2.2.0 (Yeah, 2.2.0 is not alpha) Quoting from http://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduce_Compatibility_Hadoop1_Hadoop2.htm

Re: Mahout and Hadoop 2.2.0

2013-12-09 Thread Hi There
Hi Gokhan, My project currently fetches every dependency through Maven--is there any way I can grab the version you mentioned that way? In that vein, I am using the following version of hadoop:     org.apache.hadoop     hadoop-client     2.2.0   That's not alpha, right? Thanks!

Re: Mahout and Hadoop 2.2.0

2013-12-09 Thread Gokhan Capan
Mahout actually should work with hadoop-2 stable without recompiling, not with hadoop-2 alpha though. We're, by the way, currently in the process of adding support to build mahout with hadoop-2. Please see mahout-1354 for the relevant issue Sent from my iPhone > On Dec 9, 2013, at 19:54, Hi The

Re: Mahout and Hadoop 2.2.0

2013-12-09 Thread Hi There
Is Dec 2013 still the intended release date of the next mahout release that will be compatible with Hadoop 2.2.0? On Thursday, November 21, 2013 12:36 PM, Suneel Marthi wrote: Targeted for Dec 2013. On Thursday, November 21, 2013 3:26 PM, Hi There wrote: Thanks for the reply! Is th

Re: Mahout and Hadoop 2.2.0

2013-11-21 Thread Suneel Marthi
Targeted for Dec 2013. On Thursday, November 21, 2013 3:26 PM, Hi There wrote: Thanks for the reply! Is there a timeline for then the next release will be? Thanks, Victor On Tuesday, November 19, 2013 7:30 PM, Suneel Marthi wrote: Hi Victor, Future releases of Mahout will support

Re: Mahout and Hadoop 2.2.0

2013-11-21 Thread Hi There
Thanks for the reply! Is there a timeline for then the next release will be? Thanks, Victor On Tuesday, November 19, 2013 7:30 PM, Suneel Marthi wrote: Hi Victor, Future releases of Mahout will support Hadoop 2.x, the present codebase still only supports Hadoop 1.x. On Tuesday, Nov

Re: Mahout and Hadoop 2.2.0

2013-11-19 Thread Suneel Marthi
Hi Victor, Future releases of Mahout will support Hadoop 2.x, the present codebase still only supports Hadoop 1.x. On Tuesday, November 19, 2013 1:42 PM, Hi There wrote: Hello, I recently upgraded to hadoop's newest release, and it seems one of their interfaces has changed, and when

Mahout and Hadoop 2.2.0

2013-11-19 Thread Hi There
Hello, I recently upgraded to hadoop's newest release, and it seems one of their interfaces has changed, and when I try to create sparse vectors from sequence files, I get the following exception: java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.Counter, b