On Aug 17, 2009, at 7:36 AM, deneche abdelhakim wrote:
I moved recently some of the "Decision Forest" examples from the
core project to the examples project. While in core they worked
perfectly in hadoop 0..19.1 (pseudo-distributed), but now they
don't !!!
Try 0.20.0. Also, you may need to flatten the JOB jar into a single
jar.
For example, running my org.apache.mahout.df.BuildForest gives the
following exception:
....
09/08/17 12:02:36 INFO mapred.JobClient: Running job:
job_200908171136_0020
09/08/17 12:02:37 INFO mapred.JobClient: map 0% reduce 0%
09/08/17 12:02:43 INFO mapred.JobClient: Task Id :
attempt_200908171136_0020_m_000000_0, Status : FAILED
java.lang.NoClassDefFoundError: com/thoughtworks/xstream/XStream
at org.apache.mahout.utils.StringUtils.<clinit>(StringUtils.java:28)
at org.apache.mahout.df.mapred.Builder.getTreeBuilder(Builder.java:
117)
at
org.apache.mahout.df.mapred.MapredMapper.configure(MapredMapper.java:
74)
at
org
.apache
.mahout.df.mapred.partial.Step1Mapper.configure(Step1Mapper.java:75)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:
58)
at
org
.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:
83)
...
So I tried instead to run one of mahout's : following the wiki,
kmeans gives me the following error:
...
09/08/17 11:59:27 INFO kmeans.KMeansDriver: Iteration 4
...
09/08/17 11:59:43 INFO kmeans.KMeansDriver: Clustering
09/08/17 11:59:43 INFO kmeans.KMeansDriver: Running Clustering
09/08/17 11:59:43 INFO kmeans.KMeansDriver: Input: output/data
Clusters In: output/clusters-4 Out: output/points Distance:
org.apache.mahout.utils.EuclideanDistanceMeasure
09/08/17 11:59:43 INFO kmeans.KMeansDriver: convergence: 0.5 Input
Vectors: org.apache.mahout.matrix.SparseVector
09/08/17 11:59:43 WARN mapred.JobClient: Use GenericOptionsParser
for parsing the arguments. Applications should implement Tool for
the same.
09/08/17 11:59:43 INFO mapred..FileInputFormat: Total input paths to
process : 2
09/08/17 11:59:43 INFO mapred.JobClient: Running job:
job_200908171136_0019
09/08/17 11:59:44 INFO mapred.JobClient: map 0% reduce 0%
09/08/17 11:59:54 INFO mapred.JobClient: Task Id :
attempt_200908171136_0019_m_000000_0, Status : FAILED
java.lang.NoClassDefFoundError: com/google/gson/reflect/TypeToken
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:637)
at
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:
142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:277)
at java.net.URLClassLoader.access$000(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:212)
at java.security.AccessController.doPrivileged(Native Method)
...
The problem seems related to the fact that mahout-core.jar is being
packed inside examples.jar. So I modified maven/build.xml to pack
the core classes instead (because they are available):
Index: maven/build.xml
===================================================================
--- maven/build.xml (revision 804891)
+++ maven/build.xml (working copy)
@@ -45,9 +45,9 @@
includes="**/*.jar"/>
<zipfileset dir="${core-lib}" prefix="lib"
includes="**/*.jar" excludes="hadoop-*.jar"/>
- <zipfileset dir="../core/target/" prefix="lib"
includes="apache-mahout-core-${version}.jar"/>
+ <zipfileset dir="../core/target/classes"/>
<zipfileset dir="${dest}/dependency" prefix="lib"
- includes="**/*.jar"/>
+ includes="**/*.jar" excludes="apache-mahout-core-$
{version}.jar"/>
<zipfileset dir="../core/target/dependency" prefix="lib"
includes="**/*.jar"/>
</jar>
This seems to solve the problem, but I didn't try it on all examples
__________________________________________________
Do You Yahoo!?
En finir avec le spam? Yahoo! Mail vous offre la meilleure
protection possible contre les messages non sollicités
http://mail.yahoo.fr Yahoo! Mail
--------------------------
Grant Ingersoll
http://www.lucidimagination.com/
Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)
using Solr/Lucene:
http://www.lucidimagination.com/search
