Can you share how you built and how you are running, as in command
line options, etc.? Also, JDK version, Hadoop version, etc.
On Jul 16, 2009, at 6:21 AM, Paul Ingles wrote:
Hi,
Thank you for the suggestion. Unfortunately, when I tried that I
received the same error. I've also tried copying the gson jar
directly into $HADOOP_HOME/lib (when I was running a single node
pseudo-distributed) and get the same error still.
Weirdly enough, if I try and run the Dirichlet example on the
cluster I receive another ClassNotFoundException:
09/07/16 10:27:54 INFO mapred.JobClient: Task Id :
attempt_200907161026_0002_m_000001_0, Status : FAILED
java.lang.RuntimeException: Error in configuring object
at
org
.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:
93)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:
64)
at
org
.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:
117)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:
352)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun
.reflect
.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun
.reflect
.DelegatingMethodAccessorImpl
.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org
.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:
88)
... 5 more
Caused by: java.lang.RuntimeException: Error in configuring object
at
org
.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:
93)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:
64)
at
org
.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:
117)
at
org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 10 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun
.reflect
.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun
.reflect
.DelegatingMethodAccessorImpl
.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org
.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:
88)
... 13 more
Caused by: java.lang.RuntimeException:
java.lang.ClassNotFoundException:
org
.apache
.mahout
.clustering.syntheticcontrol.dirichlet.NormalScModelDistribution
at
org
.apache
.mahout
.clustering
.dirichlet.DirichletMapper.getDirichletState(DirichletMapper.java:95)
at
org
.apache
.mahout
.clustering.dirichlet.DirichletMapper.configure(DirichletMapper.java:
60)
... 18 more
Caused by: java.lang.ClassNotFoundException:
org
.apache
.mahout
.clustering.syntheticcontrol.dirichlet.NormalScModelDistribution
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:316)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:
288)
at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
at
org
.apache
.mahout
.clustering
.dirichlet.DirichletDriver.createState(DirichletDriver.java:121)
at
org
.apache
.mahout
.clustering
.dirichlet.DirichletMapper.getDirichletState(DirichletMapper.java:71)
... 19 more
Hoping this sparks some other suggestions :)
Thanks,
Paul
On Wed Jul 15 22:08:09 UTC 2009, Adil Aijaz <a...@yahoo-inc.com>
wrote:
try hadoop --config <hod-cluster-dir> jar -libjars <path to gson.jar>
<your job/jar file> <your class> <arguments>
Adil
Paul Ingles wrote:
Hi,
Apologies for the cross-posting (I also sent this to the Hadoop user
list) but I'm still getting errors if I try and run the KMeans
examples on a cluster, whether that be my single-node Mac Pro, or
our
cluster. I've attached the stack trace at the bottom of the email.
The gson jar is definitely included in the packaged .job, and is
also
in the temporary directory when the task tracker picks up the work.
The gson jar also includes TypeToken.class in the expected path.
Again, really appreciate people's help in getting this going!
----snip----
09/07/15 17:06:38 INFO mapred.JobClient: Task Id :
attempt_200907151617_0010_m_000000_0, Status : FAILED
java.lang.NoClassDefFoundError: com/google/gson/reflect/TypeToken
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:703)
at
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:
124)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
at java.net.URLClassLoader.access$000(URLClassLoader.java:56)
at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:319)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:330)
at java.lang.ClassLoader.loadClass(ClassLoader.java:254)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:402)
at
org
.apache
.mahout.matrix.AbstractVector.asFormatString(AbstractVector.java:
374)
at
org
.apache
.mahout
.clustering.kmeans.Cluster.outputPointWithClusterInfo(Cluster.java:
198)
at
org
.apache
.mahout
.clustering
.kmeans.KMeansClusterMapper.map(KMeansClusterMapper.java:39)
at
org
.apache
.mahout
.clustering
.kmeans.KMeansClusterMapper.map(KMeansClusterMapper.java:32)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:
356)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.ClassNotFoundException:
com.google.gson.reflect.TypeToken
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:319)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:330)
at java.lang.ClassLoader.loadClass(ClassLoader.java:254)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:402)
... 20 more
----snip----
Incidentally, as part of this work I've also implemented a Pearson
distance measure, if people think it would be useful to be folded in
I'd be happy to get the SVN patch with tests and implementation
together.
Thanks,
Paul
--------------------------
Grant Ingersoll
http://www.lucidimagination.com/
Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)
using Solr/Lucene:
http://www.lucidimagination.com/search
