I always like the bin/mahout script to print the CLASSPATH since it helps
tremendously in debugging.
Here is one way to print CLASSPATH. Edit the bin/mahout script to echo
CLASSPATH soon after the #run it comment around line number 193.
if [ "$MAHOUT_JOB" = "" ]; then
for f in $MAHOUT_HOME/mahout-examples-*-job.jar; do
if [ -e "$f" ]; then
MAHOUT_JOB=$f
fi
done
fi
# run it
#add this
echo $CLASSPATH
if [ "$HADOOP_HOME" = "" ] || [ "$MAHOUT_LOCAL" != "" ] ; then
Next, invoke from $MAHOUT_HOME:
bash>bin/mahout
This should print the classpath on the screen. Here is what it gives me on
my system:
dhruv@tachyon:~/mahout$ bin/mahout
:/home/dhruv/mahout/src/conf::/usr/lib/jvm/java-6-sun/lib/tools.jar:/home/dhruv/mahout/mahout-*.jar:/home/dhruv/mahout/core/target/mahout-core-0.6-SNAPSHOT-job.jar:/home/dhruv/mahout/examples/target/mahout-examples-0.6-SNAPSHOT-job.jar:/home/dhruv/mahout/lib/*.jar:/home/dhruv/mahout/examples/target/dependency/cglib-nodep-2.2.jar:/home/dhruv/mahout/examples/target/dependency/commons-beanutils-1.7.0.jar:/home/dhruv/mahout/examples/target/dependency/commons-cli-1.2.jar:/home/dhruv/mahout/examples/target/dependency/commons-cli-2.0-mahout.jar:/home/dhruv/mahout/examples/target/dependency/commons-codec-1.3.jar:/home/dhruv/mahout/examples/target/dependency/commons-collections-3.2.1.jar:/home/dhruv/mahout/examples/target/dependency/commons-compress-1.1.jar:/home/dhruv/mahout/examples/target/dependency/commons-dbcp-1.2.2.jar:/home/dhruv/mahout/examples/target/dependency/commons-digester-1.7.jar:/home/dhruv/mahout/examples/target/dependency/commons-httpclient-3.0.1.jar:/home/dhruv/mahout/examples/target/dependency/commons-lang-2.4.jar:/home/dhruv/mahout/examples/target/dependency/commons-logging-1.1.1.jar:/home/dhruv/mahout/examples/target/dependency/commons-math-2.1.jar:/home/dhruv/mahout/examples/target/dependency/commons-pool-1.4.jar:/home/dhruv/mahout/examples/target/dependency/easymock-2.5.2.jar:/home/dhruv/mahout/examples/target/dependency/easymockclassextension-2.5.2.jar:/home/dhruv/mahout/examples/target/dependency/google-collections-1.0-rc2.jar:/home/dhruv/mahout/examples/target/dependency/guava-r09.jar:/home/dhruv/mahout/examples/target/dependency/hadoop-core-0.20.2.jar:/home/dhruv/mahout/examples/target/dependency/jakarta-regexp-1.4.jar:/home/dhruv/mahout/examples/target/dependency/junit-4.7.jar:/home/dhruv/mahout/examples/target/dependency/lucene-analyzers-3.1.0.jar:/home/dhruv/mahout/examples/target/dependency/lucene-benchmark-3.1.0.jar:/home/dhruv/mahout/examples/target/dependency/lucene-core-3.1.0.jar:/home/dhruv/mahout/examples/target/dependency/lucene-highlighter-3.1.0.jar:/home/dhruv/mahout/examples/target/dependency/lucene-memory-3.1.0.jar:/home/dhruv/mahout/examples/target/dependency/lucene-queries-3.1.0.jar:/home/dhruv/mahout/examples/target/dependency/lucene-xercesImpl-3.1.0.jar:/home/dhruv/mahout/examples/target/dependency/mahout-collections-1.0.jar:/home/dhruv/mahout/examples/target/dependency/mahout-core-0.6-SNAPSHOT.jar:/home/dhruv/mahout/examples/target/dependency/mahout-core-0.6-SNAPSHOT-tests.jar:/home/dhruv/mahout/examples/target/dependency/mahout-math-0.6-SNAPSHOT.jar:/home/dhruv/mahout/examples/target/dependency/mahout-math-0.6-SNAPSHOT-tests.jar:/home/dhruv/mahout/examples/target/dependency/mahout-utils-0.6-SNAPSHOT.jar:/home/dhruv/mahout/examples/target/dependency/objenesis-1.2.jar:/home/dhruv/mahout/examples/target/dependency/slf4j-api-1.6.0.jar:/home/dhruv/mahout/examples/target/dependency/slf4j-jcl-1.6.0.jar:/home/dhruv/mahout/examples/target/dependency/solr-commons-csv-3.1.0.jar:/home/dhruv/mahout/examples/target/dependency/uncommons-maths-1.2.jar:/home/dhruv/mahout/examples/target/dependency/watchmaker-framework-0.6.2.jar:/home/dhruv/mahout/examples/target/dependency/watchmaker-swing-0.6.2.jar:/home/dhruv/mahout/examples/target/dependency/xml-apis-1.0.b2.jar:/home/dhruv/mahout/examples/target/dependency/xpp3_min-1.1.4c.jar:/home/dhruv/mahout/examples/target/dependency/xstream-1.3.1.jar
On Tue, Jun 7, 2011 at 11:42 AM, Sean Owen <[email protected]> wrote:
> I am not terribly familiar with this script personally but one useful step
> would be to print out the CLASSPATH it is running with to spot what might
> be
> the problem. Heck it would probably be useful if the script always printed
> what it was doing on the command line.
>
> Maybe others have a clearer guess at the issue.
>
> On Tue, Jun 7, 2011 at 3:44 PM, Alex Ott <[email protected]> wrote:
>
> > Hello all
> >
> > I'm not sure is this actual error or just misconfiguration, so I'm
> > posting it here....
> >
> > I'm running wikipedia clusterization examples and found following
> > problem. if I run following command:
> >
> > bin/mahout seqwiki -all -i articles.xml -o wikipedia-seqfiles -e
> >
> > then after first part of job I get following error:
> >
> > 11/06/07 16:37:39 WARN mapred.JobClient: Use GenericOptionsParser for
> > parsing the arguments. Applications should implement Tool for the
> > same.
> > 11/06/07 16:37:39 INFO input.FileInputFormat: Total input paths to
> process
> > : 1
> > 11/06/07 16:37:39 INFO mapred.JobClient: Running job:
> job_201106071631_0006
> > 11/06/07 16:37:40 INFO mapred.JobClient: map 0% reduce 0%
> > 11/06/07 16:38:29 INFO mapred.JobClient: Task Id :
> > attempt_201106071631_0006_m_000000_0, Status : FAILED
> > Error: java.lang.ClassNotFoundException: com.google.common.base.Charsets
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> > at
> >
> org.apache.mahout.classifier.bayes.XmlInputFormat$XmlRecordReader.<init>(XmlInputFormat.java:75)
> > at
> >
> org.apache.mahout.classifier.bayes.XmlInputFormat.createRecordReader(XmlInputFormat.java:51)
> > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:588)
> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> > at org.apache.hadoop.mapred.Child.main(Child.java:170)
> >
> > Although I can find com.google.common.base.Charsets class in the
> > mahout-examples-0.5-job.jar file included into distribution.
> > And this error is fixed if I explicitly add this jar file into
> > CLASSPATH with command:
> >
> > export CLASSPATH="`pwd`/mahout-examples-0.5-job.jar"
> >
> > Is it problem with Hadoop configuration? I'm using pseudo-distributed
> > setup, where only data paths are specified additionally
> >
> > --
> > With best wishes, Alex Ott
> > http://alexott.net/
> > Tiwtter: alexott_en (English), alexott (Russian)
> > Skype: alex.ott
> >
>
