Hi
I've recently started working with Mahout. At first, I tried the
trunk, which I got to compile (both from within Eclipse with a Maven
plugin, and command line), but which apparently is in a state of flux
regarding building and running the examples (?).
I tried running the Twentynewsgroups classification example, after
copying the relevant Maven file to the examples directory, as
suggested on the mailing list some time ago. I could get the example's
data set from wikipedia, could get it processed into input data
located on the single-node/local hdfs, and could get a model trained
and output to that hdfs. However, the example class TestClassifierto
test with the trained model didn't work for me, neither in mapreduce
nor in sequential mode. In the mapreduce case, and even with quite
high JVM maximum heap sizes (I tried 2048), I get heapspace out of
memory errors / object configuration errors. In the sequential case, I
seemingly get 0 items classified, see output below. (Note that I
reduced the data set to just 8 instead of 20 newsgroups, thinking the
data size might have something to do with the problem.)
I also tried release 0.2, which I got to compile and for which I got
the example running more easily, but still with the same errors when
testing with the trained model. Any ideas what might be going wrong,
or what I might be doing wrong?
Kind regards,
Loek Cleophas
Output of TestClassifier:
bin/hadoop jar ~/Downloads/mahout-0.2/examples/target/mahout-
examples-0.2.job org.apache.mahout.classifier.bayes.TestClassifier -m
8newsmodel-0.2 -d 8newsInput -ng 3 -type bayes -source hdfs -method
sequential
<... reading all the feature weights ...>
10/01/13 10:22:08 INFO io.SequenceFileModelReader: Read 1950000
feature weights
10/01/13 10:22:11 INFO io.SequenceFileModelReader: hdfs://localhost:
9000/user/loekcleophas/8newsmodel-0.2/trainer-weights/Sigma_k/part-00000
10/01/13 10:22:11 INFO io.SequenceFileModelReader: hdfs://localhost:
9000/user/loekcleophas/8newsmodel-0.2/trainer-weights/Sigma_kSigma_j/
part-00000
10/01/13 10:22:11 INFO io.SequenceFileModelReader: 420716.6056712613
10/01/13 10:22:11 INFO io.SequenceFileModelReader: hdfs://localhost:
9000/user/loekcleophas/8newsmodel-0.2/trainer-thetaNormalizer/part-00000
10/01/13 10:22:11 INFO io.SequenceFileModelReader: hdfs://localhost:
9000/user/loekcleophas/8newsmodel-0.2/trainer-tfIdf/trainer-tfIdf/
part-00000
comp.windows.x -4443829.798557077 7727496.583973498 -0.5750671967650419
comp.graphics -3252365.124498224 7727496.583973498 -0.4208821174044246
soc.religion.christian -5106741.34456479 7727496.583973498
-0.6608532645819548
alt.atheism -3447983.6168798 7727496.583973498 -0.44619671835646907
misc.forsale -2276588.3662840202 7727496.583973498 -0.2946087832643716
comp.sys.mac.hardware -2445489.855812473 7727496.583973498
-0.31646598988918556
comp.os.ms-windows.misc -7727496.583973498 7727496.583973498 -1.0
comp.sys.ibm.pc.hardware -2687646.590023761 7727496.583973498
-0.3478030123750332
10/01/13 10:23:17 INFO bayes.TestClassifier:
nCalls = 0;
sumTime = 0.0s;
minTime = 0.0ms;
maxTime = 0.0ms;
meanTime = 0.0ms;
stdDevTime = 0.0ms;
10/01/13 10:23:18 INFO bayes.TestClassifier:
=======================================================
Summary
-------------------------------------------------------
Correctly Classified Instances : 0 ?%
Incorrectly Classified Instances : 0 ?%
Total Classified Instances : 0
=======================================================
Confusion Matrix
-------------------------------------------------------
a b c d e f g h <--Classified as
0 0 0 0 0 0 0 0 | 0 a =
comp.windows.x
0 0 0 0 0 0 0 0 | 0 b =
comp.graphics
0 0 0 0 0 0 0 0 | 0 c =
soc.religion.christian
0 0 0 0 0 0 0 0 | 0 d =
alt.atheism
0 0 0 0 0 0 0 0 | 0 e =
misc.forsale
0 0 0 0 0 0 0 0 | 0 f =
comp.sys.mac.hardware
0 0 0 0 0 0 0 0 | 0 g =
comp.os.ms-windows.misc
0 0 0 0 0 0 0 0 | 0 h =
comp.sys.ibm.pc.hardware
Default Category: unknown: 8
