See <https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters/252/changes>
Changes:
[ssc] MAHOUT-1061 mapreduce split causes ClassNotFound exception
[ssc] MAHOUT-1076 Matrix Multiplication output to user specified directory
[ssc] MAHOUT-1146 Cardinality exception bug in 'cross' method of AbstractVector
class
------------------------------------------
[...truncated 1176 lines...]
Mar 11, 2013 1:13:27 PM org.apache.hadoop.mapred.Counters log
INFO: Map input records=0
Mar 11, 2013 1:13:27 PM org.apache.hadoop.mapred.Counters log
INFO: Reduce shuffle bytes=0
Mar 11, 2013 1:13:27 PM org.apache.hadoop.mapred.Counters log
INFO: Spilled Records=0
Mar 11, 2013 1:13:27 PM org.apache.hadoop.mapred.Counters log
INFO: Map output bytes=0
Mar 11, 2013 1:13:27 PM org.apache.hadoop.mapred.Counters log
INFO: Total committed heap usage (bytes)=937951232
Mar 11, 2013 1:13:27 PM org.apache.hadoop.mapred.Counters log
INFO: SPLIT_RAW_BYTES=150
Mar 11, 2013 1:13:27 PM org.apache.hadoop.mapred.Counters log
INFO: Combine input records=0
Mar 11, 2013 1:13:27 PM org.apache.hadoop.mapred.Counters log
INFO: Reduce input records=0
Mar 11, 2013 1:13:27 PM org.apache.hadoop.mapred.Counters log
INFO: Reduce input groups=0
Mar 11, 2013 1:13:27 PM org.apache.hadoop.mapred.Counters log
INFO: Combine output records=0
Mar 11, 2013 1:13:27 PM org.apache.hadoop.mapred.Counters log
INFO: Reduce output records=0
Mar 11, 2013 1:13:27 PM org.apache.hadoop.mapred.Counters log
INFO: Map output records=0
Mar 11, 2013 1:13:29 PM org.apache.hadoop.mapreduce.lib.input.FileInputFormat
listStatus
INFO: Total input paths to process : 1
Mar 11, 2013 1:13:29 PM
org.apache.hadoop.filecache.TrackerDistributedCacheManager downloadCacheObject
INFO: Creating frequency.file-0 in
/tmp/hadoop-hudson/mapred/local/archive/-5512976599584702219_1334525619_1502974802/file/tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans-work-486901126900241769
with rwxr-xr-x
Mar 11, 2013 1:13:29 PM
org.apache.hadoop.filecache.TrackerDistributedCacheManager downloadCacheObject
INFO: Cached
/tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/frequency.file-0 as
/tmp/hadoop-hudson/mapred/local/archive/-5512976599584702219_1334525619_1502974802/file/tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/frequency.file-0
Mar 11, 2013 1:13:29 PM
org.apache.hadoop.filecache.TrackerDistributedCacheManager
localizePublicCacheObject
INFO: Cached
/tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/frequency.file-0 as
/tmp/hadoop-hudson/mapred/local/archive/-5512976599584702219_1334525619_1502974802/file/tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/frequency.file-0
Mar 11, 2013 1:13:29 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
INFO: Running job: job_local_0006
Mar 11, 2013 1:13:29 PM org.apache.hadoop.mapred.Task initialize
INFO: Using ResourceCalculatorPlugin : null
Mar 11, 2013 1:13:29 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
INFO: io.sort.mb = 100
Mar 11, 2013 1:13:30 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
INFO: map 0% reduce 0%
Mar 11, 2013 1:13:30 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
INFO: data buffer = 79691776/99614720
Mar 11, 2013 1:13:30 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
INFO: record buffer = 262144/327680
Mar 11, 2013 1:13:30 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer flush
INFO: Starting flush of map output
Mar 11, 2013 1:13:30 PM org.apache.hadoop.mapred.Task done
INFO: Task:attempt_local_0006_m_000000_0 is done. And is in the process of
commiting
Mar 11, 2013 1:13:30 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
INFO:
Mar 11, 2013 1:13:30 PM org.apache.hadoop.mapred.Task sendDone
INFO: Task 'attempt_local_0006_m_000000_0' done.
Mar 11, 2013 1:13:30 PM org.apache.hadoop.mapred.Task initialize
INFO: Using ResourceCalculatorPlugin : null
Mar 11, 2013 1:13:30 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
INFO:
Mar 11, 2013 1:13:30 PM org.apache.hadoop.mapred.Merger$MergeQueue merge
INFO: Merging 1 sorted segments
Mar 11, 2013 1:13:30 PM org.apache.hadoop.mapred.Merger$MergeQueue merge
INFO: Down to the last merge-pass, with 0 segments left of total size: 0 bytes
Mar 11, 2013 1:13:30 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
INFO:
Mar 11, 2013 1:13:30 PM org.apache.hadoop.mapred.Task done
INFO: Task:attempt_local_0006_r_000000_0 is done. And is in the process of
commiting
Mar 11, 2013 1:13:30 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
INFO:
Mar 11, 2013 1:13:30 PM org.apache.hadoop.mapred.Task commit
INFO: Task attempt_local_0006_r_000000_0 is allowed to commit now
Mar 11, 2013 1:13:30 PM
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter commitTask
INFO: Saved output of task 'attempt_local_0006_r_000000_0' to
/tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/partial-vectors-0
Mar 11, 2013 1:13:30 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
INFO: reduce > reduce
Mar 11, 2013 1:13:30 PM org.apache.hadoop.mapred.Task sendDone
INFO: Task 'attempt_local_0006_r_000000_0' done.
Mar 11, 2013 1:13:31 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
INFO: map 100% reduce 100%
Mar 11, 2013 1:13:31 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
INFO: Job complete: job_local_0006
Mar 11, 2013 1:13:31 PM org.apache.hadoop.mapred.Counters log
INFO: Counters: 17
Mar 11, 2013 1:13:31 PM org.apache.hadoop.mapred.Counters log
INFO: File Output Format Counters
Mar 11, 2013 1:13:31 PM org.apache.hadoop.mapred.Counters log
INFO: Bytes Written=102
Mar 11, 2013 1:13:31 PM org.apache.hadoop.mapred.Counters log
INFO: FileSystemCounters
Mar 11, 2013 1:13:31 PM org.apache.hadoop.mapred.Counters log
INFO: FILE_BYTES_READ=753416781
Mar 11, 2013 1:13:31 PM org.apache.hadoop.mapred.Counters log
INFO: FILE_BYTES_WRITTEN=759746118
Mar 11, 2013 1:13:31 PM org.apache.hadoop.mapred.Counters log
INFO: File Input Format Counters
Mar 11, 2013 1:13:31 PM org.apache.hadoop.mapred.Counters log
INFO: Bytes Read=102
Mar 11, 2013 1:13:31 PM org.apache.hadoop.mapred.Counters log
INFO: Map-Reduce Framework
Mar 11, 2013 1:13:31 PM org.apache.hadoop.mapred.Counters log
INFO: Map output materialized bytes=6
Mar 11, 2013 1:13:31 PM org.apache.hadoop.mapred.Counters log
INFO: Map input records=0
Mar 11, 2013 1:13:31 PM org.apache.hadoop.mapred.Counters log
INFO: Reduce shuffle bytes=0
Mar 11, 2013 1:13:31 PM org.apache.hadoop.mapred.Counters log
INFO: Spilled Records=0
Mar 11, 2013 1:13:31 PM org.apache.hadoop.mapred.Counters log
INFO: Map output bytes=0
Mar 11, 2013 1:13:31 PM org.apache.hadoop.mapred.Counters log
INFO: Total committed heap usage (bytes)=1138360320
Mar 11, 2013 1:13:31 PM org.apache.hadoop.mapred.Counters log
INFO: SPLIT_RAW_BYTES=150
Mar 11, 2013 1:13:31 PM org.apache.hadoop.mapred.Counters log
INFO: Combine input records=0
Mar 11, 2013 1:13:31 PM org.apache.hadoop.mapred.Counters log
INFO: Reduce input records=0
Mar 11, 2013 1:13:31 PM org.apache.hadoop.mapred.Counters log
INFO: Reduce input groups=0
Mar 11, 2013 1:13:31 PM org.apache.hadoop.mapred.Counters log
INFO: Combine output records=0
Mar 11, 2013 1:13:31 PM org.apache.hadoop.mapred.Counters log
INFO: Reduce output records=0
Mar 11, 2013 1:13:31 PM org.apache.hadoop.mapred.Counters log
INFO: Map output records=0
Mar 11, 2013 1:13:31 PM org.slf4j.impl.JCLLoggerAdapter info
INFO: Deleting
/tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/tfidf-vectors
Mar 11, 2013 1:13:31 PM org.apache.hadoop.mapreduce.lib.input.FileInputFormat
listStatus
INFO: Total input paths to process : 1
Mar 11, 2013 1:13:31 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
INFO: Running job: job_local_0007
Mar 11, 2013 1:13:31 PM org.apache.hadoop.mapred.Task initialize
INFO: Using ResourceCalculatorPlugin : null
Mar 11, 2013 1:13:31 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
INFO: io.sort.mb = 100
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
INFO: data buffer = 79691776/99614720
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
INFO: record buffer = 262144/327680
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer flush
INFO: Starting flush of map output
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.Task done
INFO: Task:attempt_local_0007_m_000000_0 is done. And is in the process of
commiting
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
INFO:
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.Task sendDone
INFO: Task 'attempt_local_0007_m_000000_0' done.
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.Task initialize
INFO: Using ResourceCalculatorPlugin : null
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
INFO:
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.Merger$MergeQueue merge
INFO: Merging 1 sorted segments
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.Merger$MergeQueue merge
INFO: Down to the last merge-pass, with 0 segments left of total size: 0 bytes
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
INFO:
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.Task done
INFO: Task:attempt_local_0007_r_000000_0 is done. And is in the process of
commiting
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
INFO:
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.Task commit
INFO: Task attempt_local_0007_r_000000_0 is allowed to commit now
Mar 11, 2013 1:13:32 PM
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter commitTask
INFO: Saved output of task 'attempt_local_0007_r_000000_0' to
/tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/tfidf-vectors
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
INFO: reduce > reduce
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.Task sendDone
INFO: Task 'attempt_local_0007_r_000000_0' done.
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
INFO: map 100% reduce 100%
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
INFO: Job complete: job_local_0007
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.Counters log
INFO: Counters: 17
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.Counters log
INFO: File Output Format Counters
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.Counters log
INFO: Bytes Written=102
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.Counters log
INFO: FileSystemCounters
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.Counters log
INFO: FILE_BYTES_READ=878986188
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.Counters log
INFO: FILE_BYTES_WRITTEN=886369754
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.Counters log
INFO: File Input Format Counters
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.Counters log
INFO: Bytes Read=102
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.Counters log
INFO: Map-Reduce Framework
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.Counters log
INFO: Map output materialized bytes=6
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.Counters log
INFO: Map input records=0
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.Counters log
INFO: Reduce shuffle bytes=0
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.Counters log
INFO: Spilled Records=0
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.Counters log
INFO: Map output bytes=0
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.Counters log
INFO: Total committed heap usage (bytes)=1337982976
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.Counters log
INFO: SPLIT_RAW_BYTES=157
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.Counters log
INFO: Combine input records=0
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.Counters log
INFO: Reduce input records=0
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.Counters log
INFO: Reduce input groups=0
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.Counters log
INFO: Combine output records=0
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.Counters log
INFO: Reduce output records=0
Mar 11, 2013 1:13:32 PM org.apache.hadoop.mapred.Counters log
INFO: Map output records=0
Mar 11, 2013 1:13:32 PM org.slf4j.impl.JCLLoggerAdapter info
INFO: Deleting
/tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/partial-vectors-0
Mar 11, 2013 1:13:32 PM org.slf4j.impl.JCLLoggerAdapter info
INFO: Program took 15533 ms (Minutes: 0.25888333333333335)
hadoop binary is not in PATH,HADOOP_HOME/bin,HADOOP_PREFIX/bin, running locally
Mar 11, 2013 1:13:34 PM org.slf4j.impl.JCLLoggerAdapter info
INFO: Command line arguments: {--clustering=null,
--clusters=[/tmp/mahout-work-hudson/reuters-kmeans-clusters],
--convergenceDelta=[0.5],
--distanceMeasure=[org.apache.mahout.common.distance.CosineDistanceMeasure],
--endPhase=[2147483647],
--input=[/tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/tfidf-vectors/],
--maxIter=[10], --method=[mapreduce], --numClusters=[20],
--output=[/tmp/mahout-work-hudson/reuters-kmeans], --overwrite=null,
--startPhase=[0], --tempDir=[temp]}
Mar 11, 2013 1:13:34 PM org.slf4j.impl.JCLLoggerAdapter info
INFO: Deleting /tmp/mahout-work-hudson/reuters-kmeans-clusters
Mar 11, 2013 1:13:34 PM org.apache.hadoop.util.NativeCodeLoader <clinit>
WARNING: Unable to load native-hadoop library for your platform... using
builtin-java classes where applicable
Mar 11, 2013 1:13:34 PM org.apache.hadoop.io.compress.CodecPool getCompressor
INFO: Got brand-new compressor
Mar 11, 2013 1:13:35 PM org.slf4j.impl.JCLLoggerAdapter info
INFO: Wrote 20 Klusters to
/tmp/mahout-work-hudson/reuters-kmeans-clusters/part-randomSeed
Mar 11, 2013 1:13:35 PM org.slf4j.impl.JCLLoggerAdapter info
INFO: Input:
/tmp/mahout-work-hudson/reuters-out-seqdir-sparse-kmeans/tfidf-vectors Clusters
In: /tmp/mahout-work-hudson/reuters-kmeans-clusters/part-randomSeed Out:
/tmp/mahout-work-hudson/reuters-kmeans Distance:
org.apache.mahout.common.distance.CosineDistanceMeasure
Mar 11, 2013 1:13:35 PM org.slf4j.impl.JCLLoggerAdapter info
INFO: convergence: 0.5 max Iterations: 10 num Reduce Tasks:
org.apache.mahout.math.VectorWritable Input Vectors: {}
Mar 11, 2013 1:13:35 PM org.apache.hadoop.io.compress.CodecPool getDecompressor
INFO: Got brand-new decompressor
Exception in thread "main" java.lang.IllegalStateException: No input clusters
found in /tmp/mahout-work-hudson/reuters-kmeans-clusters/part-randomSeed. Check
your -c argument.
at
org.apache.mahout.clustering.kmeans.KMeansDriver.buildClusters(KMeansDriver.java:216)
at
org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:147)
at
org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:107)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at
org.apache.mahout.clustering.kmeans.KMeansDriver.main(KMeansDriver.java:48)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:195)
Build step 'Execute shell' marked build as failure
