See <https://builds.apache.org/job/Mahout-Examples-Cluster-Reuters-II/521/changes>
Changes: [smarthi] MAHOUT-833: Make conversion to sequence files map-reduce - Checking in, tests pass ------------------------------------------ [...truncated 5859 lines...] INFO: Starting flush of map output Jun 23, 2013 6:32:20 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer sortAndSpill INFO: Finished spill 0 Jun 23, 2013 6:32:20 PM org.apache.hadoop.mapred.Task done INFO: Task:attempt_local_0019_m_000000_0 is done. And is in the process of commiting Jun 23, 2013 6:32:20 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate INFO: Jun 23, 2013 6:32:20 PM org.apache.hadoop.mapred.Task sendDone INFO: Task 'attempt_local_0019_m_000000_0' done. Jun 23, 2013 6:32:20 PM org.apache.hadoop.mapred.Task initialize INFO: Using ResourceCalculatorPlugin : null Jun 23, 2013 6:32:20 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate INFO: Jun 23, 2013 6:32:20 PM org.apache.hadoop.mapred.Merger$MergeQueue merge INFO: Merging 1 sorted segments Jun 23, 2013 6:32:20 PM org.apache.hadoop.mapred.Merger$MergeQueue merge INFO: Down to the last merge-pass, with 1 segments left of total size: 57 bytes Jun 23, 2013 6:32:20 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate INFO: Jun 23, 2013 6:32:20 PM org.apache.hadoop.mapred.Task done INFO: Task:attempt_local_0019_r_000000_0 is done. And is in the process of commiting Jun 23, 2013 6:32:20 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate INFO: Jun 23, 2013 6:32:20 PM org.apache.hadoop.mapred.Task commit INFO: Task attempt_local_0019_r_000000_0 is allowed to commit now Jun 23, 2013 6:32:20 PM org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter commitTask INFO: Saved output of task 'attempt_local_0019_r_000000_0' to /tmp/mahout-work-hudson/reuters-lda-model/model-19 Jun 23, 2013 6:32:20 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate INFO: reduce > reduce Jun 23, 2013 6:32:20 PM org.apache.hadoop.mapred.Task sendDone INFO: Task 'attempt_local_0019_r_000000_0' done. Jun 23, 2013 6:32:21 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob INFO: map 100% reduce 100% Jun 23, 2013 6:32:21 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob INFO: Job complete: job_local_0019 Jun 23, 2013 6:32:21 PM org.apache.hadoop.mapred.Counters log INFO: Counters: 17 Jun 23, 2013 6:32:21 PM org.apache.hadoop.mapred.Counters log INFO: File Output Format Counters Jun 23, 2013 6:32:21 PM org.apache.hadoop.mapred.Counters log INFO: Bytes Written=389 Jun 23, 2013 6:32:21 PM org.apache.hadoop.mapred.Counters log INFO: FileSystemCounters Jun 23, 2013 6:32:21 PM org.apache.hadoop.mapred.Counters log INFO: FILE_BYTES_READ=1614639305 Jun 23, 2013 6:32:21 PM org.apache.hadoop.mapred.Counters log INFO: FILE_BYTES_WRITTEN=1629247415 Jun 23, 2013 6:32:21 PM org.apache.hadoop.mapred.Counters log INFO: File Input Format Counters Jun 23, 2013 6:32:21 PM org.apache.hadoop.mapred.Counters log INFO: Bytes Read=152 Jun 23, 2013 6:32:21 PM org.apache.hadoop.mapred.Counters log INFO: Map-Reduce Framework Jun 23, 2013 6:32:21 PM org.apache.hadoop.mapred.Counters log INFO: Map output materialized bytes=61 Jun 23, 2013 6:32:21 PM org.apache.hadoop.mapred.Counters log INFO: Map input records=0 Jun 23, 2013 6:32:21 PM org.apache.hadoop.mapred.Counters log INFO: Reduce shuffle bytes=0 Jun 23, 2013 6:32:21 PM org.apache.hadoop.mapred.Counters log INFO: Spilled Records=40 Jun 23, 2013 6:32:21 PM org.apache.hadoop.mapred.Counters log INFO: Map output bytes=120 Jun 23, 2013 6:32:21 PM org.apache.hadoop.mapred.Counters log INFO: Total committed heap usage (bytes)=697303040 Jun 23, 2013 6:32:21 PM org.apache.hadoop.mapred.Counters log INFO: SPLIT_RAW_BYTES=119 Jun 23, 2013 6:32:21 PM org.apache.hadoop.mapred.Counters log INFO: Combine input records=20 Jun 23, 2013 6:32:21 PM org.apache.hadoop.mapred.Counters log INFO: Reduce input records=20 Jun 23, 2013 6:32:21 PM org.apache.hadoop.mapred.Counters log INFO: Reduce input groups=20 Jun 23, 2013 6:32:21 PM org.apache.hadoop.mapred.Counters log INFO: Combine output records=20 Jun 23, 2013 6:32:21 PM org.apache.hadoop.mapred.Counters log INFO: Reduce output records=20 Jun 23, 2013 6:32:21 PM org.apache.hadoop.mapred.Counters log INFO: Map output records=20 Jun 23, 2013 6:32:21 PM org.slf4j.impl.JCLLoggerAdapter info INFO: About to run iteration 20 of 20 Jun 23, 2013 6:32:21 PM org.slf4j.impl.JCLLoggerAdapter info INFO: About to run: Iteration 20 of 20, input path: /tmp/mahout-work-hudson/reuters-lda-model/model-19 Jun 23, 2013 6:32:22 PM org.apache.hadoop.mapreduce.lib.input.FileInputFormat listStatus INFO: Total input paths to process : 1 Jun 23, 2013 6:32:22 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob INFO: Running job: job_local_0020 Jun 23, 2013 6:32:22 PM org.apache.hadoop.mapred.Task initialize INFO: Using ResourceCalculatorPlugin : null Jun 23, 2013 6:32:22 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init> INFO: io.sort.mb = 100 Jun 23, 2013 6:32:22 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init> INFO: data buffer = 79691776/99614720 Jun 23, 2013 6:32:22 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init> INFO: record buffer = 262144/327680 Jun 23, 2013 6:32:22 PM org.slf4j.impl.JCLLoggerAdapter info INFO: Retrieving configuration Jun 23, 2013 6:32:22 PM org.slf4j.impl.JCLLoggerAdapter info INFO: Initializing read model Jun 23, 2013 6:32:22 PM org.slf4j.impl.JCLLoggerAdapter info INFO: Initializing write model Jun 23, 2013 6:32:22 PM org.slf4j.impl.JCLLoggerAdapter info INFO: Initializing model trainer Jun 23, 2013 6:32:22 PM org.slf4j.impl.JCLLoggerAdapter info INFO: Starting training threadpool with 4 threads Jun 23, 2013 6:32:22 PM org.slf4j.impl.JCLLoggerAdapter info INFO: Stopping model trainer Jun 23, 2013 6:32:22 PM org.slf4j.impl.JCLLoggerAdapter info INFO: Initiating stopping of training threadpool Jun 23, 2013 6:32:22 PM org.slf4j.impl.JCLLoggerAdapter info INFO: threadpool took: 0.66775ms Jun 23, 2013 6:32:23 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob INFO: map 0% reduce 0% Jun 23, 2013 6:32:23 PM org.slf4j.impl.JCLLoggerAdapter info INFO: readModel.stop() took 1001.621269ms Jun 23, 2013 6:32:24 PM org.slf4j.impl.JCLLoggerAdapter info INFO: writeModel.stop() took 1009.993033ms Jun 23, 2013 6:32:24 PM org.slf4j.impl.JCLLoggerAdapter info INFO: Writing model Jun 23, 2013 6:32:24 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer flush INFO: Starting flush of map output Jun 23, 2013 6:32:24 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer sortAndSpill INFO: Finished spill 0 Jun 23, 2013 6:32:24 PM org.apache.hadoop.mapred.Task done INFO: Task:attempt_local_0020_m_000000_0 is done. And is in the process of commiting Jun 23, 2013 6:32:24 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate INFO: Jun 23, 2013 6:32:24 PM org.apache.hadoop.mapred.Task sendDone INFO: Task 'attempt_local_0020_m_000000_0' done. Jun 23, 2013 6:32:24 PM org.apache.hadoop.mapred.Task initialize INFO: Using ResourceCalculatorPlugin : null Jun 23, 2013 6:32:24 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate INFO: Jun 23, 2013 6:32:24 PM org.apache.hadoop.mapred.Merger$MergeQueue merge INFO: Merging 1 sorted segments Jun 23, 2013 6:32:24 PM org.apache.hadoop.mapred.Merger$MergeQueue merge INFO: Down to the last merge-pass, with 1 segments left of total size: 57 bytes Jun 23, 2013 6:32:24 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate INFO: Jun 23, 2013 6:32:24 PM org.apache.hadoop.mapred.Task done INFO: Task:attempt_local_0020_r_000000_0 is done. And is in the process of commiting Jun 23, 2013 6:32:24 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate INFO: Jun 23, 2013 6:32:24 PM org.apache.hadoop.mapred.Task commit INFO: Task attempt_local_0020_r_000000_0 is allowed to commit now Jun 23, 2013 6:32:24 PM org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter commitTask INFO: Saved output of task 'attempt_local_0020_r_000000_0' to /tmp/mahout-work-hudson/reuters-lda-model/model-20 Jun 23, 2013 6:32:24 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate INFO: reduce > reduce Jun 23, 2013 6:32:24 PM org.apache.hadoop.mapred.Task sendDone INFO: Task 'attempt_local_0020_r_000000_0' done. Jun 23, 2013 6:32:25 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob INFO: map 100% reduce 100% Jun 23, 2013 6:32:25 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob INFO: Job complete: job_local_0020 Jun 23, 2013 6:32:25 PM org.apache.hadoop.mapred.Counters log INFO: Counters: 17 Jun 23, 2013 6:32:25 PM org.apache.hadoop.mapred.Counters log INFO: File Output Format Counters Jun 23, 2013 6:32:25 PM org.apache.hadoop.mapred.Counters log INFO: Bytes Written=389 Jun 23, 2013 6:32:25 PM org.apache.hadoop.mapred.Counters log INFO: FileSystemCounters Jun 23, 2013 6:32:25 PM org.apache.hadoop.mapred.Counters log INFO: FILE_BYTES_READ=1699620355 Jun 23, 2013 6:32:25 PM org.apache.hadoop.mapred.Counters log INFO: FILE_BYTES_WRITTEN=1714997335 Jun 23, 2013 6:32:25 PM org.apache.hadoop.mapred.Counters log INFO: File Input Format Counters Jun 23, 2013 6:32:25 PM org.apache.hadoop.mapred.Counters log INFO: Bytes Read=152 Jun 23, 2013 6:32:25 PM org.apache.hadoop.mapred.Counters log INFO: Map-Reduce Framework Jun 23, 2013 6:32:25 PM org.apache.hadoop.mapred.Counters log INFO: Map output materialized bytes=61 Jun 23, 2013 6:32:25 PM org.apache.hadoop.mapred.Counters log INFO: Map input records=0 Jun 23, 2013 6:32:25 PM org.apache.hadoop.mapred.Counters log INFO: Reduce shuffle bytes=0 Jun 23, 2013 6:32:25 PM org.apache.hadoop.mapred.Counters log INFO: Spilled Records=40 Jun 23, 2013 6:32:25 PM org.apache.hadoop.mapred.Counters log INFO: Map output bytes=120 Jun 23, 2013 6:32:25 PM org.apache.hadoop.mapred.Counters log INFO: Total committed heap usage (bytes)=697303040 Jun 23, 2013 6:32:25 PM org.apache.hadoop.mapred.Counters log INFO: SPLIT_RAW_BYTES=119 Jun 23, 2013 6:32:25 PM org.apache.hadoop.mapred.Counters log INFO: Combine input records=20 Jun 23, 2013 6:32:25 PM org.apache.hadoop.mapred.Counters log INFO: Reduce input records=20 Jun 23, 2013 6:32:25 PM org.apache.hadoop.mapred.Counters log INFO: Reduce input groups=20 Jun 23, 2013 6:32:25 PM org.apache.hadoop.mapred.Counters log INFO: Combine output records=20 Jun 23, 2013 6:32:25 PM org.apache.hadoop.mapred.Counters log INFO: Reduce output records=20 Jun 23, 2013 6:32:25 PM org.apache.hadoop.mapred.Counters log INFO: Map output records=20 Jun 23, 2013 6:32:25 PM org.slf4j.impl.JCLLoggerAdapter info INFO: Completed 20 iterations in 84 seconds Jun 23, 2013 6:32:25 PM org.slf4j.impl.JCLLoggerAdapter info INFO: Perplexities: () Jun 23, 2013 6:32:25 PM org.slf4j.impl.JCLLoggerAdapter info INFO: About to run: Writing final topic/term distributions from /tmp/mahout-work-hudson/reuters-lda-model/model-20 to /tmp/mahout-work-hudson/reuters-lda Jun 23, 2013 6:32:26 PM org.apache.hadoop.mapreduce.lib.input.FileInputFormat listStatus INFO: Total input paths to process : 1 Jun 23, 2013 6:32:26 PM org.slf4j.impl.JCLLoggerAdapter info INFO: About to run: Writing final document/topic inference from /tmp/mahout-work-hudson/reuters-out-matrix/matrix to /tmp/mahout-work-hudson/reuters-lda-topics Jun 23, 2013 6:32:26 PM org.apache.hadoop.mapred.Task initialize INFO: Using ResourceCalculatorPlugin : null Jun 23, 2013 6:32:26 PM org.apache.hadoop.mapred.Task done INFO: Task:attempt_local_0021_m_000000_0 is done. And is in the process of commiting Jun 23, 2013 6:32:26 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate INFO: Jun 23, 2013 6:32:26 PM org.apache.hadoop.mapred.Task commit INFO: Task attempt_local_0021_m_000000_0 is allowed to commit now Jun 23, 2013 6:32:26 PM org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter commitTask INFO: Saved output of task 'attempt_local_0021_m_000000_0' to /tmp/mahout-work-hudson/reuters-lda Jun 23, 2013 6:32:26 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate INFO: Jun 23, 2013 6:32:26 PM org.apache.hadoop.mapred.Task sendDone INFO: Task 'attempt_local_0021_m_000000_0' done. Jun 23, 2013 6:32:28 PM org.apache.hadoop.mapred.JobClient$2 run INFO: Cleaning up the staging area file:/tmp/hadoop-hudson/mapred/staging/hudson-72597339/.staging/job_local_0022 Jun 23, 2013 6:32:28 PM org.apache.hadoop.security.UserGroupInformation doAs SEVERE: PriviledgedActionException as:hudson cause:org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory /tmp/mahout-work-hudson/reuters-lda already exists Exception in thread "main" org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory /tmp/mahout-work-hudson/reuters-lda already exists at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:137) at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:949) at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:912) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149) at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:912) at org.apache.hadoop.mapreduce.Job.submit(Job.java:500) at org.apache.mahout.clustering.lda.cvb.CVB0Driver.writeDocTopicInference(CVB0Driver.java:463) at org.apache.mahout.clustering.lda.cvb.CVB0Driver.run(CVB0Driver.java:339) at org.apache.mahout.clustering.lda.cvb.CVB0Driver.run(CVB0Driver.java:198) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.mahout.clustering.lda.cvb.CVB0Driver.main(CVB0Driver.java:534) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68) at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139) at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:194) Build step 'Execute shell' marked build as failure