Hi All, I am getting problem that job is running in localrunner rather than the cluster enviormnent. And when am running the job i would not be able to see the job id in the resource manager UI
Can you please go through the issues and let me know ASAP. sudo -u hdfs hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar teragen 1000000 /benchmark/teragen/input 12/07/29 13:35:59 WARN conf.Configuration: session.id is deprecated. Instead, use dfs.metrics.session-id 12/07/29 13:35:59 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId= 12/07/29 13:35:59 INFO util.NativeCodeLoader: Loaded the native-hadoop library 12/07/29 13:35:59 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. Generating 1000000 using 1 maps with step of 1000000 12/07/29 13:35:59 INFO mapred.JobClient: Running job: job_local_0001 12/07/29 13:35:59 INFO mapred.LocalJobRunner: OutputCommitter set in config null 12/07/29 13:35:59 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter 12/07/29 13:35:59 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead 12/07/29 13:35:59 INFO util.ProcessTree: setsid exited with exit code 0 12/07/29 13:35:59 INFO mapred.Task: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@47c297a3 12/07/29 13:36:00 WARN mapreduce.Counters: Counter name MAP_INPUT_BYTES is deprecated. Use FileInputFormatCounters as group name and BYTES_READ as counter name instead 12/07/29 13:36:00 INFO mapred.MapTask: numReduceTasks: 0 12/07/29 13:36:00 INFO mapred.JobClient: map 0% reduce 0% 12/07/29 13:36:01 INFO mapred.Task: Task:attempt_local_0001_m_000000_0 is done. And is in the process of commiting 12/07/29 13:36:01 INFO mapred.LocalJobRunner: 12/07/29 13:36:01 INFO mapred.Task: Task attempt_local_0001_m_000000_0 is allowed to commit now 12/07/29 13:36:01 INFO mapred.FileOutputCommitter: Saved output of task 'attempt_local_0001_m_000000_0' to hdfs://hadoop-master-1/benchmark/teragen/input 12/07/29 13:36:01 INFO mapred.LocalJobRunner: 12/07/29 13:36:01 INFO mapred.Task: Task 'attempt_local_0001_m_000000_0' done. 12/07/29 13:36:02 INFO mapred.JobClient: map 100% reduce 0% 12/07/29 13:36:02 INFO mapred.JobClient: Job complete: job_local_0001 12/07/29 13:36:02 INFO mapred.JobClient: Counters: 19 12/07/29 13:36:02 INFO mapred.JobClient: File System Counters 12/07/29 13:36:02 INFO mapred.JobClient: FILE: Number of bytes read=142686 12/07/29 13:36:02 INFO mapred.JobClient: FILE: Number of bytes written=220956 12/07/29 13:36:02 INFO mapred.JobClient: FILE: Number of read operations=0 12/07/29 13:36:02 INFO mapred.JobClient: FILE: Number of large read operations=0 12/07/29 13:36:02 INFO mapred.JobClient: FILE: Number of write operations=0 12/07/29 13:36:02 INFO mapred.JobClient: HDFS: Number of bytes read=0 12/07/29 13:36:02 INFO mapred.JobClient: HDFS: Number of bytes written=100000000 12/07/29 13:36:02 INFO mapred.JobClient: HDFS: Number of read operations=1 12/07/29 13:36:02 INFO mapred.JobClient: HDFS: Number of large read operations=0 12/07/29 13:36:02 INFO mapred.JobClient: HDFS: Number of write operations=2 12/07/29 13:36:02 INFO mapred.JobClient: Map-Reduce Framework 12/07/29 13:36:02 INFO mapred.JobClient: Map input records=1000000 12/07/29 13:36:02 INFO mapred.JobClient: Map output records=1000000 12/07/29 13:36:02 INFO mapred.JobClient: Input split bytes=82 12/07/29 13:36:02 INFO mapred.JobClient: Spilled Records=0 12/07/29 13:36:02 INFO mapred.JobClient: CPU time spent (ms)=0 12/07/29 13:36:02 INFO mapred.JobClient: Physical memory (bytes) snapshot=0 12/07/29 13:36:02 INFO mapred.JobClient: Virtual memory (bytes) snapshot=0 12/07/29 13:36:02 INFO mapred.JobClient: Total committed heap usage (bytes)=124715008 12/07/29 13:36:02 INFO mapred.JobClient: org.apache.hadoop.mapreduce.lib.input.FileInputFormatCounter Regards Abhishek
