I have two node,one is master(172.11.12.6) and one is node1(172.11.12.7), I use following command to run a program under 172.11.12.6 ,like follows: $ hadoop jar wc2.jar WordCountPredefined abc.txt output
[hadoop@master ~]$ hadoop jar wc2.jar WordCountPredefined abc.txt output 14/02/10 00:23:23 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 14/02/10 00:23:25 INFO client.RMProxy: Connecting to ResourceManager at master/172.11.12.6:8993 14/02/10 00:23:26 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this. 14/02/10 00:23:28 INFO input.FileInputFormat: Total input paths to process : 1 14/02/10 00:23:28 INFO mapreduce.JobSubmitter: number of splits:1 14/02/10 00:23:28 INFO Configuration.deprecation: user.name is deprecated. Instead, use mapreduce.job.user.name 14/02/10 00:23:28 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar 14/02/10 00:23:28 INFO Configuration.deprecation: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class 14/02/10 00:23:28 INFO Configuration.deprecation: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class 14/02/10 00:23:28 INFO Configuration.deprecation: mapred.job.name is deprecated. Instead, use mapreduce.job.name 14/02/10 00:23:28 INFO Configuration.deprecation: mapreduce.reduce.class is deprecated. Instead, use mapreduce.job.reduce.class 14/02/10 00:23:28 INFO Configuration.deprecation: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir 14/02/10 00:23:28 INFO Configuration.deprecation: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir 14/02/10 00:23:28 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps 14/02/10 00:23:28 INFO Configuration.deprecation: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class 14/02/10 00:23:29 INFO Configuration.deprecation: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir 14/02/10 00:23:29 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1391993915383_0003 14/02/10 00:23:30 INFO impl.YarnClientImpl: Submitted application application_1391993915383_0003 to ResourceManager at master/172.11.12.6:8993 14/02/10 00:23:31 INFO mapreduce.Job: The url to track the job: http://master:8088/proxy/application_1391993915383_0003/ 14/02/10 00:23:31 INFO mapreduce.Job: Running job: job_1391993915383_0003 14/02/10 00:23:54 INFO mapreduce.Job: Job job_1391993915383_0003 running in uber mode : false 14/02/10 00:23:54 INFO mapreduce.Job: map 0% reduce 0% 14/02/10 00:26:09 INFO mapreduce.Job: map 67% reduce 0% 14/02/10 00:26:12 INFO mapreduce.Job: map 100% reduce 0% 14/02/10 00:26:43 INFO mapreduce.Job: map 100% reduce 100% 14/02/10 00:26:45 INFO mapreduce.Job: Job job_1391993915383_0003 completed successfully 14/02/10 00:26:46 INFO mapreduce.Job: Counters: 43 File System Counters FILE: Number of bytes read=30 FILE: Number of bytes written=156945 FILE: Number of read operations=0 FILE: Number of large read operations=0 FILE: Number of write operations=0 HDFS: Number of bytes read=115 HDFS: Number of bytes written=16 HDFS: Number of read operations=6 HDFS: Number of large read operations=0 HDFS: Number of write operations=2 Job Counters Launched map tasks=1 Launched reduce tasks=1 Data-local map tasks=1 Total time spent by all maps in occupied slots (ms)=139690 Total time spent by all reduces in occupied slots (ms)=20127 Map-Reduce Framework Map input records=1 Map output records=2 Map output bytes=20 Map output materialized bytes=30 Input split bytes=103 Combine input records=0 Combine output records=0 Reduce input groups=2 Reduce shuffle bytes=30 Reduce input records=2 Reduce output records=2 Spilled Records=4 Shuffled Maps =1 Failed Shuffles=0 Merged Map outputs=1 GC time elapsed (ms)=396 CPU time spent (ms)=2130 Physical memory (bytes) snapshot=199540736 Virtual memory (bytes) snapshot=709742592 Total committed heap usage (bytes)=124510208 Shuffle Errors BAD_ID=0 CONNECTION=0 IO_ERROR=0 WRONG_LENGTH=0 WRONG_MAP=0 WRONG_REDUCE=0 File Input Format Counters Bytes Read=12 File Output Format Counters Bytes Written=16 I can view http://172.11.12.6:9002/dfsnodelist.jsp?whatNodes=LIVE in firefox NameNode 'master:9000' Started: Sun Feb 09 16:57:50 PST 2014 Version: 2.2.0, 1529768 Compiled: 2013-10-07T06:28Z by hortonmu from branch-2.2.0 Cluster ID: CID-e47f6b2e-ca1f-4eda-b6f7-dd9d5314c251 Block Pool ID: BP-1227723652-172.11.12.6-1390358446283 Live Datanodes : 2 Node Transferring Address Last Contact Admin State Configured Capacity (GB) Used(GB) Non DFS Used (GB) Remaining master 172.11.12.6:50010 1 In Service 16.15 0.00 2.76 13.39 node1 172.11.12.7:50010 1 In Service 16.15 0.00 2.75 13.40 How can I know which node will be called ,master(172.11.12.6) or node1(172.11.12.6) ? Thanks. --------------------------------------------------------------------------------------------------- Confidentiality Notice: The information contained in this e-mail and any accompanying attachment(s) is intended only for the use of the intended recipient and may be confidential and/or privileged of Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader of this communication is not the intended recipient, unauthorized use, forwarding, printing, storing, disclosure or copying is strictly prohibited, and may be unlawful.If you have received this communication in error,please immediately notify the sender by return e-mail, and delete the original message and all copies from your system. Thank you. ---------------------------------------------------------------------------------------------------