I suppose you are suggesting something like below which i tried and did not help .
yarn jar /opt/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar terasort -Dmapreduce.totalorderpartitioner.path =_sortPartitioning input output Thanks Ashish From: Bing Jiang <jiangbinglo...@gmail.com> To: user@hadoop.apache.org Date: 12/01/2014 11:13 AM Subject: Re: Hadoop Terasort Benchmark Failure - Need Inputs hi, Ashish I have ever seen a similar issue, and reported the issue https://issues.apache.org/jira/browse/MAPREDUCE-5782 I have some workaround from that jira. -Bing 2014-11-30 4:07 GMT+08:00 Ashish Kumar9 <ashis...@in.ibm.com>: Hi , I am facing issue when i run teragen / terasort benchmark . Can someone suggest if you have also faced the same issue Command Used yarn jar /opt/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar terasort input output Exception 14/11/29 07:03:10 INFO Configuration.deprecation: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class 14/11/29 07:03:10 INFO Configuration.deprecation: mapred.compress.map.output is deprecated. Instead, use mapreduce.map.output.compress 14/11/29 07:03:10 INFO Configuration.deprecation: min.num.spills.for.combine is deprecated. Instead, use mapreduce.map.combine.minspills 14/11/29 07:03:10 WARN mapred.LocalJobRunner: job_local_0001 java.lang.IllegalArgumentException: can't read paritions file at org.apache.hadoop.examples.terasort.TeraSort$TotalOrderPartitioner.setConf(TeraSort.java:216) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:675) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:740) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:368) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:270) Caused by: java.io.FileNotFoundException: File _partition.lst does not exist at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:520) at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:398) at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:142) at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:344) at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:772) at org.apache.hadoop.examples.terasort.TeraSort$TotalOrderPartitioner.readPartitions(TeraSort.java:158) at org.apache.hadoop.examples.terasort.TeraSort$TotalOrderPartitioner.setConf(TeraSort.java:213) ... 6 more 14/11/29 07:03:10 INFO Configuration.deprecation: job.end.notification.url is deprecated. Instead, use mapreduce.job.end-notification.url 14/11/29 07:03:11 INFO mapred.JobClient: map 0% reduce 0% 14/11/29 07:03:11 INFO mapred.JobClient: Job complete: job_local_0001 14/11/29 07:03:11 INFO mapred.JobClient: Counters: 0 14/11/29 07:03:11 INFO terasort.TeraSort: done Investigations done so far thoroughly validated mapred-site.xml and am completely in sync with below recommendations http://hadoop.apache.org/docs/r2.2.0/hadoop-project-dist/hadoop-common/ClusterSetup.html teragen executes successfully but terasort fails with above exception some sites suggest that i should use the property "" in mapred-site.xml but looks like this property is no more valid . Granted full access to hdfs directory hadoop fs -chmod -R 775 / Thanks and Regards, Ashish Kumar
