I have a Hadoop cluster on which I have generated some data using Teragen. But while running Terasort on this data, it gives following error.
java.lang.RuntimeException: Error in configuring object at org.apache.hadoop.util. ReflectionUtils.setJobConf(ReflectionUtils.java:93) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.MapTask$OldOutputCollector.<init>(MapTask.java:481) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88) ... 6 more I have generated some data using Teragen on my Hadoop cluster. But when I run Caused by: java.lang.IllegalArgumentException: can't read paritions file at org.apache.hadoop.examples.terasort.TeraSort$TotalOrderPartitioner.configure(TeraSort.java:213) ... 11 more Caused by: java.io.FileNotFoundException: File _partition.lst does not exist. at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:383) at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251) at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:776) at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1424) at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1419) at org.apache.hadoop.examples.terasort.TeraSort$TotalOrderPartitioner.readPartitions(TeraSort.java:153) at org.apache.hadoop.examples.terasort.TeraSort$TotalOrderPartitioner.configure(TeraSort.java:210) ... 11 more HDFS *does* shows this file in its listing and I’m unable to understand why is Terasort unable to find this flle. bash-3.2$ hadoop dfs -lsr /user/hduser/ drwxrwxrwx - hdfs supergroup 0 2012-01-24 14:12 /user/hduser/terasort-input1 -rw-r--r-- 1 hdfs supergroup 0 2012-01-24 00:38 /user/hduser/terasort-input1/_SUCCESS -rw-r--r-- 1 hdfs supergroup 129 2012-01-24 14:12 /user/hduser/terasort-input1/_partition.lst -rw-r--r-- 1 hdfs supergroup 1000000000000 2012-01-23 15:25 /user/hduser/terasort-input1/part-00000 I tried changing the file permissions, ownership and copying the _partition.lst file at root of HDFS (so that relative path does not matter) but nothing seems to work. Online forums/mailing lists also don't help. Any help/pointers on this will be appreciated. ~Utkarsh