Hi all,
I'm having some trouble running the importtsv tool on CDH3B4 configured in
pseudo distributed mode.
The tool works fine unless I add the option importtsv.bulk.output.
Does importtsv with the option importtsv.bulk.output work in pseudo distributed
mode or do I maybe have something configured incorrectly?
Here is some info on the error:
Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.LocalJobRunner$Job <init>
WARNING: LocalJobRunner does not support symlinking into current working dir.
Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.TaskRunner symlink
INFO: Creating symlink:
/tmp/hadoop-hadoop/mapred/local/archive/953502662101888516_-198765657_2115049918/file/home/hadoop/test/java/partitions_1303490140287
<- /tmp/hadoop-hadoop/mapred/local/localRunner/_partition.lst
Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.TaskRunner symlink
WARNING: Failed to create symlink:
/tmp/hadoop-hadoop/mapred/local/archive/953502662101888516_-198765657_2115049918/file/home/hadoop/test/java/partitions_1303490140287
<- /tmp/hadoop-hadoop/mapred/local/localRunner/_partition.lst
Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
INFO: Running job: job_local_0001
Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
INFO: io.sort.mb = 100
Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
INFO: data buffer = 79691776/99614720
Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
INFO: record buffer = 262144/327680
Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.LocalJobRunner$Job run
WARNING: job_local_0001
java.lang.IllegalArgumentException: Can't read partitions file
at
org.apache.hadoop.hbase.mapreduce.hadoopbackport.TotalOrderPartitioner.setConf(TotalOrderPartitioner.java:111)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:559)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:638)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:322)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)
Caused by: java.io.FileNotFoundException: File _partition.lst does not exist.
at
org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:383)
at
org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:776)
at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1424)
at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1419)
at
org.apache.hadoop.hbase.mapreduce.hadoopbackport.TotalOrderPartitioner.readPartitions(TotalOrderPartitioner.java:296)
at
org.apache.hadoop.hbase.mapreduce.hadoopbackport.TotalOrderPartitioner.setConf(TotalOrderPartitioner.java:82)
... 6 more
Apr 22, 2011 9:35:40 AM
org.apache.hadoop.filecache.TrackerDistributedCacheManager deleteLocalPath
INFO: Deleted path
/tmp/hadoop-hadoop/mapred/local/archive/-8364002144339543919_194806607_1507402918/file/home/hadoop/test/java/lib/guava-r06.jar
Apr 22, 2011 9:35:40 AM
org.apache.hadoop.filecache.TrackerDistributedCacheManager deleteLocalPath
INFO: Deleted path
/tmp/hadoop-hadoop/mapred/local/archive/-7608337350018775429_-1267154261_925648918/file/home/hadoop/test/java/lib/hadoop-core-0.20.2-CDH3B4.jar
Apr 22, 2011 9:35:40 AM
org.apache.hadoop.filecache.TrackerDistributedCacheManager deleteLocalPath
INFO: Deleted path
/tmp/hadoop-hadoop/mapred/local/archive/6475934364733173115_-1837084859_925493918/file/home/hadoop/test/java/lib/hbase-0.90.1-CDH3B4.jar
Apr 22, 2011 9:35:40 AM
org.apache.hadoop.filecache.TrackerDistributedCacheManager deleteLocalPath
INFO: Deleted path
/tmp/hadoop-hadoop/mapred/local/archive/-5268899720351360254_-17093236_1440710918/file/home/hadoop/test/java/lib/zookeeper-3.3.2-CDH3B4.jar
Apr 22, 2011 9:35:40 AM
org.apache.hadoop.filecache.TrackerDistributedCacheManager deleteLocalPath
INFO: Deleted path
/tmp/hadoop-hadoop/mapred/local/archive/953502662101888516_-198765657_2115049918/file/home/hadoop/test/java/partitions_1303490140287
Apr 22, 2011 9:35:41 AM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
INFO: map 0% reduce 0%
Apr 22, 2011 9:35:41 AM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
INFO: Job complete: job_local_0001
Apr 22, 2011 9:35:41 AM org.apache.hadoop.mapred.Counters log
INFO: Counters: 0
Thanks,
Eric
