Hi Eric, Unfortunately, the LocalJobRunner is missing a feature that is causing the bulk load option to fail.
Are you running a MapReduce cluster? Make sure that you've configured the jobtracker address in your mapred-site.xml if so. -Todd On Fri, Apr 22, 2011 at 11:09 AM, Eric Ross <[email protected]>wrote: > Hi all, > > I'm having some trouble running the importtsv tool on CDH3B4 configured in > pseudo distributed mode. > The tool works fine unless I add the option importtsv.bulk.output. > > Does importtsv with the option importtsv.bulk.output work in pseudo > distributed mode or do I maybe have something configured incorrectly? > > Here is some info on the error: > > Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.LocalJobRunner$Job <init> > WARNING: LocalJobRunner does not support symlinking into current working > dir. > Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.TaskRunner symlink > INFO: Creating symlink: > /tmp/hadoop-hadoop/mapred/local/archive/953502662101888516_-198765657_2115049918/file/home/hadoop/test/java/partitions_1303490140287 > <- /tmp/hadoop-hadoop/mapred/local/localRunner/_partition.lst > Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.TaskRunner symlink > WARNING: Failed to create symlink: > /tmp/hadoop-hadoop/mapred/local/archive/953502662101888516_-198765657_2115049918/file/home/hadoop/test/java/partitions_1303490140287 > <- /tmp/hadoop-hadoop/mapred/local/localRunner/_partition.lst > Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.JobClient > monitorAndPrintJob > INFO: Running job: job_local_0001 > Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.MapTask$MapOutputBuffer > <init> > INFO: io.sort.mb = 100 > Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.MapTask$MapOutputBuffer > <init> > INFO: data buffer = 79691776/99614720 > Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.MapTask$MapOutputBuffer > <init> > INFO: record buffer = 262144/327680 > Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.LocalJobRunner$Job run > WARNING: job_local_0001 > java.lang.IllegalArgumentException: Can't read partitions file > at > org.apache.hadoop.hbase.mapreduce.hadoopbackport.TotalOrderPartitioner.setConf(TotalOrderPartitioner.java:111) > at > org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62) > at > org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) > at > org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:559) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:638) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:322) > at > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210) > Caused by: java.io.FileNotFoundException: File _partition.lst does not > exist. > at > org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:383) > at > org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251) > at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:776) > at > org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1424) > at > org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1419) > at > org.apache.hadoop.hbase.mapreduce.hadoopbackport.TotalOrderPartitioner.readPartitions(TotalOrderPartitioner.java:296) > at > org.apache.hadoop.hbase.mapreduce.hadoopbackport.TotalOrderPartitioner.setConf(TotalOrderPartitioner.java:82) > ... 6 more > Apr 22, 2011 9:35:40 AM > org.apache.hadoop.filecache.TrackerDistributedCacheManager deleteLocalPath > INFO: Deleted path > /tmp/hadoop-hadoop/mapred/local/archive/-8364002144339543919_194806607_1507402918/file/home/hadoop/test/java/lib/guava-r06.jar > Apr 22, 2011 9:35:40 AM > org.apache.hadoop.filecache.TrackerDistributedCacheManager deleteLocalPath > INFO: Deleted path > /tmp/hadoop-hadoop/mapred/local/archive/-7608337350018775429_-1267154261_925648918/file/home/hadoop/test/java/lib/hadoop-core-0.20.2-CDH3B4.jar > Apr 22, 2011 9:35:40 AM > org.apache.hadoop.filecache.TrackerDistributedCacheManager deleteLocalPath > INFO: Deleted path > /tmp/hadoop-hadoop/mapred/local/archive/6475934364733173115_-1837084859_925493918/file/home/hadoop/test/java/lib/hbase-0.90.1-CDH3B4.jar > Apr 22, 2011 9:35:40 AM > org.apache.hadoop.filecache.TrackerDistributedCacheManager deleteLocalPath > INFO: Deleted path > /tmp/hadoop-hadoop/mapred/local/archive/-5268899720351360254_-17093236_1440710918/file/home/hadoop/test/java/lib/zookeeper-3.3.2-CDH3B4.jar > Apr 22, 2011 9:35:40 AM > org.apache.hadoop.filecache.TrackerDistributedCacheManager deleteLocalPath > INFO: Deleted path > /tmp/hadoop-hadoop/mapred/local/archive/953502662101888516_-198765657_2115049918/file/home/hadoop/test/java/partitions_1303490140287 > Apr 22, 2011 9:35:41 AM org.apache.hadoop.mapred.JobClient > monitorAndPrintJob > INFO: map 0% reduce 0% > Apr 22, 2011 9:35:41 AM org.apache.hadoop.mapred.JobClient > monitorAndPrintJob > INFO: Job complete: job_local_0001 > Apr 22, 2011 9:35:41 AM org.apache.hadoop.mapred.Counters log > INFO: Counters: 0 > > Thanks, > Eric > > -- Todd Lipcon Software Engineer, Cloudera
