Hello, I am trying to run HBase's ImportTsv against Yarn (Hadoop 2.2.0).
I can run the Hadoop TestDFSIO Yarn job with no problems: hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.2.0-tests.jar TestDFSIO -write -nrFiles 20 -fileSize 10 I cannot run the HBase ImportTsv job: hadoop jar $HBASE_HOME/lib/hbase-server-0.96.0-hadoop2.jar importtsv | -Dimporttsv.separator= \ -Dimporttsv.columns=$IMPORTTSV_COLUMNS \ -Dimporttsv.bulk.output=hdfs://$HADOOP_NAMENODE:54310/user/bihadoop/importtsv-output \ bulkloaded-xdremail \ hdfs://$HADOOP_NAMENODE:54310/user/bihadoop/kv-data-files ... because the job.jar file is never successfully downloaded to any yarn node: 2014-05-05 19:44:06,641 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Writing credentials to the nmPrivate file /tmp/hadoop-bihadoop/nm-local-dir/nmPrivate/container_1399316513410_0007_01_000001.tokens. Credentials list: 2014-05-05 19:44:06,642 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Initializing user bihadoop 2014-05-05 19:44:06,690 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Copying from /tmp/hadoop-bihadoop/nm-local-dir/nmPrivate/container_1399316513410_0007_01_000001.tokens to /tmp/hadoop-bihadoop/nm-local-dir/usercache/bihadoop/appcache/application_1399316513410_0007/container_1399316513410_0007_01_000001.tokens 2014-05-05 19:44:06,690 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: CWD set to /tmp/hadoop-bihadoop/nm-local-dir/usercache/bihadoop/appcache/application_1399316513410_0007 = file:/tmp/hadoop-bihadoop/nm-local-dir/usercache/bihadoop/appcache/application_1399316513410_0007 2014-05-05 19:44:06,791 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:bihadoop (auth:SIMPLE) cause:java.io.FileNotFoundException: File file:/tmp/hadoop-yarn/staging/bihadoop/.staging/job_1399316513410_0007/job.jar does not exist 2014-05-05 19:44:06,810 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: DEBUG: FAILED { file:/tmp/hadoop-yarn/staging/bihadoop/.staging/job_1399316513410_0007/job.jar, 1399319045000, PATTERN, (?:classes/|lib/).* }, File file:/tmp/hadoop-yarn/staging/bihadoop/.staging/job_1399316513410_0007/job.jar does not exist 2014-05-05 19:44:06,812 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource file:/tmp/hadoop-yarn/staging/bihadoop/.staging/job_1399316513410_0007/job.jar transitioned from DOWNLOADING to FAILED I have also tried running importtsv from the hbase script, i.e.: hbase org.apache.hadoop.hbase.mapreduce.ImportTsv \ ... ... The end result is the same: job.jar not found in staging dir. (The job.jar is found on the master node, where I run the job.) There seems to be no problems distributing job.jar to the staging dirs when I run the TestDFSIO job. But the job.jar containing ImportTsv is never distributed. Very frustrating, time eating... ;/ Does anyone have any suggestions? I am runinng Hadoop 2.2.0 w/ HBase 0.96.0 Thanks, Stan