Hi Job,
This seems CDH-specific, so I've moved the thread over to the cdh-users
mailing list (BCC common-user)
Thanks
-Todd
On Thu, Feb 24, 2011 at 2:52 AM, Job j...@gridline.nl wrote:
Hi all,
This issue could very well be related to the Cloudera distribution
(CDH3b4) I use, but maybe someone knows the solution:
I configured a Job, something like this:
Configuration conf = getConf();
// ... set configuration
conf.set(mapred.jar, localJarFile.toString())
// tracker, zookeeper, hbase etc.
Job job = new Job(conf);
// map:
job.setMapperClass(DataImportMap.class);
job.setMapOutputKeyClass(LongWritable.class);
job.setMapOutputValueClass(Put.class);
// reduce:
TableMapReduceUtil.initTableReducerJob(MyTable,
DataImportReduce.class, job);
FileInputFormat.addInputPath(job, new Path(inputData));
// execute:
job.waitForCompletion(true);
Now the server throws a strange exception below, see the stacktrace
below.
When i take look at the hdfs file system - through hdfs fuse - the file
is there, it really is the jar that contains my mapred classes.
Any clue wat goes wrong here?
Thanks,
Job
-
java.io.FileNotFoundException:
File
/var/lib/hadoop-0.20/cache/mapred/mapred/staging/job/.staging/job_201102241026_0002/job.jar
does not exist.
at
org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:383)
at
org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:207)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:157)
at
org.apache.hadoop.fs.LocalFileSystem.copyToLocalFile(LocalFileSystem.java:61)
at
org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1303)
at
org.apache.hadoop.mapred.JobLocalizer.localizeJobJarFile(JobLocalizer.java:273)
at
org.apache.hadoop.mapred.JobLocalizer.localizeJobFiles(JobLocalizer.java:381)
at
org.apache.hadoop.mapred.JobLocalizer.localizeJobFiles(JobLocalizer.java:371)
at
org.apache.hadoop.mapred.DefaultTaskController.initializeJob(DefaultTaskController.java:198)
at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1154)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
at
org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1129)
at
org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1055)
at
org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:2212)
at org.apache.hadoop.mapred.TaskTracker
$TaskLauncher.run(TaskTracker.java:2176)
--
Drs. Job Tiel Groenestege
GridLine - Intranet en Zoeken
GridLine
Keizersgracht 520
1017 EK Amsterdam
www: http://www.gridline.nl
mail: j...@gridline.nl
tel: +31 20 616 2050
fax: +31 20 616 2051
De inhoud van dit bericht en de eventueel daarbij behorende bijlagen zijn
persoonlijk gericht aan en derhalve uitsluitend bestemd voor de
geadresseerde. Zij kunnen gegevens met betrekking tot een derde bevatten. De
ontvanger die niet de geadresseerde is, noch bevoegd is dit bericht namens
geadresseerde te ontvangen, wordt verzocht de afzender onmiddellijk op de
hoogte te stellen van de ontvangst. Elk gebruik van de inhoud van dit
bericht en/of van de daarbij behorende bijlagen door een ander dan de
geadresseerde is onrechtmatig jegens afzender respectievelijk de hiervoor
bedoelde derde.
--
Todd Lipcon
Software Engineer, Cloudera
