Hi All,
I'm quite new at using Hadoop and I'm facing a weird problem.
When running Hadoop in *local mode*, there is no problem getting a
resource within the Mapper from *myJob.jar*, all works like a charm.
*this.getClass().getResource("/test1.txt").getFile()*
But when running in *pseudo-dist mode*, the above sentence is always
unable to find the resource so there is no way for me to get the
resource required in the Mapper.
As you can see below, the TaskTracker tries to get the resource from a
weird place when running in pseudo-dist mode.
*Dir layout for the job attempt:*
josocas
??? distcache
??? jobcache
??? job_201202132214_0003
??? attempt_201202132214_0003_m_000000_1
? ??? work
??? attempt_201202132214_0003_m_000001_1
? ??? work
??? jars
? ??? job.jar
??? jobToken
??? job.xml
??? work
*Stack trace:*
12/02/13 22:19:30 INFO mapred.JobClient: Task Id :
attempt_201202132214_0003_m_000000_1, Status : FAILED
java.io.IOException: Resources not found
!!!!*file:/tmp/hadoop-josocas/mapred/local/taskTracker/josocas/jobcache/job_201202132214_0003/jars/job.jar!/test1.txt*
at es.josocas.hadoop.TokenizingMapper.map(TokenizingMapper.java:41)
at es.josocas.hadoop.TokenizingMapper.map(TokenizingMapper.java:20)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
at org.apache.hadoop.mapred.Child.main(Child.java:264)
Thanks in advance for your help.
Regards,
Jose
