Yes, I did, the myMapper.sh is executed, the problem is inside this myMapper.sh, it calls a java program named myJava, the myJava did not get executed on slaves, and I shipped myJava.class too.
Thanks, Boyu On Wed, May 2, 2012 at 1:20 AM, 黄 山 <thuhuang...@gmail.com> wrote: > have you shipped myMapper.sh to each node? > > thuhuang...@gmail.com > > > > 在 2012-5-2，下午1:17， Boyu Zhang 写道： > > > Hi All, > > > > I am in a little bit strange situation, I am using Hadoop streaming to > run > > a bash shell program myMapper.sh, and in the myMapper.sh, it calls a java > > program, then a R program, then output intermediate key, values. I used > > -file option to ship the java and R files, but the java program was not > > executed by the streaming. The myMapper.sh has something like this: > > > > java myJava arguments > > > > And in the streaming command, I use something like this: > > > > hadoop jar /opt/hadoop/hadoop-0.20.2-streaming.jar -D > mapred.reduce.tasks=0 > > -input /user/input -output /user/output7 -mapper ./myMapper.sh -file > > myJava.class -verbose > > > > And the myJava program is not run when I execute like this, and if I go > to > > the actual slave node to check the files, the myMapper.sh is shipped to > the > > slave node, but the myJava.class is not, it is inside the job.jar file. > > > > Can someone provide some insights on how to run a java program through > > hadoop streaming? Thanks! > > > > Boyu > >
