Hi Bobby, Thanks for the response.
After I tried the following comannd:
bin/hadoop jar $HADOOP_HOME/hadoop-0.20.2-streaming.jar -mapper
/data/yehdego/hadoop-0.20.2/pknotsRG-1.3/src/pknotsRG - -file
/data/yehdego/hadoop-0.20.2/pknotsRG-1.3/src/pknotsRG -reducer NONE -input
/user/yehdego/RNAData/RF00028_B.bpseqL3G5_seg_Centered_Method.txt -output
/user/yehdego/RF-out - verbose
I got a stderr logs :
java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed
with code 139
at
org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:311)
at
org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:545)
at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:132)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:36)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
syslog logs
2011-07-22 13:02:27,467 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
Initializing JVM Metrics with processName=MAP, sessionId=
2011-07-22 13:02:27,913 INFO org.apache.hadoop.mapred.MapTask: numReduceTasks: 0
2011-07-22 13:02:28,149 INFO org.apache.hadoop.streaming.PipeMapRed: PipeMapRed
exec
[/data/yehdego/hadoop_tmp/dfs/local/taskTracker/jobcache/job_201107181535_0079/attempt_201107181535_0079_m_000000_0/work/./pknotsRG]
2011-07-22 13:02:28,242 INFO org.apache.hadoop.streaming.PipeMapRed:
R/W/S=1/0/0 in:NA [rec/s] out:NA [rec/s]
2011-07-22 13:02:28,267 INFO org.apache.hadoop.streaming.PipeMapRed:
MROutputThread done
2011-07-22 13:02:28,267 INFO org.apache.hadoop.streaming.PipeMapRed:
MRErrorThread done
2011-07-22 13:02:28,267 INFO org.apache.hadoop.streaming.PipeMapRed: PipeMapRed
failed!
2011-07-22 13:02:28,361 WARN org.apache.hadoop.mapred.TaskTracker: Error
running child
java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed
with code 139
at
org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:311)
at
org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:545)
at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:132)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:36)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
2011-07-22 13:02:28,395 INFO org.apache.hadoop.mapred.TaskRunner: Runnning
cleanup for the task
Regards,
Daniel T. Yehdego
Computational Science Program
University of Texas at El Paso, UTEP
dtyehd...@miners.utep.edu
> From: ev...@yahoo-inc.com
> To: common-user@hadoop.apache.org; dtyehd...@miners.utep.edu
> Date: Fri, 22 Jul 2011 09:12:18 -0700
> Subject: Re: Hadoop-streaming using binary executable c program
>
> It looks like it tried to run your program and the program exited with a 1
> not a 0. What are the stderr logs like for the mappers that were launched,
> you should be able to access them through the Web GUI? You might want to add
> in some stderr log messages to you c program too. To be able to debug how far
> along it is going before exiting.
>
> --Bobby Evans
>
> On 7/22/11 9:19 AM, "Daniel Yehdego" <dtyehd...@miners.utep.edu> wrote:
>
> I am trying to parallelize some very long RNA sequence for the sake of
> predicting their RNA 2D structures. I am using a binary executable c
> program called pknotsRG as my mapper. I tried the following bin/hadoop
> command:
>
> HADOOP_HOME$ bin/hadoop
> jar /data/yehdego/hadoop-0.20.2/hadoop-0.20.2-streaming.jar
> -mapper /data/yehdego/hadoop-0.20.2/pknotsRG
> -file /data/yehdego/hadoop-0.20.2/pknotsRG
> -input /user/yehdego/RF00028_B.bpseqL3G5_seg_Centered_Method.txt
> -output /user/yehdego/RF-out -reducer NONE -verbose
>
> but i keep getting the following error message:
>
> java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess
> failed with code 1
> at
> org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:311)
> at
> org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:545)
> at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:132)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
> at
> org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:36)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
>
> FYI: my input file is RF00028_B.bpseqL3G5_seg_Centered_Method.txt which
> is a chunk of RNA sequences and the mapper is expected to get the input
> and execute the input file line by line and out put the predicted
> structure for each line of sequence for a specified number of maps. Any
> help on this problem is really appreciated. Thanks.
>
>
