Ravi wrote:
Hi,
I have designed a mapreduce algorithm for all pairs shortest paths
problem. As a part of the implementation of this algorithm, I have
written the following mapreduce job. It is running well and producing
desired output in pseudo distributed mode. I have used a machine with
ubuntu 8.04 and hadoop-0.18.3 to run the job in pseudo distributed mode.
When I tried to run the same program on a cluster of 4 machines(each
running Redhat linux 9) with the same version of hadoop(hadooop-0.18.3),
the program is not giving any errors but its not giving any output as
well(The output file is blank). This is the first time I am facing this
kind of problem.
I am attaching the jar file of the program and sample inputs: out1 and
out2 as well.(The program need to read input from these two files)
I have searched the archive but didn't find any mail mentioning this
problem. I have googled, but it was of no use.
I am not able to find out what am I missing in the code.
First, the bad news: nobody is going to debug your program for you. It's
your program, you get to learn about distributed debugging. We all have
our own programs and their bugs to deal with, and if its a problem with
your physical cluster, then nobody but you are in a psition to fix.
Now, the good news: the skills you learn on this simple app scale well
to bigger clusters and more complex programs. Accordingly, it is
absolutely essential that you do learn this process now, while your
problem is still small.
http://wiki.apache.org/hadoop/HowToDebugMapReducePrograms
* Tell Hadoop to save all failed outputs with keep.failed.task.files
* See what gets retained on the worker machines, and their logs
* log at log4j's debug level in your code, run the nodes with your
classes set to log at debug level (leave the rest at info, for now).
Leave the logging settings in, they may come in useful later, just check
with log.isDebugEnabled() before constructing strings or other work to
create the log entries.
> Should I be using hadoop-0.20?
>
It's not likely to magically make your problem go away, if that is what
you were wondering.
