Re: How to call method after all map jobs on slaves nodes are done

is your job a single map job without any reduce? if it is ,i think you could set the num of reduce to 0 then the map intermediate data will directly output to hdfs from local. 2009/11/14, Hrishikesh Agashe : > Hi, > > I am implementing the MapRunnable interface to create the Map jobs. > I have la

Re: How to call method after all map jobs on slaves nodes are done

I don't know anything about MapRunnable but this would be pretty easy to do with a bash script. All you do it list out your bash commands in a text file and run that text file... http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO.html It sounds like you're going to want to do something like... #!/bin/

How to call method after all map jobs on slaves nodes are done

Hi, I am implementing the MapRunnable interface to create the Map jobs. I have large data set for processing. (Data size is around 10 GB). I have 1 master and 10 slaves cluster. When I run my program, hadoop will process data successfully. After processing, I am collecting all data (all are files