Re: How to stop a MR Job when a certain condition is met?

2011-09-30 Thread Arun Murthy
Not sure what you are trying, or if MR is the right tool... Anyway, the basic idea is to write a map-only job with a custom RecordReader to stop emitting inputs to your map when the mapper/recordreader finds the needle in the haystack. Arun Sent from my iPhone On Sep 30, 2011, at 8:39 PM,

Re: How to stop a MR Job when a certain condition is met?

2011-09-30 Thread Praveen Sripati
Arun, Let's say there is 1PB of data in HDFS and the requirement is to look for a particular sequence in the whole data. If the sequence is found then the MR job could be aborted. There is no meaning in keeping the Map/Reduce tasks running. There was a query in one of the forums on how a MR job