Re: Dynamically terminate a job once Reporter hits a threshold

[Arun C Murthy]

On Nov 7, 2008, at 12:12 PM, Brian MacKay wrote:

Looking for a way to dynamically terminate a job once Reporter in a  
Map
job hits a threshold,

Example:

public void map(WritableComparable key, Text values, Output
Collector<Text, Text> output, Reporter reporter) throws IOException {

    if( reporter.getCount()  > SomeConfigValue) {
               return;
           }
         ....  map job code
       }

Obviously,  reporter.getCount() doesn't exist.  Open to other ideas,  
and
any advice would be appreciated.

If you _really_ need this, you could do this from your JobClient...  
use JobClient.submitJob (rather than runJob: http://hadoop.apache.org/core/docs/current/mapred_tutorial.html#Job+Submission+and+Monitoring) 
, manually fetch the Counters you need and terminate the Job via  
JobClient.getJob(jobId).killJob().

Arun