[ http://issues.apache.org/jira/browse/HADOOP-808?page=all ]
Arun C Murthy updated HADOOP-808:
---------------------------------
Attachment: HADOOP-808_1_20061208.patch
myjob.tgz
Here is a simple fix to JobClient.java
It handles the above deficiency by 'mapred.jar' in the jobconf, loads the
'mapred.job.class' (this class is assumed to extend
org.apache.hadoop.util.ToolBase') and calls it's 'doMain'; there-by providing a
hook for user code before the job is fired.
Hence with this one can completey 'configure' the map-reduce job (job-name,
key/value classes, output-directory, speculative-execution, no. of maps/reduces
etc.), specify the 'jar' with the mapper/reducer classes and fire the job. Thus
any change needed for 'configuration' is now done completely via the job.xml,
and shouldn't need be done via JobConf.set* methods.
With the attached myjob.jar/myjob.xml one can:
$ hadoop job -submit myjob.xml
to run a modified version of 'random-writer'... there-by illustrating it's
usefulness.
> Enhance 'bin/hadoop job -submit <>'
> -----------------------------------
>
> Key: HADOOP-808
> URL: http://issues.apache.org/jira/browse/HADOOP-808
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Reporter: Arun C Murthy
> Assigned To: Arun C Murthy
> Priority: Minor
> Attachments: HADOOP-808_1_20061208.patch, myjob.tgz
>
>
> Today 'bin/hadoop job -submit' takes a config file and just does a
> JobClient.submitJob(jobConf), this isn't flexible enough for cases where the
> user wants to submit a job which calls his main(), does some work and then
> calls 'JobClient.runJob', I propose we enhance this subcommand to handle that
> scenario. This could be really useful for the time we decide to setup a
> simple webpage with 'form' where the user uploads a job.xml & job.jar and we
> can fire the job.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
