[
https://issues.apache.org/jira/browse/MAPREDUCE-2836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13101266#comment-13101266
]
Todd Lipcon commented on MAPREDUCE-2836:
----------------------------------------
Hi Ahmed. Which branch are you looking at and testing on? The code I see in
JobTracker.java is:
{code}
synchronized (jobs) {
synchronized (taskScheduler) {
jobs.put(job.getProfile().getJobID(), job);
for (JobInProgressListener listener : jobInProgressListeners) {
try {
listener.jobAdded(job);
} catch (IOException ioe) {
LOG.warn("Failed to add and so skipping the job : "
+ job.getJobID() + ". Exception : " + ioe);
}
}
}
}
{code}
which would ignore the error.
and in FairScheduler.java:
{code}
eventLog.log("JOB_ADDED", job.getJobID());
JobInfo info = new JobInfo(new JobSchedulable(FairScheduler.this, job,
TaskType.MAP),
new JobSchedulable(FairScheduler.this, job, TaskType.REDUCE));
infos.put(job, info);
poolMgr.addJob(job); // Also adds job into the right PoolScheduable
update();
{code}
which would put into infos before calling poolMgr.
> Provide option to fail jobs when submitted to non-existent pools.
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-2836
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2836
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: contrib/fair-share
> Reporter: Jeff Bean
> Assignee: Ahmed Radwan
> Priority: Minor
> Attachments: MAPREDUCE-2836.patch, MAPREDUCE-2836_rev2.patch,
> MAPREDUCE-2836_rev3.patch, MAPREDUCE-2836_rev4.patch,
> MAPREDUCE-2836_rev5.patch
>
>
> In some environments, it might be desirable to explicitly specify the fair
> scheduler pools and to explicitly fail jobs that are not submitted to any of
> the pools.
> Current behavior of the fair scheduler is to submit jobs to a default pool if
> a pool name isn't specified or to create a pool with the new name if the pool
> name doesn't already exist. There should be a configuration option for the
> fair scheduler that causes it to noisily fail the job if it's submitted to a
> pool that isn't pre-specified or if the specified pool doesn't exist.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
