[
https://issues.apache.org/jira/browse/MAPREDUCE-5870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14737035#comment-14737035
]
Jason Lowe commented on MAPREDUCE-5870:
---------------------------------------
bq. AppReport has the information about the priority of the application. So a
second call has to be made to fetch the same from YarnRunner#getJobStatus. And
it can be set to JobStatus separately.
The problem is that the ApplicationMaster doesn't actually know what its job
priority actually is, as that's a concept tracked by the RM not the AM. So to
report the priority we'd have to make a separate call to the RM to determine
the priority and meld that into the rest of the information returned for the
job status. I'm not thrilled with the thought of adding an additional call to
the RM for every time a client wants to get the job status, especially for a
field that most clients won't need. Clients often bang on the job status to
see when a job is done, and that means they'll start banging on the RM as well
if we make the delegate start hitting both the AM and the RM to generate the
JobStatus.
There's only two ways I can currently think of to get around that. Either we
need to obsolete the priority field from the job status and provide a separate
call to get the priority when the client really wants to know it, or we have to
find a way for the AM to know its job priority so it can return it in its
JobStatus responses. For the latter we could have the RM send it down in
heartbeat responses, but there would be a delay between when a client updates
the priority and the AM reports the updated value. We could have the AM
proactively ask the RM what its priority is, but then we're back to banging on
the RM for every job status call (only this time from AMs instead of the
client, but same number of RM calls overall).
Any other thoughts on what we should do here? I'm not thrilled about the
thought of ditching the priority in the JobStatus field. However it's been
worthless ever since we went to YARN (i.e.: broken since 0.23 and 2.x), so it's
tempting to declare it officially obsolete (as it in practice already is) and
provide a different API to retrieve it.
bq. And YARN throws excpetion for change priority application if its not
RUNNING or ACCEPTED.
Yes, this to me is a bug. We don't throw exceptions when clients try to kill
apps that have terminated, and similarly we shouldn't throw exceptions if
clients try to change the priority of terminated apps.
> Support for passing Job priority through Application Submission Context in
> Mapreduce Side
> -----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-5870
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5870
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: client
> Reporter: Sunil G
> Assignee: Sunil G
> Attachments: 0001-MAPREDUCE-5870.patch, 0002-MAPREDUCE-5870.patch,
> 0003-MAPREDUCE-5870.patch, 0004-MAPREDUCE-5870.patch,
> 0005-MAPREDUCE-5870.patch, 0006-MAPREDUCE-5870.patch, Yarn-2002.1.patch
>
>
> Job Prioirty can be set from client side as below [Configuration and api].
> a. JobConf.getJobPriority() and
> Job.setPriority(JobPriority priority)
> b. We can also use configuration
> "mapreduce.job.priority".
> Now this Job priority can be passed in Application Submission
> context from Client side.
> Here we can reuse the MRJobConfig.PRIORITY configuration.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
