[
https://issues.apache.org/jira/browse/MAPREDUCE-5870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14736993#comment-14736993
]
Sunil G commented on MAPREDUCE-5870:
------------------------------------
Thank you [~jlowe] for sharing the comments.
As discussed offline, I had doubts on couple of points here.
bq.We are already returning the job priority via getJobStatus
As I see, we are invoking {{YARNRunner#getJobStatus}} to get the job status
while we try to call {{Job#getPriority}}.
And in {{ClientServiceDelegate#getJobStatus}}, *JobStatus* is created from
*JobReport* (from AM).
{code}
public JobStatus getJobStatus(JobID oldJobID) throws IOException {
org.apache.hadoop.mapreduce.v2.api.records.JobId jobId =
TypeConverter.toYarn(oldJobID);
GetJobReportRequest request =
recordFactory.newRecordInstance(GetJobReportRequest.class);
request.setJobId(jobId);
JobReport report = ((GetJobReportResponse) invoke("getJobReport",
GetJobReportRequest.class, request)).getJobReport();
...
...
jobStatus = TypeConverter.fromYarn(report, url);
...
{code}
As this JobReport doesnt have priority information, current code in
{{TypeConverter#fromYarn}} is to hardcode the status.
{code}
public static JobStatus fromYarn(JobReport jobreport, String trackingUrl) {
JobPriority jobPriority = JobPriority.NORMAL;
JobStatus jobStatus = new org.apache.hadoop.mapred.JobStatus(
fromYarn(jobreport.getJobId()), jobreport.getSetupProgress(), jobreport
....
{code}
AppReport has the information about the priority of the application. So a
second call has to be made to fetch the same from {{YarnRunner#getJobStatus}}.
And it can be set to JobStatus separately.
Kindly help to share your thoughts here if I am missing something important.
bq.Why was the call to testChangingJobPriority removed from TestMRJobClient?
{{TestMRJobClient#testChangingJobPriority}} was invoked after job was
completed. And YARN throws excpetion for change priority application if its not
RUNNING or ACCEPTED. As updated offline, I agree that YARN need not have to
throw excpetion if the app is completed, rather it can be skipped. I will file
a ticket in YARN to fix the same. Thoughts?
For LocalJobRunner exception case, I will file another ticket to fix the
exception and to support correct priority.
Also I will add all possible test cases for Job and JobConf to see all possible
integer and enums are set/get correctly.
Thank you for the support.
> Support for passing Job priority through Application Submission Context in
> Mapreduce Side
> -----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-5870
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5870
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: client
> Reporter: Sunil G
> Assignee: Sunil G
> Attachments: 0001-MAPREDUCE-5870.patch, 0002-MAPREDUCE-5870.patch,
> 0003-MAPREDUCE-5870.patch, 0004-MAPREDUCE-5870.patch,
> 0005-MAPREDUCE-5870.patch, 0006-MAPREDUCE-5870.patch, Yarn-2002.1.patch
>
>
> Job Prioirty can be set from client side as below [Configuration and api].
> a. JobConf.getJobPriority() and
> Job.setPriority(JobPriority priority)
> b. We can also use configuration
> "mapreduce.job.priority".
> Now this Job priority can be passed in Application Submission
> context from Client side.
> Here we can reuse the MRJobConfig.PRIORITY configuration.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
