[jira] [Commented] (MAPREDUCE-6110) JobHistoryServer CLI throws NullPointerException with job ids that do not exist
[
https://issues.apache.org/jira/browse/MAPREDUCE-6110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15192145#comment-15192145
]
Li Lu commented on MAPREDUCE-6110:
--
Sure, will take a look at it soon... Sorry about the delay...
> JobHistoryServer CLI throws NullPointerException with job ids that do not
> exist
> ---
>
> Key: MAPREDUCE-6110
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6110
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Reporter: Li Lu
>Assignee: Kai Sasaki
>Priority: Minor
> Attachments: MAPREDUCE-6110.01.patch, MAPREDUCE-6110.02.patch
>
>
> When using JobHistoryServer CLI to query a job id that does not exist on the
> server, it may throw NullPointerException sometimes.
> I tried "mapred job -events 0 100", and the result was:
> Exception in thread "main" java.lang.NullPointerException
> at org.apache.hadoop.mapreduce.tools.CLI.listEvents(CLI.java:487)
> at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:316)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237)
> Similar symptoms also appear with -list-attempt-ids, but were fine with
> -status and -set-priority.
> I traced back to CLI.listEvents, and line 487 is:
> {code}
> TaskCompletionEvent[] events = job.
> getTaskCompletionEvents(fromEventId, numEvents);
> {code}
> The job object is obtained from JobID.forName(jobid)) (line 316), which will
> return null if the job does not exist on server.
> Maybe we want to have some behaviors consistent with -status here, by simply
> reporting jobId does not exist?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6110) JobHistoryServer CLI throws NullPointerException with job ids that do not exist
[
https://issues.apache.org/jira/browse/MAPREDUCE-6110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15192132#comment-15192132
]
Kai Sasaki commented on MAPREDUCE-6110:
---
[~gtCarrera] Could you review it when you have time?
> JobHistoryServer CLI throws NullPointerException with job ids that do not
> exist
> ---
>
> Key: MAPREDUCE-6110
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6110
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Reporter: Li Lu
>Assignee: Kai Sasaki
>Priority: Minor
> Attachments: MAPREDUCE-6110.01.patch, MAPREDUCE-6110.02.patch
>
>
> When using JobHistoryServer CLI to query a job id that does not exist on the
> server, it may throw NullPointerException sometimes.
> I tried "mapred job -events 0 100", and the result was:
> Exception in thread "main" java.lang.NullPointerException
> at org.apache.hadoop.mapreduce.tools.CLI.listEvents(CLI.java:487)
> at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:316)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237)
> Similar symptoms also appear with -list-attempt-ids, but were fine with
> -status and -set-priority.
> I traced back to CLI.listEvents, and line 487 is:
> {code}
> TaskCompletionEvent[] events = job.
> getTaskCompletionEvents(fromEventId, numEvents);
> {code}
> The job object is obtained from JobID.forName(jobid)) (line 316), which will
> return null if the job does not exist on server.
> Maybe we want to have some behaviors consistent with -status here, by simply
> reporting jobId does not exist?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-4522) DBOutputFormat Times out on large batch inserts
[ https://issues.apache.org/jira/browse/MAPREDUCE-4522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shyam Gavulla updated MAPREDUCE-4522: - Status: Open (was: Patch Available) Submitting for code review before submitting the patch > DBOutputFormat Times out on large batch inserts > --- > > Key: MAPREDUCE-4522 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4522 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: task-controller >Affects Versions: 0.20.205.0 >Reporter: Nathan Jarus >Assignee: Shyam Gavulla > Labels: newbie > Fix For: 3.0.0 > > Attachments: MAPREDUCE-4522.001.patch > > > In DBRecordWriter#close(), progress is never updated. In large batch inserts, > this can cause the reduce task to time out due to the amount of time it takes > the SQL engine to process that insert. > Potential solutions I can see: > Don't batch inserts; do the insert when DBRecordWriter#write() is called > (awful) > Spin up a thread in DBRecordWriter#close() and update progress in that. > (gross) > I can provide code for either if you're interested. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-4522) DBOutputFormat Times out on large batch inserts
[ https://issues.apache.org/jira/browse/MAPREDUCE-4522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shyam Gavulla updated MAPREDUCE-4522: - Attachment: MAPREDUCE-4522.001.patch > DBOutputFormat Times out on large batch inserts > --- > > Key: MAPREDUCE-4522 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4522 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: task-controller >Affects Versions: 0.20.205.0 >Reporter: Nathan Jarus >Assignee: Shyam Gavulla > Labels: newbie > Fix For: 3.0.0 > > Attachments: MAPREDUCE-4522.001.patch > > > In DBRecordWriter#close(), progress is never updated. In large batch inserts, > this can cause the reduce task to time out due to the amount of time it takes > the SQL engine to process that insert. > Potential solutions I can see: > Don't batch inserts; do the insert when DBRecordWriter#write() is called > (awful) > Spin up a thread in DBRecordWriter#close() and update progress in that. > (gross) > I can provide code for either if you're interested. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-4522) DBOutputFormat Times out on large batch inserts
[ https://issues.apache.org/jira/browse/MAPREDUCE-4522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shyam Gavulla updated MAPREDUCE-4522: - Fix Version/s: 3.0.0 Status: Patch Available (was: Open) I made the changes and applied the patch. Please review > DBOutputFormat Times out on large batch inserts > --- > > Key: MAPREDUCE-4522 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4522 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: task-controller >Affects Versions: 0.20.205.0 >Reporter: Nathan Jarus >Assignee: Shyam Gavulla > Labels: newbie > Fix For: 3.0.0 > > > In DBRecordWriter#close(), progress is never updated. In large batch inserts, > this can cause the reduce task to time out due to the amount of time it takes > the SQL engine to process that insert. > Potential solutions I can see: > Don't batch inserts; do the insert when DBRecordWriter#write() is called > (awful) > Spin up a thread in DBRecordWriter#close() and update progress in that. > (gross) > I can provide code for either if you're interested. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
