[jira] [Updated] (MAPREDUCE-2470) Receiving NPE occasionally on RunningJob.getCounters() call
[ https://issues.apache.org/jira/browse/MAPREDUCE-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Douglas updated MAPREDUCE-2470: - Resolution: Fixed Status: Resolved (was: Patch Available) I committed the revised patch. Thanks, Robert! > Receiving NPE occasionally on RunningJob.getCounters() call > --- > > Key: MAPREDUCE-2470 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2470 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: client >Affects Versions: 0.21.0 > Environment: FreeBSD, Java6, Hadoop r0.21.0 >Reporter: Aaron Baff >Assignee: Robert Joseph Evans > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2470-v1.patch, MAPREDUCE-2470-v2.patch, > MAPREDUCE-2470-v3.patch, counters_null_data.pcap > > > This is running in a Java daemon that is used as an interface (Thrift) to get > information and data from MR Jobs. Using JobClient.getJob(JobID) I > successfully get a RunningJob object (I'm checking for NULL), and then rarely > I get an NPE when I do RunningJob.getCounters(). This seems to occur after > the daemon has been up and running for a while, and in the event of an > Exception, I close the JobClient, set it to NULL, and a new one should then > be created on the next request for data. Yet, I still seem to be unable to > fetch the Counters. Below is the stack trace. > java.lang.NullPointerException > at org.apache.hadoop.mapred.Counters.downgrade(Counters.java:77) > at > org.apache.hadoop.mapred.JobClient$NetworkedJob.getCounters(JobClient.java:381) > at > com.telescope.HadoopThrift.service.ServiceImpl.getReportResults(ServiceImpl.java:350) > at > com.telescope.HadoopThrift.gen.HadoopThrift$Processor$getReportResults.process(HadoopThrift.java:545) > at > com.telescope.HadoopThrift.gen.HadoopThrift$Processor.process(HadoopThrift.java:421) > at > org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:697) > at > org.apache.thrift.server.THsHaServer$Invocation.run(THsHaServer.java:317) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:619) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2470) Receiving NPE occasionally on RunningJob.getCounters() call
[ https://issues.apache.org/jira/browse/MAPREDUCE-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Joseph Evans updated MAPREDUCE-2470: --- Attachment: MAPREDUCE-2470-v3.patch Fixed issues with fault injection build. > Receiving NPE occasionally on RunningJob.getCounters() call > --- > > Key: MAPREDUCE-2470 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2470 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: client >Affects Versions: 0.21.0 > Environment: FreeBSD, Java6, Hadoop r0.21.0 >Reporter: Aaron Baff >Assignee: Robert Joseph Evans > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2470-v1.patch, MAPREDUCE-2470-v2.patch, > MAPREDUCE-2470-v3.patch, counters_null_data.pcap > > > This is running in a Java daemon that is used as an interface (Thrift) to get > information and data from MR Jobs. Using JobClient.getJob(JobID) I > successfully get a RunningJob object (I'm checking for NULL), and then rarely > I get an NPE when I do RunningJob.getCounters(). This seems to occur after > the daemon has been up and running for a while, and in the event of an > Exception, I close the JobClient, set it to NULL, and a new one should then > be created on the next request for data. Yet, I still seem to be unable to > fetch the Counters. Below is the stack trace. > java.lang.NullPointerException > at org.apache.hadoop.mapred.Counters.downgrade(Counters.java:77) > at > org.apache.hadoop.mapred.JobClient$NetworkedJob.getCounters(JobClient.java:381) > at > com.telescope.HadoopThrift.service.ServiceImpl.getReportResults(ServiceImpl.java:350) > at > com.telescope.HadoopThrift.gen.HadoopThrift$Processor$getReportResults.process(HadoopThrift.java:545) > at > com.telescope.HadoopThrift.gen.HadoopThrift$Processor.process(HadoopThrift.java:421) > at > org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:697) > at > org.apache.thrift.server.THsHaServer$Invocation.run(THsHaServer.java:317) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:619) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2470) Receiving NPE occasionally on RunningJob.getCounters() call
[ https://issues.apache.org/jira/browse/MAPREDUCE-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Joseph Evans updated MAPREDUCE-2470: --- Status: Patch Available (was: Reopened) > Receiving NPE occasionally on RunningJob.getCounters() call > --- > > Key: MAPREDUCE-2470 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2470 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: client >Affects Versions: 0.21.0 > Environment: FreeBSD, Java6, Hadoop r0.21.0 >Reporter: Aaron Baff >Assignee: Robert Joseph Evans > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2470-v1.patch, MAPREDUCE-2470-v2.patch, > MAPREDUCE-2470-v3.patch, counters_null_data.pcap > > > This is running in a Java daemon that is used as an interface (Thrift) to get > information and data from MR Jobs. Using JobClient.getJob(JobID) I > successfully get a RunningJob object (I'm checking for NULL), and then rarely > I get an NPE when I do RunningJob.getCounters(). This seems to occur after > the daemon has been up and running for a while, and in the event of an > Exception, I close the JobClient, set it to NULL, and a new one should then > be created on the next request for data. Yet, I still seem to be unable to > fetch the Counters. Below is the stack trace. > java.lang.NullPointerException > at org.apache.hadoop.mapred.Counters.downgrade(Counters.java:77) > at > org.apache.hadoop.mapred.JobClient$NetworkedJob.getCounters(JobClient.java:381) > at > com.telescope.HadoopThrift.service.ServiceImpl.getReportResults(ServiceImpl.java:350) > at > com.telescope.HadoopThrift.gen.HadoopThrift$Processor$getReportResults.process(HadoopThrift.java:545) > at > com.telescope.HadoopThrift.gen.HadoopThrift$Processor.process(HadoopThrift.java:421) > at > org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:697) > at > org.apache.thrift.server.THsHaServer$Invocation.run(THsHaServer.java:317) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:619) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2470) Receiving NPE occasionally on RunningJob.getCounters() call
[ https://issues.apache.org/jira/browse/MAPREDUCE-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Douglas updated MAPREDUCE-2470: - Resolution: Fixed Fix Version/s: 0.23.0 Hadoop Flags: [Reviewed] Status: Resolved (was: Patch Available) bq. The difference is that with Job.getStatus it calls internally updateStatus which will throw an IOException if it is unable to get an updated status from the JobTracker. The only time you can get an NPE from that is if you catch the IOException, ignore it, and keep trying to use the Job object what was originally created. Got it. Thanks for the explanation. +1 I committed this. Thanks! > Receiving NPE occasionally on RunningJob.getCounters() call > --- > > Key: MAPREDUCE-2470 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2470 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: client >Affects Versions: 0.21.0 > Environment: FreeBSD, Java6, Hadoop r0.21.0 >Reporter: Aaron Baff >Assignee: Robert Joseph Evans > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2470-v1.patch, MAPREDUCE-2470-v2.patch, > counters_null_data.pcap > > > This is running in a Java daemon that is used as an interface (Thrift) to get > information and data from MR Jobs. Using JobClient.getJob(JobID) I > successfully get a RunningJob object (I'm checking for NULL), and then rarely > I get an NPE when I do RunningJob.getCounters(). This seems to occur after > the daemon has been up and running for a while, and in the event of an > Exception, I close the JobClient, set it to NULL, and a new one should then > be created on the next request for data. Yet, I still seem to be unable to > fetch the Counters. Below is the stack trace. > java.lang.NullPointerException > at org.apache.hadoop.mapred.Counters.downgrade(Counters.java:77) > at > org.apache.hadoop.mapred.JobClient$NetworkedJob.getCounters(JobClient.java:381) > at > com.telescope.HadoopThrift.service.ServiceImpl.getReportResults(ServiceImpl.java:350) > at > com.telescope.HadoopThrift.gen.HadoopThrift$Processor$getReportResults.process(HadoopThrift.java:545) > at > com.telescope.HadoopThrift.gen.HadoopThrift$Processor.process(HadoopThrift.java:421) > at > org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:697) > at > org.apache.thrift.server.THsHaServer$Invocation.run(THsHaServer.java:317) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:619) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2470) Receiving NPE occasionally on RunningJob.getCounters() call
[ https://issues.apache.org/jira/browse/MAPREDUCE-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Joseph Evans updated MAPREDUCE-2470: --- Status: Open (was: Patch Available) Canceling the patch to address the comments to V1. Will post V2 shortly. > Receiving NPE occasionally on RunningJob.getCounters() call > --- > > Key: MAPREDUCE-2470 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2470 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: client >Affects Versions: 0.21.0 > Environment: FreeBSD, Java6, Hadoop r0.21.0 >Reporter: Aaron Baff >Assignee: Robert Joseph Evans > Attachments: MAPREDUCE-2470-v1.patch, MAPREDUCE-2470-v2.patch, > counters_null_data.pcap > > > This is running in a Java daemon that is used as an interface (Thrift) to get > information and data from MR Jobs. Using JobClient.getJob(JobID) I > successfully get a RunningJob object (I'm checking for NULL), and then rarely > I get an NPE when I do RunningJob.getCounters(). This seems to occur after > the daemon has been up and running for a while, and in the event of an > Exception, I close the JobClient, set it to NULL, and a new one should then > be created on the next request for data. Yet, I still seem to be unable to > fetch the Counters. Below is the stack trace. > java.lang.NullPointerException > at org.apache.hadoop.mapred.Counters.downgrade(Counters.java:77) > at > org.apache.hadoop.mapred.JobClient$NetworkedJob.getCounters(JobClient.java:381) > at > com.telescope.HadoopThrift.service.ServiceImpl.getReportResults(ServiceImpl.java:350) > at > com.telescope.HadoopThrift.gen.HadoopThrift$Processor$getReportResults.process(HadoopThrift.java:545) > at > com.telescope.HadoopThrift.gen.HadoopThrift$Processor.process(HadoopThrift.java:421) > at > org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:697) > at > org.apache.thrift.server.THsHaServer$Invocation.run(THsHaServer.java:317) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:619) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2470) Receiving NPE occasionally on RunningJob.getCounters() call
[ https://issues.apache.org/jira/browse/MAPREDUCE-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Joseph Evans updated MAPREDUCE-2470: --- Attachment: MAPREDUCE-2470-v2.patch > Receiving NPE occasionally on RunningJob.getCounters() call > --- > > Key: MAPREDUCE-2470 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2470 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: client >Affects Versions: 0.21.0 > Environment: FreeBSD, Java6, Hadoop r0.21.0 >Reporter: Aaron Baff >Assignee: Robert Joseph Evans > Attachments: MAPREDUCE-2470-v1.patch, MAPREDUCE-2470-v2.patch, > counters_null_data.pcap > > > This is running in a Java daemon that is used as an interface (Thrift) to get > information and data from MR Jobs. Using JobClient.getJob(JobID) I > successfully get a RunningJob object (I'm checking for NULL), and then rarely > I get an NPE when I do RunningJob.getCounters(). This seems to occur after > the daemon has been up and running for a while, and in the event of an > Exception, I close the JobClient, set it to NULL, and a new one should then > be created on the next request for data. Yet, I still seem to be unable to > fetch the Counters. Below is the stack trace. > java.lang.NullPointerException > at org.apache.hadoop.mapred.Counters.downgrade(Counters.java:77) > at > org.apache.hadoop.mapred.JobClient$NetworkedJob.getCounters(JobClient.java:381) > at > com.telescope.HadoopThrift.service.ServiceImpl.getReportResults(ServiceImpl.java:350) > at > com.telescope.HadoopThrift.gen.HadoopThrift$Processor$getReportResults.process(HadoopThrift.java:545) > at > com.telescope.HadoopThrift.gen.HadoopThrift$Processor.process(HadoopThrift.java:421) > at > org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:697) > at > org.apache.thrift.server.THsHaServer$Invocation.run(THsHaServer.java:317) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:619) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2470) Receiving NPE occasionally on RunningJob.getCounters() call
[ https://issues.apache.org/jira/browse/MAPREDUCE-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Joseph Evans updated MAPREDUCE-2470: --- Status: Patch Available (was: Open) > Receiving NPE occasionally on RunningJob.getCounters() call > --- > > Key: MAPREDUCE-2470 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2470 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: client >Affects Versions: 0.21.0 > Environment: FreeBSD, Java6, Hadoop r0.21.0 >Reporter: Aaron Baff >Assignee: Robert Joseph Evans > Attachments: MAPREDUCE-2470-v1.patch, MAPREDUCE-2470-v2.patch, > counters_null_data.pcap > > > This is running in a Java daemon that is used as an interface (Thrift) to get > information and data from MR Jobs. Using JobClient.getJob(JobID) I > successfully get a RunningJob object (I'm checking for NULL), and then rarely > I get an NPE when I do RunningJob.getCounters(). This seems to occur after > the daemon has been up and running for a while, and in the event of an > Exception, I close the JobClient, set it to NULL, and a new one should then > be created on the next request for data. Yet, I still seem to be unable to > fetch the Counters. Below is the stack trace. > java.lang.NullPointerException > at org.apache.hadoop.mapred.Counters.downgrade(Counters.java:77) > at > org.apache.hadoop.mapred.JobClient$NetworkedJob.getCounters(JobClient.java:381) > at > com.telescope.HadoopThrift.service.ServiceImpl.getReportResults(ServiceImpl.java:350) > at > com.telescope.HadoopThrift.gen.HadoopThrift$Processor$getReportResults.process(HadoopThrift.java:545) > at > com.telescope.HadoopThrift.gen.HadoopThrift$Processor.process(HadoopThrift.java:421) > at > org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:697) > at > org.apache.thrift.server.THsHaServer$Invocation.run(THsHaServer.java:317) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:619) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2470) Receiving NPE occasionally on RunningJob.getCounters() call
[ https://issues.apache.org/jira/browse/MAPREDUCE-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Joseph Evans updated MAPREDUCE-2470: --- Status: Patch Available (was: Open) > Receiving NPE occasionally on RunningJob.getCounters() call > --- > > Key: MAPREDUCE-2470 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2470 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: client >Affects Versions: 0.21.0 > Environment: FreeBSD, Java6, Hadoop r0.21.0 >Reporter: Aaron Baff >Assignee: Robert Joseph Evans > Attachments: MAPREDUCE-2470-v1.patch, counters_null_data.pcap > > > This is running in a Java daemon that is used as an interface (Thrift) to get > information and data from MR Jobs. Using JobClient.getJob(JobID) I > successfully get a RunningJob object (I'm checking for NULL), and then rarely > I get an NPE when I do RunningJob.getCounters(). This seems to occur after > the daemon has been up and running for a while, and in the event of an > Exception, I close the JobClient, set it to NULL, and a new one should then > be created on the next request for data. Yet, I still seem to be unable to > fetch the Counters. Below is the stack trace. > java.lang.NullPointerException > at org.apache.hadoop.mapred.Counters.downgrade(Counters.java:77) > at > org.apache.hadoop.mapred.JobClient$NetworkedJob.getCounters(JobClient.java:381) > at > com.telescope.HadoopThrift.service.ServiceImpl.getReportResults(ServiceImpl.java:350) > at > com.telescope.HadoopThrift.gen.HadoopThrift$Processor$getReportResults.process(HadoopThrift.java:545) > at > com.telescope.HadoopThrift.gen.HadoopThrift$Processor.process(HadoopThrift.java:421) > at > org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:697) > at > org.apache.thrift.server.THsHaServer$Invocation.run(THsHaServer.java:317) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:619) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2470) Receiving NPE occasionally on RunningJob.getCounters() call
[ https://issues.apache.org/jira/browse/MAPREDUCE-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Joseph Evans updated MAPREDUCE-2470: --- Status: Open (was: Patch Available) Looks like Jenkins did not see the patch, so trying again > Receiving NPE occasionally on RunningJob.getCounters() call > --- > > Key: MAPREDUCE-2470 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2470 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: client >Affects Versions: 0.21.0 > Environment: FreeBSD, Java6, Hadoop r0.21.0 >Reporter: Aaron Baff >Assignee: Robert Joseph Evans > Attachments: MAPREDUCE-2470-v1.patch, counters_null_data.pcap > > > This is running in a Java daemon that is used as an interface (Thrift) to get > information and data from MR Jobs. Using JobClient.getJob(JobID) I > successfully get a RunningJob object (I'm checking for NULL), and then rarely > I get an NPE when I do RunningJob.getCounters(). This seems to occur after > the daemon has been up and running for a while, and in the event of an > Exception, I close the JobClient, set it to NULL, and a new one should then > be created on the next request for data. Yet, I still seem to be unable to > fetch the Counters. Below is the stack trace. > java.lang.NullPointerException > at org.apache.hadoop.mapred.Counters.downgrade(Counters.java:77) > at > org.apache.hadoop.mapred.JobClient$NetworkedJob.getCounters(JobClient.java:381) > at > com.telescope.HadoopThrift.service.ServiceImpl.getReportResults(ServiceImpl.java:350) > at > com.telescope.HadoopThrift.gen.HadoopThrift$Processor$getReportResults.process(HadoopThrift.java:545) > at > com.telescope.HadoopThrift.gen.HadoopThrift$Processor.process(HadoopThrift.java:421) > at > org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:697) > at > org.apache.thrift.server.THsHaServer$Invocation.run(THsHaServer.java:317) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:619) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2470) Receiving NPE occasionally on RunningJob.getCounters() call
[ https://issues.apache.org/jira/browse/MAPREDUCE-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Joseph Evans updated MAPREDUCE-2470: --- Attachment: MAPREDUCE-2470-v1.patch Added Version 1 of the patch. It simply avoids the NPE by returning null from getCounters after the job has retired, just like the Job class does. > Receiving NPE occasionally on RunningJob.getCounters() call > --- > > Key: MAPREDUCE-2470 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2470 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: client >Affects Versions: 0.21.0 > Environment: FreeBSD, Java6, Hadoop r0.21.0 >Reporter: Aaron Baff >Assignee: Robert Joseph Evans > Attachments: MAPREDUCE-2470-v1.patch, counters_null_data.pcap > > > This is running in a Java daemon that is used as an interface (Thrift) to get > information and data from MR Jobs. Using JobClient.getJob(JobID) I > successfully get a RunningJob object (I'm checking for NULL), and then rarely > I get an NPE when I do RunningJob.getCounters(). This seems to occur after > the daemon has been up and running for a while, and in the event of an > Exception, I close the JobClient, set it to NULL, and a new one should then > be created on the next request for data. Yet, I still seem to be unable to > fetch the Counters. Below is the stack trace. > java.lang.NullPointerException > at org.apache.hadoop.mapred.Counters.downgrade(Counters.java:77) > at > org.apache.hadoop.mapred.JobClient$NetworkedJob.getCounters(JobClient.java:381) > at > com.telescope.HadoopThrift.service.ServiceImpl.getReportResults(ServiceImpl.java:350) > at > com.telescope.HadoopThrift.gen.HadoopThrift$Processor$getReportResults.process(HadoopThrift.java:545) > at > com.telescope.HadoopThrift.gen.HadoopThrift$Processor.process(HadoopThrift.java:421) > at > org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:697) > at > org.apache.thrift.server.THsHaServer$Invocation.run(THsHaServer.java:317) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:619) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2470) Receiving NPE occasionally on RunningJob.getCounters() call
[ https://issues.apache.org/jira/browse/MAPREDUCE-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Joseph Evans updated MAPREDUCE-2470: --- Status: Patch Available (was: Open) > Receiving NPE occasionally on RunningJob.getCounters() call > --- > > Key: MAPREDUCE-2470 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2470 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: client >Affects Versions: 0.21.0 > Environment: FreeBSD, Java6, Hadoop r0.21.0 >Reporter: Aaron Baff >Assignee: Robert Joseph Evans > Attachments: MAPREDUCE-2470-v1.patch, counters_null_data.pcap > > > This is running in a Java daemon that is used as an interface (Thrift) to get > information and data from MR Jobs. Using JobClient.getJob(JobID) I > successfully get a RunningJob object (I'm checking for NULL), and then rarely > I get an NPE when I do RunningJob.getCounters(). This seems to occur after > the daemon has been up and running for a while, and in the event of an > Exception, I close the JobClient, set it to NULL, and a new one should then > be created on the next request for data. Yet, I still seem to be unable to > fetch the Counters. Below is the stack trace. > java.lang.NullPointerException > at org.apache.hadoop.mapred.Counters.downgrade(Counters.java:77) > at > org.apache.hadoop.mapred.JobClient$NetworkedJob.getCounters(JobClient.java:381) > at > com.telescope.HadoopThrift.service.ServiceImpl.getReportResults(ServiceImpl.java:350) > at > com.telescope.HadoopThrift.gen.HadoopThrift$Processor$getReportResults.process(HadoopThrift.java:545) > at > com.telescope.HadoopThrift.gen.HadoopThrift$Processor.process(HadoopThrift.java:421) > at > org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:697) > at > org.apache.thrift.server.THsHaServer$Invocation.run(THsHaServer.java:317) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:619) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2470) Receiving NPE occasionally on RunningJob.getCounters() call
[ https://issues.apache.org/jira/browse/MAPREDUCE-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Baff updated MAPREDUCE-2470: -- Attachment: counters_null_data.pcap Wireshark capture of RunningJob.getCounters() call. > Receiving NPE occasionally on RunningJob.getCounters() call > --- > > Key: MAPREDUCE-2470 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2470 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: client >Affects Versions: 0.21.0 > Environment: FreeBSD, Java6, Hadoop r0.21.0 >Reporter: Aaron Baff > Attachments: counters_null_data.pcap > > > This is running in a Java daemon that is used as an interface (Thrift) to get > information and data from MR Jobs. Using JobClient.getJob(JobID) I > successfully get a RunningJob object (I'm checking for NULL), and then rarely > I get an NPE when I do RunningJob.getCounters(). This seems to occur after > the daemon has been up and running for a while, and in the event of an > Exception, I close the JobClient, set it to NULL, and a new one should then > be created on the next request for data. Yet, I still seem to be unable to > fetch the Counters. Below is the stack trace. > java.lang.NullPointerException > at org.apache.hadoop.mapred.Counters.downgrade(Counters.java:77) > at > org.apache.hadoop.mapred.JobClient$NetworkedJob.getCounters(JobClient.java:381) > at > com.telescope.HadoopThrift.service.ServiceImpl.getReportResults(ServiceImpl.java:350) > at > com.telescope.HadoopThrift.gen.HadoopThrift$Processor$getReportResults.process(HadoopThrift.java:545) > at > com.telescope.HadoopThrift.gen.HadoopThrift$Processor.process(HadoopThrift.java:421) > at > org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:697) > at > org.apache.thrift.server.THsHaServer$Invocation.run(THsHaServer.java:317) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:619) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
