[jira] [Commented] (OOZIE-3578) MapReduce counters cannot be used over 120
[
https://issues.apache.org/jira/browse/OOZIE-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010649#comment-17010649
]
ASF subversion and git services commented on OOZIE-3578:
Commit f0af3a01c68e373cba3122ac7a005b987d84c729 in oozie's branch
refs/heads/master from Gezapeti Cseh
[ https://gitbox.apache.org/repos/asf?p=oozie.git;h=f0af3a0 ]
OOZIE-3578 MapReduce counters cannot be used over 120 (dionusos via pbacsko,
gezapeti)
> MapReduce counters cannot be used over 120
> --
>
> Key: OOZIE-3578
> URL: https://issues.apache.org/jira/browse/OOZIE-3578
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.1.0
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Attachments: OOZIE-3578-001.patch, OOZIE-3578-002.patch,
> OOZIE-3578-003.patch
>
>
> When we create a mapreduce action which then creates more than 120 counters
> then the following exception is thrown:
> {noformat}
> org.apache.hadoop.mapreduce.counters.Limits.checkCounters(Limits.java:101)
> org.apache.hadoop.mapreduce.counters.Limits.incrCounters(Limits.java:108)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.addCounter(AbstractCounterGroup.java:78)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.addCounterImpl(AbstractCounterGroup.java:95)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounterImpl(AbstractCounterGroup.java:123)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounter(AbstractCounterGroup.java:113)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounter(AbstractCounterGroup.java:130)
> org.apache.hadoop.mapreduce.counters.AbstractCounters.findCounter(AbstractCounters.java:155)
> org.apache.hadoop.mapreduce.TypeConverter.fromYarn(TypeConverter.java:264)
> org.apache.hadoop.mapred.ClientServiceDelegate.getJobCounters(ClientServiceDelegate.java:383)
> org.apache.hadoop.mapred.YARNRunner.getJobCounters(YARNRunner.java:859)
> org.apache.hadoop.mapreduce.Job$8.run(Job.java:820)
> org.apache.hadoop.mapreduce.Job$8.run(Job.java:817)
> java.security.AccessController.doPrivileged(Native Method)
> javax.security.auth.Subject.doAs(Subject.java:422)
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
> org.apache.hadoop.mapreduce.Job.getCounters(Job.java:817)
> org.apache.hadoop.mapred.JobClient$NetworkedJob.getCounters(JobClient.java:379)
> org.apache.oozie.action.hadoop.MapReduceActionExecutor.end(MapReduceActionExecutor.java:252)
> org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.java:183)
> org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.java:62)
> org.apache.oozie.command.XCommand.call(XCommand.java:291)
> org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.java:244)
> org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.java:56)
> org.apache.oozie.command.XCommand.call(XCommand.java:291)
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:210)
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> java.lang.Thread.run(Thread.java:748)
> {noformat}
> It turned out if we use Oozie with Hadoop 3 the MR class called {{Limits}} is
> not initialised properly but with default values:
> https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/Limits.java#L40
> If we set the "mapreduce.job.counters.max" to 500 in mapred-site.xml or in
> core-site.xml has no positive effect.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3578) MapReduce counters cannot be used over 120
[
https://issues.apache.org/jira/browse/OOZIE-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010648#comment-17010648
]
Gézapeti commented on OOZIE-3578:
-
+1
committed to master!
Thanks for the contribution [~dionusos]!
> MapReduce counters cannot be used over 120
> --
>
> Key: OOZIE-3578
> URL: https://issues.apache.org/jira/browse/OOZIE-3578
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.1.0
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Attachments: OOZIE-3578-001.patch, OOZIE-3578-002.patch,
> OOZIE-3578-003.patch
>
>
> When we create a mapreduce action which then creates more than 120 counters
> then the following exception is thrown:
> {noformat}
> org.apache.hadoop.mapreduce.counters.Limits.checkCounters(Limits.java:101)
> org.apache.hadoop.mapreduce.counters.Limits.incrCounters(Limits.java:108)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.addCounter(AbstractCounterGroup.java:78)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.addCounterImpl(AbstractCounterGroup.java:95)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounterImpl(AbstractCounterGroup.java:123)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounter(AbstractCounterGroup.java:113)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounter(AbstractCounterGroup.java:130)
> org.apache.hadoop.mapreduce.counters.AbstractCounters.findCounter(AbstractCounters.java:155)
> org.apache.hadoop.mapreduce.TypeConverter.fromYarn(TypeConverter.java:264)
> org.apache.hadoop.mapred.ClientServiceDelegate.getJobCounters(ClientServiceDelegate.java:383)
> org.apache.hadoop.mapred.YARNRunner.getJobCounters(YARNRunner.java:859)
> org.apache.hadoop.mapreduce.Job$8.run(Job.java:820)
> org.apache.hadoop.mapreduce.Job$8.run(Job.java:817)
> java.security.AccessController.doPrivileged(Native Method)
> javax.security.auth.Subject.doAs(Subject.java:422)
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
> org.apache.hadoop.mapreduce.Job.getCounters(Job.java:817)
> org.apache.hadoop.mapred.JobClient$NetworkedJob.getCounters(JobClient.java:379)
> org.apache.oozie.action.hadoop.MapReduceActionExecutor.end(MapReduceActionExecutor.java:252)
> org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.java:183)
> org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.java:62)
> org.apache.oozie.command.XCommand.call(XCommand.java:291)
> org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.java:244)
> org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.java:56)
> org.apache.oozie.command.XCommand.call(XCommand.java:291)
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:210)
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> java.lang.Thread.run(Thread.java:748)
> {noformat}
> It turned out if we use Oozie with Hadoop 3 the MR class called {{Limits}} is
> not initialised properly but with default values:
> https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/Limits.java#L40
> If we set the "mapreduce.job.counters.max" to 500 in mapred-site.xml or in
> core-site.xml has no positive effect.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3578) MapReduce counters cannot be used over 120
[
https://issues.apache.org/jira/browse/OOZIE-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010636#comment-17010636
]
Hadoop QA commented on OOZIE-3578:
--
Testing JIRA OOZIE-3578
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any star imports
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} Javadoc generation succeeded with the patch
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1{color} There are [20] new bugs found below threshold in total
that must be fixed.
.{color:red}-1{color} There are [5] new bugs found below threshold in
[core] that must be fixed.
.You can find the SpotBugs diff here (look for the red and orange ones):
core/findbugs-new.html
.The most important SpotBugs errors are:
.At BulkJPAExecutor.java:[line 206]: This use of
javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query;
can be vulnerable to SQL/JPQL injection
.At BulkJPAExecutor.java:[line 176]: At BulkJPAExecutor.java:[line 175]
.At BulkJPAExecutor.java:[line 205]: At BulkJPAExecutor.java:[line 199]
.This use of
javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query;
can be vulnerable to SQL/JPQL injection: At BulkJPAExecutor.java:[line 206]
.At CoordJobGetActionsSubsetJPAExecutor.java:[line 76]: At
CoordJobGetActionsSubsetJPAExecutor.java:[line 111]
.{color:red}-1{color} There are [15] new bugs found below threshold in
[tools] that must be fixed, listing only the first [5] ones.
.You can find the SpotBugs diff here (look for the red and orange ones):
tools/findbugs-new.html
.The top [5] most important SpotBugs errors are:
.At OozieDBCLI.java:[line 584]: This use of
java/sql/Statement.executeUpdate(Ljava/lang/String;)I can be vulnerable to SQL
injection
.At OozieDBCLI.java:[line 574]: At OozieDBCLI.java:[line 573]
.At OozieDBCLI.java:[line 577]: At OozieDBCLI.java:[line 575]
.At OozieDBCLI.java:[line 579]: At OozieDBCLI.java:[line 578]
.At OozieDBCLI.java:[line 584]: At OozieDBCLI.java:[line 581]
.{color:green}+1{color} There are no new bugs found in [docs].
.{color:green}+1{color} There are no new bugs found in [sharelib/streaming].
.{color:green}+1{color} There are no new bugs found in [sharelib/spark].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive2].
.{color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
.{color:green}+1{color} There are no new bugs found in [sharelib/distcp].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive].
.{color:green}+1{color} There are no new bugs found in [sharelib/oozie].
.{color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
.{color:green}+1{color} There are no new bugs found in [sharelib/pig].
.{color:green}+1{color} There are no new bugs found in [sharelib/git].
.{color:green}+1{color} There are no new bugs found in [webapp].
.{color:green}+1{color} There are no new bugs found in [client].
.{color:green}+1{color} There are no new bugs found in [examples].
.{color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
.{color:orange}0{color} There are [4] new bugs found in [server] that would
be nice to have fixed.
.You can find the SpotBugs diff here: server/findbugs-new.html
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 3200
.{color:orange}Tests failed at first run:{color}
TestBlockingInputStream#testFastWritingBlockingInputStream
TestPriorityDelayQueue#testPeek
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:green}+1 MODERNIZER{color}
[jira] [Commented] (OOZIE-3578) MapReduce counters cannot be used over 120
[
https://issues.apache.org/jira/browse/OOZIE-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010556#comment-17010556
]
Hadoop QA commented on OOZIE-3578:
--
PreCommit-OOZIE-Build started
> MapReduce counters cannot be used over 120
> --
>
> Key: OOZIE-3578
> URL: https://issues.apache.org/jira/browse/OOZIE-3578
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.1.0
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Attachments: OOZIE-3578-001.patch, OOZIE-3578-002.patch,
> OOZIE-3578-003.patch
>
>
> When we create a mapreduce action which then creates more than 120 counters
> then the following exception is thrown:
> {noformat}
> org.apache.hadoop.mapreduce.counters.Limits.checkCounters(Limits.java:101)
> org.apache.hadoop.mapreduce.counters.Limits.incrCounters(Limits.java:108)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.addCounter(AbstractCounterGroup.java:78)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.addCounterImpl(AbstractCounterGroup.java:95)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounterImpl(AbstractCounterGroup.java:123)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounter(AbstractCounterGroup.java:113)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounter(AbstractCounterGroup.java:130)
> org.apache.hadoop.mapreduce.counters.AbstractCounters.findCounter(AbstractCounters.java:155)
> org.apache.hadoop.mapreduce.TypeConverter.fromYarn(TypeConverter.java:264)
> org.apache.hadoop.mapred.ClientServiceDelegate.getJobCounters(ClientServiceDelegate.java:383)
> org.apache.hadoop.mapred.YARNRunner.getJobCounters(YARNRunner.java:859)
> org.apache.hadoop.mapreduce.Job$8.run(Job.java:820)
> org.apache.hadoop.mapreduce.Job$8.run(Job.java:817)
> java.security.AccessController.doPrivileged(Native Method)
> javax.security.auth.Subject.doAs(Subject.java:422)
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
> org.apache.hadoop.mapreduce.Job.getCounters(Job.java:817)
> org.apache.hadoop.mapred.JobClient$NetworkedJob.getCounters(JobClient.java:379)
> org.apache.oozie.action.hadoop.MapReduceActionExecutor.end(MapReduceActionExecutor.java:252)
> org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.java:183)
> org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.java:62)
> org.apache.oozie.command.XCommand.call(XCommand.java:291)
> org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.java:244)
> org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.java:56)
> org.apache.oozie.command.XCommand.call(XCommand.java:291)
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:210)
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> java.lang.Thread.run(Thread.java:748)
> {noformat}
> It turned out if we use Oozie with Hadoop 3 the MR class called {{Limits}} is
> not initialised properly but with default values:
> https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/Limits.java#L40
> If we set the "mapreduce.job.counters.max" to 500 in mapred-site.xml or in
> core-site.xml has no positive effect.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3578) MapReduce counters cannot be used over 120
[
https://issues.apache.org/jira/browse/OOZIE-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010531#comment-17010531
]
Peter Bacsko commented on OOZIE-3578:
-
Thanks [~dionusos] IMO the patch is good.
Just a couple of nits:
# Instead of hardcoding ""mapreduce.job.counters.max", you can use
{{MRJobConfig.COUNTERS_MAX_KEY}}
# Instead of {{get("mapreduce.job.counters.max")}}, you can use {{getInt()}}
so you don't need {{Integer.parseInt()}}
> MapReduce counters cannot be used over 120
> --
>
> Key: OOZIE-3578
> URL: https://issues.apache.org/jira/browse/OOZIE-3578
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.1.0
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Attachments: OOZIE-3578-001.patch, OOZIE-3578-002.patch
>
>
> When we create a mapreduce action which then creates more than 120 counters
> then the following exception is thrown:
> {noformat}
> org.apache.hadoop.mapreduce.counters.Limits.checkCounters(Limits.java:101)
> org.apache.hadoop.mapreduce.counters.Limits.incrCounters(Limits.java:108)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.addCounter(AbstractCounterGroup.java:78)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.addCounterImpl(AbstractCounterGroup.java:95)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounterImpl(AbstractCounterGroup.java:123)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounter(AbstractCounterGroup.java:113)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounter(AbstractCounterGroup.java:130)
> org.apache.hadoop.mapreduce.counters.AbstractCounters.findCounter(AbstractCounters.java:155)
> org.apache.hadoop.mapreduce.TypeConverter.fromYarn(TypeConverter.java:264)
> org.apache.hadoop.mapred.ClientServiceDelegate.getJobCounters(ClientServiceDelegate.java:383)
> org.apache.hadoop.mapred.YARNRunner.getJobCounters(YARNRunner.java:859)
> org.apache.hadoop.mapreduce.Job$8.run(Job.java:820)
> org.apache.hadoop.mapreduce.Job$8.run(Job.java:817)
> java.security.AccessController.doPrivileged(Native Method)
> javax.security.auth.Subject.doAs(Subject.java:422)
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
> org.apache.hadoop.mapreduce.Job.getCounters(Job.java:817)
> org.apache.hadoop.mapred.JobClient$NetworkedJob.getCounters(JobClient.java:379)
> org.apache.oozie.action.hadoop.MapReduceActionExecutor.end(MapReduceActionExecutor.java:252)
> org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.java:183)
> org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.java:62)
> org.apache.oozie.command.XCommand.call(XCommand.java:291)
> org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.java:244)
> org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.java:56)
> org.apache.oozie.command.XCommand.call(XCommand.java:291)
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:210)
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> java.lang.Thread.run(Thread.java:748)
> {noformat}
> It turned out if we use Oozie with Hadoop 3 the MR class called {{Limits}} is
> not initialised properly but with default values:
> https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/Limits.java#L40
> If we set the "mapreduce.job.counters.max" to 500 in mapred-site.xml or in
> core-site.xml has no positive effect.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3578) MapReduce counters cannot be used over 120
[
https://issues.apache.org/jira/browse/OOZIE-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009826#comment-17009826
]
Hadoop QA commented on OOZIE-3578:
--
Testing JIRA OOZIE-3578
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any star imports
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} Javadoc generation succeeded with the patch
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1{color} There are [20] new bugs found below threshold in total
that must be fixed.
.{color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
.{color:green}+1{color} There are no new bugs found in [docs].
.{color:red}-1{color} There are [5] new bugs found below threshold in
[core] that must be fixed.
.You can find the SpotBugs diff here (look for the red and orange ones):
core/findbugs-new.html
.The most important SpotBugs errors are:
.At BulkJPAExecutor.java:[line 206]: This use of
javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query;
can be vulnerable to SQL/JPQL injection
.At BulkJPAExecutor.java:[line 176]: At BulkJPAExecutor.java:[line 175]
.At BulkJPAExecutor.java:[line 205]: At BulkJPAExecutor.java:[line 199]
.This use of
javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query;
can be vulnerable to SQL/JPQL injection: At BulkJPAExecutor.java:[line 206]
.At CoordJobGetActionsSubsetJPAExecutor.java:[line 76]: At
CoordJobGetActionsSubsetJPAExecutor.java:[line 111]
.{color:green}+1{color} There are no new bugs found in [sharelib/spark].
.{color:green}+1{color} There are no new bugs found in [sharelib/git].
.{color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive2].
.{color:green}+1{color} There are no new bugs found in [sharelib/streaming].
.{color:green}+1{color} There are no new bugs found in [sharelib/pig].
.{color:green}+1{color} There are no new bugs found in [sharelib/oozie].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive].
.{color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
.{color:green}+1{color} There are no new bugs found in [sharelib/distcp].
.{color:red}-1{color} There are [15] new bugs found below threshold in
[tools] that must be fixed, listing only the first [5] ones.
.You can find the SpotBugs diff here (look for the red and orange ones):
tools/findbugs-new.html
.The top [5] most important SpotBugs errors are:
.At OozieDBCLI.java:[line 584]: This use of
java/sql/Statement.executeUpdate(Ljava/lang/String;)I can be vulnerable to SQL
injection
.At OozieDBCLI.java:[line 574]: At OozieDBCLI.java:[line 573]
.At OozieDBCLI.java:[line 577]: At OozieDBCLI.java:[line 575]
.At OozieDBCLI.java:[line 579]: At OozieDBCLI.java:[line 578]
.At OozieDBCLI.java:[line 584]: At OozieDBCLI.java:[line 581]
.{color:orange}0{color} There are [4] new bugs found in [server] that would
be nice to have fixed.
.You can find the SpotBugs diff here: server/findbugs-new.html
.{color:green}+1{color} There are no new bugs found in [client].
.{color:green}+1{color} There are no new bugs found in [examples].
.{color:green}+1{color} There are no new bugs found in [webapp].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 3200
.{color:orange}Tests failed at first run:{color}
TestBlockingInputStream#testFailure
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:green}+1 MODERNIZER{color}
{color:red}*-1 Overall result, please check the reported
[jira] [Commented] (OOZIE-3578) MapReduce counters cannot be used over 120
[
https://issues.apache.org/jira/browse/OOZIE-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009727#comment-17009727
]
Hadoop QA commented on OOZIE-3578:
--
PreCommit-OOZIE-Build started
> MapReduce counters cannot be used over 120
> --
>
> Key: OOZIE-3578
> URL: https://issues.apache.org/jira/browse/OOZIE-3578
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.1.0
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Attachments: OOZIE-3578-001.patch, OOZIE-3578-002.patch
>
>
> When we create a mapreduce action which then creates more than 120 counters
> then the following exception is thrown:
> {noformat}
> org.apache.hadoop.mapreduce.counters.Limits.checkCounters(Limits.java:101)
> org.apache.hadoop.mapreduce.counters.Limits.incrCounters(Limits.java:108)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.addCounter(AbstractCounterGroup.java:78)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.addCounterImpl(AbstractCounterGroup.java:95)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounterImpl(AbstractCounterGroup.java:123)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounter(AbstractCounterGroup.java:113)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounter(AbstractCounterGroup.java:130)
> org.apache.hadoop.mapreduce.counters.AbstractCounters.findCounter(AbstractCounters.java:155)
> org.apache.hadoop.mapreduce.TypeConverter.fromYarn(TypeConverter.java:264)
> org.apache.hadoop.mapred.ClientServiceDelegate.getJobCounters(ClientServiceDelegate.java:383)
> org.apache.hadoop.mapred.YARNRunner.getJobCounters(YARNRunner.java:859)
> org.apache.hadoop.mapreduce.Job$8.run(Job.java:820)
> org.apache.hadoop.mapreduce.Job$8.run(Job.java:817)
> java.security.AccessController.doPrivileged(Native Method)
> javax.security.auth.Subject.doAs(Subject.java:422)
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
> org.apache.hadoop.mapreduce.Job.getCounters(Job.java:817)
> org.apache.hadoop.mapred.JobClient$NetworkedJob.getCounters(JobClient.java:379)
> org.apache.oozie.action.hadoop.MapReduceActionExecutor.end(MapReduceActionExecutor.java:252)
> org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.java:183)
> org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.java:62)
> org.apache.oozie.command.XCommand.call(XCommand.java:291)
> org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.java:244)
> org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.java:56)
> org.apache.oozie.command.XCommand.call(XCommand.java:291)
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:210)
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> java.lang.Thread.run(Thread.java:748)
> {noformat}
> It turned out if we use Oozie with Hadoop 3 the MR class called {{Limits}} is
> not initialised properly but with default values:
> https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/Limits.java#L40
> If we set the "mapreduce.job.counters.max" to 500 in mapred-site.xml or in
> core-site.xml has no positive effect.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3578) MapReduce counters cannot be used over 120
[
https://issues.apache.org/jira/browse/OOZIE-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009720#comment-17009720
]
Denes Bodo commented on OOZIE-3578:
---
Thanks [~asalamon74] for letting me know. I did run an incorrect git command.
Sorry.
> MapReduce counters cannot be used over 120
> --
>
> Key: OOZIE-3578
> URL: https://issues.apache.org/jira/browse/OOZIE-3578
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.1.0
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Attachments: OOZIE-3578-001.patch, OOZIE-3578-002.patch
>
>
> When we create a mapreduce action which then creates more than 120 counters
> then the following exception is thrown:
> {noformat}
> org.apache.hadoop.mapreduce.counters.Limits.checkCounters(Limits.java:101)
> org.apache.hadoop.mapreduce.counters.Limits.incrCounters(Limits.java:108)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.addCounter(AbstractCounterGroup.java:78)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.addCounterImpl(AbstractCounterGroup.java:95)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounterImpl(AbstractCounterGroup.java:123)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounter(AbstractCounterGroup.java:113)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounter(AbstractCounterGroup.java:130)
> org.apache.hadoop.mapreduce.counters.AbstractCounters.findCounter(AbstractCounters.java:155)
> org.apache.hadoop.mapreduce.TypeConverter.fromYarn(TypeConverter.java:264)
> org.apache.hadoop.mapred.ClientServiceDelegate.getJobCounters(ClientServiceDelegate.java:383)
> org.apache.hadoop.mapred.YARNRunner.getJobCounters(YARNRunner.java:859)
> org.apache.hadoop.mapreduce.Job$8.run(Job.java:820)
> org.apache.hadoop.mapreduce.Job$8.run(Job.java:817)
> java.security.AccessController.doPrivileged(Native Method)
> javax.security.auth.Subject.doAs(Subject.java:422)
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
> org.apache.hadoop.mapreduce.Job.getCounters(Job.java:817)
> org.apache.hadoop.mapred.JobClient$NetworkedJob.getCounters(JobClient.java:379)
> org.apache.oozie.action.hadoop.MapReduceActionExecutor.end(MapReduceActionExecutor.java:252)
> org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.java:183)
> org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.java:62)
> org.apache.oozie.command.XCommand.call(XCommand.java:291)
> org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.java:244)
> org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.java:56)
> org.apache.oozie.command.XCommand.call(XCommand.java:291)
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:210)
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> java.lang.Thread.run(Thread.java:748)
> {noformat}
> It turned out if we use Oozie with Hadoop 3 the MR class called {{Limits}} is
> not initialised properly but with default values:
> https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/Limits.java#L40
> If we set the "mapreduce.job.counters.max" to 500 in mapred-site.xml or in
> core-site.xml has no positive effect.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3578) MapReduce counters cannot be used over 120
[
https://issues.apache.org/jira/browse/OOZIE-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009700#comment-17009700
]
Hadoop QA commented on OOZIE-3578:
--
Testing JIRA OOZIE-3578
Cleaning local git workspace
{color:red}-1{color} Patch failed to apply to head of branch
> MapReduce counters cannot be used over 120
> --
>
> Key: OOZIE-3578
> URL: https://issues.apache.org/jira/browse/OOZIE-3578
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.1.0
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Attachments: OOZIE-3578-001.patch
>
>
> When we create a mapreduce action which then creates more than 120 counters
> then the following exception is thrown:
> {noformat}
> org.apache.hadoop.mapreduce.counters.Limits.checkCounters(Limits.java:101)
> org.apache.hadoop.mapreduce.counters.Limits.incrCounters(Limits.java:108)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.addCounter(AbstractCounterGroup.java:78)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.addCounterImpl(AbstractCounterGroup.java:95)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounterImpl(AbstractCounterGroup.java:123)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounter(AbstractCounterGroup.java:113)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounter(AbstractCounterGroup.java:130)
> org.apache.hadoop.mapreduce.counters.AbstractCounters.findCounter(AbstractCounters.java:155)
> org.apache.hadoop.mapreduce.TypeConverter.fromYarn(TypeConverter.java:264)
> org.apache.hadoop.mapred.ClientServiceDelegate.getJobCounters(ClientServiceDelegate.java:383)
> org.apache.hadoop.mapred.YARNRunner.getJobCounters(YARNRunner.java:859)
> org.apache.hadoop.mapreduce.Job$8.run(Job.java:820)
> org.apache.hadoop.mapreduce.Job$8.run(Job.java:817)
> java.security.AccessController.doPrivileged(Native Method)
> javax.security.auth.Subject.doAs(Subject.java:422)
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
> org.apache.hadoop.mapreduce.Job.getCounters(Job.java:817)
> org.apache.hadoop.mapred.JobClient$NetworkedJob.getCounters(JobClient.java:379)
> org.apache.oozie.action.hadoop.MapReduceActionExecutor.end(MapReduceActionExecutor.java:252)
> org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.java:183)
> org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.java:62)
> org.apache.oozie.command.XCommand.call(XCommand.java:291)
> org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.java:244)
> org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.java:56)
> org.apache.oozie.command.XCommand.call(XCommand.java:291)
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:210)
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> java.lang.Thread.run(Thread.java:748)
> {noformat}
> It turned out if we use Oozie with Hadoop 3 the MR class called {{Limits}} is
> not initialised properly but with default values:
> https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/Limits.java#L40
> If we set the "mapreduce.job.counters.max" to 500 in mapred-site.xml or in
> core-site.xml has no positive effect.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3578) MapReduce counters cannot be used over 120
[
https://issues.apache.org/jira/browse/OOZIE-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009696#comment-17009696
]
Andras Salamon commented on OOZIE-3578:
---
[~dionusos] Can you please check the attached patch, it seems to me it's the
patch for OOZIE-3573.
> MapReduce counters cannot be used over 120
> --
>
> Key: OOZIE-3578
> URL: https://issues.apache.org/jira/browse/OOZIE-3578
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.1.0
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Attachments: OOZIE-3578-001.patch
>
>
> When we create a mapreduce action which then creates more than 120 counters
> then the following exception is thrown:
> {noformat}
> org.apache.hadoop.mapreduce.counters.Limits.checkCounters(Limits.java:101)
> org.apache.hadoop.mapreduce.counters.Limits.incrCounters(Limits.java:108)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.addCounter(AbstractCounterGroup.java:78)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.addCounterImpl(AbstractCounterGroup.java:95)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounterImpl(AbstractCounterGroup.java:123)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounter(AbstractCounterGroup.java:113)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounter(AbstractCounterGroup.java:130)
> org.apache.hadoop.mapreduce.counters.AbstractCounters.findCounter(AbstractCounters.java:155)
> org.apache.hadoop.mapreduce.TypeConverter.fromYarn(TypeConverter.java:264)
> org.apache.hadoop.mapred.ClientServiceDelegate.getJobCounters(ClientServiceDelegate.java:383)
> org.apache.hadoop.mapred.YARNRunner.getJobCounters(YARNRunner.java:859)
> org.apache.hadoop.mapreduce.Job$8.run(Job.java:820)
> org.apache.hadoop.mapreduce.Job$8.run(Job.java:817)
> java.security.AccessController.doPrivileged(Native Method)
> javax.security.auth.Subject.doAs(Subject.java:422)
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
> org.apache.hadoop.mapreduce.Job.getCounters(Job.java:817)
> org.apache.hadoop.mapred.JobClient$NetworkedJob.getCounters(JobClient.java:379)
> org.apache.oozie.action.hadoop.MapReduceActionExecutor.end(MapReduceActionExecutor.java:252)
> org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.java:183)
> org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.java:62)
> org.apache.oozie.command.XCommand.call(XCommand.java:291)
> org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.java:244)
> org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.java:56)
> org.apache.oozie.command.XCommand.call(XCommand.java:291)
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:210)
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> java.lang.Thread.run(Thread.java:748)
> {noformat}
> It turned out if we use Oozie with Hadoop 3 the MR class called {{Limits}} is
> not initialised properly but with default values:
> https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/Limits.java#L40
> If we set the "mapreduce.job.counters.max" to 500 in mapred-site.xml or in
> core-site.xml has no positive effect.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3578) MapReduce counters cannot be used over 120
[
https://issues.apache.org/jira/browse/OOZIE-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009692#comment-17009692
]
Hadoop QA commented on OOZIE-3578:
--
PreCommit-OOZIE-Build started
> MapReduce counters cannot be used over 120
> --
>
> Key: OOZIE-3578
> URL: https://issues.apache.org/jira/browse/OOZIE-3578
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.1.0
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Attachments: OOZIE-3578-001.patch
>
>
> When we create a mapreduce action which then creates more than 120 counters
> then the following exception is thrown:
> {noformat}
> org.apache.hadoop.mapreduce.counters.Limits.checkCounters(Limits.java:101)
> org.apache.hadoop.mapreduce.counters.Limits.incrCounters(Limits.java:108)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.addCounter(AbstractCounterGroup.java:78)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.addCounterImpl(AbstractCounterGroup.java:95)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounterImpl(AbstractCounterGroup.java:123)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounter(AbstractCounterGroup.java:113)
> org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounter(AbstractCounterGroup.java:130)
> org.apache.hadoop.mapreduce.counters.AbstractCounters.findCounter(AbstractCounters.java:155)
> org.apache.hadoop.mapreduce.TypeConverter.fromYarn(TypeConverter.java:264)
> org.apache.hadoop.mapred.ClientServiceDelegate.getJobCounters(ClientServiceDelegate.java:383)
> org.apache.hadoop.mapred.YARNRunner.getJobCounters(YARNRunner.java:859)
> org.apache.hadoop.mapreduce.Job$8.run(Job.java:820)
> org.apache.hadoop.mapreduce.Job$8.run(Job.java:817)
> java.security.AccessController.doPrivileged(Native Method)
> javax.security.auth.Subject.doAs(Subject.java:422)
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
> org.apache.hadoop.mapreduce.Job.getCounters(Job.java:817)
> org.apache.hadoop.mapred.JobClient$NetworkedJob.getCounters(JobClient.java:379)
> org.apache.oozie.action.hadoop.MapReduceActionExecutor.end(MapReduceActionExecutor.java:252)
> org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.java:183)
> org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.java:62)
> org.apache.oozie.command.XCommand.call(XCommand.java:291)
> org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.java:244)
> org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.java:56)
> org.apache.oozie.command.XCommand.call(XCommand.java:291)
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:210)
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> java.lang.Thread.run(Thread.java:748)
> {noformat}
> It turned out if we use Oozie with Hadoop 3 the MR class called {{Limits}} is
> not initialised properly but with default values:
> https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/Limits.java#L40
> If we set the "mapreduce.job.counters.max" to 500 in mapred-site.xml or in
> core-site.xml has no positive effect.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
