houman babai created OOZIE-3604:
-----------------------------------
Summary: Oozie Luncher & Map-Reduce Action Complete Successfully
However Oozie Still Fails the Action
Key: OOZIE-3604
URL: https://issues.apache.org/jira/browse/OOZIE-3604
Project: Oozie
Issue Type: Bug
Components: core
Affects Versions: 5.1.0
Environment: 5.1.0-cdh-6.3.1
5.1.0-cdh-6.3.2
Reporter: houman babai
I am using oozie 5.1.0-cdh6.3.1
In my workflow I have a mapreduce action, which generates over 300 counters.
The oozie launcher & the mapreduce job successfully complete, however, oozie
reports that:
{code:java}
Error Code: LimitExceededException
LimitExceededException: Too many counters: 121 max=120
{code}
I have updated mapred-site.xml.
The log for the *mapreduce job* reports success, in fact I can see all the
counters & the actual output of the mapreduce job on hdfs.
In the oozie *launcher log* I can see:
* mapreduce.job.counters.max : 8192
* mapreduce.job.counters.groups.max : 100
I set the oozie logs to TRACE, and I can see the following WARN:
{code:java}
WARN org.apache.oozie.command.wf.ActionEndXCommand: SERVER[SERVER-NAME]
USER[MY-NAME] GROUP[-] TOKEN[] APP[APP-NAME]
JOB[0000012-200714223028181-oozie-oozi-W]
ACTION[0000012-200714223028181-oozie-oozi-W@ACTION-NAME] Error ending action
[ACTION-NAME]. ErrorType [ERROR], ErrorCode [LimitExceededException], Message
[LimitExceededException: Too many counters: 121 max=120]
WARN org.apache.oozie.command.wf.ActionEndXCommand: SERVER[SERVER-NAME]
USER[MY-NAME] GROUP[-] TOKEN[] APP[APP-NAME]
JOB[0000012-200714223028181-oozie-oozi-W]
ACTION[0000012-200714223028181-oozie-oozi-W@ACTION-NAME] Setting Action Status
to [ERROR]
{code}
Looking through the oozie source code, I think the issue may be traced to [line
in
MapReduceActionExecutor|https://github.com/apache/oozie/blob/a40ab5361372aa73c9e4aa386a9c81bd21742aa4/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java#L250]
I am guessing that for some reason, the context or the config passed into the
[createJobClient|https://github.com/apache/oozie/blob/a40ab5361372aa73c9e4aa386a9c81bd21742aa4/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java#L242]
lack the mapreduce.job.counters.max configuration.
I tried several other things hoping that it would somehow work:
* I added the mapreduce.job.counters.max as configuration into the action
(inside the workflow)
* I added the mapreduce.job.counters.max to action config default.xml
Nothing has helped.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
