[jira] [Commented] (OOZIE-2818) Can't overwrite oozie.action.max.output.data on a per-workflow basis
[
https://issues.apache.org/jira/browse/OOZIE-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15974478#comment-15974478
]
Peter Bacsko commented on OOZIE-2818:
-
Thanks [~asasvari], committed to master
> Can't overwrite oozie.action.max.output.data on a per-workflow basis
>
>
> Key: OOZIE-2818
> URL: https://issues.apache.org/jira/browse/OOZIE-2818
> Project: Oozie
> Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Fix For: 5.0.0
>
> Attachments: OOZIE-2818-01.patch
>
>
> It's not possible to raise the limit of the output on a per-action basis.
> With the workflow:
> {quote}
>
>
>
> Action failed, error
> message\[$\{wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> $\{jobTracker}
> $\{nameNode}
>
>
> oozie.action.max.output.data
> 10
>
>
> oozie.launcher.oozie.action.max.output.data
> 10
>
>
> /user/admin/large_output.sh
> 10
> /user/admin/large_output.sh#large_output.sh
>
>
>
>
>
>
>
> {quote}
> In the localized files in the job.xml the value of this property gets
> overwritten:
> {quote}
> # cat
> /yarn/nm/usercache/admin/appcache/application_1489145190857_0001/container_1489145190857_0001_01_02/job.xml
> |grep max.outp
> oozie.launcher.oozie.action.max.output.data10programaticallyjob.xml
> oozie.action.max.output.data2048programaticallyjob.xml
> # cat
> /yarn/nm/usercache/admin/appcache/application_1489145190857_0001/container_1489145190857_0001_01_02/action.xml
> |grep max.outp
> oozie.launcher.oozie.action.max.output.data10programatically
> oozie.action.max.output.data10programatically
> {quote}
> I could not find a reason for this behavior, so I think it's a bug.
> The value is read from
> [ConfigurationService|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L175]
> and set in
> [createLauncherConf|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L986]
> in JavaActionExecutor. There could be a check to see if it's already set.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2818) Can't overwrite oozie.action.max.output.data on a per-workflow basis
[
https://issues.apache.org/jira/browse/OOZIE-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15974461#comment-15974461
]
Peter Bacsko commented on OOZIE-2818:
-
+1 LGTM
> Can't overwrite oozie.action.max.output.data on a per-workflow basis
>
>
> Key: OOZIE-2818
> URL: https://issues.apache.org/jira/browse/OOZIE-2818
> Project: Oozie
> Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2818-01.patch
>
>
> It's not possible to raise the limit of the output on a per-action basis.
> With the workflow:
> {quote}
>
>
>
> Action failed, error
> message\[$\{wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> $\{jobTracker}
> $\{nameNode}
>
>
> oozie.action.max.output.data
> 10
>
>
> oozie.launcher.oozie.action.max.output.data
> 10
>
>
> /user/admin/large_output.sh
> 10
> /user/admin/large_output.sh#large_output.sh
>
>
>
>
>
>
>
> {quote}
> In the localized files in the job.xml the value of this property gets
> overwritten:
> {quote}
> # cat
> /yarn/nm/usercache/admin/appcache/application_1489145190857_0001/container_1489145190857_0001_01_02/job.xml
> |grep max.outp
> oozie.launcher.oozie.action.max.output.data10programaticallyjob.xml
> oozie.action.max.output.data2048programaticallyjob.xml
> # cat
> /yarn/nm/usercache/admin/appcache/application_1489145190857_0001/container_1489145190857_0001_01_02/action.xml
> |grep max.outp
> oozie.launcher.oozie.action.max.output.data10programatically
> oozie.action.max.output.data10programatically
> {quote}
> I could not find a reason for this behavior, so I think it's a bug.
> The value is read from
> [ConfigurationService|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L175]
> and set in
> [createLauncherConf|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L986]
> in JavaActionExecutor. There could be a check to see if it's already set.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2818) Can't overwrite oozie.action.max.output.data on a per-workflow basis
[
https://issues.apache.org/jira/browse/OOZIE-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15972196#comment-15972196
]
Attila Sasvari commented on OOZIE-2818:
---
[~pbacsko] could you review the patch set? thanks
> Can't overwrite oozie.action.max.output.data on a per-workflow basis
>
>
> Key: OOZIE-2818
> URL: https://issues.apache.org/jira/browse/OOZIE-2818
> Project: Oozie
> Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2818-01.patch
>
>
> It's not possible to raise the limit of the output on a per-action basis.
> With the workflow:
> {quote}
>
>
>
> Action failed, error
> message\[$\{wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> $\{jobTracker}
> $\{nameNode}
>
>
> oozie.action.max.output.data
> 10
>
>
> oozie.launcher.oozie.action.max.output.data
> 10
>
>
> /user/admin/large_output.sh
> 10
> /user/admin/large_output.sh#large_output.sh
>
>
>
>
>
>
>
> {quote}
> In the localized files in the job.xml the value of this property gets
> overwritten:
> {quote}
> # cat
> /yarn/nm/usercache/admin/appcache/application_1489145190857_0001/container_1489145190857_0001_01_02/job.xml
> |grep max.outp
> oozie.launcher.oozie.action.max.output.data10programaticallyjob.xml
> oozie.action.max.output.data2048programaticallyjob.xml
> # cat
> /yarn/nm/usercache/admin/appcache/application_1489145190857_0001/container_1489145190857_0001_01_02/action.xml
> |grep max.outp
> oozie.launcher.oozie.action.max.output.data10programatically
> oozie.action.max.output.data10programatically
> {quote}
> I could not find a reason for this behavior, so I think it's a bug.
> The value is read from
> [ConfigurationService|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L175]
> and set in
> [createLauncherConf|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L986]
> in JavaActionExecutor. There could be a check to see if it's already set.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2818) Can't overwrite oozie.action.max.output.data on a per-workflow basis
[
https://issues.apache.org/jira/browse/OOZIE-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15950740#comment-15950740
]
Attila Sasvari commented on OOZIE-2818:
---
[~abhishekbafna] can you take a look?
> Can't overwrite oozie.action.max.output.data on a per-workflow basis
>
>
> Key: OOZIE-2818
> URL: https://issues.apache.org/jira/browse/OOZIE-2818
> Project: Oozie
> Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2818-01.patch
>
>
> It's not possible to raise the limit of the output on a per-action basis.
> With the workflow:
> {quote}
>
>
>
> Action failed, error
> message\[$\{wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> $\{jobTracker}
> $\{nameNode}
>
>
> oozie.action.max.output.data
> 10
>
>
> oozie.launcher.oozie.action.max.output.data
> 10
>
>
> /user/admin/large_output.sh
> 10
> /user/admin/large_output.sh#large_output.sh
>
>
>
>
>
>
>
> {quote}
> In the localized files in the job.xml the value of this property gets
> overwritten:
> {quote}
> # cat
> /yarn/nm/usercache/admin/appcache/application_1489145190857_0001/container_1489145190857_0001_01_02/job.xml
> |grep max.outp
> oozie.launcher.oozie.action.max.output.data10programaticallyjob.xml
> oozie.action.max.output.data2048programaticallyjob.xml
> # cat
> /yarn/nm/usercache/admin/appcache/application_1489145190857_0001/container_1489145190857_0001_01_02/action.xml
> |grep max.outp
> oozie.launcher.oozie.action.max.output.data10programatically
> oozie.action.max.output.data10programatically
> {quote}
> I could not find a reason for this behavior, so I think it's a bug.
> The value is read from
> [ConfigurationService|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L175]
> and set in
> [createLauncherConf|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L986]
> in JavaActionExecutor. There could be a check to see if it's already set.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2818) Can't overwrite oozie.action.max.output.data on a per-workflow basis
[
https://issues.apache.org/jira/browse/OOZIE-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15946957#comment-15946957
]
Hadoop QA commented on OOZIE-2818:
--
Testing JIRA OOZIE-2818
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in
[hadooplibs/hadoop-utils-2].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [examples].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1908
.Tests rerun: 46
.Tests failed at first run:
org.apache.oozie.service.TestStatusTransitService,org.apache.oozie.command.wf.TestWorkflowActionRetryInfoXCommand,org.apache.oozie.util.TestMetricsInstrumentation,
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:green}*+1 Overall result, good!, no -1s*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/3747/
> Can't overwrite oozie.action.max.output.data on a per-workflow basis
>
>
> Key: OOZIE-2818
> URL: https://issues.apache.org/jira/browse/OOZIE-2818
> Project: Oozie
> Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2818-01.patch
>
>
> It's not possible to raise the limit of the output on a per-action basis.
> With the workflow:
> {quote}
>
>
>
> Action failed, error
> message\[$\{wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> $\{jobTracker}
> $\{nameNode}
>
>
> oozie.action.max.output.data
> 10
>
>
> oozie.launcher.oozie.action.max.output.data
> 10
>
>
> /user/admin/large_output.sh
> 10
> /user/admin/large_output.sh#large_output.sh
>
>
>
>
>
>
>
> {quote}
> In the localized files in the job.xml the value of this property gets
> overwritten:
> {quote}
> # cat
> /yarn/nm/usercache/admin/appcache/application_1489145190857_0001/container_1489145190857_0001_01_02/job.xml
> |grep max.outp
> oozie.launcher.oozie.action.max.output.data10programaticallyjob.xml
> oozie.action.max.output.data2048programaticallyjob.xml
> # cat
> /yarn/nm/usercache/admin/appcache/application_1489145190857_0001/container_1489145190857_0001_01_02/action.xml
> |grep max.outp
> oozie.launcher.oozie.action.max.output.data10programatically
[jira] [Commented] (OOZIE-2818) Can't overwrite oozie.action.max.output.data on a per-workflow basis
[
https://issues.apache.org/jira/browse/OOZIE-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15946789#comment-15946789
]
Attila Sasvari commented on OOZIE-2818:
---
[~gezapeti] can you take a look?
> Can't overwrite oozie.action.max.output.data on a per-workflow basis
>
>
> Key: OOZIE-2818
> URL: https://issues.apache.org/jira/browse/OOZIE-2818
> Project: Oozie
> Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2818-01.patch
>
>
> It's not possible to raise the limit of the output on a per-action basis.
> With the workflow:
> {quote}
>
>
>
> Action failed, error
> message\[$\{wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> $\{jobTracker}
> $\{nameNode}
>
>
> oozie.action.max.output.data
> 10
>
>
> oozie.launcher.oozie.action.max.output.data
> 10
>
>
> /user/admin/large_output.sh
> 10
> /user/admin/large_output.sh#large_output.sh
>
>
>
>
>
>
>
> {quote}
> In the localized files in the job.xml the value of this property gets
> overwritten:
> {quote}
> # cat
> /yarn/nm/usercache/admin/appcache/application_1489145190857_0001/container_1489145190857_0001_01_02/job.xml
> |grep max.outp
> oozie.launcher.oozie.action.max.output.data10programaticallyjob.xml
> oozie.action.max.output.data2048programaticallyjob.xml
> # cat
> /yarn/nm/usercache/admin/appcache/application_1489145190857_0001/container_1489145190857_0001_01_02/action.xml
> |grep max.outp
> oozie.launcher.oozie.action.max.output.data10programatically
> oozie.action.max.output.data10programatically
> {quote}
> I could not find a reason for this behavior, so I think it's a bug.
> The value is read from
> [ConfigurationService|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L175]
> and set in
> [createLauncherConf|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L986]
> in JavaActionExecutor. There could be a check to see if it's already set.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2818) Can't overwrite oozie.action.max.output.data on a per-workflow basis
[
https://issues.apache.org/jira/browse/OOZIE-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905032#comment-15905032
]
Matyas Orhidi commented on OOZIE-2818:
--
This did not work either:
{code}
oozie.action.max.output.data
2
oozie.launcher.oozie.action.max.output.data
2
Action failed, error
message[${wf:errorMessage(wf:lastErrorNode())}]
${jobTracker}
${nameNode}
/user/admin/large_output.sh
10
/user/admin/large_output.sh#large_output.sh
{code}
> Can't overwrite oozie.action.max.output.data on a per-workflow basis
>
>
> Key: OOZIE-2818
> URL: https://issues.apache.org/jira/browse/OOZIE-2818
> Project: Oozie
> Issue Type: Bug
>Reporter: Peter Cseh
>
> It's not possible to raise the limit of the output on a per-action basis.
> With the workflow:
> {quote}
>
>
>
> Action failed, error
> message\[$\{wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> $\{jobTracker}
> $\{nameNode}
>
>
> oozie.action.max.output.data
> 10
>
>
> oozie.launcher.oozie.action.max.output.data
> 10
>
>
> /user/admin/large_output.sh
> 10
> /user/admin/large_output.sh#large_output.sh
>
>
>
>
>
>
>
> {quote}
> In the localized files in the job.xml the value of this property gets
> overwritten:
> {quote}
> # cat
> /yarn/nm/usercache/admin/appcache/application_1489145190857_0001/container_1489145190857_0001_01_02/job.xml
> |grep max.outp
> oozie.launcher.oozie.action.max.output.data10programaticallyjob.xml
> oozie.action.max.output.data2048programaticallyjob.xml
> # cat
> /yarn/nm/usercache/admin/appcache/application_1489145190857_0001/container_1489145190857_0001_01_02/action.xml
> |grep max.outp
> oozie.launcher.oozie.action.max.output.data10programatically
> oozie.action.max.output.data10programatically
> {quote}
> I could not find a reason for this behavior, so I think it's a bug.
> The value is read from
> [ConfigurationService|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L175]
> and set in
> [createLauncherConf|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L986]
> in JavaActionExecutor. There could be a check to see if it's already set.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
