[jira] Subscription: Oozie Patch Available
Issue Subscription Filter: Oozie Patch Available (105 issues) Subscriber: ooziedaily Key Summary OOZIE-3693 [examples] Fix current SpotBugs discovered issues in Oozie's examples module https://issues.apache.org/jira/browse/OOZIE-3693 OOZIE-3692 [sharelib-spark] Fix current SpotBugs discovered issues in Oozie's sharelib-spark module https://issues.apache.org/jira/browse/OOZIE-3692 OOZIE-3690 [server] Fix current SpotBugs discovered issues in Oozie's server module https://issues.apache.org/jira/browse/OOZIE-3690 OOZIE-3687 Fix Oozie client always using the current system username instead the one specified by the user (e.g.: via kerberos or via explicit basic authentication) https://issues.apache.org/jira/browse/OOZIE-3687 OOZIE-3680 Add default value to custom configuration of all the supported file systems in Oozie https://issues.apache.org/jira/browse/OOZIE-3680 OOZIE-3663 Upgrade Apache Xerces Java to 2.12.2 https://issues.apache.org/jira/browse/OOZIE-3663 OOZIE-3654 update to httpclient 4.5.13 https://issues.apache.org/jira/browse/OOZIE-3654 OOZIE-3635 Reduce nest of code in RecoveryService https://issues.apache.org/jira/browse/OOZIE-3635 OOZIE-3623 Fix Typos in Distro Assembly https://issues.apache.org/jira/browse/OOZIE-3623 OOZIE-3621 Make TestECPolicyDisabler work with Hadoop 3 https://issues.apache.org/jira/browse/OOZIE-3621 OOZIE-3620 hadoopId is not sent to eventHandlerService (listener) for workflow action events https://issues.apache.org/jira/browse/OOZIE-3620 OOZIE-3609 Zookeeper SSL/TLS support https://issues.apache.org/jira/browse/OOZIE-3609 OOZIE-3596 When the SSH action is killed, it must be changed to the kill command that can terminate the related subprocess. https://issues.apache.org/jira/browse/OOZIE-3596 OOZIE-3568 Have large amount of log information “WARN messages [main] openjpa.MetaData” in jetty.log need to clean https://issues.apache.org/jira/browse/OOZIE-3568 OOZIE-3567 Oozie ShellAction should support absolute bash file path https://issues.apache.org/jira/browse/OOZIE-3567 OOZIE-3560 IDEA shows have some error in index.jsp https://issues.apache.org/jira/browse/OOZIE-3560 OOZIE-3554 Add asf.yaml to git repo https://issues.apache.org/jira/browse/OOZIE-3554 OOZIE-3545 Upgrade jQuery https://issues.apache.org/jira/browse/OOZIE-3545 OOZIE-3482 Fix bug in CoordSubmitXCommand#validateCoordinatorJob https://issues.apache.org/jira/browse/OOZIE-3482 OOZIE-3480 Add windowactionstatus metrics in DBLiteWorkflowStoreService https://issues.apache.org/jira/browse/OOZIE-3480 OOZIE-3461 CoordMaterializeTriggerService code cleanup https://issues.apache.org/jira/browse/OOZIE-3461 OOZIE-3449 Make spark-2 as the default profile https://issues.apache.org/jira/browse/OOZIE-3449 OOZIE-3447 Run test case in local : It shows oozie-hsqldb-orm.xml exception https://issues.apache.org/jira/browse/OOZIE-3447 OOZIE-3434 Filtering for invalid jobtype should give error message https://issues.apache.org/jira/browse/OOZIE-3434 OOZIE-3418 Upgrade to Guava 27 https://issues.apache.org/jira/browse/OOZIE-3418 OOZIE-3404 The env variable of SPARK_HOME needs to be set when running pySpark https://issues.apache.org/jira/browse/OOZIE-3404 OOZIE-3375 Can't use empty in coordinator https://issues.apache.org/jira/browse/OOZIE-3375 OOZIE-3367 Using && in EL expressions in oozie bundle.xml files generates parse errors https://issues.apache.org/jira/browse/OOZIE-3367 OOZIE-3366 Update workflow status and subworkflow status on suspend command https://issues.apache.org/jira/browse/OOZIE-3366 OOZIE-3364 Rerunning Oozie bundle jobs starts the coordinators in indeterminate order https://issues.apache.org/jira/browse/OOZIE-3364 OOZIE-3362 When killed, SSH action should kill the spawned processes on target host https://issues.apache.org/jira/browse/OOZIE-3362 OOZIE-3335 Cleanup parseFilter methods https://issues.apache.org/jira/browse/OOZIE-3335 OOZIE-3328 Create Hive compatibility action executor to run hive actions using beeline https://issues.apache.org/jira/browse/OOZIE-3328 OOZIE-3319 Log SSH action callback error output https://issues.apache.org/jira/browse/OOZIE-3319 OOZIE-3301 Update NOTICE file https://issues.apache.org/jira/browse/OOZIE-3301 OOZIE-3274 Remove slf4j https://issues.apache.org/jira/browse/OOZIE-3274 OOZIE-3266 Coord action rerun support RERUN_SKIP_NODES option https://issues.apache.org/jira/browse/OOZIE-3266 OOZIE-3256 refactor OozieCLI class https://issues.apache.org/jira/browse/OOZIE-3256 OOZIE-3196
[jira] [Commented] (OOZIE-3690) [server] Fix current SpotBugs discovered issues in Oozie's server module
[
https://issues.apache.org/jira/browse/OOZIE-3690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655506#comment-17655506
]
Hadoop QA commented on OOZIE-3690:
--
Testing JIRA OOZIE-3690
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any star imports
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} Javadoc generation succeeded with the patch
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
.{color:green}+1{color} There are no new bugs found in [core].
.{color:green}+1{color} There are no new bugs found in [client].
.{color:green}+1{color} There are no new bugs found in [docs].
.{color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
.{color:green}+1{color} There are no new bugs found in [server].
.{color:green}+1{color} There are no new bugs found in [examples].
.{color:green}+1{color} There are no new bugs found in [tools].
.{color:green}+1{color} There are no new bugs found in [webapp].
.{color:green}+1{color} There are no new bugs found in [sharelib/distcp].
.{color:green}+1{color} There are no new bugs found in [sharelib/spark].
.{color:green}+1{color} There are no new bugs found in [sharelib/oozie].
.{color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
.{color:green}+1{color} There are no new bugs found in [sharelib/streaming].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive2].
.{color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive].
.{color:green}+1{color} There are no new bugs found in [sharelib/pig].
.{color:green}+1{color} There are no new bugs found in [sharelib/git].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 3260
.Tests failed : 0
.Tests in error : 1
.Tests timed out : 0
{color:red}-1{color} [ERROR] There are [1] test errors in [server]. Listing
only the first [5] ones
unnecessary Mockito stubbings:org.apache.oozie.server.TestJspHandler
Check console output for the full list of errors/failures
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:green}+1 MODERNIZER{color}
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://ci-hadoop.apache.org/job/PreCommit-OOZIE-Build/184/
> [server] Fix current SpotBugs discovered issues in Oozie's server module
>
>
> Key: OOZIE-3690
> URL: https://issues.apache.org/jira/browse/OOZIE-3690
> Project: Oozie
> Issue Type: Sub-task
> Components: build
>Affects Versions: 5.2.1
>Reporter: János Makai
>Assignee: János Makai
>Priority: Major
> Attachments: OOZIE-3690-001.patch, OOZIE-3690-002.patch,
> OOZIE-3690-003.patch, Oozie-server-SpotBugs-issues.txt
>
>
> This ticket is created for fixing or filtering the SpotBugs related issues in
> Oozie's server module.
> The exact SpotBugs output is attached for this module.
> There are currently 10 SpotBugs discovered issues in this module.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (OOZIE-3690) [server] Fix current SpotBugs discovered issues in Oozie's server module
[ https://issues.apache.org/jira/browse/OOZIE-3690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655468#comment-17655468 ] Hadoop QA commented on OOZIE-3690: -- PreCommit-OOZIE-Build started > [server] Fix current SpotBugs discovered issues in Oozie's server module > > > Key: OOZIE-3690 > URL: https://issues.apache.org/jira/browse/OOZIE-3690 > Project: Oozie > Issue Type: Sub-task > Components: build >Affects Versions: 5.2.1 >Reporter: János Makai >Assignee: János Makai >Priority: Major > Attachments: OOZIE-3690-001.patch, OOZIE-3690-002.patch, > OOZIE-3690-003.patch, Oozie-server-SpotBugs-issues.txt > > > This ticket is created for fixing or filtering the SpotBugs related issues in > Oozie's server module. > The exact SpotBugs output is attached for this module. > There are currently 10 SpotBugs discovered issues in this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OOZIE-3692) [sharelib-spark] Fix current SpotBugs discovered issues in Oozie's sharelib-spark module
[ https://issues.apache.org/jira/browse/OOZIE-3692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655463#comment-17655463 ] János Makai commented on OOZIE-3692: Hi [~dionusos] could you please check my latest patch set which is changed based on your finding? Thanks. > [sharelib-spark] Fix current SpotBugs discovered issues in Oozie's > sharelib-spark module > > > Key: OOZIE-3692 > URL: https://issues.apache.org/jira/browse/OOZIE-3692 > Project: Oozie > Issue Type: Sub-task > Components: build >Affects Versions: 5.2.1 >Reporter: János Makai >Assignee: János Makai >Priority: Major > Attachments: OOZIE-3692-001.patch, OOZIE-3692-002.patch, > Oozie-oozie-sharelib-spark-SpotBugs-issues.txt > > > This ticket is created for fixing or filtering the SpotBugs related issues in > Oozie's sharelib-spark module. > The exact SpotBugs output is attached for this module. > There are currently 4 SpotBugs discovered issues in this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OOZIE-3693) [examples] Fix current SpotBugs discovered issues in Oozie's examples module
[ https://issues.apache.org/jira/browse/OOZIE-3693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655462#comment-17655462 ] János Makai commented on OOZIE-3693: Hello [~dionusos] can you please take a look at my latest patch set? NOTE: the failed test and the raw patch analysis' results are unrelated to this change. Thanks in advance > [examples] Fix current SpotBugs discovered issues in Oozie's examples module > > > Key: OOZIE-3693 > URL: https://issues.apache.org/jira/browse/OOZIE-3693 > Project: Oozie > Issue Type: Sub-task > Components: build >Affects Versions: 5.2.1 >Reporter: János Makai >Assignee: János Makai >Priority: Major > Attachments: OOZIE-3693-001.patch, OOZIE-3693-002.patch, > Oozie-examples-SpotBugs-issues.txt > > > This ticket is created for fixing or filtering the SpotBugs related issues in > Oozie's examples module. > The exact SpotBugs output is attached for this module. > There are currently 3 SpotBugs discovered issues in this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (OOZIE-3712) Oozie Verbose Option at Workflow Level which sets Java Verbose to all JVMs involved in the actions
[
https://issues.apache.org/jira/browse/OOZIE-3712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655423#comment-17655423
]
Prabhu Joseph edited comment on OOZIE-3712 at 1/6/23 12:30 PM:
---
Thanks [~dionusos] for reviewing this.
{quote} May I suggest to have this property fine tuned for system level, for
workflow level and for action level too?
{quote}
Yes this gives more control to user. Will use this way.
{quote}Also what do you suggest as a "verbose" option, what debug level should
be turned on? Verbose class loading? Debug/Trace level logging for the given
action? Should we allow the user to somehow configure in an easy way (switches)
what verbose options they want?
{quote}
I had only Java verbose class loading option (-verbose:class) in mind, as that
is useful to find jar conflicts leading to NoSuchMethodErrors. But we could use
it to configure any common JVM flags which is needed across all JVMs part of a
specific Action or all Actions.
Examples:
A. Enable Java Verbose Class Loading
{code:java}
oozie.workflow.action.common.jvmflags=-verbose:class
{code}
B. Enable log4j debug option
{code:java}
oozie.workflow.action.common.jvmflags=-Dlog4j.debug
{code}
C. Print GC Details
{code:java}
oozie.workflow.action.common.jvmflags=-XX:+PrintGCDetails and
-XX:+PrintGCTimeStamps
{code}
bq. Anyway, I like the idea. Do you plan working on it?
Yes, my teammate has started working on it. Will assign it to her once JIRA
Account is created.
was (Author: prabhu joseph):
Thanks [~dionusos] for reviewing this.
{quote} May I suggest to have this property fine tuned for system level, for
workflow level and for action level too?
{quote}
Yes this gives more control to user. Will use this way.
{quote}Also what do you suggest as a "verbose" option, what debug level should
be turned on? Verbose class loading? Debug/Trace level logging for the given
action? Should we allow the user to somehow configure in an easy way (switches)
what verbose options they want?
{quote}
I had only Java verbose class loading option (-verbose:class) in mind, as that
is useful to find jar conflicts leading to NoSuchMethodErrors. But we could use
it configure any common JVM flags with default maybe -verbose:class and which
will be set across all JVMs part of specific Action or all Actions.
Examples:
A. Enable Java Verbose Class Loading
{code:java}
oozie.workflow.action.common.jvmflags=-verbose:class
{code}
B. Enable log4j debug option
{code:java}
oozie.workflow.action.common.jvmflags=-Dlog4j.debug
{code}
C. Print GC Details
{code:java}
oozie.workflow.action.common.jvmflags=-XX:+PrintGCDetails and
-XX:+PrintGCTimeStamps
{code}
bq. Anyway, I like the idea. Do you plan working on it?
Yes, my teammate has started working on it. Will assign it to her once JIRA
Account is created.
> Oozie Verbose Option at Workflow Level which sets Java Verbose to all JVMs
> involved in the actions
> --
>
> Key: OOZIE-3712
> URL: https://issues.apache.org/jira/browse/OOZIE-3712
> Project: Oozie
> Issue Type: Improvement
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
>
> The most common issue in Oozie is NoSuhMethodError. And the best way to debug
> is to set the JVM option -verbose to the right JVM which has failed. The
> challenge is to identify which JVM failed and what is the OPTS option through
> which -verbose can be set. In MapReduce Action, MRAppMaster, MapTask and
> ReduceTask each have different way of passing OPTS. In Spark Action, Spark
> ApplicationMaster, Driver and Executors have different way of passing OPTS.
> This Jira intends to expose one config say oozie.workflow.enable.verbose=true
> in job.properties to user and internally Oozie sets -verbose to all the JVMs
> involved in the Actions (SparkAction, SqoopAction or MapReduceAction). This
> will be very useful for debugging the most common issue NoSuhMethodError.
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (OOZIE-3712) Oozie Verbose Option at Workflow Level which sets Java Verbose to all JVMs involved in the actions
[
https://issues.apache.org/jira/browse/OOZIE-3712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655423#comment-17655423
]
Prabhu Joseph commented on OOZIE-3712:
--
Thanks [~dionusos] for reviewing this.
{quote} May I suggest to have this property fine tuned for system level, for
workflow level and for action level too?
{quote}
Yes this gives more control to user. Will use this way.
{quote}Also what do you suggest as a "verbose" option, what debug level should
be turned on? Verbose class loading? Debug/Trace level logging for the given
action? Should we allow the user to somehow configure in an easy way (switches)
what verbose options they want?
{quote}
I had only Java verbose class loading option (-verbose:class) in mind, as that
is useful to find jar conflicts leading to NoSuchMethodErrors. But we could use
it configure any common JVM flags with default maybe -verbose:class and which
will be set across all JVMs part of specific Action or all Actions.
Examples:
A. Enable Java Verbose Class Loading
{code:java}
oozie.workflow.action.common.jvmflags=-verbose:class
{code}
B. Enable log4j debug option
{code:java}
oozie.workflow.action.common.jvmflags=-Dlog4j.debug
{code}
C. Print GC Details
{code:java}
oozie.workflow.action.common.jvmflags=-XX:+PrintGCDetails and
-XX:+PrintGCTimeStamps
{code}
bq. Anyway, I like the idea. Do you plan working on it?
Yes, my teammate has started working on it. Will assign it to her once JIRA
Account is created.
> Oozie Verbose Option at Workflow Level which sets Java Verbose to all JVMs
> involved in the actions
> --
>
> Key: OOZIE-3712
> URL: https://issues.apache.org/jira/browse/OOZIE-3712
> Project: Oozie
> Issue Type: Improvement
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
>
> The most common issue in Oozie is NoSuhMethodError. And the best way to debug
> is to set the JVM option -verbose to the right JVM which has failed. The
> challenge is to identify which JVM failed and what is the OPTS option through
> which -verbose can be set. In MapReduce Action, MRAppMaster, MapTask and
> ReduceTask each have different way of passing OPTS. In Spark Action, Spark
> ApplicationMaster, Driver and Executors have different way of passing OPTS.
> This Jira intends to expose one config say oozie.workflow.enable.verbose=true
> in job.properties to user and internally Oozie sets -verbose to all the JVMs
> involved in the Actions (SparkAction, SqoopAction or MapReduceAction). This
> will be very useful for debugging the most common issue NoSuhMethodError.
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (OOZIE-3696) [sharelib-git] Fix current SpotBugs discovered issues in Oozie's sharelib-git module
[ https://issues.apache.org/jira/browse/OOZIE-3696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655405#comment-17655405 ] János Makai commented on OOZIE-3696: Thank you [~dionusos] for the contribution, the change looks good and I've just pushed it to master. ;) > [sharelib-git] Fix current SpotBugs discovered issues in Oozie's sharelib-git > module > > > Key: OOZIE-3696 > URL: https://issues.apache.org/jira/browse/OOZIE-3696 > Project: Oozie > Issue Type: Sub-task > Components: build >Affects Versions: 5.2.1 >Reporter: János Makai >Assignee: Dénes Bodó >Priority: Major > Attachments: OOZIE-3696-001.patch, > Oozie-oozie-sharelib-git-SpotBugs-issues.txt > > > This ticket is created for fixing or filtering the SpotBugs related issues in > Oozie's sharelib-git module. > The exact SpotBugs output is attached for this module. > There are currently 2 SpotBugs discovered issues in this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OOZIE-3696) [sharelib-git] Fix current SpotBugs discovered issues in Oozie's sharelib-git module
[ https://issues.apache.org/jira/browse/OOZIE-3696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655402#comment-17655402 ] ASF subversion and git services commented on OOZIE-3696: Commit a36bd2f8fa5b471957242727876d8de9f2b317c2 in oozie's branch refs/heads/master from Janos Makai [ https://gitbox.apache.org/repos/asf?p=oozie.git;h=a36bd2f8f ] OOZIE-3696 [sharelib-git] Fix current SpotBugs discovered issues in Oozie's sharelib-git module (dionusos via jmakai) > [sharelib-git] Fix current SpotBugs discovered issues in Oozie's sharelib-git > module > > > Key: OOZIE-3696 > URL: https://issues.apache.org/jira/browse/OOZIE-3696 > Project: Oozie > Issue Type: Sub-task > Components: build >Affects Versions: 5.2.1 >Reporter: János Makai >Assignee: Dénes Bodó >Priority: Major > Attachments: OOZIE-3696-001.patch, > Oozie-oozie-sharelib-git-SpotBugs-issues.txt > > > This ticket is created for fixing or filtering the SpotBugs related issues in > Oozie's sharelib-git module. > The exact SpotBugs output is attached for this module. > There are currently 2 SpotBugs discovered issues in this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OOZIE-3696) [sharelib-git] Fix current SpotBugs discovered issues in Oozie's sharelib-git module
[ https://issues.apache.org/jira/browse/OOZIE-3696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655385#comment-17655385 ] ASF subversion and git services commented on OOZIE-3696: Commit 5e29ce6efdf3ea47dcc7cc7a9caacaa0564fdbf6 in oozie's branch refs/heads/origin/master from Janos Makai [ https://gitbox.apache.org/repos/asf?p=oozie.git;h=5e29ce6ef ] OOZIE-3696 [sharelib-git] Fix current SpotBugs discovered issues in Oozie's sharelib-git module (dionusos via jmakai) > [sharelib-git] Fix current SpotBugs discovered issues in Oozie's sharelib-git > module > > > Key: OOZIE-3696 > URL: https://issues.apache.org/jira/browse/OOZIE-3696 > Project: Oozie > Issue Type: Sub-task > Components: build >Affects Versions: 5.2.1 >Reporter: János Makai >Assignee: Dénes Bodó >Priority: Major > Attachments: OOZIE-3696-001.patch, > Oozie-oozie-sharelib-git-SpotBugs-issues.txt > > > This ticket is created for fixing or filtering the SpotBugs related issues in > Oozie's sharelib-git module. > The exact SpotBugs output is attached for this module. > There are currently 2 SpotBugs discovered issues in this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OOZIE-3712) Oozie Verbose Option at Workflow Level which sets Java Verbose to all JVMs involved in the actions
[
https://issues.apache.org/jira/browse/OOZIE-3712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655364#comment-17655364
]
Dénes Bodó commented on OOZIE-3712:
---
Hey [~prabhujoseph]
Thanks for mentioning this issue. I agree that now turning on the debug level
logging in Oozie actions is not like turning on a switch.
You mentioned that {{oozie.workflow.enable.verbose=true}} would be a
job.properties configuration which means it will be applied to all the action
in the workflow. I saw cases where a workflow contained dozens of actions and
maybe the user does not want to turn on verbose class loading and such for all
these actions but to only some of them. May I suggest to have this property
fine tuned for system level, for workflow level and for action level too? So
* if we set it in oozie-site.xml then all the action will be in debug.
* if we set it in job.properties or workflow.xml global configuration then all
the action in the given workflow will be in debug.
* if we set it in a workflow action's configuration then only that action will
be in debug.
Also what do you suggest as a "verbose" option, what debug level should be
turned on? Verbose class loading? Debug/Trace level logging for the given
action? Should we allow the user to somehow configure in an easy way (switches)
what verbose options they want?
Anyway, I like the idea. Do you plan working on it?
Thanks,
Denes
> Oozie Verbose Option at Workflow Level which sets Java Verbose to all JVMs
> involved in the actions
> --
>
> Key: OOZIE-3712
> URL: https://issues.apache.org/jira/browse/OOZIE-3712
> Project: Oozie
> Issue Type: Improvement
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
>
> The most common issue in Oozie is NoSuhMethodError. And the best way to debug
> is to set the JVM option -verbose to the right JVM which has failed. The
> challenge is to identify which JVM failed and what is the OPTS option through
> which -verbose can be set. In MapReduce Action, MRAppMaster, MapTask and
> ReduceTask each have different way of passing OPTS. In Spark Action, Spark
> ApplicationMaster, Driver and Executors have different way of passing OPTS.
> This Jira intends to expose one config say oozie.workflow.enable.verbose=true
> in job.properties to user and internally Oozie sets -verbose to all the JVMs
> involved in the Actions (SparkAction, SqoopAction or MapReduceAction). This
> will be very useful for debugging the most common issue NoSuhMethodError.
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
