[jira] [Assigned] (OOZIE-3719) Apache Oozie Regex Denial of Service (ReDoS) Vulnerability by Low Privilege Users Disrupting Access for Intended Users
[
https://issues.apache.org/jira/browse/OOZIE-3719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kinga Marton reassigned OOZIE-3719:
---
Assignee: Sanjay Kumar Sahu
> Apache Oozie Regex Denial of Service (ReDoS) Vulnerability by Low Privilege
> Users Disrupting Access for Intended Users
> --
>
> Key: OOZIE-3719
> URL: https://issues.apache.org/jira/browse/OOZIE-3719
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.2.1
>Reporter: Sanjay Kumar Sahu
>Assignee: Sanjay Kumar Sahu
>Priority: Major
> Attachments: image-2023-09-15-02-47-52-819.png,
> image-2023-09-15-02-49-14-531.png, image-2023-09-15-02-52-09-320.png
>
>
> !image-2023-09-15-02-47-52-819.png!
>
> Looking further into the code focusing on the action and type query strings.
> We can see that the filter variable is getting its value from the
> requestsParameters .
> once the Filter parameter is being populated, an If loop checking whether
> Scope and Type are not Null and next
> the code checks the logRetrievalType is equal to the JOB_LOG_ACTION (which is
> the action query string).
>
> Next the values of logRetrievalScope gets split by , and entering the the if
> loop.
> In the block where ranges of actions are processed ( if (s.contains("-")) \{
> ... } ), an attacker could potentially
> send a specially crafted request with a massive range, such as "1-100".
> This would create a for loop
> iterating and adding that many actions to the actionSet , consuming CPU and
> memory resources.
> Though there is a subsequent check against maxNumActionsForLog , this check
> only happens after all the iterations,
> allowing an attacker to consume resources before this check is made -
>
> !image-2023-09-15-02-52-09-320.png!
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (OOZIE-3611) Oozie remote command with SSH action cannot support stdout redirect to a file and quotes in the command string
[ https://issues.apache.org/jira/browse/OOZIE-3611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17197563#comment-17197563 ] Kinga Marton commented on OOZIE-3611: - [~wangdo] can you please upload your patch here to trigger the precommit build? As described here: [https://cwiki.apache.org/confluence/display/OOZIE/How+To+Contribute] > Oozie remote command with SSH action cannot support stdout redirect to a file > and quotes in the command string > -- > > Key: OOZIE-3611 > URL: https://issues.apache.org/jira/browse/OOZIE-3611 > Project: Oozie > Issue Type: Bug >Reporter: wangdo >Assignee: wangdo >Priority: Critical > > when executing command on remote machines with SSH action, such as: hostname > >/tmp/bbb.txt && date '+%F %T' >>/tmp/bbb.txt && date "+%s" >>/tmp/bbb.txt, > currently, we have 2 issues > issue 1: the ">" will cause no pid obtained( in pid = > getFirstLine(inputBuffer); of SshActionExecutor.java); > issue 2: the double quotes and single quote of the command will also lead to > failure as well -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (OOZIE-3611) Oozie remote command with SSH action cannot support stdout redirect to a file and quotes in the command string
[ https://issues.apache.org/jira/browse/OOZIE-3611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kinga Marton reassigned OOZIE-3611: --- Assignee: wangdo > Oozie remote command with SSH action cannot support stdout redirect to a file > and quotes in the command string > -- > > Key: OOZIE-3611 > URL: https://issues.apache.org/jira/browse/OOZIE-3611 > Project: Oozie > Issue Type: Bug >Reporter: wangdo >Assignee: wangdo >Priority: Critical > > when executing command on remote machines with SSH action, such as: hostname > >/tmp/bbb.txt && date '+%F %T' >>/tmp/bbb.txt && date "+%s" >>/tmp/bbb.txt, > currently, we have 2 issues > issue 1: the ">" will cause no pid obtained( in pid = > getFirstLine(inputBuffer); of SshActionExecutor.java); > issue 2: the double quotes and single quote of the command will also lead to > failure as well -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OOZIE-3601) Upgrade quartz to 2.3.2
[ https://issues.apache.org/jira/browse/OOZIE-3601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151491#comment-17151491 ] Kinga Marton commented on OOZIE-3601: - [~dengliming], thank you for fixing this. We do not support PRs yet. Can you please create a Patch and upload it here, to trigger the precommit checks. See more details about how to contribute in the following page: [https://cwiki.apache.org/confluence/display/OOZIE/How+To+Contribute] > Upgrade quartz to 2.3.2 > --- > > Key: OOZIE-3601 > URL: https://issues.apache.org/jira/browse/OOZIE-3601 > Project: Oozie > Issue Type: Bug >Affects Versions: 5.2.0 >Reporter: Andras Salamon >Assignee: dengliming >Priority: Major > > We should upgrade quartz to 2.3.2: > https://nvd.nist.gov/vuln/detail/CVE-2019-13990 > Most likely it's just a simple version bump. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (OOZIE-3601) Upgrade quartz to 2.3.2
[ https://issues.apache.org/jira/browse/OOZIE-3601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kinga Marton reassigned OOZIE-3601: --- Assignee: dengliming > Upgrade quartz to 2.3.2 > --- > > Key: OOZIE-3601 > URL: https://issues.apache.org/jira/browse/OOZIE-3601 > Project: Oozie > Issue Type: Bug >Affects Versions: 5.2.0 >Reporter: Andras Salamon >Assignee: dengliming >Priority: Major > > We should upgrade quartz to 2.3.2: > https://nvd.nist.gov/vuln/detail/CVE-2019-13990 > Most likely it's just a simple version bump. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (OOZIE-3459) [Java 11] Build and test Oozie with Java 11
[
https://issues.apache.org/jira/browse/OOZIE-3459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kinga Marton reassigned OOZIE-3459:
---
Assignee: (was: Kinga Marton)
> [Java 11] Build and test Oozie with Java 11
> ---
>
> Key: OOZIE-3459
> URL: https://issues.apache.org/jira/browse/OOZIE-3459
> Project: Oozie
> Issue Type: Bug
> Components: core, fluent-job
>Affects Versions: 5.1.0
>Reporter: Dénes Bodó
>Priority: Major
>
> Using OpenJDK 11 I am not able to build Oozie using {{mvn clean install}}.
> I found two issues:
> * Fluent job API build fails due to Jaxb2 maven plugin.
> * No {{com.sun.tools.}} package is available so *TestMetricsInstrumentation*
> will not work.
> * Maven surefire plugin has to be updated. It works with 3.0.0-M3
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (OOZIE-3445) [build] Replace findbugs-diff jar
[ https://issues.apache.org/jira/browse/OOZIE-3445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kinga Marton reassigned OOZIE-3445: --- Assignee: (was: Kinga Marton) > [build] Replace findbugs-diff jar > - > > Key: OOZIE-3445 > URL: https://issues.apache.org/jira/browse/OOZIE-3445 > Project: Oozie > Issue Type: Task >Reporter: Kinga Marton >Priority: Minor > > During precommit, we use an extrenal[ > jar|https://repo1.maven.org/maven2/me/andrz/findbugs/findbugs-diff/0.1.0/]. > We should search for alternatives and replace it, because it not so nice to > store the md5, hardwiring the jar address and downloading the jar again and > again. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (OOZIE-3509) [Java 11] Fix TestMapReduceActionExecutor test failures
[
https://issues.apache.org/jira/browse/OOZIE-3509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kinga Marton reassigned OOZIE-3509:
---
Assignee: (was: Kinga Marton)
> [Java 11] Fix TestMapReduceActionExecutor test failures
> ---
>
> Key: OOZIE-3509
> URL: https://issues.apache.org/jira/browse/OOZIE-3509
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Kinga Marton
>Priority: Major
>
> {{TestMapReduceActionExecutor#testJobNameSetForMapReduceChild}} and
> {{TestMapReduceActionExecutor#testMapReduceWithUberJarEnabled}} are failing:
>
> {code:bash}
> TestMapReduceActionExecutor.testJobNameSetForMapReduceChild:801->_testSubmit:423
> expected:<[SUCCEED]ED> but was:<[FAILED/KILL]ED>
> [ERROR]
> TestMapReduceActionExecutor.testMapReduceWithUberJarEnabled:820->_testMapReduceWithUberJar:736->_testSubmit:423
> expected:<[SUCCEED]ED> but was:<[FAILED/KILL]ED>
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (OOZIE-3534) Refactor ActionExecutorTestCase to not extend HCatTestCase
[ https://issues.apache.org/jira/browse/OOZIE-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kinga Marton reassigned OOZIE-3534: --- Assignee: (was: Kinga Marton) > Refactor ActionExecutorTestCase to not extend HCatTestCase > -- > > Key: OOZIE-3534 > URL: https://issues.apache.org/jira/browse/OOZIE-3534 > Project: Oozie > Issue Type: Improvement > Components: tests >Reporter: Kinga Marton >Priority: Major > > ActionExecutorTestCase extends HCatTestCase but there are only two test cases > where we need HCatalog related settings as well. > Let's try to refactor this part and make the ActionExecturTestCase > independent from the HCatalog part. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OOZIE-3568) Have large amount of log information “WARN messages [main] openjpa.MetaData” in jetty.log need to clean
[ https://issues.apache.org/jira/browse/OOZIE-3568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040923#comment-17040923 ] Kinga Marton commented on OOZIE-3568: - [~nobigo], how did you renamed your file? Using git -mv, or simply renamed it? > Have large amount of log information “WARN messages [main] openjpa.MetaData” > in jetty.log need to clean > --- > > Key: OOZIE-3568 > URL: https://issues.apache.org/jira/browse/OOZIE-3568 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 5.1.0 >Reporter: duan xiong >Assignee: duan xiong >Priority: Major > Fix For: 5.3.0 > > Attachments: OOZIE-3568-001.patch, OOZIE-3568-002.patch, > OOZIE-3568-003.patch, OOZIE-3568-004.patch > > > Have large amount of log information “WARN messages [main] openjpa.MetaData” > in jetty.out need to clean: > some not exists class need to enhanced in openjpa: > org.apache.oozie.client.rest.JsonWorkflowJob > org.apache.oozie.client.rest.JsonWorkflowAction > org.apache.oozie.client.rest.JsonCoordinatorJob > org.apache.oozie.client.rest.JsonCoordinatorAction > org.apache.oozie.client.rest.JsonBundleJob > for example: > > 3493 oozie-postgresql WARN [main] openjpa.MetaData - The class > "org.apache.oozie.client.rest.JsonBundleJob" listed in the > openjpa.MetaDataFactory configuration property could not be loaded by > sun.misc.Launcher$AppClassLoader@33909752; ignoring. > 3494 oozie-postgresql WARN [main] openjpa.MetaData - The class > "org.apache.oozie.client.rest.JsonCoordinatorJob" listed in the > openjpa.MetaDataFactory configuration property could not be loaded by > sun.misc.Launcher$AppClassLoader@33909752; ignoring. > 3495 oozie-postgresql WARN [main] openjpa.MetaData - The class > "org.apache.oozie.client.rest.JsonWorkflowJob" listed in the > openjpa.MetaDataFactory configuration property could not be loaded by > sun.misc.Launcher$AppClassLoader@33909752; ignoring. > 3495 oozie-postgresql WARN [main] openjpa.MetaData - The class > "org.apache.oozie.client.rest.JsonWorkflowAction" listed in the > openjpa.MetaDataFactory configuration property could not be loaded by > sun.misc.Launcher$AppClassLoader@33909752; ignoring. > 3496 oozie-postgresql WARN [main] openjpa.MetaData - The class > "org.apache.oozie.client.rest.JsonCoordinatorAction" listed in the > openjpa.MetaDataFactory configuration property could not be loaded by > sun.misc.Launcher$AppClassLoader@33909752; ignoring. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OOZIE-3560) IDEA shows have some error in index.jsp
[
https://issues.apache.org/jira/browse/OOZIE-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040915#comment-17040915
]
Kinga Marton commented on OOZIE-3560:
-
[~nobigo], did you had any change to check the generated HTML?
> IDEA shows have some error in index.jsp
>
>
> Key: OOZIE-3560
> URL: https://issues.apache.org/jira/browse/OOZIE-3560
> Project: Oozie
> Issue Type: Bug
> Components: ui
>Affects Versions: 5.1.0
>Reporter: duan xiong
>Assignee: duan xiong
>Priority: Major
> Fix For: 5.3.0
>
> Attachments: OOZIE-3560-001.patch
>
>
> :
> :
> style="width:1048">:
>
> Mismatched property value ( | |
> | |
> *{color:#ff6464}[initial | inherit | unset | revert]{color}*)
> Inspection info: This inspection detects illegal property's values.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3091) Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: org/apache/avro/mapred/AvroWrapper"
[
https://issues.apache.org/jira/browse/OOZIE-3091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040914#comment-17040914
]
Kinga Marton commented on OOZIE-3091:
-
[~prabhujoseph] is it still an issue in Oozie 5.2 as well?
> Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError:
> org/apache/avro/mapred/AvroWrapper"
> ---
>
> Key: OOZIE-3091
> URL: https://issues.apache.org/jira/browse/OOZIE-3091
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: 4.2.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: OOZIE-3091.1.patch, OOZIE-3091.2.patch,
> Oozie_Sqoop_Avro_import
>
>
> Oozie Sqoop Action which does Import as avro fails with below.
> avro-mapred-1.8.0-hadoop2.jar need to be included in Oozie Sqoop Sharelib
> {code}
> 2017-10-19 09:45:25,349 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.lang.RuntimeException:
> java.lang.reflect.InvocationTargetException
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:134)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:745)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:170)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:164)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:132)
> ... 7 more
> Caused by: java.lang.NoClassDefFoundError: org/apache/avro/mapred/AvroWrapper
> at
> org.apache.sqoop.mapreduce.AvroImportMapper.(AvroImportMapper.java:43)
> ... 12 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.avro.mapred.AvroWrapper
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 13 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (OOZIE-3586) Oozie spark actions using --keytab fail due to duplicate dist. cache
[ https://issues.apache.org/jira/browse/OOZIE-3586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040882#comment-17040882 ] Kinga Marton edited comment on OOZIE-3586 at 2/20/20 11:41 AM: --- [~jmakai] thank you for reporting this. Please attach your patch to this Jira when you have one and set the status to patch available. Also link the review Board review to it in case of bigger changes that may need RB. was (Author: kmarton): [~jmakai] thank you for reporting this. Please attach your patch to this Jira when you have one and set the status to patch available. Also link the review Board review to it in case of bigger changes that need RB. > Oozie spark actions using --keytab fail due to duplicate dist. cache > > > Key: OOZIE-3586 > URL: https://issues.apache.org/jira/browse/OOZIE-3586 > Project: Oozie > Issue Type: Bug >Reporter: Janos Makai >Assignee: Janos Makai >Priority: Major > > In CDH 6.x, on the Hadoop 3 rebase, it is now a failure if items are added > multiple times to the distributed cache. In CDH5, this was a warning. This is > not an issue for most uses, as adding multiple times typically is user error, > but this completely breaks Spark actions with keytabs (--keytab). > Oozie spark actions add everything in the distributed cache of the launcher > job to the distributed cache of the spark job, meaning the keytab is already > there, then the --keytab argument tries to add it again causing the failure > [0]. > [0] > Caused by: java.lang.IllegalArgumentException: Attempt to add > (.../filename.keytab#filename.keytab) multiple times to the distributed cache. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OOZIE-3586) Oozie spark actions using --keytab fail due to duplicate dist. cache
[ https://issues.apache.org/jira/browse/OOZIE-3586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040882#comment-17040882 ] Kinga Marton commented on OOZIE-3586: - [~jmakai] thank you for reporting this. Please attach your patch to this Jira when you have one and set the status to patch available. Also link the review Board review to it in case of bigger changes that need RB. > Oozie spark actions using --keytab fail due to duplicate dist. cache > > > Key: OOZIE-3586 > URL: https://issues.apache.org/jira/browse/OOZIE-3586 > Project: Oozie > Issue Type: Bug >Reporter: Janos Makai >Assignee: Janos Makai >Priority: Major > > In CDH 6.x, on the Hadoop 3 rebase, it is now a failure if items are added > multiple times to the distributed cache. In CDH5, this was a warning. This is > not an issue for most uses, as adding multiple times typically is user error, > but this completely breaks Spark actions with keytabs (--keytab). > Oozie spark actions add everything in the distributed cache of the launcher > job to the distributed cache of the spark job, meaning the keytab is already > there, then the --keytab argument tries to add it again causing the failure > [0]. > [0] > Caused by: java.lang.IllegalArgumentException: Attempt to add > (.../filename.keytab#filename.keytab) multiple times to the distributed cache. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OOZIE-3582) Rerun end point returning 401 error
[ https://issues.apache.org/jira/browse/OOZIE-3582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031472#comment-17031472 ] Kinga Marton commented on OOZIE-3582: - [~mdbilaly2k], have you faced this issue with the latest Oozie (5.2.0) as well? > Rerun end point returning 401 error > --- > > Key: OOZIE-3582 > URL: https://issues.apache.org/jira/browse/OOZIE-3582 > Project: Oozie > Issue Type: Bug > Components: security >Affects Versions: 4.2.0 >Reporter: Mohammed Bilal >Priority: Critical > Fix For: 4.2.0 > > > Hi, > I am making HTTP put request to rerun oozie job, but I am seeing > authorization issue. > I added basic authorization, but no luck. > I referred web API and followed all the steps mentioned there. > [https://oozie.apache.org/docs/4.2.0/WebServicesAPI.html#Re-Running_a_Workflow_Job] > Here there is no information about any kind of security mechanism which we > need to follow before calling rerun end point. I was able to invoke post > method without any issue to start a oozie job without any authentication. > Could you please provide me more details on how I can fix this issue? > Error Log: > org.springframework.web.client.HttpClientErrorException: 401 > Unauthorizedorg.springframework.web.client.HttpClientErrorException: 401 > Unauthorized at > org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:94) > ~[spring-web-5.0.5.RELEASE.jar:5.0.5.RELEASE] at > org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:79) > ~[spring-web-5.0.5.RELEASE.jar:5.0.5.RELEASE] at > org.springframework.web.client.ResponseErrorHandler.handleError(ResponseErrorHandler.java:63) > ~[spring-web-5.0.5.RELEASE.jar:5.0.5.RELEASE] at > org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:775) > ~[spring-web-5.0.5.RELEASE.jar:5.0.5.RELEASE] at > org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:728) > ~[spring-web-5.0.5.RELEASE.jar:5.0.5.RELEASE] at > org.springframework.web.client.RestTemplate.execute(RestTemplate.java:694) > ~[spring-web-5.0.5.RELEASE.jar:5.0.5.RELEASE] at > org.springframework.web.client.RestTemplate.put(RestTemplate.java:503) > ~[spring-web-5.0.5.RELEASE.jar:5.0.5.RELEASE] > Thank you! > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OOZIE-3568) Have large amount of log information “WARN messages [main] openjpa.MetaData” in jetty.log need to clean
[ https://issues.apache.org/jira/browse/OOZIE-3568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16994760#comment-16994760 ] Kinga Marton commented on OOZIE-3568: - [~nobigo] I think this is a valid issue. Those files were removed with OOZIE-1463. I didn't found any explanation related the reason of keeping them in persistence.xml, so I think that this change is OK. Also I think that we should remove the test classes as well, if they don't have any added value. [~nobigo] can you please check what exactly are those classes testing and check if they can be removed as well, or we should keep those. Since the proposed changes are in persistence.xml, the precommit check fails at backwards compatibility part. However I don't think that this should be a problem, since there were invalid references. [~asalamon74] what is you opinion about this? > Have large amount of log information “WARN messages [main] openjpa.MetaData” > in jetty.log need to clean > --- > > Key: OOZIE-3568 > URL: https://issues.apache.org/jira/browse/OOZIE-3568 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 5.1.0 >Reporter: duan xiong >Assignee: duan xiong >Priority: Major > Fix For: 5.3.0 > > Attachments: OOZIE-3568-001.patch, OOZIE-3568-002.patch > > > Have large amount of log information “WARN messages [main] openjpa.MetaData” > in jetty.out need to clean: > some not exists class need to enhanced in openjpa: > org.apache.oozie.client.rest.JsonWorkflowJob > org.apache.oozie.client.rest.JsonWorkflowAction > org.apache.oozie.client.rest.JsonCoordinatorJob > org.apache.oozie.client.rest.JsonCoordinatorAction > org.apache.oozie.client.rest.JsonBundleJob > for example: > > 3493 oozie-postgresql WARN [main] openjpa.MetaData - The class > "org.apache.oozie.client.rest.JsonBundleJob" listed in the > openjpa.MetaDataFactory configuration property could not be loaded by > sun.misc.Launcher$AppClassLoader@33909752; ignoring. > 3494 oozie-postgresql WARN [main] openjpa.MetaData - The class > "org.apache.oozie.client.rest.JsonCoordinatorJob" listed in the > openjpa.MetaDataFactory configuration property could not be loaded by > sun.misc.Launcher$AppClassLoader@33909752; ignoring. > 3495 oozie-postgresql WARN [main] openjpa.MetaData - The class > "org.apache.oozie.client.rest.JsonWorkflowJob" listed in the > openjpa.MetaDataFactory configuration property could not be loaded by > sun.misc.Launcher$AppClassLoader@33909752; ignoring. > 3495 oozie-postgresql WARN [main] openjpa.MetaData - The class > "org.apache.oozie.client.rest.JsonWorkflowAction" listed in the > openjpa.MetaDataFactory configuration property could not be loaded by > sun.misc.Launcher$AppClassLoader@33909752; ignoring. > 3496 oozie-postgresql WARN [main] openjpa.MetaData - The class > "org.apache.oozie.client.rest.JsonCoordinatorAction" listed in the > openjpa.MetaDataFactory configuration property could not be loaded by > sun.misc.Launcher$AppClassLoader@33909752; ignoring. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OOZIE-3557) [build] Update required minimum maven version
[
https://issues.apache.org/jira/browse/OOZIE-3557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16989567#comment-16989567
]
Kinga Marton commented on OOZIE-3557:
-
Thank you [~asalamon74] for the fix! +1, committed to master
> [build] Update required minimum maven version
> -
>
> Key: OOZIE-3557
> URL: https://issues.apache.org/jira/browse/OOZIE-3557
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.1.0, 5.2.0
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3557-01.patch
>
>
> According to the Oozie [build
> docs|https://github.com/apache/oozie/blob/master/docs/src/site/markdown/ENG_Building.md],
> the minimum required maven version is 3.0.1.
> According to the pom.xml the minimum required version is 3.0.0
> According to the test of [~asasvari] Oozie cannot be compiled using maven
> 3.3.9, which means we need to find the minimum required version and update
> the documentation and the pom.xml
> The error message for maven 3.3.9:
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-dependency-plugin:3.0.2:copy (copy-sharelib)
> on project oozie-webapp: Unable to find/resolve artifact.
> com.google.inject.ProvisionException: Unable to provision, see the following
> errors:
> [ERROR]
> [ERROR] 1) Error injecting: private org.eclipse.aether.spi.log.Logger
> org.apache.maven.repository.internal.DefaultVersionRangeResolver.logger
> [ERROR] while locating
> org.apache.maven.repository.internal.DefaultVersionRangeResolver
> [ERROR] while locating java.lang.Object annotated with *
> [ERROR] at org.eclipse.sisu.wire.LocatorWiring
> [ERROR] while locating org.eclipse.aether.impl.VersionRangeResolver
> [ERROR] for parameter 2 at
> org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.(Unknown
> Source)
> [ERROR] while locating
> org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector
> [ERROR] while locating java.lang.Object annotated with *
> [ERROR] at org.eclipse.sisu.wire.LocatorWiring {noformat}
>
> According to my tests Oozie compiles using maven 3.5.3
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3562) Fix Hive example
[
https://issues.apache.org/jira/browse/OOZIE-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kinga Marton updated OOZIE-3562:
Component/s: examples
Issue Type: Bug (was: Improvement)
> Fix Hive example
>
>
> Key: OOZIE-3562
> URL: https://issues.apache.org/jira/browse/OOZIE-3562
> Project: Oozie
> Issue Type: Bug
> Components: examples
>Affects Versions: 5.1.0
>Reporter: Kinga Marton
>Priority: Major
> Attachments: hive1StachTrace.txt
>
>
> Hive example in pseudo distributed mode is failing with the following error
> message:
> {code:java}
> ACTION[008-191121145732587-oozie-mart-W@hive-node] Launcher exception:
> java.lang.RuntimeException: Unable to instantiate
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
> {code}
> See the full stacktrace attached.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (OOZIE-3563) Fix HIve2 example
Kinga Marton created OOZIE-3563:
---
Summary: Fix HIve2 example
Key: OOZIE-3563
URL: https://issues.apache.org/jira/browse/OOZIE-3563
Project: Oozie
Issue Type: Bug
Components: examples
Reporter: Kinga Marton
Hive 2 example is getting Killed:
{code:java}
2019-11-21 15:18:31,603 WARN Hive2ActionExecutor:523 - SERVER[kmarton-MBP]
USER[martonjuliakinga] GROUP[-] TOKEN[] APP[hive2-wf]
JOB[009-191121145732587-oozie-mart-W]
ACTION[009-191121145732587-oozie-mart-W@hive2-node] Launcher ERROR, reason:
Main Class [org.apache.oozie.action.hadoop.Hive2Main], exit code [2]
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3562) Fix Hive example
[
https://issues.apache.org/jira/browse/OOZIE-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kinga Marton updated OOZIE-3562:
Attachment: hive1StachTrace.txt
> Fix Hive example
>
>
> Key: OOZIE-3562
> URL: https://issues.apache.org/jira/browse/OOZIE-3562
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: 5.1.0
>Reporter: Kinga Marton
>Priority: Major
> Attachments: hive1StachTrace.txt
>
>
> Hive example in pseudo distributed mode is failing with the following error
> message:
> {code:java}
> ACTION[008-191121145732587-oozie-mart-W@hive-node] Launcher exception:
> java.lang.RuntimeException: Unable to instantiate
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
> {code}
> See the full stacktrace attached.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (OOZIE-3562) Fix Hive example
Kinga Marton created OOZIE-3562:
---
Summary: Fix Hive example
Key: OOZIE-3562
URL: https://issues.apache.org/jira/browse/OOZIE-3562
Project: Oozie
Issue Type: Improvement
Affects Versions: 5.1.0
Reporter: Kinga Marton
Hive example in pseudo distributed mode is failing with the following error
message:
{code:java}
ACTION[008-191121145732587-oozie-mart-W@hive-node] Launcher exception:
java.lang.RuntimeException: Unable to instantiate
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
{code}
See the full stacktrace attached.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3553) [examples] Fix sqoop example
[
https://issues.apache.org/jira/browse/OOZIE-3553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16959781#comment-16959781
]
Kinga Marton commented on OOZIE-3553:
-
Thank you [~asalamon74] for reporting and fixing it!
+1, committed to master
> [examples] Fix sqoop example
>
>
> Key: OOZIE-3553
> URL: https://issues.apache.org/jira/browse/OOZIE-3553
> Project: Oozie
> Issue Type: Bug
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Critical
> Attachments: OOZIE-3553-01.patch
>
>
> The built-in sqoop example fails with the following error:
> {noformat}
> java.lang.NoClassDefFoundError: org/apache/avro/LogicalType
> at
> org.apache.sqoop.manager.DefaultManagerFactory.accept(DefaultManagerFactory.java:76)
> at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:184)
> at org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:272)
> at org.apache.sqoop.tool.ImportTool.init(ImportTool.java:96)
> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:616)
> at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
> at
> org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:165)
> at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:155)
> at
> org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:107)
> at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:47)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:412)
> at
> org.apache.oozie.action.hadoop.LauncherAM.access$400(LauncherAM.java:54)
> at
> org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:225)
> at java.base/java.security.AccessController.doPrivileged(Native
> Method)
> at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:219)
> at
> org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:155)
> at java.base/java.security.AccessController.doPrivileged(Native
> Method)
> at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:143)
> Caused by: java.lang.ClassNotFoundException: org.apache.avro.LogicalType
> at
> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583)
> at
> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
> ... 31 more {noformat}
> Might be related to OOZIE-3515. Maybe the example works differently in dbd
> and in the pseudo-distributed environment.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (OOZIE-3552) OozieCLI: missing separator in coordinator job output
[
https://issues.apache.org/jira/browse/OOZIE-3552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kinga Marton reassigned OOZIE-3552:
---
Assignee: Alexander Litvinov
> OozieCLI: missing separator in coordinator job output
> -
>
> Key: OOZIE-3552
> URL: https://issues.apache.org/jira/browse/OOZIE-3552
> Project: Oozie
> Issue Type: Bug
> Components: client
>Reporter: Alexander Litvinov
>Assignee: Alexander Litvinov
>Priority: Major
> Attachments:
> 0001-OOZIE-3552-Missing-delimiter-added-to-coord-job-info.patch
>
>
> In the output, you can see the values for *Nominal Time* and *Status*, that
> there is no separator '\ t' between *2019-10-21 03:00:00 GMTKILLED*
> {code}oozie job -oozie http://oozie.mydomain.ru/oozie -info
> 0207943-190826193438338-oozie-oozi-C -verbose
> {code}
> *output*
> {code}
> Job ID : 0207943-190826193438338-oozie-oozi-C
>
> Job Name: my_job
> App Path: /user/bob/oozie/my_job
> Status : KILLED
> Start Time : 2019-10-21 03:00 GMT
> End Time: 2099-10-10 10:00 GMT
> Pause Time : -
> Concurrency : 1
>
> ID Action Number Console URL Error Code Error Message
> External ID External Status Job ID Tracker URI Created Nominal Time
> Status Last Modified Missing Dependencies
>
> 0207943-190826193438338-oozie-oozi-C@1 1 - - - -
> - 0207943-190826193438338-oozie-oozi-C- 2019-10-21 09:51:55
> GMT 2019-10-21 03:00:00 GMTKILLED 2019-10-21 10:11:26 GMT -
>
> {code}
> And in the
> [code|https://github.com/apache/oozie/blob/8002acf772c0a383025d4afb0f70acea2c0c424b/client/src/main/java/org/apache/oozie/cli/OozieCLI.java#L1532]
> you can see the absence of a separator.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3552) OozieCLI: missing separator in coordinator job output
[
https://issues.apache.org/jira/browse/OOZIE-3552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16956752#comment-16956752
]
Kinga Marton commented on OOZIE-3552:
-
[~dvigal] thank you for reporting and fixing this. Unfortunately in Oozie we
are not using pull requests yet. Can you please generate a patch file and
upload it here? I have added you to the contributors list, so you can assign
the Jira as well.
> OozieCLI: missing separator in coordinator job output
> -
>
> Key: OOZIE-3552
> URL: https://issues.apache.org/jira/browse/OOZIE-3552
> Project: Oozie
> Issue Type: Bug
> Components: client
>Reporter: Alexander Litvinov
>Priority: Major
>
> In the output, you can see the values for *Nominal Time* and *Status*, that
> there is no separator '\ t' between *2019-10-21 03:00:00 GMTKILLED*
> {code}oozie job -oozie http://oozie.mydomain.ru/oozie -info
> 0207943-190826193438338-oozie-oozi-C -verbose
> {code}
> *output*
> {code}
> Job ID : 0207943-190826193438338-oozie-oozi-C
>
> Job Name: my_job
> App Path: /user/bob/oozie/my_job
> Status : KILLED
> Start Time : 2019-10-21 03:00 GMT
> End Time: 2099-10-10 10:00 GMT
> Pause Time : -
> Concurrency : 1
>
> ID Action Number Console URL Error Code Error Message
> External ID External Status Job ID Tracker URI Created Nominal Time
> Status Last Modified Missing Dependencies
>
> 0207943-190826193438338-oozie-oozi-C@1 1 - - - -
> - 0207943-190826193438338-oozie-oozi-C- 2019-10-21 09:51:55
> GMT 2019-10-21 03:00:00 GMTKILLED 2019-10-21 10:11:26 GMT -
>
> {code}
> And in the
> [code|https://github.com/apache/oozie/blob/8002acf772c0a383025d4afb0f70acea2c0c424b/client/src/main/java/org/apache/oozie/cli/OozieCLI.java#L1532]
> you can see the absence of a separator.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3542) Handle better old Hdfs implementations in ECPolicyDisabler
[ https://issues.apache.org/jira/browse/OOZIE-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16952751#comment-16952751 ] Kinga Marton commented on OOZIE-3542: - Thank you [~zsombor] for the fix! Unfortunately the Spotbugs check is broken. The reported issues are unrelated. +1, committed to master > Handle better old Hdfs implementations in ECPolicyDisabler > -- > > Key: OOZIE-3542 > URL: https://issues.apache.org/jira/browse/OOZIE-3542 > Project: Oozie > Issue Type: Bug > Components: tools >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Fix For: 5.2.0 > > Attachments: OOZIE-3542-2.patch, OOZIE-3542-3.patch, > OOZIE-3542-4.patch, OOZIE-3542-amend-01.patch, OOZIE-3542-amend-02.patch > > > Currently, ECPolicyDisabler checks if the local hdfs implementation has the > necessary methods to get and set erasure coding policy. However, if the > namenode implementation is old, it could throw a > org.apache.hadoop.ipc.RemoteException with > RpcErrorCodeProto.ERROR_NO_SUCH_METHOD value in it. > In this case, ECPolicyDisabler fails, and prevents the installation to > succeed. > This case should be handled just like, when erasure coding is not supported. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OOZIE-3533) Flaky test TestXLogService.testLog4jReload
[
https://issues.apache.org/jira/browse/OOZIE-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938491#comment-16938491
]
Kinga Marton commented on OOZIE-3533:
-
Thank you [~asalamon74] for the fix! +1, committed to master
> Flaky test TestXLogService.testLog4jReload
> --
>
> Key: OOZIE-3533
> URL: https://issues.apache.org/jira/browse/OOZIE-3533
> Project: Oozie
> Issue Type: Sub-task
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Minor
> Attachments: OOZIE-3533-01.patch
>
>
> Sometimes this test fails with the following error message:
> {noformat}
> junit.framework.AssertionFailedError
> at junit.framework.Assert.fail(Assert.java:55)
> at junit.framework.Assert.assertTrue(Assert.java:22)
> at junit.framework.Assert.assertFalse(Assert.java:39)
> at junit.framework.Assert.assertFalse(Assert.java:47)
> at junit.framework.TestCase.assertFalse(TestCase.java:219)
> at
> org.apache.oozie.service.TestXLogService.testLog4jReload(TestXLogService.java:111)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at org.junit.runners.Suite.runChild(Suite.java:127)
> at org.junit.runners.Suite.runChild(Suite.java:26)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runners.Suite.runChild(Suite.java:127)
> at org.junit.runners.Suite.runChild(Suite.java:26)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at
> org.apache.maven.surefire.junitcore.pc.Scheduler$1.run(Scheduler.java:410)
> at
> org.apache.maven.surefire.junitcore.pc.InvokerStrategy.schedule(InvokerStrategy.java:54)
> at
> org.apache.maven.surefire.junitcore.pc.Scheduler.schedule(Scheduler.java:367)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at
> org.apache.maven.surefire.junitcore.pc.ParallelComputerBuilder$PC$1.run(ParallelComputerBuilder.java:593)
> at org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeEager(JUnitCoreWrapper.java:107)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:83)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:373)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:334)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:119)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:407)
> {noformat}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (OOZIE-3542) Handle better old Hdfs implementations in ECPolicyDisabler
[
https://issues.apache.org/jira/browse/OOZIE-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938437#comment-16938437
]
Kinga Marton edited comment on OOZIE-3542 at 9/26/19 9:35 AM:
--
Thank you [~zsombor] for the clarification! The fix looks good to me. Can you
please extend the test with the scenarios when the other two methods are not
supported? As you did for getErasureCodingPolicy method in
{{testServerNotSupportsGetErasureCodingPolicyMethod()}}
was (Author: kmarton):
Thank you @zsombor for the clarification! The fix looks good to me. Can you
please extend the test with the scenarios when the other two methods are not
supported? As you did for getErasureCodingPolicy method in
{{testServerNotSupportsGetErasureCodingPolicyMethod()}}
> Handle better old Hdfs implementations in ECPolicyDisabler
> --
>
> Key: OOZIE-3542
> URL: https://issues.apache.org/jira/browse/OOZIE-3542
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Zsombor Gegesy
>Assignee: Zsombor Gegesy
>Priority: Major
> Fix For: 5.2.0
>
> Attachments: OOZIE-3542-2.patch, OOZIE-3542-3.patch,
> OOZIE-3542-4.patch, OOZIE-3542-amend-01.patch
>
>
> Currently, ECPolicyDisabler checks if the local hdfs implementation has the
> necessary methods to get and set erasure coding policy. However, if the
> namenode implementation is old, it could throw a
> org.apache.hadoop.ipc.RemoteException with
> RpcErrorCodeProto.ERROR_NO_SUCH_METHOD value in it.
> In this case, ECPolicyDisabler fails, and prevents the installation to
> succeed.
> This case should be handled just like, when erasure coding is not supported.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3542) Handle better old Hdfs implementations in ECPolicyDisabler
[
https://issues.apache.org/jira/browse/OOZIE-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938437#comment-16938437
]
Kinga Marton commented on OOZIE-3542:
-
Thank you @zsombor for the clarification! The fix looks good to me. Can you
please extend the test with the scenarios when the other two methods are not
supported? As you did for getErasureCodingPolicy method in
{{testServerNotSupportsGetErasureCodingPolicyMethod()}}
> Handle better old Hdfs implementations in ECPolicyDisabler
> --
>
> Key: OOZIE-3542
> URL: https://issues.apache.org/jira/browse/OOZIE-3542
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Zsombor Gegesy
>Assignee: Zsombor Gegesy
>Priority: Major
> Fix For: 5.2.0
>
> Attachments: OOZIE-3542-2.patch, OOZIE-3542-3.patch,
> OOZIE-3542-4.patch, OOZIE-3542-amend-01.patch
>
>
> Currently, ECPolicyDisabler checks if the local hdfs implementation has the
> necessary methods to get and set erasure coding policy. However, if the
> namenode implementation is old, it could throw a
> org.apache.hadoop.ipc.RemoteException with
> RpcErrorCodeProto.ERROR_NO_SUCH_METHOD value in it.
> In this case, ECPolicyDisabler fails, and prevents the installation to
> succeed.
> This case should be handled just like, when erasure coding is not supported.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3542) Handle better old Hdfs implementations in ECPolicyDisabler
[ https://issues.apache.org/jira/browse/OOZIE-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16936909#comment-16936909 ] Julia Kinga Marton commented on OOZIE-3542: --- [~zsombor], can you please share what was the problem with the initial patch? > Handle better old Hdfs implementations in ECPolicyDisabler > -- > > Key: OOZIE-3542 > URL: https://issues.apache.org/jira/browse/OOZIE-3542 > Project: Oozie > Issue Type: Bug > Components: tools >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Fix For: 5.2.0 > > Attachments: OOZIE-3542-2.patch, OOZIE-3542-3.patch, > OOZIE-3542-4.patch, OOZIE-3542-amend-01.patch > > > Currently, ECPolicyDisabler checks if the local hdfs implementation has the > necessary methods to get and set erasure coding policy. However, if the > namenode implementation is old, it could throw a > org.apache.hadoop.ipc.RemoteException with > RpcErrorCodeProto.ERROR_NO_SUCH_METHOD value in it. > In this case, ECPolicyDisabler fails, and prevents the installation to > succeed. > This case should be handled just like, when erasure coding is not supported. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OOZIE-3533) Flaky test TestXLogService.testLog4jReload
[
https://issues.apache.org/jira/browse/OOZIE-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16935681#comment-16935681
]
Julia Kinga Marton commented on OOZIE-3533:
---
[~asalamon74] Option 2 is the most sympathetic to me.
Since only sometimes the 1000 ms is not enough, I don't think that we should
force a bigger sleep value for every case.
> Flaky test TestXLogService.testLog4jReload
> --
>
> Key: OOZIE-3533
> URL: https://issues.apache.org/jira/browse/OOZIE-3533
> Project: Oozie
> Issue Type: Sub-task
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Minor
>
> Sometimes this test fails with the following error message:
> {noformat}
> junit.framework.AssertionFailedError
> at junit.framework.Assert.fail(Assert.java:55)
> at junit.framework.Assert.assertTrue(Assert.java:22)
> at junit.framework.Assert.assertFalse(Assert.java:39)
> at junit.framework.Assert.assertFalse(Assert.java:47)
> at junit.framework.TestCase.assertFalse(TestCase.java:219)
> at
> org.apache.oozie.service.TestXLogService.testLog4jReload(TestXLogService.java:111)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at org.junit.runners.Suite.runChild(Suite.java:127)
> at org.junit.runners.Suite.runChild(Suite.java:26)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runners.Suite.runChild(Suite.java:127)
> at org.junit.runners.Suite.runChild(Suite.java:26)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at
> org.apache.maven.surefire.junitcore.pc.Scheduler$1.run(Scheduler.java:410)
> at
> org.apache.maven.surefire.junitcore.pc.InvokerStrategy.schedule(InvokerStrategy.java:54)
> at
> org.apache.maven.surefire.junitcore.pc.Scheduler.schedule(Scheduler.java:367)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at
> org.apache.maven.surefire.junitcore.pc.ParallelComputerBuilder$PC$1.run(ParallelComputerBuilder.java:593)
> at org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeEager(JUnitCoreWrapper.java:107)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:83)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:373)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:334)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:119)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:407)
> {noformat}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3542) Handle better old Hdfs implementations in ECPolicyDisabler
[ https://issues.apache.org/jira/browse/OOZIE-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928278#comment-16928278 ] Julia Kinga Marton commented on OOZIE-3542: --- Thank you [~zsombor] and [~dionusos] for the fix! +1, committed to master > Handle better old Hdfs implementations in ECPolicyDisabler > -- > > Key: OOZIE-3542 > URL: https://issues.apache.org/jira/browse/OOZIE-3542 > Project: Oozie > Issue Type: Bug > Components: tools >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Attachments: OOZIE-3542-2.patch, OOZIE-3542-3.patch, > OOZIE-3542-4.patch > > > Currently, ECPolicyDisabler checks if the local hdfs implementation has the > necessary methods to get and set erasure coding policy. However, if the > namenode implementation is old, it could throw a > org.apache.hadoop.ipc.RemoteException with > RpcErrorCodeProto.ERROR_NO_SUCH_METHOD value in it. > In this case, ECPolicyDisabler fails, and prevents the installation to > succeed. > This case should be handled just like, when erasure coding is not supported. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (OOZIE-3542) Handle better old Hdfs implementations in ECPolicyDisabler
[
https://issues.apache.org/jira/browse/OOZIE-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16927534#comment-16927534
]
Julia Kinga Marton commented on OOZIE-3542:
---
[~dionusos], I yesterday [~zsombor] mentioned offline that the
{{SystemErasureCodingPolicies}} is called using reflection, so is not a good
idea to rename it. I think that the test failures are caused by this rename.
Sorry for the confusion.
> Handle better old Hdfs implementations in ECPolicyDisabler
> --
>
> Key: OOZIE-3542
> URL: https://issues.apache.org/jira/browse/OOZIE-3542
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Zsombor Gegesy
>Assignee: Zsombor Gegesy
>Priority: Major
> Attachments: OOZIE-3542-2.patch, OOZIE-3542-3.patch
>
>
> Currently, ECPolicyDisabler checks if the local hdfs implementation has the
> necessary methods to get and set erasure coding policy. However, if the
> namenode implementation is old, it could throw a
> org.apache.hadoop.ipc.RemoteException with
> RpcErrorCodeProto.ERROR_NO_SUCH_METHOD value in it.
> In this case, ECPolicyDisabler fails, and prevents the installation to
> succeed.
> This case should be handled just like, when erasure coding is not supported.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (OOZIE-3542) Handle better old Hdfs implementations in ECPolicyDisabler
[ https://issues.apache.org/jira/browse/OOZIE-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16927333#comment-16927333 ] Julia Kinga Marton commented on OOZIE-3542: --- Thanks [~dionusos]! LGTM, +1, let's wait for the pending Jenkins and I will merge the change. > Handle better old Hdfs implementations in ECPolicyDisabler > -- > > Key: OOZIE-3542 > URL: https://issues.apache.org/jira/browse/OOZIE-3542 > Project: Oozie > Issue Type: Bug > Components: tools >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Attachments: OOZIE-3542-2.patch, OOZIE-3542-3.patch > > > Currently, ECPolicyDisabler checks if the local hdfs implementation has the > necessary methods to get and set erasure coding policy. However, if the > namenode implementation is old, it could throw a > org.apache.hadoop.ipc.RemoteException with > RpcErrorCodeProto.ERROR_NO_SUCH_METHOD value in it. > In this case, ECPolicyDisabler fails, and prevents the installation to > succeed. > This case should be handled just like, when erasure coding is not supported. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Comment Edited] (OOZIE-3542) Handle better old Hdfs implementations in ECPolicyDisabler
[ https://issues.apache.org/jira/browse/OOZIE-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926474#comment-16926474 ] Julia Kinga Marton edited comment on OOZIE-3542 at 9/10/19 9:40 AM: Thank you [~zsombor] for the fix. I would have two small comments related your patch: * Please rename the test class to TestECPolicyDisabler.java * Please mark somewhere in the name of SystemErasureCodingPolicies that is created for tested purposes, so when we are doing class search to know it even without opening it. (for example I would rename it to SystemErasureCodingPoliciesForTest) was (Author: kmarton): Thank you [~zsombor] for the fix. I would have two small comments related this: * Please rename the test class to TestECPolicyDisabler.java * Please mark somewhere in the name of SystemErasureCodingPolicies that is created for tested purposes, so when we are doing class search to know it even without opening it. (for example I would rename it to SystemErasureCodingPoliciesForTest) > Handle better old Hdfs implementations in ECPolicyDisabler > -- > > Key: OOZIE-3542 > URL: https://issues.apache.org/jira/browse/OOZIE-3542 > Project: Oozie > Issue Type: Bug > Components: tools >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Attachments: OOZIE-3542-2.patch > > > Currently, ECPolicyDisabler checks if the local hdfs implementation has the > necessary methods to get and set erasure coding policy. However, if the > namenode implementation is old, it could throw a > org.apache.hadoop.ipc.RemoteException with > RpcErrorCodeProto.ERROR_NO_SUCH_METHOD value in it. > In this case, ECPolicyDisabler fails, and prevents the installation to > succeed. > This case should be handled just like, when erasure coding is not supported. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (OOZIE-3542) Handle better old Hdfs implementations in ECPolicyDisabler
[ https://issues.apache.org/jira/browse/OOZIE-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926474#comment-16926474 ] Julia Kinga Marton commented on OOZIE-3542: --- Thank you [~zsombor] for the fix. I would have two small comments related this: * Please rename the test class to TestECPolicyDisabler.java * Please mark somewhere in the name of SystemErasureCodingPolicies that is created for tested purposes, so when we are doing class search to know it even without opening it. (for example I would rename it to SystemErasureCodingPoliciesForTest) > Handle better old Hdfs implementations in ECPolicyDisabler > -- > > Key: OOZIE-3542 > URL: https://issues.apache.org/jira/browse/OOZIE-3542 > Project: Oozie > Issue Type: Bug > Components: tools >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Attachments: OOZIE-3542-2.patch > > > Currently, ECPolicyDisabler checks if the local hdfs implementation has the > necessary methods to get and set erasure coding policy. However, if the > namenode implementation is old, it could throw a > org.apache.hadoop.ipc.RemoteException with > RpcErrorCodeProto.ERROR_NO_SUCH_METHOD value in it. > In this case, ECPolicyDisabler fails, and prevents the installation to > succeed. > This case should be handled just like, when erasure coding is not supported. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Comment Edited] (OOZIE-3405) SSH action shows empty error Message and Error code
[
https://issues.apache.org/jira/browse/OOZIE-3405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16924366#comment-16924366
]
Julia Kinga Marton edited comment on OOZIE-3405 at 9/6/19 4:16 PM:
---
[~matijhs], I can see in the review that you have uploaded multiple patches.
Can you please upload your patch here as well, to trigger the precommit script,
or this one is the last one?
was (Author: kmarton):
[~matijhs], I can see in the review that you have uploaded multiple patches.
Can you please upload your patch here as well, to trigger the precommit script?
> SSH action shows empty error Message and Error code
> ---
>
> Key: OOZIE-3405
> URL: https://issues.apache.org/jira/browse/OOZIE-3405
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: 5.1.0
>Reporter: Peter Orova
>Assignee: Mate Juhasz
>Priority: Minor
> Fix For: trunk
>
> Attachments: OOZIE-3405-V1.patch
>
>
> Currently, when an SSH action fails the only message that is returned is the
> Status. Neither the {{error Message}} nor {{Error code}} fields are filled.
> This makes reporting on the causes of SSH Action failures via Oozie highly
> impractical: the only meaningful bit of information there is on a failed SSH
> Action is the Status.
> The Status is filled based on the presence (or lack of) the {{.error file}}
> that is produced in case the user submitted script returns with any other
> value than 0.
> {noformat}
> SshActionExecutor#getActionStatus
> ...
> String outFile = getRemoteFileName(context, action, "error", false, true);
> String checkErrorCmd = SSH_COMMAND_BASE + action.getTrackerUri() + " ls " +
> outFile;
> int retVal = getReturnValue(checkErrorCmd);
> ...
> {noformat}
>
> User requirement is to provide some more detailed information on the
> success/failure of the user-submitted script. That could be at a minimum the
> return value, optionally the last ~1K of the stderr that is drained. This
> information could then be communicated via {{errorMessage}} and {{ErrorCode}}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (OOZIE-3405) SSH action shows empty error Message and Error code
[
https://issues.apache.org/jira/browse/OOZIE-3405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16924366#comment-16924366
]
Julia Kinga Marton commented on OOZIE-3405:
---
[~matijhs], I can see in the review that you have uploaded multiple patches.
Can you please upload your patch here as well, to trigger the precommit script?
> SSH action shows empty error Message and Error code
> ---
>
> Key: OOZIE-3405
> URL: https://issues.apache.org/jira/browse/OOZIE-3405
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: 5.1.0
>Reporter: Peter Orova
>Assignee: Mate Juhasz
>Priority: Minor
> Fix For: trunk
>
> Attachments: OOZIE-3405-V1.patch
>
>
> Currently, when an SSH action fails the only message that is returned is the
> Status. Neither the {{error Message}} nor {{Error code}} fields are filled.
> This makes reporting on the causes of SSH Action failures via Oozie highly
> impractical: the only meaningful bit of information there is on a failed SSH
> Action is the Status.
> The Status is filled based on the presence (or lack of) the {{.error file}}
> that is produced in case the user submitted script returns with any other
> value than 0.
> {noformat}
> SshActionExecutor#getActionStatus
> ...
> String outFile = getRemoteFileName(context, action, "error", false, true);
> String checkErrorCmd = SSH_COMMAND_BASE + action.getTrackerUri() + " ls " +
> outFile;
> int retVal = getReturnValue(checkErrorCmd);
> ...
> {noformat}
>
> User requirement is to provide some more detailed information on the
> success/failure of the user-submitted script. That could be at a minimum the
> return value, optionally the last ~1K of the stderr that is drained. This
> information could then be communicated via {{errorMessage}} and {{ErrorCode}}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (OOZIE-3540) Use StringBuilder instead of StringBuffer if concurrent access is not required
[
https://issues.apache.org/jira/browse/OOZIE-3540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16920666#comment-16920666
]
Julia Kinga Marton commented on OOZIE-3540:
---
[~zsombor], unfortunately the spotbugs check is buggy and there are a lot of
old bugs reported as new ones. (we have an open Jira for replacing the spotbugs
diff jar OOZIE-3445).
The "newly" reported bug in this case is an old one in the {{ShellMain.java}}
file [line 92].
> Use StringBuilder instead of StringBuffer if concurrent access is not required
> --
>
> Key: OOZIE-3540
> URL: https://issues.apache.org/jira/browse/OOZIE-3540
> Project: Oozie
> Issue Type: Improvement
> Components: core
>Affects Versions: trunk
>Reporter: Zsombor Gegesy
>Assignee: Zsombor Gegesy
>Priority: Minor
> Fix For: 5.2.0
>
> Attachments: OOZIE-3540.patch
>
>
> StringBuffer is an old relic from the distant past, Oozie shouldnt' need to
> use it, unless concurrent access is expected.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (OOZIE-3360) Upgrade to Log4j2
[ https://issues.apache.org/jira/browse/OOZIE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton reassigned OOZIE-3360: - Assignee: (was: Julia Kinga Marton) > Upgrade to Log4j2 > - > > Key: OOZIE-3360 > URL: https://issues.apache.org/jira/browse/OOZIE-3360 > Project: Oozie > Issue Type: Improvement >Reporter: Julia Kinga Marton >Priority: Major > > The Log4j™ 1.x logging framework has reached its end of life (EOL) and is no > longer officially supported. > We should upgrade the Oozie server from Log4j1 to Log4j2 and add Log4j2 > support for Sharelib. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (OOZIE-3359) Check for Process#waitFor() usage and fix it in order to avoid indefinite waiting
[
https://issues.apache.org/jira/browse/OOZIE-3359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton reassigned OOZIE-3359:
-
Assignee: (was: Julia Kinga Marton)
> Check for Process#waitFor() usage and fix it in order to avoid indefinite
> waiting
> -
>
> Key: OOZIE-3359
> URL: https://issues.apache.org/jira/browse/OOZIE-3359
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Julia Kinga Marton
>Priority: Major
> Fix For: 5.2.0
>
>
> {{Process#waitFor()}} will block until the process finishes. There are
> situations where this will wait indefinitely. A similar case was in
> SshActionExecutor, fixed in OOZIE-3354.
> Let's check the code for other usages, and fix it.
> Maybe we should check if we can use somehow the Process#waitFor(long timeout,
> TimeUnit timeUnit) instead of it. The tricky part is that this method is
> available only in Java 8.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (OOZIE-3536) oozie-main(pom.xml) plugin maven-javadoc-plugin upgrade version caused configuration can't find the Tag
[
https://issues.apache.org/jira/browse/OOZIE-3536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918479#comment-16918479
]
Julia Kinga Marton commented on OOZIE-3536:
---
[~nobigo], According to the [stackoverflow
link|https://stackoverflow.com/questions/52547306/maven-javadoc-plugin-not-accepting-additionalparam-xdoclintnone-additionalpa]
you have shared, the javadoc generation should fail because of this issue, but
{{mvn javadoc:javadoc }}succeeds.
Can you please share what failed, and how can we reproduce the issue?
> oozie-main(pom.xml) plugin maven-javadoc-plugin upgrade version caused
> configuration can't find the Tag
> --
>
> Key: OOZIE-3536
> URL: https://issues.apache.org/jira/browse/OOZIE-3536
> Project: Oozie
> Issue Type: Bug
> Components: build
>Affects Versions: trunk, 5.1.0, 5.2.0
>Reporter: duan xiong
>Assignee: duan xiong
>Priority: Major
> Fix For: 5.2.0
>
> Attachments: OOZIE-3536-001.patch
>
>
> oozie-main(pom.xml) plugin maven-javadoc-plugin upgrade 3.0.1version caused
> configuration can't find the Tag . In early version
> 2.10.4,this plugin has this tag
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (OOZIE-3468) [build] Use modernizer plugin
[
https://issues.apache.org/jira/browse/OOZIE-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918458#comment-16918458
]
Julia Kinga Marton commented on OOZIE-3468:
---
hank you [~asalamon74] for the contribution! +1, committed to master
> [build] Use modernizer plugin
> -
>
> Key: OOZIE-3468
> URL: https://issues.apache.org/jira/browse/OOZIE-3468
> Project: Oozie
> Issue Type: Improvement
> Components: build
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3468-01-wip.patch, OOZIE-3468-02-wip.patch,
> OOZIE-3468-03-wip.patch, OOZIE-3468-04.patch, OOZIE-3468-06.patch,
> OOZIE-3468-07.patch
>
>
> Recently I've opened a few jiras which suggested to use standard java classes
> instead of external libraries ( OOZIE-3463, OOZIE-3467). There is a tool
> which can find such technical depts: [maven modernizer
> plugin|https://github.com/gaul/modernizer-maven-plugin].
> The usage is quite simple:
> {noformat}
> $ mvn modernizer:modernizer
> ...
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:88:
> Prefer java.lang.String.(byte[], java.nio.charset.Charset)
>
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:122:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:85:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:92:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1JobsServlet.java:188:
> Prefer java.util.ArrayList<>()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:91:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:101:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:110:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:119:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/VersionServlet.java:36:
> Prefer java.util.Collections.emptyList()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1AdminServlet.java:49:
> Prefer java.util.Collections.emptyList()
> ...
> {noformat}
> There are several ways to use this plugin:
> # Add the plugin to the root pom and let developers manually use this
> plugin. It's the simplest solution, but it will be easy to forget it.
> # Add this to the precommit script similarly to findbugs and at least avoid
> to insert new code which uses old style API. Probably we will have the same
> problems like we have with findbugs, we will have lots of false positive
> warnings.
> # Turn the plugin on by default and fail the compilation if it finds any
> problem. I think this is too strict.
> If we choose option 2 or 3 we should probably specify an ignore list, I don't
> think for instance that we really want to change all the {{new Long(10)}}
> code to {{Long.valueOf(10)}}.
> By default this plugin checks the target java version (1.8 right now) but
> it's possible to specify 1.7 instead if we want to focus on those problems
> first. (It was not working for me without specifying the java version.)
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Comment Edited] (OOZIE-3468) [build] Use modernizer plugin
[
https://issues.apache.org/jira/browse/OOZIE-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918458#comment-16918458
]
Julia Kinga Marton edited comment on OOZIE-3468 at 8/29/19 9:53 AM:
Thank you [~asalamon74] for the contribution! +1, committed to master
was (Author: kmarton):
hank you [~asalamon74] for the contribution! +1, committed to master
> [build] Use modernizer plugin
> -
>
> Key: OOZIE-3468
> URL: https://issues.apache.org/jira/browse/OOZIE-3468
> Project: Oozie
> Issue Type: Improvement
> Components: build
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3468-01-wip.patch, OOZIE-3468-02-wip.patch,
> OOZIE-3468-03-wip.patch, OOZIE-3468-04.patch, OOZIE-3468-06.patch,
> OOZIE-3468-07.patch
>
>
> Recently I've opened a few jiras which suggested to use standard java classes
> instead of external libraries ( OOZIE-3463, OOZIE-3467). There is a tool
> which can find such technical depts: [maven modernizer
> plugin|https://github.com/gaul/modernizer-maven-plugin].
> The usage is quite simple:
> {noformat}
> $ mvn modernizer:modernizer
> ...
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:88:
> Prefer java.lang.String.(byte[], java.nio.charset.Charset)
>
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:122:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:85:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:92:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1JobsServlet.java:188:
> Prefer java.util.ArrayList<>()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:91:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:101:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:110:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:119:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/VersionServlet.java:36:
> Prefer java.util.Collections.emptyList()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1AdminServlet.java:49:
> Prefer java.util.Collections.emptyList()
> ...
> {noformat}
> There are several ways to use this plugin:
> # Add the plugin to the root pom and let developers manually use this
> plugin. It's the simplest solution, but it will be easy to forget it.
> # Add this to the precommit script similarly to findbugs and at least avoid
> to insert new code which uses old style API. Probably we will have the same
> problems like we have with findbugs, we will have lots of false positive
> warnings.
> # Turn the plugin on by default and fail the compilation if it finds any
> problem. I think this is too strict.
> If we choose option 2 or 3 we should probably specify an ignore list, I don't
> think for instance that we really want to change all the {{new Long(10)}}
> code to {{Long.valueOf(10)}}.
> By default this plugin checks the target java version (1.8 right now) but
> it's possible to specify 1.7 instead if we want to focus on those problems
> first. (It was not working for me without specifying the java version.)
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (OOZIE-3468) [build] Use modernizer plugin
[
https://issues.apache.org/jira/browse/OOZIE-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton updated OOZIE-3468:
--
Summary: [build] Use modernizer plugin (was: Use modernizer plugin)
> [build] Use modernizer plugin
> -
>
> Key: OOZIE-3468
> URL: https://issues.apache.org/jira/browse/OOZIE-3468
> Project: Oozie
> Issue Type: Improvement
> Components: build
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3468-01-wip.patch, OOZIE-3468-02-wip.patch,
> OOZIE-3468-03-wip.patch, OOZIE-3468-04.patch, OOZIE-3468-06.patch,
> OOZIE-3468-07.patch
>
>
> Recently I've opened a few jiras which suggested to use standard java classes
> instead of external libraries ( OOZIE-3463, OOZIE-3467). There is a tool
> which can find such technical depts: [maven modernizer
> plugin|https://github.com/gaul/modernizer-maven-plugin].
> The usage is quite simple:
> {noformat}
> $ mvn modernizer:modernizer
> ...
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:88:
> Prefer java.lang.String.(byte[], java.nio.charset.Charset)
>
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:122:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:85:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:92:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1JobsServlet.java:188:
> Prefer java.util.ArrayList<>()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:91:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:101:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:110:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:119:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/VersionServlet.java:36:
> Prefer java.util.Collections.emptyList()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1AdminServlet.java:49:
> Prefer java.util.Collections.emptyList()
> ...
> {noformat}
> There are several ways to use this plugin:
> # Add the plugin to the root pom and let developers manually use this
> plugin. It's the simplest solution, but it will be easy to forget it.
> # Add this to the precommit script similarly to findbugs and at least avoid
> to insert new code which uses old style API. Probably we will have the same
> problems like we have with findbugs, we will have lots of false positive
> warnings.
> # Turn the plugin on by default and fail the compilation if it finds any
> problem. I think this is too strict.
> If we choose option 2 or 3 we should probably specify an ignore list, I don't
> think for instance that we really want to change all the {{new Long(10)}}
> code to {{Long.valueOf(10)}}.
> By default this plugin checks the target java version (1.8 right now) but
> it's possible to specify 1.7 instead if we want to focus on those problems
> first. (It was not working for me without specifying the java version.)
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (OOZIE-3468) Use modernizer plugin
[
https://issues.apache.org/jira/browse/OOZIE-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16902065#comment-16902065
]
Julia Kinga Marton commented on OOZIE-3468:
---
[~asalamon74], I have some small comments related the actual patch:
* in the modernizer script the PATCHFILE option is not used. can you please
remove it?
* _"REPORT+=("\{color:red}-1\{color} the patch introduces ${newErrorInClass}
new warnings in ${class_name}")",_ please replace warning with warning(s).
> Use modernizer plugin
> -
>
> Key: OOZIE-3468
> URL: https://issues.apache.org/jira/browse/OOZIE-3468
> Project: Oozie
> Issue Type: Improvement
> Components: build
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3468-01-wip.patch, OOZIE-3468-02-wip.patch,
> OOZIE-3468-03-wip.patch, OOZIE-3468-04.patch, OOZIE-3468-06.patch
>
>
> Recently I've opened a few jiras which suggested to use standard java classes
> instead of external libraries ( OOZIE-3463, OOZIE-3467). There is a tool
> which can find such technical depts: [maven modernizer
> plugin|https://github.com/gaul/modernizer-maven-plugin].
> The usage is quite simple:
> {noformat}
> $ mvn modernizer:modernizer
> ...
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:88:
> Prefer java.lang.String.(byte[], java.nio.charset.Charset)
>
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:122:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:85:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:92:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1JobsServlet.java:188:
> Prefer java.util.ArrayList<>()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:91:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:101:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:110:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:119:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/VersionServlet.java:36:
> Prefer java.util.Collections.emptyList()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1AdminServlet.java:49:
> Prefer java.util.Collections.emptyList()
> ...
> {noformat}
> There are several ways to use this plugin:
> # Add the plugin to the root pom and let developers manually use this
> plugin. It's the simplest solution, but it will be easy to forget it.
> # Add this to the precommit script similarly to findbugs and at least avoid
> to insert new code which uses old style API. Probably we will have the same
> problems like we have with findbugs, we will have lots of false positive
> warnings.
> # Turn the plugin on by default and fail the compilation if it finds any
> problem. I think this is too strict.
> If we choose option 2 or 3 we should probably specify an ignore list, I don't
> think for instance that we really want to change all the {{new Long(10)}}
> code to {{Long.valueOf(10)}}.
> By default this plugin checks the target java version (1.8 right now) but
> it's possible to specify 1.7 instead if we want to focus on those problems
> first. (It was not working for me without specifying the java version.)
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Comment Edited] (OOZIE-3468) Use modernizer plugin
[
https://issues.apache.org/jira/browse/OOZIE-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16891862#comment-16891862
]
Julia Kinga Marton edited comment on OOZIE-3468 at 8/7/19 11:59 AM:
[~asalamon74] can you please check your code with ShellCheck and fix the
reported issues?
was (Author: kmarton):
[~asalamon74] can you please check tour code with ShellCheck and fix the
reported issues?
> Use modernizer plugin
> -
>
> Key: OOZIE-3468
> URL: https://issues.apache.org/jira/browse/OOZIE-3468
> Project: Oozie
> Issue Type: Improvement
> Components: build
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3468-01-wip.patch, OOZIE-3468-02-wip.patch,
> OOZIE-3468-03-wip.patch, OOZIE-3468-04.patch, OOZIE-3468-06.patch
>
>
> Recently I've opened a few jiras which suggested to use standard java classes
> instead of external libraries ( OOZIE-3463, OOZIE-3467). There is a tool
> which can find such technical depts: [maven modernizer
> plugin|https://github.com/gaul/modernizer-maven-plugin].
> The usage is quite simple:
> {noformat}
> $ mvn modernizer:modernizer
> ...
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:88:
> Prefer java.lang.String.(byte[], java.nio.charset.Charset)
>
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:122:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:85:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:92:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1JobsServlet.java:188:
> Prefer java.util.ArrayList<>()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:91:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:101:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:110:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:119:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/VersionServlet.java:36:
> Prefer java.util.Collections.emptyList()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1AdminServlet.java:49:
> Prefer java.util.Collections.emptyList()
> ...
> {noformat}
> There are several ways to use this plugin:
> # Add the plugin to the root pom and let developers manually use this
> plugin. It's the simplest solution, but it will be easy to forget it.
> # Add this to the precommit script similarly to findbugs and at least avoid
> to insert new code which uses old style API. Probably we will have the same
> problems like we have with findbugs, we will have lots of false positive
> warnings.
> # Turn the plugin on by default and fail the compilation if it finds any
> problem. I think this is too strict.
> If we choose option 2 or 3 we should probably specify an ignore list, I don't
> think for instance that we really want to change all the {{new Long(10)}}
> code to {{Long.valueOf(10)}}.
> By default this plugin checks the target java version (1.8 right now) but
> it's possible to specify 1.7 instead if we want to focus on those problems
> first. (It was not working for me without specifying the java version.)
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (OOZIE-3535) mapreduce.job.acl-view-job property in Oozie workflow.xml not taking full effect
[
https://issues.apache.org/jira/browse/OOZIE-3535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton updated OOZIE-3535:
--
Attachment: OOZIE-3535-001.patch
> mapreduce.job.acl-view-job property in Oozie workflow.xml not taking full
> effect
>
>
> Key: OOZIE-3535
> URL: https://issues.apache.org/jira/browse/OOZIE-3535
> Project: Oozie
> Issue Type: Bug
>Reporter: Julia Kinga Marton
>Assignee: Julia Kinga Marton
>Priority: Major
> Attachments: OOZIE-3535-001.patch
>
>
> mapreduce.job.acl-view-job property in Oozie workflow.xml will be ignored if
> only {{yarn.acl.enable}} = true, but {{mapreduce.cluster.acls.enabled}} =
> false.
> This is caused by the following check:
> [https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/YarnACLHandler.java#L39]
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (OOZIE-3535) mapreduce.job.acl-view-job property in Oozie workflow.xml not taking full effect
Julia Kinga Marton created OOZIE-3535:
-
Summary: mapreduce.job.acl-view-job property in Oozie workflow.xml
not taking full effect
Key: OOZIE-3535
URL: https://issues.apache.org/jira/browse/OOZIE-3535
Project: Oozie
Issue Type: Bug
Reporter: Julia Kinga Marton
Assignee: Julia Kinga Marton
mapreduce.job.acl-view-job property in Oozie workflow.xml will be ignored if
only {{yarn.acl.enable}} = true, but {{mapreduce.cluster.acls.enabled}} = false.
This is caused by the following check:
[https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/YarnACLHandler.java#L39]
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (OOZIE-3534) Refactor ActionExecutorTestCase to not extend HCatTestCase
[ https://issues.apache.org/jira/browse/OOZIE-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16900039#comment-16900039 ] Julia Kinga Marton commented on OOZIE-3534: --- The task is not so trivial, because after OOZIE-2287 a lot of classes got dependent on this HCatTestCase even if only a few ones really needs it. > Refactor ActionExecutorTestCase to not extend HCatTestCase > -- > > Key: OOZIE-3534 > URL: https://issues.apache.org/jira/browse/OOZIE-3534 > Project: Oozie > Issue Type: Improvement > Components: tests >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > > ActionExecutorTestCase extends HCatTestCase but there are only two test cases > where we need HCatalog related settings as well. > Let's try to refactor this part and make the ActionExecturTestCase > independent from the HCatalog part. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Assigned] (OOZIE-3214) Allow configurable timezone for coordinators
[
https://issues.apache.org/jira/browse/OOZIE-3214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton reassigned OOZIE-3214:
-
Assignee: (was: Julia Kinga Marton)
> Allow configurable timezone for coordinators
>
>
> Key: OOZIE-3214
> URL: https://issues.apache.org/jira/browse/OOZIE-3214
> Project: Oozie
> Issue Type: New Feature
> Components: coordinator
>Affects Versions: trunk
>Reporter: Julia Kinga Marton
>Priority: Major
>
> Right now in case of coordinators only daylight saving time corrections are
> applied, the processing timezone for the coordinators is always the Oozie
> processing timezone.
> It would be more transparent to have an option for changing the {{timezone}}
> attribute itself, such as an additional attribute in the {{coordinator.xml}}
> (meaning a new version of {{coordinator.xsd}}). I would make this option
> switched off by default(to have the actual behavior).
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Assigned] (OOZIE-3177) Unclear ooziedb -sqlFile option behaviour
[ https://issues.apache.org/jira/browse/OOZIE-3177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton reassigned OOZIE-3177: - Assignee: (was: Julia Kinga Marton) > Unclear ooziedb -sqlFile option behaviour > -- > > Key: OOZIE-3177 > URL: https://issues.apache.org/jira/browse/OOZIE-3177 > Project: Oozie > Issue Type: Bug > Components: scripts >Reporter: Julia Kinga Marton >Priority: Minor > > There are two scenarios when running ooziedb.sh with -sqlscript option can > lead to misunderstandings: > full command: ooziedb.sh upgrade -sqlfile oozie-upgrade.sql > # if there are no changes, the following information is printed out: "The > SQL commands have been written to: oozie-upgrade.sql", however if there is no > SQL command, the file will be not created. > # If there are changes and the file is created, it is stored in the /tmp/ > directory. In the log message it would be good to print out the full path to > the file, not only the file name. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Assigned] (OOZIE-2231) Upgrade curator to latest version 2.13.0
[ https://issues.apache.org/jira/browse/OOZIE-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton reassigned OOZIE-2231: - Assignee: (was: Julia Kinga Marton) > Upgrade curator to latest version 2.13.0 > > > Key: OOZIE-2231 > URL: https://issues.apache.org/jira/browse/OOZIE-2231 > Project: Oozie > Issue Type: Bug > Components: HA >Reporter: Purshotam Shah >Priority: Blocker > Fix For: 5.2.0 > > Attachments: OOZIE-2231-00.patch, OOZIE-2231-01.patch, > OOZIE-2231-02.patch, OOZIE-2231-02.patch, OOZIE-2231-03.patch, > OOZIE-2231-04.patch, OOZIE-2231-05.patch, OOZIE-2231-06.patch > > > It have some fix related to InterProcessReadWriteLock, ChildReaper, > LeaderSelector which we use. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Assigned] (OOZIE-3350) Forkjoin validation does not fail if a node is reachable from two different forks
[
https://issues.apache.org/jira/browse/OOZIE-3350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton reassigned OOZIE-3350:
-
Assignee: (was: Julia Kinga Marton)
> Forkjoin validation does not fail if a node is reachable from two different
> forks
> -
>
> Key: OOZIE-3350
> URL: https://issues.apache.org/jira/browse/OOZIE-3350
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.3.1
>Reporter: wang jinyin
>Priority: Major
> Fix For: trunk
>
> Original Estimate: 120h
> Remaining Estimate: 120h
>
> when "multiple ok to same node" under decision node, forkjoin validation
> error.
>
> such as below example, 'action_C' and 'action_D' both transition to
> 'action_E'.
> But, because they are under same topDecisionParent 'decision_A', validator
> will not throw any exception.
>
> {quote}
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Assigned] (OOZIE-3136) Upgrade from Log4j 1.x to 2.x
[
https://issues.apache.org/jira/browse/OOZIE-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton reassigned OOZIE-3136:
-
Assignee: (was: Julia Kinga Marton)
> Upgrade from Log4j 1.x to 2.x
> -
>
> Key: OOZIE-3136
> URL: https://issues.apache.org/jira/browse/OOZIE-3136
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Attila Sasvari
>Priority: Major
>
> {{5 August 2015 --The Apache Logging Services™ Project Management Committee
> (PMC) has announced that the Log4j™ 1.x logging framework has reached its end
> of life (EOL) and is no longer officially supported.}}
> https://blogs.apache.org/foundation/entry/apache_logging_services_project_announces
> We should upgrade from Log4j 1.x to 2.x . Perhaps we could use slf4j .
> Related tickets: MAPREDUCE-6983, HADOOP-12956, OOZIE-3135
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Assigned] (OOZIE-3336) [persistence] Refactor entity classes to feature PK, FK, and UQ constraints
[
https://issues.apache.org/jira/browse/OOZIE-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton reassigned OOZIE-3336:
-
Assignee: (was: Julia Kinga Marton)
> [persistence] Refactor entity classes to feature PK, FK, and UQ constraints
> ---
>
> Key: OOZIE-3336
> URL: https://issues.apache.org/jira/browse/OOZIE-3336
> Project: Oozie
> Issue Type: Improvement
> Components: core
>Affects Versions: 5.0.0
>Reporter: Andras Piros
>Priority: Major
> Fix For: 5.2.0
>
>
> When an Oozie database grows substantial in size, let's say, over a few
> hundred thousands of {{WorkflowActionBean}}, {{CoordinatorActionBean}}
> instances, we face a couple of performance issues. Here is an analysis why.
> Current Oozie JPA {{@Entity}} usage, and the resulting database DDL, suffers
> from a couple of drawback from a performance point of view:
> * {{@Id}} fields are {{String}}:
> ** leaving no space for database primary key indices to work effectively
> ** those values are calculated in case of {{WorkflowActionBean}},
> {{CoordinatorActionBean}}, and {{BundleActionBean}} instances
> * no foreign constraint is set from {{WorkflowActionBean}} to
> {{WorkflowJobBean}}, from {{CoordinatorActionBean}} to
> {{CoordinatorJobBean}}, or from {{BundleActionBean}} to {{BundleJobBean}}
> instances:
> ** have to assess JPA queries discovering parent-child relationships by hand
> ** no database indices are created, and hence, those queries that contain any
> {{JOIN}} instances are slower
> * no use of unique constraints whatsoever
> * JPA queries are created by hand instead of relying on OpenJPA
> * JPA entities are filled by hand instead of relying on OpenJPA
> Following enhancements are necessary:
> # keeping the existing {{String compositeId}} fields, let's break down the
> contents to following new fields:
> ## {{@Id long id}} - an auto-increment value that is unique across Oozie
> database
> ## {{long currentSequence}} - the sequence number of the current run since
> last Oozie server restart. The first part of the {{compositeId}}
> ## {{Timestamp serverStartupTimestamp}} - the timestamp when the Oozie server
> was last started. The second part of the {{compositeId}}
> ## {{String serverName}} - the third part of the {{compositeId}}
> ## {{String name}} - the fourth and last part of the {{compositeId}}
> ## {{compositeId}} might be calculated when an entity is loaded / persisted,
> and then stored
> # FK constraints:
> ## {{@OneToMany}} fields where we have a list of child references inside
> parent
> ## {{@ManyToOne}} fields where we have a parent reference inside child
> ## pay attention to {{FetchType}}, most of the times {{LAZY}} will be needed
> ## the containment fields should not be {{@Transient}} anymore
> # UQ constraints:
> ## on {{currentSequence}} and {{serverStartupTimestamp}}
> ## on {{currentSequence}} and {{name}}
> # new JPQL queries:
> ## to cover changed parent-child relationships
> ## to get use of each disassembled part of {{originalId}} when doing e.g.
> filtering
> # let JPA fill entities instead performing this by hand
> Following enhancements can be considered as nice-to-have:
> * upgrade to an OpenJPA version that features JPA 2.1's composite indexing
> capability
> * see whether to have an optimistic locking field using {{@Version}} instead
> of ZooKeeper based pessimistic locking would increase High Availability
> characteristics
> * refactor also SLA related entity classes
> It's necessary to have performance benchmarks with some database types like
> MySQL/MariaDB, and PostgreSQL before and after the changes for following use
> cases:
> * {{CoordinatorJobBean}} and {{WorkflowJobBean}} instances up to millions
> * {{CoordinatorActionBean}} and {{WorkflowActionBean}} instances up to tens
> of millions
> * performance for JPQLs that get a list of entities
> * performance of persisting a new entity
> * performance of querying lists of entities based on popular / possible
> filters like the ones used by {{VxJobsServlet}}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Assigned] (OOZIE-3469) Extract common methods, fields from oozie-core to a new oozie-common module
[ https://issues.apache.org/jira/browse/OOZIE-3469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton reassigned OOZIE-3469: - Assignee: (was: Julia Kinga Marton) > Extract common methods, fields from oozie-core to a new oozie-common module > --- > > Key: OOZIE-3469 > URL: https://issues.apache.org/jira/browse/OOZIE-3469 > Project: Oozie > Issue Type: Improvement >Reporter: Julia Kinga Marton >Priority: Major > > Oozie sharelib needs oozie-core as dependency, what will bring in a lot of > transitive dependencies, what may cause conflicts and we need to exclude them > one by one. In a lot of cases we need only some constants from oozie-core. > Let's investigate whether we can extract this common fields, > (methods/classes) into a new common module. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Assigned] (OOZIE-3135) Configure log4j2 in SqoopMain
[
https://issues.apache.org/jira/browse/OOZIE-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton reassigned OOZIE-3135:
-
Assignee: (was: Julia Kinga Marton)
> Configure log4j2 in SqoopMain
> -
>
> Key: OOZIE-3135
> URL: https://issues.apache.org/jira/browse/OOZIE-3135
> Project: Oozie
> Issue Type: Sub-task
>Affects Versions: 5.0.0b1
>Reporter: Attila Sasvari
>Priority: Major
> Attachments: OOZIE-3135-001.patch
>
>
> In Hadoop 3, MAPREDUCE-6983 switched to use slfj4 & log4j2 in
> {{org.apache.hadoop.mapreduce.Job}} (that prints out MR job id-s needed for
> Oozie). We need to setup log4j accordingly (it is also related to
> HADOOP-12956).
> Without proper configuration in the Sqoop action, we won't be able to get
> external job id-s (SqoopActionExecutor unit tests and real action would be
> also affected).
>
> [The API for Log4j 2 is not compatible with Log4j
> 1.x|https://logging.apache.org/log4j/2.x/], but we will need to support both
> hadoop 2 and hadoop 3 profiles for a while.
> We could use reflection to determine the type of the logger object in
> {{org.apache.hadoop.mapreduce.Job}} and configure log4j settings based on it,
> but there might be a better way.
> For example we could do something like this:
> - add a new method for configuring log4j2:
> {code}
> private String setUpSqoopLog4J2(final String rootLogLevel) throws
> IOException {
> System.out.println("Setting up log4j2");
> final String logFile = getSqoopLogFile();
> final File log4j2Xml = new File(SQOOP_LOG4J2_XML);
> try (Writer writer = new FileWriter(log4j2Xml))
> {
> final String logj2SettingsXml = " encoding=\"UTF-8\"?>\n" +
> "\n" +
> "\n" +
> " target=\"SYSTEM_OUT\">\n" +
> " [%t] %-5level %logger{36} - %msg%n\"/>\n" +
> "\n" +
> " "\"> \n" +
> " [%t] %-5level %logger{36} - %msg%n\"/>\n" +
> " \n" +
> "\n" +
> "\n" +
> " "\">\n" +
> "\n" +
> "\n" +
> "\n" +
> "\n" +
> "";
> writer.write(logj2SettingsXml);
> }
> System.out.printf("log4j2 configuration file created at %s%n",
> log4j2Xml.getAbsolutePath());
> final LoggerContext context = (LoggerContext)
> LogManager.getContext(false);
> context.setConfigLocation(log4j2Xml.toURI()); // forces log4j2
> reconfiguration
> return logFile;
> }
> {code}
> and call it in the {{run()}} method if the mapreduce client is using slf4j
> for logging:
> {code}
> String logFile;
> // MAPREDUCE-6983 switches to slfj4 & log4j2. Need to setup log4j
> accordingly
> if
> (org.apache.hadoop.mapreduce.Job.class.getDeclaredField("LOG").getType().
> isAssignableFrom(org.slf4j.Logger.class)) {
> logFile = setUpSqoopLog4J2(rootLogLevel);
> }
> else {
> logFile = setUpSqoopLog4J(rootLogLevel, logLevel);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Assigned] (OOZIE-3356) Add blacklist for EL evaluation
[ https://issues.apache.org/jira/browse/OOZIE-3356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton reassigned OOZIE-3356: - Assignee: (was: Julia Kinga Marton) > Add blacklist for EL evaluation > - > > Key: OOZIE-3356 > URL: https://issues.apache.org/jira/browse/OOZIE-3356 > Project: Oozie > Issue Type: Improvement >Affects Versions: trunk >Reporter: Julia Kinga Marton >Priority: Major > > OOZIE-1580 fixed that the EL variables included using got fixed, > but there are cases when we pass some *-site.xml, and the variables have > different format. > For example when talking to hbase, we can pass the hbase-site.xml using > tag, and if there are some variables defined, we need to modify > them manually. > By adding a blacklist, when we don't want to evaluate the EL expressions, we > can avoid this manual interaction. > By default only the *-site.xml files should be ignored, but this should be > configurable. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Assigned] (OOZIE-3416) Incorporate dbd and dbd testing into Apache Oozie/tools
[ https://issues.apache.org/jira/browse/OOZIE-3416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton reassigned OOZIE-3416: - Assignee: (was: Julia Kinga Marton) > Incorporate dbd and dbd testing into Apache Oozie/tools > --- > > Key: OOZIE-3416 > URL: https://issues.apache.org/jira/browse/OOZIE-3416 > Project: Oozie > Issue Type: Task > Components: tools >Affects Versions: 5.1.0 >Reporter: Julia Kinga Marton >Priority: Major > > [~daniel.becker] implemented a dockerised integration testing framework. > It consists of two repositories: > * dbd - [https://github.com/d-becker/dbd]This tool’s aim is to make it easy > to set up a working dockerized big data infrastructure where the versions of > the components (Hadoop, Hive, Oozie etc.) can be set individually, and even > unreleased snapshot builds can be used. > * oozie-dbd-testing - [https://github.com/d-becker/oozie-dbd-testing] This > is the repository that contains the scripts and the Jenkins job definition > used to test Oozie. It runs the Oozie examples and the fluent job examples > and produces a report. To set up a Jenkins job, use the "Multibranch > pipeline" project type. > Now, that this tools are working, we should integrate them in the Apache > Oozie repository next to the other tools. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Assigned] (OOZIE-3301) Update NOTICE file
[ https://issues.apache.org/jira/browse/OOZIE-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton reassigned OOZIE-3301: - Assignee: (was: Julia Kinga Marton) > Update NOTICE file > -- > > Key: OOZIE-3301 > URL: https://issues.apache.org/jira/browse/OOZIE-3301 > Project: Oozie > Issue Type: Bug >Affects Versions: 5.0.0 >Reporter: Attila Sasvari >Priority: Major > Attachments: OOZIE-3301-001.patch, OOZIE-3301-002-reupload.patch, > THIRD-PARTY.txt > > > Oozie NOTICE file is not up to date. Please overview and update it as per > http://www.apache.org/licenses/LICENSE-2.0.html#redistribution > For example: Oozie uses embedded jetty, however Eclipse license > (http://www.eclipse.org/jetty/documentation/9.3.x/embedding-jetty.html) is > not listed in > [NOTICE.txt|https://github.com/apache/oozie/blob/a299d4a6d435a2c92cd1d0ffce7f35a2ef8d639b/NOTICE.txt#L9]. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (OOZIE-3534) Refactor ActionExecutorTestCase to not extend HCatTestCase
Julia Kinga Marton created OOZIE-3534: - Summary: Refactor ActionExecutorTestCase to not extend HCatTestCase Key: OOZIE-3534 URL: https://issues.apache.org/jira/browse/OOZIE-3534 Project: Oozie Issue Type: Improvement Components: tests Reporter: Julia Kinga Marton Assignee: Julia Kinga Marton ActionExecutorTestCase extends HCatTestCase but there are only two test cases where we need HCatalog related settings as well. Let's try to refactor this part and make the ActionExecturTestCase independent from the HCatalog part. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (OOZIE-3496) Upgrade ActiveMQ to 5.15.9
[ https://issues.apache.org/jira/browse/OOZIE-3496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898648#comment-16898648 ] Julia Kinga Marton commented on OOZIE-3496: --- Thank you [~asalamon74]! +1, committed to master > Upgrade ActiveMQ to 5.15.9 > -- > > Key: OOZIE-3496 > URL: https://issues.apache.org/jira/browse/OOZIE-3496 > Project: Oozie > Issue Type: Bug >Affects Versions: trunk >Reporter: Andras Salamon >Assignee: Andras Salamon >Priority: Minor > Attachments: OOZIE-3496-01-wip.patch, OOZIE-3496-02-wip.patch, > OOZIE-3496-03-wip.patch, OOZIE-3496-04.patch > > > Although we only use it for tests we should still upgrade ActiveMQ to the > latest 5.15.9 version from the current 5.15.3 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (OOZIE-3468) Use modernizer plugin
[
https://issues.apache.org/jira/browse/OOZIE-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16891862#comment-16891862
]
Julia Kinga Marton commented on OOZIE-3468:
---
[~asalamon74] can you please check tour code with ShellCheck and fix the
reported issues?
> Use modernizer plugin
> -
>
> Key: OOZIE-3468
> URL: https://issues.apache.org/jira/browse/OOZIE-3468
> Project: Oozie
> Issue Type: Improvement
> Components: build
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3468-01-wip.patch, OOZIE-3468-02-wip.patch,
> OOZIE-3468-03-wip.patch, OOZIE-3468-04.patch
>
>
> Recently I've opened a few jiras which suggested to use standard java classes
> instead of external libraries ( OOZIE-3463, OOZIE-3467). There is a tool
> which can find such technical depts: [maven modernizer
> plugin|https://github.com/gaul/modernizer-maven-plugin].
> The usage is quite simple:
> {noformat}
> $ mvn modernizer:modernizer
> ...
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:88:
> Prefer java.lang.String.(byte[], java.nio.charset.Charset)
>
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:122:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:85:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:92:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1JobsServlet.java:188:
> Prefer java.util.ArrayList<>()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:91:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:101:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:110:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:119:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/VersionServlet.java:36:
> Prefer java.util.Collections.emptyList()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1AdminServlet.java:49:
> Prefer java.util.Collections.emptyList()
> ...
> {noformat}
> There are several ways to use this plugin:
> # Add the plugin to the root pom and let developers manually use this
> plugin. It's the simplest solution, but it will be easy to forget it.
> # Add this to the precommit script similarly to findbugs and at least avoid
> to insert new code which uses old style API. Probably we will have the same
> problems like we have with findbugs, we will have lots of false positive
> warnings.
> # Turn the plugin on by default and fail the compilation if it finds any
> problem. I think this is too strict.
> If we choose option 2 or 3 we should probably specify an ignore list, I don't
> think for instance that we really want to change all the {{new Long(10)}}
> code to {{Long.valueOf(10)}}.
> By default this plugin checks the target java version (1.8 right now) but
> it's possible to specify 1.7 instead if we want to focus on those problems
> first. (It was not working for me without specifying the java version.)
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (OOZIE-3504) [Java 11] Fix tests using PowerMock
[ https://issues.apache.org/jira/browse/OOZIE-3504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16891716#comment-16891716 ] Julia Kinga Marton commented on OOZIE-3504: --- The test error is unrelated, however I retriggered the build. Let's wait for the new results. > [Java 11] Fix tests using PowerMock > --- > > Key: OOZIE-3504 > URL: https://issues.apache.org/jira/browse/OOZIE-3504 > Project: Oozie > Issue Type: Sub-task >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > Attachments: OOZIE-3504-001.patch > > > With JDK 11 the following tests are failing with initialisation error: > * TestScriptLanguageActionExecutor > * TestHCatCredentials > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (OOZIE-3504) [Java 11] Fix tests using PowerMock
[ https://issues.apache.org/jira/browse/OOZIE-3504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton updated OOZIE-3504: -- Attachment: OOZIE-3504-001.patch > [Java 11] Fix tests using PowerMock > --- > > Key: OOZIE-3504 > URL: https://issues.apache.org/jira/browse/OOZIE-3504 > Project: Oozie > Issue Type: Sub-task >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > Attachments: OOZIE-3504-001.patch > > > With JDK 11 the following tests are failing with initialisation error: > * TestScriptLanguageActionExecutor > * TestHCatCredentials > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (OOZIE-3504) [Java 11] Fix tests using PowerMock
[ https://issues.apache.org/jira/browse/OOZIE-3504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890971#comment-16890971 ] Julia Kinga Marton commented on OOZIE-3504: --- TestHCatCredentials is failing because of classloading issues: it seems that PowerMock has a custom classloader what nows nothing about the modules in JDK11. We can instruct Powermock to let the parent classloader to load some classes by using @PowerMockIgnore annotation. > [Java 11] Fix tests using PowerMock > --- > > Key: OOZIE-3504 > URL: https://issues.apache.org/jira/browse/OOZIE-3504 > Project: Oozie > Issue Type: Sub-task >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > Attachments: OOZIE-3504-001.patch > > > With JDK 11 the following tests are failing with initialisation error: > * TestScriptLanguageActionExecutor > * TestHCatCredentials > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (OOZIE-3504) [Java 11] Fix tests using PowerMock
[ https://issues.apache.org/jira/browse/OOZIE-3504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890889#comment-16890889 ] Julia Kinga Marton commented on OOZIE-3504: --- OOZIE-3528 solved the TestScriptLanguageActionExecutor failure, but TestHCatCredentials is still failing. > [Java 11] Fix tests using PowerMock > --- > > Key: OOZIE-3504 > URL: https://issues.apache.org/jira/browse/OOZIE-3504 > Project: Oozie > Issue Type: Sub-task >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > > With JDK 11 the following tests are failing with initialisation error: > * TestScriptLanguageActionExecutor > * TestHCatCredentials > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (OOZIE-3528) Migrate to PowerMock 2 and Mockito2
[ https://issues.apache.org/jira/browse/OOZIE-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890119#comment-16890119 ] Julia Kinga Marton commented on OOZIE-3528: --- testActionKillCommandDate:org.apache.oozie.command.coord.TestCoordActionsKillXCommand passed locally, so I am retriggering the precommit build > Migrate to PowerMock 2 and Mockito2 > --- > > Key: OOZIE-3528 > URL: https://issues.apache.org/jira/browse/OOZIE-3528 > Project: Oozie > Issue Type: Improvement >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > Attachments: OOZIE-3528-001.patch, OOZIE-3528-002.patch, > OOZIE-3528-003.patch, WIP_OOZIE-3528.patch > > > PowerMock 1 does not support JDK11. If we want to . support it we will need > to migrate to PoweMock 2 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (OOZIE-3528) Migrate to PowerMock 2 and Mockito2
[ https://issues.apache.org/jira/browse/OOZIE-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890016#comment-16890016 ] Julia Kinga Marton edited comment on OOZIE-3528 at 7/22/19 9:16 AM: [~asalamon74] below you can find my answers to your questions: * I will check with JDK11 during OOZIE-3504. In this fix my purpose was to do the upgrade, and to make everything with the new versions using JDK8 * You are right, we should use the newer version from branch 2. I have run some tests manually with 2.28.2, they passed, let's see what the precommit will show. * variable creation: done was (Author: kmarton): [~asalamon74] below you can find my answers to your questions: * I will check with JDK11 during OOZIE-3504. In this fix my purpose was to do the upgrade, and to make everything with the new versions using JDK8 * You are right, we should use the newer version from line2. I have run some tests manually with 2.28.2, they passed, let's see what the precommit will show. * variable creation: done > Migrate to PowerMock 2 and Mockito2 > --- > > Key: OOZIE-3528 > URL: https://issues.apache.org/jira/browse/OOZIE-3528 > Project: Oozie > Issue Type: Improvement >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > Attachments: OOZIE-3528-001.patch, OOZIE-3528-002.patch, > OOZIE-3528-003.patch, WIP_OOZIE-3528.patch > > > PowerMock 1 does not support JDK11. If we want to . support it we will need > to migrate to PoweMock 2 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (OOZIE-3528) Migrate to PowerMock 2 and Mockito2
[ https://issues.apache.org/jira/browse/OOZIE-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890016#comment-16890016 ] Julia Kinga Marton commented on OOZIE-3528: --- [~asalamon74] below you can find my answers to your questions: * I will check with JDK11 during OOZIE-3504. In this fix my purpose was to do the upgrade, and to make everything with the new versions using JDK8 * You are right, we should use the newer version from line2. I have run some tests manually with 2.28.2, they passed, let's see what the precommit will show. * variable creation: done > Migrate to PowerMock 2 and Mockito2 > --- > > Key: OOZIE-3528 > URL: https://issues.apache.org/jira/browse/OOZIE-3528 > Project: Oozie > Issue Type: Improvement >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > Attachments: OOZIE-3528-001.patch, OOZIE-3528-002.patch, > OOZIE-3528-003.patch, WIP_OOZIE-3528.patch > > > PowerMock 1 does not support JDK11. If we want to . support it we will need > to migrate to PoweMock 2 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (OOZIE-3528) Migrate to PowerMock 2 and Mockito2
[ https://issues.apache.org/jira/browse/OOZIE-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton updated OOZIE-3528: -- Attachment: OOZIE-3528-003.patch > Migrate to PowerMock 2 and Mockito2 > --- > > Key: OOZIE-3528 > URL: https://issues.apache.org/jira/browse/OOZIE-3528 > Project: Oozie > Issue Type: Improvement >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > Attachments: OOZIE-3528-001.patch, OOZIE-3528-002.patch, > OOZIE-3528-003.patch, WIP_OOZIE-3528.patch > > > PowerMock 1 does not support JDK11. If we want to . support it we will need > to migrate to PoweMock 2 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Resolved] (OOZIE-2882) Rerun workflow fails Error: E0404
[
https://issues.apache.org/jira/browse/OOZIE-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton resolved OOZIE-2882.
---
Resolution: Fixed
Fix Version/s: 5.2.0
This issue was fixed with OOZIE-3265
> Rerun workflow fails Error: E0404
> -
>
> Key: OOZIE-2882
> URL: https://issues.apache.org/jira/browse/OOZIE-2882
> Project: Oozie
> Issue Type: Improvement
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.2.0
>
>
> Only one of the properties are allowed [oozie.wf.rerun.skip.nodes OR
> oozie.wf.rerun.failnodes]
> Reproduction:
> 1. Create a workflow with more than 1 node. Eg: Fork - with three parallel
> shell actions. Make sure one of them fails
> 2. Rerun with 'oozie.wf.rerun.failnodes' set.
> 3. Rerun again with 'oozie.wf.rerun.skip.nodes' and check 'Skip all
> successful nodes'.
> You will get the following error.
> Error: E0404 : E0404: Only one of the properties are allowed
> [oozie.wf.rerun.skip.nodes OR oozie.wf.rerun.failnodes]
> When a user reruns a workflow job with oozie.wf.rerun.failnode=true and if
> the job fails in subsequent steps, we do not have an option to resubmit the
> workflow using oozie.wf.rerun.skip.node=action1,action2 to allow submission
> from predecessor steps.
> Currently, once the workflow fails and one of the rerun options is used for
> job rerun it gets merged and there is no way to override like regular oozie
> configurations or variables.
> We have a few options:
> 1. If fail.nodes and skip.nodes are specified at the same time (or one of
> them was carried over from a previous wf run), we can add {generate
> skip.nodes by discovering nodes that did not fail} union {skip.nodes}
> 2. Add a way to remove properties (this is also is potentially helpful for
> other use cases)
> 3. The "newest" property (oozie.wf.rerun.skip.nodes or
> oozie.wf.rerun.failnodes) takes priority and the previous is ignored
> 4. Make oozie.wf.rerun.skip.nodes or oozie.wf.rerun.failnodes somehow not
> persist in the DB
> Part of this JIRA would be to figure out which is the best option.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (OOZIE-3528) Migrate to PowerMock 2 and Mockito2
[ https://issues.apache.org/jira/browse/OOZIE-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton updated OOZIE-3528: -- Attachment: OOZIE-3528-002.patch > Migrate to PowerMock 2 and Mockito2 > --- > > Key: OOZIE-3528 > URL: https://issues.apache.org/jira/browse/OOZIE-3528 > Project: Oozie > Issue Type: Improvement >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > Attachments: OOZIE-3528-001.patch, OOZIE-3528-002.patch, > WIP_OOZIE-3528.patch > > > PowerMock 1 does not support JDK11. If we want to . support it we will need > to migrate to PoweMock 2 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (OOZIE-3528) Migrate to PowerMock 2 and Mockito2
[ https://issues.apache.org/jira/browse/OOZIE-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton updated OOZIE-3528: -- Attachment: OOZIE-3528-001.patch > Migrate to PowerMock 2 and Mockito2 > --- > > Key: OOZIE-3528 > URL: https://issues.apache.org/jira/browse/OOZIE-3528 > Project: Oozie > Issue Type: Improvement >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > Attachments: OOZIE-3528-001.patch, WIP_OOZIE-3528.patch > > > PowerMock 1 does not support JDK11. If we want to . support it we will need > to migrate to PoweMock 2 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (OOZIE-3265) properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together
[ https://issues.apache.org/jira/browse/OOZIE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton updated OOZIE-3265: -- Attachment: OOZIE-3265-007.patch > properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear > together > -- > > Key: OOZIE-3265 > URL: https://issues.apache.org/jira/browse/OOZIE-3265 > Project: Oozie > Issue Type: Task >Affects Versions: 5.0.0 >Reporter: TIAN XING >Assignee: Julia Kinga Marton >Priority: Minor > Attachments: OOZIE-3265-004.patch, OOZIE-3265-005.patch, > OOZIE-3265-006.patch, OOZIE-3265-007.patch, OOZIE-3265-v1.patch, > OOZIE-3265-v2.patch, OOZIE-3265-v3.patch, rerun.patch > > > Currently when you re-run a workflow with property "oozie.wf.rerun.failnodes" > set to true, > you can no longer re-run it again with "oozie.wf.rerun.skip.nodes" property > specified, even if you set "oozie.wf.rerun.failnodes" to false. > This kind of limitation is not reasonable. There is only one case where > "oozie.wf.rerun.failnodes" is true and "oozie.wf.rerun.skip.nodes" is not > null or empty, that should be disallowed. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (OOZIE-2755) Oozie HA: ZKJobsConcurrencyService throws runtime exception when numOozies is zero
[
https://issues.apache.org/jira/browse/OOZIE-2755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886894#comment-16886894
]
Julia Kinga Marton commented on OOZIE-2755:
---
committed to master
> Oozie HA: ZKJobsConcurrencyService throws runtime exception when numOozies is
> zero
> --
>
> Key: OOZIE-2755
> URL: https://issues.apache.org/jira/browse/OOZIE-2755
> Project: Oozie
> Issue Type: Bug
> Components: HA
>Reporter: Dongying Jiao
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-2755-01.patch, OOZIE-2755-02.patch
>
>
> Setting up Oozie HA using virtual IP, {{server-1}} and {{server-2}}
> (active-active), when we take down {{server-1}} any Oozie job submitted fails
> with below stacktrace. If both are up , there is no issue.
> {code:java}
> ERROR RecoveryService$RecoveryRunnable:517 - SERVER[XXX] USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] Exception, / by zero
> java.lang.ArithmeticException: / by zero
> at
> org.apache.oozie.service.ZKJobsConcurrencyService.checkJobIdForServer(ZKJobsConcurrencyService.java:167)
> at
> org.apache.oozie.service.ZKJobsConcurrencyService.isJobIdForThisServer(ZKJobsConcurrencyService.java:129)
> at
> org.apache.oozie.service.RecoveryService$RecoveryRunnable.runWFRecovery(RecoveryService.java:362)
> at
> org.apache.oozie.service.RecoveryService$RecoveryRunnable.run(RecoveryService.java:146)
> at
> org.apache.oozie.service.SchedulerService$2.run(SchedulerService.java:175)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (OOZIE-2755) Oozie HA: ZKJobsConcurrencyService throws runtime exception when numOozies is zero
[
https://issues.apache.org/jira/browse/OOZIE-2755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886890#comment-16886890
]
Julia Kinga Marton commented on OOZIE-2755:
---
I agree with [~asalamon74] that simply returning true may cause other issues.
+1 for this specific warning approach.
> Oozie HA: ZKJobsConcurrencyService throws runtime exception when numOozies is
> zero
> --
>
> Key: OOZIE-2755
> URL: https://issues.apache.org/jira/browse/OOZIE-2755
> Project: Oozie
> Issue Type: Bug
> Components: HA
>Reporter: Dongying Jiao
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-2755-01.patch, OOZIE-2755-02.patch
>
>
> Setting up Oozie HA using virtual IP, {{server-1}} and {{server-2}}
> (active-active), when we take down {{server-1}} any Oozie job submitted fails
> with below stacktrace. If both are up , there is no issue.
> {code:java}
> ERROR RecoveryService$RecoveryRunnable:517 - SERVER[XXX] USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] Exception, / by zero
> java.lang.ArithmeticException: / by zero
> at
> org.apache.oozie.service.ZKJobsConcurrencyService.checkJobIdForServer(ZKJobsConcurrencyService.java:167)
> at
> org.apache.oozie.service.ZKJobsConcurrencyService.isJobIdForThisServer(ZKJobsConcurrencyService.java:129)
> at
> org.apache.oozie.service.RecoveryService$RecoveryRunnable.runWFRecovery(RecoveryService.java:362)
> at
> org.apache.oozie.service.RecoveryService$RecoveryRunnable.run(RecoveryService.java:146)
> at
> org.apache.oozie.service.SchedulerService$2.run(SchedulerService.java:175)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (OOZIE-3265) properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together
[ https://issues.apache.org/jira/browse/OOZIE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885147#comment-16885147 ] Julia Kinga Marton commented on OOZIE-3265: --- Sure [~asalamon74], done > properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear > together > -- > > Key: OOZIE-3265 > URL: https://issues.apache.org/jira/browse/OOZIE-3265 > Project: Oozie > Issue Type: Task >Affects Versions: 5.0.0 >Reporter: TIAN XING >Assignee: Julia Kinga Marton >Priority: Minor > Attachments: OOZIE-3265-004.patch, OOZIE-3265-005.patch, > OOZIE-3265-006.patch, OOZIE-3265-v1.patch, OOZIE-3265-v2.patch, > OOZIE-3265-v3.patch, rerun.patch > > > Currently when you re-run a workflow with property "oozie.wf.rerun.failnodes" > set to true, > you can no longer re-run it again with "oozie.wf.rerun.skip.nodes" property > specified, even if you set "oozie.wf.rerun.failnodes" to false. > This kind of limitation is not reasonable. There is only one case where > "oozie.wf.rerun.failnodes" is true and "oozie.wf.rerun.skip.nodes" is not > null or empty, that should be disallowed. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (OOZIE-3528) Migrate to PowerMock 2 and Mockito2
[ https://issues.apache.org/jira/browse/OOZIE-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16883758#comment-16883758 ] Julia Kinga Marton edited comment on OOZIE-3528 at 7/12/19 12:26 PM: - *DO NOT MERGE WIP_OOZIE-3528.patch*! I expect to have a couple of test failure. I have uploaded it to check the precommit result. was (Author: kmarton): DO NOT MERGE YET! I expect to have a couple of test failure. I have uploaded it to check the precommit result. > Migrate to PowerMock 2 and Mockito2 > --- > > Key: OOZIE-3528 > URL: https://issues.apache.org/jira/browse/OOZIE-3528 > Project: Oozie > Issue Type: Improvement >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > Attachments: WIP_OOZIE-3528.patch > > > PowerMock 1 does not support JDK11. If we want to . support it we will need > to migrate to PoweMock 2 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (OOZIE-3528) Migrate to PowerMock 2 and Mockito2
[ https://issues.apache.org/jira/browse/OOZIE-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton updated OOZIE-3528: -- Attachment: WIP_OOZIE-3528.patch > Migrate to PowerMock 2 and Mockito2 > --- > > Key: OOZIE-3528 > URL: https://issues.apache.org/jira/browse/OOZIE-3528 > Project: Oozie > Issue Type: Improvement >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > Attachments: WIP_OOZIE-3528.patch > > > PowerMock 1 does not support JDK11. If we want to . support it we will need > to migrate to PoweMock 2 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (OOZIE-3528) Migrate to PowerMock 2 and Mockito2
[ https://issues.apache.org/jira/browse/OOZIE-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16883758#comment-16883758 ] Julia Kinga Marton commented on OOZIE-3528: --- DO NOT MERGE YET! I expect to have a couple of test failure. I have uploaded it to check the precommit result. > Migrate to PowerMock 2 and Mockito2 > --- > > Key: OOZIE-3528 > URL: https://issues.apache.org/jira/browse/OOZIE-3528 > Project: Oozie > Issue Type: Improvement >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > Attachments: WIP_OOZIE-3528.patch > > > PowerMock 1 does not support JDK11. If we want to . support it we will need > to migrate to PoweMock 2 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (OOZIE-3528) Migrate to PowerMock 2 and Mockito2
[ https://issues.apache.org/jira/browse/OOZIE-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton updated OOZIE-3528: -- Attachment: (was: OOZIE-3528-001.patch) > Migrate to PowerMock 2 and Mockito2 > --- > > Key: OOZIE-3528 > URL: https://issues.apache.org/jira/browse/OOZIE-3528 > Project: Oozie > Issue Type: Improvement >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > > PowerMock 1 does not support JDK11. If we want to . support it we will need > to migrate to PoweMock 2 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (OOZIE-3528) Migrate to PowerMock 2 and Mockito2
[ https://issues.apache.org/jira/browse/OOZIE-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton updated OOZIE-3528: -- Attachment: OOZIE-3528-001.patch > Migrate to PowerMock 2 and Mockito2 > --- > > Key: OOZIE-3528 > URL: https://issues.apache.org/jira/browse/OOZIE-3528 > Project: Oozie > Issue Type: Improvement >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > > PowerMock 1 does not support JDK11. If we want to . support it we will need > to migrate to PoweMock 2 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (OOZIE-3528) Migrate to Powermock 2 and Mockito2
[ https://issues.apache.org/jira/browse/OOZIE-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton updated OOZIE-3528: -- Description: PowerMock 1 does not support JDK11. If we want to . support it we will need to migrate to PoweMock 2 > Migrate to Powermock 2 and Mockito2 > --- > > Key: OOZIE-3528 > URL: https://issues.apache.org/jira/browse/OOZIE-3528 > Project: Oozie > Issue Type: Improvement >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > > PowerMock 1 does not support JDK11. If we want to . support it we will need > to migrate to PoweMock 2 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (OOZIE-3528) Migrate to PowerMock 2 and Mockito2
[ https://issues.apache.org/jira/browse/OOZIE-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton updated OOZIE-3528: -- Summary: Migrate to PowerMock 2 and Mockito2 (was: Migrate to Powermock 2 and Mockito2) > Migrate to PowerMock 2 and Mockito2 > --- > > Key: OOZIE-3528 > URL: https://issues.apache.org/jira/browse/OOZIE-3528 > Project: Oozie > Issue Type: Improvement >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > > PowerMock 1 does not support JDK11. If we want to . support it we will need > to migrate to PoweMock 2 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (OOZIE-3504) [Java 11] Fix tests using PowerMock
[ https://issues.apache.org/jira/browse/OOZIE-3504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16883748#comment-16883748 ] Julia Kinga Marton commented on OOZIE-3504: --- It seems that Powermock2 supports JDK11, so I have opened a Jira to make this upgrade: OOZIE-3528 > [Java 11] Fix tests using PowerMock > --- > > Key: OOZIE-3504 > URL: https://issues.apache.org/jira/browse/OOZIE-3504 > Project: Oozie > Issue Type: Sub-task >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > > With JDK 11 the following tests are failing with initialisation error: > * TestScriptLanguageActionExecutor > * TestHCatCredentials > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (OOZIE-3528) Migrate to Powermock 2 and Mockito2
Julia Kinga Marton created OOZIE-3528: - Summary: Migrate to Powermock 2 and Mockito2 Key: OOZIE-3528 URL: https://issues.apache.org/jira/browse/OOZIE-3528 Project: Oozie Issue Type: Improvement Reporter: Julia Kinga Marton Assignee: Julia Kinga Marton -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (OOZIE-3265) properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together
[ https://issues.apache.org/jira/browse/OOZIE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton updated OOZIE-3265: -- Attachment: OOZIE-3265-006.patch > properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear > together > -- > > Key: OOZIE-3265 > URL: https://issues.apache.org/jira/browse/OOZIE-3265 > Project: Oozie > Issue Type: Task >Affects Versions: 5.0.0 >Reporter: TIAN XING >Assignee: Julia Kinga Marton >Priority: Minor > Attachments: OOZIE-3265-004.patch, OOZIE-3265-005.patch, > OOZIE-3265-006.patch, OOZIE-3265-v1.patch, OOZIE-3265-v2.patch, > OOZIE-3265-v3.patch, rerun.patch > > > Currently when you re-run a workflow with property "oozie.wf.rerun.failnodes" > set to true, > you can no longer re-run it again with "oozie.wf.rerun.skip.nodes" property > specified, even if you set "oozie.wf.rerun.failnodes" to false. > This kind of limitation is not reasonable. There is only one case where > "oozie.wf.rerun.failnodes" is true and "oozie.wf.rerun.skip.nodes" is not > null or empty, that should be disallowed. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (OOZIE-2836) Remove .ps1 and .cmd scripts
[
https://issues.apache.org/jira/browse/OOZIE-2836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton updated OOZIE-2836:
--
Attachment: OOZIE-2836-002.patch
> Remove .ps1 and .cmd scripts
>
>
> Key: OOZIE-2836
> URL: https://issues.apache.org/jira/browse/OOZIE-2836
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Minor
> Attachments: OOZIE-2836-002.patch, OOZIE-2836-01.patch
>
>
> With OOZIE-2666 Oozie was migrated from Tomcat to embedded jetty. Distro
> shell scripts were updated, but Windows scripts were not adjusted in
> {{distro/src/main/bin}}.
> It would be nice if someone with a Windows machine could fix the following
> scripts:
> - oozie-setup.cmd
> - oozie-setup.ps1
> - oozie-sys.cmd
> - oozie.cmd
> - oozied.cmd
> and verify that Oozie works on Windows.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-2836) Remove .ps1 and .cmd scripts
[
https://issues.apache.org/jira/browse/OOZIE-2836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton updated OOZIE-2836:
--
Summary: Remove .ps1 and .cmd scripts (was: Fix ps1 and cmd scripts to
make Oozie runnable on Windows)
> Remove .ps1 and .cmd scripts
>
>
> Key: OOZIE-2836
> URL: https://issues.apache.org/jira/browse/OOZIE-2836
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Minor
> Attachments: OOZIE-2836-01.patch
>
>
> With OOZIE-2666 Oozie was migrated from Tomcat to embedded jetty. Distro
> shell scripts were updated, but Windows scripts were not adjusted in
> {{distro/src/main/bin}}.
> It would be nice if someone with a Windows machine could fix the following
> scripts:
> - oozie-setup.cmd
> - oozie-setup.ps1
> - oozie-sys.cmd
> - oozie.cmd
> - oozied.cmd
> and verify that Oozie works on Windows.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (OOZIE-2836) Fix ps1 and cmd scripts to make Oozie runnable on Windows
[
https://issues.apache.org/jira/browse/OOZIE-2836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton reassigned OOZIE-2836:
-
Assignee: Julia Kinga Marton
> Fix ps1 and cmd scripts to make Oozie runnable on Windows
> -
>
> Key: OOZIE-2836
> URL: https://issues.apache.org/jira/browse/OOZIE-2836
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Minor
> Attachments: OOZIE-2836-01.patch
>
>
> With OOZIE-2666 Oozie was migrated from Tomcat to embedded jetty. Distro
> shell scripts were updated, but Windows scripts were not adjusted in
> {{distro/src/main/bin}}.
> It would be nice if someone with a Windows machine could fix the following
> scripts:
> - oozie-setup.cmd
> - oozie-setup.ps1
> - oozie-sys.cmd
> - oozie.cmd
> - oozied.cmd
> and verify that Oozie works on Windows.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3468) Use modernizer plugin
[
https://issues.apache.org/jira/browse/OOZIE-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16878327#comment-16878327
]
Julia Kinga Marton commented on OOZIE-3468:
---
+1 for adding the modrnizer to our precommit script.
> Use modernizer plugin
> -
>
> Key: OOZIE-3468
> URL: https://issues.apache.org/jira/browse/OOZIE-3468
> Project: Oozie
> Issue Type: Improvement
> Components: build
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3468-01-wip.patch, OOZIE-3468-02-wip.patch
>
>
> Recently I've opened a few jiras which suggested to use standard java classes
> instead of external libraries ( OOZIE-3463, OOZIE-3467). There is a tool
> which can find such technical depts: [maven modernizer
> plugin|https://github.com/gaul/modernizer-maven-plugin].
> The usage is quite simple:
> {noformat}
> $ mvn modernizer:modernizer
> ...
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:88:
> Prefer java.lang.String.(byte[], java.nio.charset.Charset)
>
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:122:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:85:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:92:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1JobsServlet.java:188:
> Prefer java.util.ArrayList<>()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:91:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:101:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:110:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:119:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/VersionServlet.java:36:
> Prefer java.util.Collections.emptyList()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1AdminServlet.java:49:
> Prefer java.util.Collections.emptyList()
> ...
> {noformat}
> There are several ways to use this plugin:
> # Add the plugin to the root pom and let developers manually use this
> plugin. It's the simplest solution, but it will be easy to forget it.
> # Add this to the precommit script similarly to findbugs and at least avoid
> to insert new code which uses old style API. Probably we will have the same
> problems like we have with findbugs, we will have lots of false positive
> warnings.
> # Turn the plugin on by default and fail the compilation if it finds any
> problem. I think this is too strict.
> If we choose option 2 or 3 we should probably specify an ignore list, I don't
> think for instance that we really want to change all the {{new Long(10)}}
> code to {{Long.valueOf(10)}}.
> By default this plugin checks the target java version (1.8 right now) but
> it's possible to specify 1.7 instead if we want to focus on those problems
> first. (It was not working for me without specifying the java version.)
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3506) Flaky test TestOozieRollingPolicy
[
https://issues.apache.org/jira/browse/OOZIE-3506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16877810#comment-16877810
]
Julia Kinga Marton commented on OOZIE-3506:
---
Thank you [~asalamon74]! +1, committed to master
> Flaky test TestOozieRollingPolicy
> -
>
> Key: OOZIE-3506
> URL: https://issues.apache.org/jira/browse/OOZIE-3506
> Project: Oozie
> Issue Type: Sub-task
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Minor
> Attachments: OOZIE-3506-01.patch
>
>
> {{TestOozieRollingPolicy}} sometimes fails with the following error message:
> {noformat}2019-05-30 13:15:21 [INFO] Running
> org.apache.oozie.util.TestOozieRollingPolicy
> 2019-05-30 13:15:21 [ERROR] Tests run: 3, Failures: 1, Errors: 0, Skipped: 0,
> Time elapsed: 87.025 s <<< FAILURE! - in
> org.apache.oozie.util.TestOozieRollingPolicy
> 2019-05-30 13:15:21 [ERROR]
> testDeletingOldFiles(org.apache.oozie.util.TestOozieRollingPolicy) Time
> elapsed: 60.337 s <<< FAILURE!
> 2019-05-30 13:15:21 junit.framework.AssertionFailedError
> 2019-05-30 13:15:21 at junit.framework.Assert.fail(Assert.java:55)
> 2019-05-30 13:15:21 at junit.framework.Assert.assertTrue(Assert.java:22)
> 2019-05-30 13:15:21 at junit.framework.Assert.assertTrue(Assert.java:31)
> 2019-05-30 13:15:21 at
> junit.framework.TestCase.assertTrue(TestCase.java:201)
> 2019-05-30 13:15:21 at
> org.apache.oozie.util.TestOozieRollingPolicy._testDeletingOldFiles(TestOozieRollingPolicy.java:125)
> 2019-05-30 13:15:21 at
> org.apache.oozie.util.TestOozieRollingPolicy._testDeletingOldFiles(TestOozieRollingPolicy.java:59)
> 2019-05-30 13:15:21 at
> org.apache.oozie.util.TestOozieRollingPolicy.testDeletingOldFiles(TestOozieRollingPolicy.java:47)
> ...
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3476) Migrate classes without setup/tearDown to JUnit4
[ https://issues.apache.org/jira/browse/OOZIE-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16877787#comment-16877787 ] Julia Kinga Marton commented on OOZIE-3476: --- thank you [~asalamon74] for the contribution! +1, committed to master > Migrate classes without setup/tearDown to JUnit4 > > > Key: OOZIE-3476 > URL: https://issues.apache.org/jira/browse/OOZIE-3476 > Project: Oozie > Issue Type: Sub-task > Components: tests >Reporter: Andras Salamon >Assignee: Andras Salamon >Priority: Minor > Attachments: OOZIE-3476-01.patch, OOZIE-3476-02.patch, > OOZIE-3476-03.patch > > > There are several classes where we really don't need to extend XTestCase or > even TestCase, we can easily convert them to JUnit4. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3476) Migrate classes without setup/tearDown to JUnit4
[ https://issues.apache.org/jira/browse/OOZIE-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton updated OOZIE-3476: -- Summary: Migrate classes without setup/tearDown to JUnit4 (was: Migrate several classes to JUnit4) > Migrate classes without setup/tearDown to JUnit4 > > > Key: OOZIE-3476 > URL: https://issues.apache.org/jira/browse/OOZIE-3476 > Project: Oozie > Issue Type: Sub-task > Components: tests >Reporter: Andras Salamon >Assignee: Andras Salamon >Priority: Minor > Attachments: OOZIE-3476-01.patch, OOZIE-3476-02.patch, > OOZIE-3476-03.patch > > > There are several classes where we really don't need to extend XTestCase or > even TestCase, we can easily convert them to JUnit4. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3522) Migrate from Guava's Joiner
[ https://issues.apache.org/jira/browse/OOZIE-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16877756#comment-16877756 ] Julia Kinga Marton commented on OOZIE-3522: --- Thank you [~asalamon74] for the contribution! +1, committed to master > Migrate from Guava's Joiner > --- > > Key: OOZIE-3522 > URL: https://issues.apache.org/jira/browse/OOZIE-3522 > Project: Oozie > Issue Type: Sub-task >Reporter: Andras Salamon >Assignee: Andras Salamon >Priority: Major > Attachments: OOZIE-3522-01.patch > > > As mentioned in OOZIE-3488 we might use standard JDK classes instead of the > [Joiner|https://guava.dev/releases/11.0.2/api/docs/index.html?com/google/common/base/Joiner.MapJoiner.html] > class of Guava as suggested here: https://stackoverflow.com/a/22577565/21348 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3266) Coord action rerun support RERUN_SKIP_NODES option
[ https://issues.apache.org/jira/browse/OOZIE-3266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16877740#comment-16877740 ] Julia Kinga Marton commented on OOZIE-3266: --- [~zuston], please upload the new patch here as well, to trigger the precommit job. Also please cover the new feature with test cases and update the documentation as well (DG_CoordinatorRerun.md) > Coord action rerun support RERUN_SKIP_NODES option > -- > > Key: OOZIE-3266 > URL: https://issues.apache.org/jira/browse/OOZIE-3266 > Project: Oozie > Issue Type: New Feature >Affects Versions: 5.1.0 >Reporter: TIAN XING >Assignee: Junfan Zhang >Priority: Major > Fix For: 5.2.0 > > Attachments: OOZIE-3266-v1.txt, OOZIE-3266-v2.txt > > > currently, when you re-run a workflow job, you have 3 options > # re-run all of its action nodes > # re-run failed nodes only > # re-run with specified nodes skipped > if this workflow job is generated by a coord action. you can re-run this > coord action with 2 options > # re-run all of the workflow action nodes (generate a new workflow job id) > # re-run failed workflow action nodes only (workflow job id not changed) > now we want to add a another option > - re-run with specified workflow nodes skipped (workflow job id not changed) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3513) Migrate from Preconditions.checkNotNull and ParamChecker.notNull
[
https://issues.apache.org/jira/browse/OOZIE-3513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876045#comment-16876045
]
Julia Kinga Marton commented on OOZIE-3513:
---
Thank you @asalamon for the contribution! +1, committed to master
> Migrate from Preconditions.checkNotNull and ParamChecker.notNull
>
>
> Key: OOZIE-3513
> URL: https://issues.apache.org/jira/browse/OOZIE-3513
> Project: Oozie
> Issue Type: Sub-task
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3513-01.patch, OOZIE-3513-02.patch,
> OOZIE-3513-03.patch, OOZIE-3513-04.patch, OOZIE-3513-05.patch
>
>
> We currently use both Guava's {{Preconditions.checkNotNull}} and our own
> {{ParamChecker.notNull}} to check for null arguments. Instead we should use
> the standard {{Objects.requireNonNull}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3266) Coord action rerun support RERUN_SKIP_NODES option
[ https://issues.apache.org/jira/browse/OOZIE-3266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16875971#comment-16875971 ] Julia Kinga Marton commented on OOZIE-3266: --- Thank you [~zuston]! I have left some comments on the RB. > Coord action rerun support RERUN_SKIP_NODES option > -- > > Key: OOZIE-3266 > URL: https://issues.apache.org/jira/browse/OOZIE-3266 > Project: Oozie > Issue Type: New Feature >Affects Versions: 5.1.0 >Reporter: TIAN XING >Assignee: Junfan Zhang >Priority: Major > Fix For: 5.2.0 > > Attachments: OOZIE-3266-v1.txt, OOZIE-3266-v2.txt > > > currently, when you re-run a workflow job, you have 3 options > # re-run all of its action nodes > # re-run failed nodes only > # re-run with specified nodes skipped > if this workflow job is generated by a coord action. you can re-run this > coord action with 2 options > # re-run all of the workflow action nodes (generate a new workflow job id) > # re-run failed workflow action nodes only (workflow job id not changed) > now we want to add a another option > - re-run with specified workflow nodes skipped (workflow job id not changed) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2907) Delete PrepareActionsDriver from oozie-sharelib
[ https://issues.apache.org/jira/browse/OOZIE-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16874734#comment-16874734 ] Julia Kinga Marton commented on OOZIE-2907: --- Thank you [~asalamon74] for the contribution! +1, committed to master > Delete PrepareActionsDriver from oozie-sharelib > --- > > Key: OOZIE-2907 > URL: https://issues.apache.org/jira/browse/OOZIE-2907 > Project: Oozie > Issue Type: Sub-task >Affects Versions: 5.0.0 >Reporter: Peter Bacsko >Assignee: Andras Salamon >Priority: Minor > Attachments: OOZIE-2907-01.patch, OOZIE-2907-02.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
